Join us for an evening of technical presentations featuring two exciting talks on simulation & database observability!

Date: Tuesday, October 1, 2025

Time: 6:00 PM - 8:00 PM

Venue: Datadog NYC Office

Recording: This event will be recorded, including Q&A

Agenda:

Talk 1: Observability-Based Index Recommendations - Alex Weisberger

Alex, a full stack software engineer at Datadog, will discuss database observability & index optimization techniques.

Speaker Bio - Alex Weisberger:

Alex is a full stack software engineer at Datadog with experience across diverse environments including trucking navigation, sheet music recognition, karaoke apps, payment processing, commercial real estate SaaS, & database observability. Passionate about correctness & reliability, with expertise in TLA+, property-based testing, & discrete event simulation.

Talk 2: Persona Based Evaluation of Search Systems - Uri Goren

Search evaluation is notoriously complex, with even minor tweaks to hyperparameters or embeddings creating widespread ripple effects across retrieved results. Mitigating this inherent instability in search algorithm changes has long been a challenge. Traditional approaches, such as composing test cases, offer a degree of control & consistency. However, writing & maintaining test cases is a painstaking task, particularly in dynamic environments where catalogs are constantly updated with new items. Our proposed method introduces a groundbreaking solution: leveraging user modeling inspired by the LLM as a judge paradigm to automate query & result-set generation. This approach dynamically creates realistic query-result pairs by simulating diverse user personas, each designed to evaluate different modalities (image, video, audio) within the catalog under varying prompts.

The innovation lies in the adaptability & efficiency of the system:

• Dynamic Test Coverage: Personas adapt to catalog changes, ensuring that new items are immediately evaluated without manual intervention.


• Multi-Modality Testing: By supporting multiple content types, the system mirrors the diversity of modern search use cases.


• Fully Automated Pipeline: The process eliminates the need for manually written test cases, reducing overhead & accelerating iteration cycles.

This talk will provide an in-depth look at the methodology, the benefits of dynamic user modeling, & real-world results from applying this system. Attendees will leave with actionable insights into transforming their search evaluation strategies, unlocking new levels of stability & precision in their algorithms.

***

Registration: Please register here by September 30th at 12:00 PM as we need to give the guest list to security in advance. Please make sure you are using your real name (check your ID) to speed up the process of getting in.

This event is open to all levels. Newcomers & beginners are welcome. All NYC Python events are governed by our Code of Conduct.