| |
|
| |
Join us for Agentic + AI Observability meetup on Tuesday, October 13th from 5pm - 8pm PST at the Databricks SF office, an evening focused on evaluating, debugging, & optimizing agentic architectures. We'll explore how to leverage observed agentic traces to continuously iterate on & improve complex AI systems in production.
This meetup is built for software engineers, ML/AI developers, system architects, & AI startup founders who are building or deploying autonomous agents & want to go deeper into the tech. We'll dive into real-world patterns, failure modes, & engineering solutions for composing multi-step agent workflows, extracting actionable insights from production traces, & establishing robust evaluation loops to refine agent behavior over time.
Whether you're at an early-stage startup or an established company, if you care about getting AI agents into production & keeping multi-agent systems healthy, this meetup is for you.
Why you should attend
Debug & Iterate on Production Agents: Move past generic logs to structured agentic traces, evaluation metrics, & real-time observability pipelines that isolate system failures & track complex model reasoning.
Engineer Multi-Step Agent Workflows: Discover practical architectural patterns for composing, running, & managing multi-step agentic systems safely at scale.
Turn Observed Traces Into Model Improvements: Learn how to analyze execution traces to fine-tune prompts, refine tool selection, & continuously iterate on your AI workflows post-deployment.
Connect with Other Builders: Meet engineers, founders, & practitioners to discuss production observability challenges, swap agent architectural patterns, & network with builders.
Agenda
5:00pm: Registration/Mingling
6:00pm: Welcome Remarks
6:15pm: Talk #1: Hidden Gems in Agent Traces: Automatically Improving Your Agent with Open Source MLflow Observability (Yuki Watanabe, Sr. Software Engineer at Databricks)
6:45pm: Talk #2 with Mandeep Singh, AI Deployment Engineer at OpenAI
7:15pm: Mingling with bites + dessert
8:00pm: Night Ends
Session Descriptions:
Hidden Gems in Agent Traces: Automatically Improving Your Agent with Open Source MLflow Observability
Agent traces capture where your agent gets stuck, which mistakes repeat, & where users get frustrated. In this talk, we'll walk through how to use those signals to improve agent quality. We'll show how automatic trace analysis surfaces patterns & quality issues, & how to turn those findings into improvements with repeatable verification. We'll demonstrate the workflow with MLflow, an open-source observability & evaluation tool.
|
|
|
|
|
|
|
|
