| |
AI Agent Evaluations
|
| Wth Claire Longo (Lead AI Researcher, Comet), Shrilakshmi Chavali (Arize), Ranjan Sinha (IBM Fellow). |
| Venue, To Be Announced, San Francisco |
|
Jul 08 (Tue) , 2025 @ 05:00 PM
| |
FREE |
|
|
|
|
|
|
|
|
| |
| DETAILS |
|
Our 4th AI Agent Meetup is sponsored by BeeAI.
The SF AI Agent Meetup is a new & growing community for AI Agent Developers, Engineers, UX, Ops, & Applied Researchers exploring & leading the evolution of AI agents. Whether you're building autonomous systems, experimenting with LLM-powered assistants, or integrating AI agents into real-world applications, this meetup is the place to discover, share insights, & collaborate with your peers. Join us for talks, demos, discussions & networking with like-minded innovators shaping the next generation of AI.
Agenda
5:00 pm - Checkin & Networking
5:30 pm - Introductions & Welcome
5:45 pm - Lightning Talk
Natural Language Interaction Protocol (NLIP) by Ranjan Sinha, IBM Fellow
6:00 pm - Talks:
"Best Practices for Monitoring Agent Applications in Production" by Claire Longo, Lead AI Researcher at Comet
Agent Evaluation by Shrilakshmi Chavali of Arize
Open RAG Eval by Ofer Mendelevitch at Vectara
7:30 pm - Open Networking w/ Pizza & Sodas
8:30 pm - Close
Talk Details:
Talk #1 - "Best Practices for Monitoring Agent Applications in Production" by Claire Longo, Lead AI Researcher at Comet - Building AI agents requires a shift from monolithic to modular architectures, especially when dealing with the non-deterministic nature of LLMs. Traditional monitoring falls short as Agents can silently escalate costs, hallucinate, & generate biased outputs. In this talk, Claire will share practical strategies for AI Agent Observability, including cost tracking, eval metrics, & automated Agent Optimization. She will also introduce Comet Opik, an open-source tool designed to optimize & monitor AI Agents in real time.
We look forward to building San Francisco's AI Agent community together!
Food & drink, engaging conversation, & incredible company will all be provided!
This AI Alliance event is sponsored by BeeAI.
Why join?
This is your chance to dive deep into the world of AI agents, connect with a new & emerging developer community. This event is designed for experienced developers, & those who are curious about agents.
Please be advised: Unfortunately, space is very limited at these community events & we can not always accept everyone we would like to. We appreciate your application & are looking forward to seeing you at a future event!
By coming to this event you agree to be part of group photographs which are used for community building & promoting future events.
Code of Conduct: We expect our members to treat each other like family, so please click here to read our Code of Conduct. If you see someone violating it, please speak with your event organizer or reach out to dnielsen@thealliance.ai.
The AI Alliance is a non-profit, grassroots community of 150+ companies, startups, researchers, & individuals. Through world-class events, online events, & workshops, members leverage their diverse perspectives & innovative minds to foster meaningful relationships, solve challenging problems, & define the future of AI.
|
|
|
|
|
|
|
|
