| |
|
| |
Join us in San Francisco (SoMa) for a night of talks on building with decision models & using them to evaluate agents. Sydney Runkle, a Product Manager on our OSS team, will show how to pair Jev with traditional LLMs to build effective agents. Sen Roche, an Education Engineer, will cover Jev as a judge for agent evals.
Talk 1: Decision Models & LangGraph with Sydney
Most agents send every judgment call to a frontier LLM. That covers which model to route to, which tools to load, whether an action is safe & whether the work is good enough, so each step is slower & more expensive than it needs to be. Decision models like TypeSafe AI's Jev are built for these calls. They don't generate text, instead they answer typed questions about an agent's state with calibrated probabilities. In this talk, we cover how decision models work, where they fit in an agent harness, & why getting good answers from them is still a context engineering problem. We'll close with a pattern for pairing them with LLMs: cheap by default, frontier on exception.
Talk 2: Jev as a judge with Sen
Code-based evals are fast but narrow, & LLM judges are flexible but slow, expensive, & inconsistent. We tested a third option: Jev, a model from TypeSafe AI that returns typed answers & probabilities instead of generating text. Using Deep Agents & LangSmith, we replayed the same agent runs against Jev, GPT-5.6 Luna, GPT-5.6 Terra, & Claude Sonnet 4.6, comparing accuracy, consistency, latency, & cost. In this talk we'll cover the methodology behind this experiment, dive into the results, & discuss their implications on the future of agent evals.
Agenda
6:00 PM: Welcome + Refreshments
6:30 PM: Presentation with Sydney - Decision Models & LangGraph
6:45 PM: Q&A
6:55 PM: Presentation with Sen - Jev as a judge
7:10 PM: Q&A
7:20 PM: Networking
8:30 PM: Event Ends
Event info:
Please note this event is fully in-person & will not be live-streamed or recorded.
Location: SoMa - Address provided upon approval.
We can only admit guests with approved registrations.
About the host:
LangChain enables every company to own their intelligence. LangChain's open, model-agnostic harnesses give teams choice & control over how they build their agent architecture. LangSmith brings testing, deployment, & monitoring together so teams can continuously improve their agents, compound intelligence, & govern them at scale. More than 7,000 customers, including Nvidia, Bridgewater, LinkedIn, Workday, Harvey, & Rippling trust LangSmith to improve their agents across the Agent Development Lifecycle. Learn more: www.langchain.com
|
|
|
|
|
|
|
|
