Evals for AI Agents: How to Get the Most Out of Every New Model
## A live session on evaluating AI agents, for Claude Startups members.
New models ship fast, and many teams that are shipping agents can't tell whether the latest one makes their product better. Agents call tools, pull in context and take several steps before a customer sees anything, and each step can fail in its own way.
We'll show how the startups we work with evaluate agents end to end and decide quickly whether a new model is worth switching to. You'll see real examples from Anthropic's work with teams building on Claude and leave with something you can use the same week.
## What you'll learn
Why single-turn evals miss most of what goes wrong in an agent, and what to measure instead
How to build your first eval set from real production failures
How to decide quickly whether a new model is worth adopting for your product
Speakers
Anmol Tukrel
Member of Technical Staff, Anthropic
May 22, 2025
San Francisco, California
AI Innovation in Finance
An Executive Dinner with Anthropic
Join Anthropic’s first developer conference, Code with Claude. You’ll hear from our product leaders, participate in hands-on workshops, and connect with fellow developers who are building responsible AI applications. Limited spots available.
Join Anthropic’s first developer conference, Code with Claude. You’ll hear from our product leaders, participate in hands-on workshops, and connect with fellow developers who are building responsible AI applications. Limited spots available.
