One lucky run is not an agent evaluation.
Turn an impressive demo into a repeatable evaluation with explicit outcomes, multiple trials, and readable traces.
Grade the outcome
An agent saying that it completed a task is a claim. Verify the actual result: the requested file exists, the edited code passes the relevant checks, or the intended record has the correct state.
Anthropic’s agent evaluation guidance distinguishes tasks, repeated trials, transcripts, and outcomes. That distinction is useful when a convincing final response hides a failed action.
Keep a small, repeatable suite
Build cases from real failure modes: ambiguous input, a missing dependency, a tool failure, and a task that needs more than one step. Fix the environment and record the model, prompt, tool configuration, and grading rules.
For an initial smoke comparison, five trials per case can reveal obvious variability. This is a suggested starting point, not a statistical guarantee. Expand the run budget according to the failure rate you need to detect and the confidence required for the decision.
suite: agent-regression
trials_per_case: 5 # initial smoke check
record:
- model_and_prompt_version
- tool_inputs_and_results
- final_environment_state
grade:
- requested_outcome_exists
- required_checks_pass
- unintended_changes_absentCompare changes on the same cases
Run the baseline and candidate on the same case set. Report success counts per case alongside latency and failure types. An aggregate improvement can conceal a regression in a critical workflow.
Use deterministic checks where the outcome can be measured directly. When using a model to judge open-ended quality, define the rubric and calibrate it against human judgments.
Read the failures
Inspect tool calls, intermediate results, and the final state for failed trials. Separate a reasoning mistake from a missing permission, a tool error, or a broken fixture. Each requires a different fix.