Treat a prompt change like a release.
Version the behavior you expect, compare it against a baseline, and keep enough evidence to understand regressions.
Make the change reproducible
Give each candidate a readable version such as reference-resolution-v3. Save the exact prompt, model identifier, runtime settings, tool schemas, and fixture revision. Comparing different prompts on different fixtures answers a different question from testing the prompt itself.
Exercise the boundaries
For a prompt that combines images and text, include text-only, image-only, mixed, missing, conflicting, and repeated references. Define the expected resolution rules before looking at the output.
Check both the assembled request and the final result. If the request is wrong, a plausible output can still make the demo look successful.
Write the release decision before the run
Choose the blocking cases, repeat budget, acceptable error rate, and cost or latency constraints in advance. Compare the candidate with the current version on those same cases, and report disagreements instead of smoothing them into a single score.
Keep a stable regression suite and a separate set of harder capability cases. Passing the old cases protects existing behavior; the harder cases show where improvement is still needed.
Further reading: Anthropic’s capability and regression evaluation guidance ↗