core-concept · Evaluation & verification · maturing · Reviewed
Eval-driven development
An engineering loop that converts expected behavior and observed failures into repeatable evaluations combining deterministic checks, environment inspection, security testing, model graders, repeated trials, and human judgment.
Definition
An engineering loop that converts expected behavior and observed failures into repeatable evaluations combining deterministic checks, environment inspection, security testing, model graders, repeated trials, and human judgment. The test target is the whole agent system: outcome, trajectory, environment, side effects, and policy adherence.
The discipline is becoming concrete while measurement standards remain nascent.
Why it matters
The test target is the whole agent system: outcome, trajectory, environment, side effects, and policy adherence.
The operating shift is from “A benchmark score” to “A living system-level assurance suite.”
System anatomy
- Practice 1
- Evaluate outcomes and trajectories
- Practice 2
- Calibrate graders against expert judgment
- Practice 3
- Turn production corrections into regression cases
Important distinctions
- A benchmark score
- This concept moves the engineering system toward a living system-level assurance suite.
Implementation signals
- Evaluate outcomes and trajectories
- Calibrate graders against expert judgment
- Turn production corrections into regression cases
Failure modes
- Broken tasks producing authoritative scores
- Optimizing the benchmark instead of the operating outcome
Sources and further study
- Anthropic — Demystifying evals for AI agents
A practical treatment of evaluating trajectories, outcomes, graders, tasks, and agent-environment interaction.
Use in this library: Official guidance. This is first-party engineering guidance drawn from Anthropic deployments. Evaluation designs remain task- and environment-specific, and the article does not establish independent verifier ownership by itself.
engineering · engineering · Published 2026-01-09 - NIST — Towards best practices for automated benchmark evaluations
Guidance on the integrity and limitations of automated benchmark evaluation practices.
Use in this library: Official guidance. Guidance describes recommended practice; citation does not prove that a control is implemented or effective in a particular environment.
guidance · guidance · Published 2026-01-30 - OpenAI — Building self-improving tax agents with Codex
A feedback-loop pattern that turns practitioner corrections and production traces into evaluations and scoped engineering work.
Use in this library: First-party case study. This first-party account documents one organization, product, or implementation context and should not be generalized without local evidence.
engineering · engineering · Published 2026-05-27