Latest
Why Anthropic Says the Old Way of Testing AI No Longer Works for Agents
According to Anthropic, the autonomy, intelligence, and flexibility that make AI agents useful are the same properties that make them harder to evaluate—pushing the field toward multi-turn evaluations that examine full environment state rather than single responses.
Why a single prompt-and-response test no longer works
For most of the short history of large language models, evaluating an AI meant giving it a question and checking the answer. That approach worked when a model's job was to return a single response. Agents operate differently: they call tools, modify state, and adapt across many turns—and that is exactly what makes the old testing method break down. According to Anthropic's engineering post on agent evaluations, the autonomy, intelligence, and flexibility that make agents useful are the same properties that make them harder to evaluate.
The practical consequence is error compounding. Because an agent takes action after action, a mistake early in a run does not stay isolated—it propagates through subsequent steps. Anthropic's post also notes that frontier models can find creative solutions that exceed the limits of static evaluations: Opus 4.5 reportedly solved a flight-booking benchmark problem by discovering a policy loophole, technically failing the evaluation as written while producing a better outcome for the user. That example, cited by Anthropic, illustrates a genuine challenge: a grader designed around expected paths can penalise genuinely good reasoning.
What multi-turn evaluation adds—and what the dossier leaves open
As agent capabilities have expanded, Anthropic reports that multi-turn evaluations have become increasingly common. In place of a single grader checking one output, a multi-turn eval supplies the agent with tools, a task, and an environment, then examines what the environment looks like after the full run. Anthropic's post describes a coding-agent example in which the agent is asked to build an MCP server (a type of integration endpoint); success is verified afterwards by running unit tests against the resulting implementation—not by inspecting any single reply.
The shift in vocabulary reflects the shift in logic. Anthropic's framework distinguishes a task (the test case with defined success criteria), a trial (one attempt at that task), a transcript (the complete record of tool calls, reasoning, and results), and an outcome (the final environment state). Running multiple trials per task is recommended because model outputs vary between runs. These definitions are drawn from Anthropic's own published guidance; independent verification of their effectiveness across other architectures or organisations is not part of the available evidence. Whether the same approach transfers cleanly to agents with longer horizons, external APIs, or different risk profiles remains an open question worth investigating before applying the framework wholesale.
Sources
Change history
- rev.1Published
