A benchmark score is a point estimate on a curated, static, in-distribution task set. Production reliability is a distribution under drift and stress — and the measured gap between the two widens exactly as conditions get harder.
An agent posts a strong number on a well-known agent benchmark — task completion in the high eighties, a tier from the top of the leaderboard — and the team ships it. The number was real. It came out of an honest evaluation on a respected task set, and nobody fudged anything.
Then the agent meets production, and its measured success rate is visibly lower than the leaderboard said. Worse than the offset, it is jumpy: it sails through a task on Monday and fails the same kind of task on Tuesday because a field arrived phrased a little differently, a tool answered slower than usual, the user changed their mind halfway through. The benchmark reported one confident number. Production is serving a distribution, and the distribution has a long, ugly tail the number never mentioned.
A benchmark score is a point estimate. It is one number, produced by running a fixed set of tasks — usually once each — in a fixed and cooperative environment. Production reliability is not a point estimate. It is a distribution: the spread of outcomes an agent produces across rephrased inputs, flaky tools, adversarial content, and tasks whose goal moves while the agent is working. The score and the reliability are different mathematical objects, and the recent reliability-focused benchmarks exist precisely to measure the distance between them. They find it large, and they find it grows as conditions get harder.
A leaderboard number is necessary. It has never been sufficient. This post is about what else you have to measure before that number is allowed to gate a launch.
An agent benchmark, to be a benchmark at all, has to hold things constant. It holds the task set constant — a curated collection, often hand-checked to be well-posed. It holds the inputs constant — each task phrased once, and phrased clearly. It holds the environment constant — tools that respond, latencies that behave, no outages. It usually holds the run count at one. And it contains no adversary, because a reproducible benchmark cannot have someone actively attacking it.
Every one of those is a silent assumption, and the score is conditioned on all of them at once. Production honors none of them. Users phrase the same request five different ways, half of them badly. Tools time out, rate-limit, and return yesterday’s data. Goals change mid-task. Some inputs are hostile. The benchmark score is an accurate measurement of agent behavior in a world that has been swept clean of all of that — which is to say, it is a measurement of a good day. The mistake is not running the benchmark. The mistake is reading “84% on a good day” as “84% reliable.”
The single most expensive habit in agent evaluation is reporting pass@1 — did the agent succeed on its one attempt — and calling the result reliability. Pass@1 collapses a distribution to a mean and then hides that it did so.
Consider two agents that both score 80%. The first succeeds on the same 80% of tasks every run and fails the same 20%. The second succeeds on a different, randomly varying 80% each run. Their leaderboard numbers are identical. Their reliability is not remotely identical. For the first agent a user on a given task has a fixed, knowable outcome. For the second — flaky rather than reliably good — two independent runs of a task land on success both times only 64% of the time. The leaderboard cannot tell these two agents apart, and the difference between them is the entire question a production owner is asking.
ReliabilityBench (arXiv 2601.06112) is built around exactly this correction. It refuses to evaluate a tool-using agent on a single axis and instead measures three: consistency across repeated runs, robustness to variations of the task, and tolerance to API failures — 1,280 episodes spanning two models, two agent architectures, and four domains. Read the design as an argument. It is saying that “reliability” is not a number you can have; it is a shape, and you only see the shape if you run the task more than once, vary it, and break the tools underneath it on purpose.
Here is the part that should change how you read a leaderboard. The score-to-reliability gap is not a fixed discount you can mentally subtract. It widens as conditions move away from the benchmark’s curated center, and the recent work measures the widening directly.
Start with the gentlest possible stressor: rephrasing. Not an attack, not noise — just a user saying the same thing in different words. ReliabilityBench applied semantic perturbations of that kind and watched success fall from 96.9% to 88.1%. Roughly nine points of reliability, paid out for nothing but rewording — and rewording is the most ordinary thing a real user does.
Now let the goal itself move. AgentChangeBench (arXiv 2510.18170) evaluates how agents handle an objective that shifts mid-conversation, the way a customer changes their mind partway through a booking. On airline-booking goal shifts, GPT-4o recovered 92.2% of the time; Gemini managed 48.6%. Two models a reader would call comparably capable from their headline scores, and a forty-point spread the moment the task stopped sitting still. The paper’s own framing is the thesis of this post stated as a finding: raw accuracy does not guarantee robustness under dynamic goals.
Now make the environment realistic. WAREX (arXiv 2510.03285) takes browser agents off the clean benchmark and runs them under real-world instability — network issues, web attacks — across three existing benchmarks, and reports significant drops in task success rates against the very numbers those benchmarks otherwise produce. The pattern across all three papers is the same and it is not subtle: introduce any realistic stressor and the score does not survive it, and the harsher the stressor, the wider the gap. The benchmark measured the good day. The gap is the rest of the week.
Drift is benign — inputs that are merely unlike the benchmark. Adversarial stress is someone deliberately attacking, and it is a separate axis the leaderboard does not sample at all, because a static reproducible task set has no attacker inside it.
Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harms (arXiv 2508.16481) measures that axis. Its BAD-ACTS benchmark assembles 188 harmful-action examples across four agentic system implementations, and finds that an adversarial agent operating inside a multi-agent system achieves a high attack-success rate — and, usefully, that message monitoring defends considerably better than prompt-level instructions do. The result that matters for benchmark-reading is the bare fact that the number exists. A leaderboard score is not a low adversarial-robustness number. It is no adversarial-robustness number — the entire axis is absent from the measurement. An agent can top a benchmark and have an adversarial robustness you have literally never measured.
The fix is not to distrust benchmarks. It is to build the evaluation whose distribution actually resembles production, and to report the leaderboard number with the conditions still attached.
Measure consistency, not pass@1. Run every eval task several times and report the distribution — success rate and its spread. An agent that is jumpy at 80% and an agent that is steady at 80% must not be allowed to print the same number.
Perturb the eval set on purpose. Add rephrasings of in-distribution tasks, noisier inputs, and injected tool failures — the ReliabilityBench shape. The drift you simulate in the harness is drift you will not discover in an incident.
Keep an adversarial slice. It does not have to be large. It has to exist, so that “adversarial robustness” is a number you have rather than an axis you forgot.
Maintain the golden set. A benchmark frozen at launch drifts away from production every week that production changes; a rotting golden set makes the gap silently worse, because the pass rate stays high while the relevance falls.
Sample production traces and score them continuously. This is the only evaluation whose distribution is the real one. Agent-native observability — per-step traces fed back into regression evals — is how the leaderboard number gets corrected by the truth on an ongoing basis rather than once, before launch.
One caveat threads through all five: most of these evaluations are scored by a model, and an LLM-as-judge is itself a model you have to evaluate. A reliability harness graded by an uncalibrated judge has only moved the point estimate, not replaced it.
Before an agent benchmark score is allowed to gate a ship decision:
If the only number you can give a launch review is a benchmark score, you have measured the good day and called it the forecast. Reliability is the rest of the distribution, and the rest of the distribution is where your users live.
A score answers one question: how did this agent do on a curated task set, once, on a good day. Reliability answers the question the launch review is actually asking. They are not the same question, and the gap between them is not a constant — it is a function of how hard the day gets.
Our taxonomy argued prompt injection is a vulnerability class you contain, not a bug you fix. Here is the quantitative half — against the strongest published defenses, indirect-injection attack success stays high, and for agents that can act it stays alarming.
10 min →
A quality-ascending curriculum beats random shuffling — until a decaying schedule delivers your best data exactly when the learning rate is too small to absorb it. The same coupling flips proxy-model ablations: which dataset wins depends on the schedule, not the data alone.
13 min →
An agent workflow that pauses for human approval and then resumes looks safe — a person clicked approve. But the approval decision and the authority the resumed step runs under are two different objects, and most systems conflate them by carrying the approval as an in-band signal the resumed call trusts. That is a confused-deputy bug: a forged resume, a replayed request, or an approval granted at one gate authorizes an action it was never meant to. The fix is to stop transporting approval and start deriving authority from the trusted approval record at resume time, scoped to the exact suspended step — capabilities enforced at the tool boundary, fail-closed by default.
14 min →