The cost of 63,000 AI agent runs

AI agents are increasingly sold at fixed prices ($0.99 per resolved support ticket, $20/month for a coding assistant, $2 per conversation) while the cost of producing each unit of work is anything but fixed. A hard ticket can burn 60× the tokens of an easy one. Providers repriced frontier models upward for the first time this year. Somebody is holding that variance on every contract, usually without having measured it.

We measured it. Using only public trajectory datasets (67k coding-agent rollouts released by Nebius, Princeton's HAL leaderboard traces with per-task dollar costs, and Exgentic's OpenTelemetry corpus) we assembled per-unit cost distributions for ~63,000 agent runs across support tickets, software engineering, research tasks, and multi-app operation. Total data cost: $0. Five findings follow, several of which surprised us.

1. Rerunning the same task varies the bill 2–3×, and 30× is real but rare

The SWE-rebench dump contains a median of ten rollouts of the same task under the same agent and model: a natural experiment in pure cost nondeterminism. Across 4,737 tasks: the median max/min cost ratio per task is 2.3×, the 90th percentile is 3.6×, and the worst task we found spanned 25×. A widely-cited Stanford result ("repeated runs differ by up to 30×") is confirmed, as a tail event rather than the typical case. Run-to-run noise is real, and bounded.

2. Cost is predictable: the task explains twice as much as the noise

The question that decides whether agent costs can be priced at all: does which task you got explain cost, or is every run a coin flip? Decomposing 57,281 rollouts into between-task and within-task variance gives an intraclass correlation of 0.68 on log cost. Task identity explains roughly twice the variance that rerun noise does. The result survives homogenization: restricted to single repositories (the closest analogue to one customer's workload), median ICC is 0.65, and no repository in the data drops below 0.27.

Agent costs are heteroskedastic in a way a classifier can learn, which means they can be underwritten.

3. The tail atlas: research agents are the fat-tail archetype

The ordering surprised us: the intuitive "insurable" workload, support tickets, has the thinnest tails when run on strong models, while open-ended research agents produce individual tasks costing 66× the median. If you're pricing per-outcome research or coding work at a flat rate, your tail is an order of magnitude heavier than a support vendor's.

4. Cheaper models buy their savings with tail

Across three independent datasets, the same pattern: on identical tasks, budget-tier models carry 2–5× the relative tail (p99/mean) of frontier models. The mean drops, and the dispersion around it grows. This matters because cost-optimal routing, the thing every serious agent shop does, systematically pushes portfolios toward exactly the models whose cost distributions are hardest to predict. The act of saving money manufactures the variance.

5. Across model generations, dispersion doesn't die. It migrates

Comparing like-for-like model successions (GPT-4.1→5, Opus 4→4.1, o3-mini→o4-mini) on fixed benchmarks: frontier successions compress relative dispersion by ~19–32% per generation while mean cost per task stays flat (newer models spend more tokens even as tokens get cheaper). Meanwhile the one tier that got cheaper (mini-class reasoners, −36% mean cost) got 45% fatter-tailed. The variance isn't disappearing. It's moving down the price curve, into the tier routing favors.

One more, from assembling 69 API price events (2023–2026) against GPU rental indices: the monthly correlation between token prices and GPU rental is ≈ 0, but every explicit list-price increase in 3.5 years of data clusters inside the Oct 2025–Aug 2026 capacity crunch. Pass-through from compute to API prices exists only as a regime, not a beta.

What this implies

A rapidly growing share of AI is sold per-outcome, and per-outcome sellers are short cost volatility they mostly haven't measured. The numbers above say that volatility is (a) large, (b) predictable enough to price, and (c) growing in exactly the model tiers the market is moving toward. In any other industry, that combination produces an underwriter. In this one, so far, it has produced dashboards.