fix(codex): report per-turn tokens instead of the thread-cumulative total

[bai-uipath]

What's wrong

The Codex SDK's thread token total counts the whole thread, not the turn. The Codex thread is created once per task and reused for every turn, so by turn N that total still carries turns 1 through N-1. Each turn was reporting the running total, and the orchestrator sums per-turn usages into the task total. A task's reported tokens therefore became a sum of prefix sums, inflating a task by roughly (N+1)/2 in the number of turns.

Scope: model calls within a turn are fine, simulation turns are not

These are two different levels of nesting, and only the outer one is affected.

Model calls inside a single turn (each tool call leading to the next generation) were always correct. Those are booked from the SDK's per-generation delta, and the thread total at the end of turn 1 is exactly the sum of turn 1's generations. A task that runs 14 model calls in one turn reports all 14 correctly.

Simulation dialog turns (each communicate() call, driven by a task's simulation: block) are where the double-count happens, because the thread persists across them while the orchestrator adds up what each turn claims to have spent.

So single-turn tasks were never affected regardless of how many tool calls they made. On the 2026-08-13 nightly, 490 of the 618 tasks that generated anything ran a single turn and were exactly right. The damage was confined to the 128 multi-turn tasks.

Impact

The affected tasks are a minority, but an expensive one. On the 2026-08-13 nightly the 128 multi-turn tasks were 21% of the tasks that generated anything, yet carried 46% of the reported agent spend, because simulation dialogs are the long tasks to begin with. They were then inflated on top of that.

bucket	tasks	reported tokens	ratio	reported cost	true cost	overstated by
single-turn	490 (79%)	211,816,827	1.02x	$96.14	$94.86	$1.28
multi-turn	128 (21%)	166,092,110	2.07x	$80.28	$39.11	$41.16
total	618	377,908,937	1.31x	$176.41	$133.97	$42.44

Multi-turn tasks reported 2.07x their real tokens on average, and individual tasks ran as high as 5.3x. At the run level that is roughly $41 of a $175.03 nightly, about 23.5%.

The $1.28 in the single-turn row is not part of the defect. It is sub-agent tokens that the per-generation ground truth does not see, since children run on their own threads. The real figure is the multi-turn $41.16.

Token and USD run budgets read the same aggregate, so a multi-turn Codex task could also trip max_total_tokens or max_usd well before it actually should.

The distortion is not a constant

It tracks how much of the suite is multi-turn, which has been moving:

nightly	multi-turn share	overstated by
07-28	92/1002 (9%)	$84.47
08-12	133/650 (20%)	$52.63
08-13	128/618 (21%)	$41.16

07-28 carries the largest dollar error despite the smallest multi-turn share, because that run was bigger and those 92 tasks were deeper dialogs. So run-over-run Codex cost comparisons have been reading a distortion that changes size each night, which is harder to spot than a fixed bias would be.

Why it went unnoticed

Every task stayed internally consistent. The synthetic reconciliation message absorbed the gap turn by turn, so the message stream still summed to the turn total, just an inflated one. Any audit checking "do the messages sum to the turn total?" passed. Summing every reconciliation row across that run recovers the double-counted amount to within 0.06%, which is the clearest signal that the residual was the bug rather than the expected small prompt slice it is documented to carry.

It is also Codex-only, so in cross-harness comparisons it read as "Codex costs more than Claude" rather than as a defect.

Verification

Live end-to-end run

skill-troubleshoot-no-host-pending run on the eval VM against a real Codex thread and a real Bedrock-backed simulator, once on main and once on this branch. Same task, same command, same model.

Before, 5 simulation turns. Turn 1 is exact; every later turn reports the running total:

sim turn	model calls	actually spent (cache_read)	reported
1	12	338,598	338,598
2	1	43,367	381,965
3	4	196,445	578,410
4	3	179,695	758,105
5	2	132,921	891,026

Task total 2,948,104 reported against 891,026 spent, 3.36x, priced at $1.4503.

After, 3 simulation turns (the simulated dialog ends where it ends, so the two runs are not the same trajectory and the dollar figures are not directly comparable — the invariant is):

sim turn	model calls	actually spent (cache_read)	reported
1	15	447,013	447,013
2	2	83,407	83,407
3	1	42,314	42,314

Every turn now reports exactly what it spent, and the task total matches ground truth exactly: 1.0000x, priced at $0.2585. On this same trajectory the old code would have booked 1,676,555 input tokens against 615,260 real, 2.72x, or about $0.70.

The reconciliation rows corroborate it independently. They ran 382,894 / 425,633 / 635,114 / 824,522 across the turns of the pre-fix run, and collapse to 0 / 977 / 2 after — back to the near-zero residual Codex is documented to produce, since its message stream is already complete.

Worked example from the 2026-08-13 nightly

skill-troubleshoot-export-pdf-com-hang, 4 simulation turns, 21 model calls in total:

sim turn	model calls in the turn	actually spent	reported	reconciliation row
1	14	386,026	386,026	0
2	1	37,262	423,288	387,384
3	5	246,294	669,582	423,290
4	1	60,768	730,350	669,584

Every row's reported figure is the previous row's reported figure plus that turn's own spend, and each reconciliation row is almost exactly the previous turn's cumulative total. Turn 1 is correct including all 14 of its model calls. The last row, 730,350, is the correct task total; summing the reported column instead gives 2,209,246 (3.02x), and the task was priced at $1.0660 instead of ~$0.3524.

Replay against production data

All 128 multi-turn tasks from that nightly, replayed through the corrected path:

	input tokens	error vs ground truth
shipped	166,092,110	+107.1%
with this change	80,033,311	-0.2%
ground truth (per-generation)	80,213,611	—

123 of the 128 land within 0.5% of ground truth. The remaining 0.2% is sub-agent tokens, which are folded in separately and are not part of the parent thread's total.

Tests

tests/test_codex_token_mapping.py covers the multi-turn delta, the invariant that summed turns equal the thread's final cumulative total, per-turn cost, per-agent baseline isolation, a restarted thread, and the turn following a crash. Four of them were confirmed to fail against the pre-fix behavior rather than passing vacuously.

make format, make check, make typecheck and make lint are clean. make test has 4 failures, all pre-existing and unrelated: 3 in test_reports_stats_nonfinite that reproduce on an unmodified origin/main (a Python 3.13.7 statistics change), and 1 in test_litellm_route that depends on a local .env model pin.

Other harnesses

Unaffected and unchanged. Claude Code and Antigravity already report per-turn figures, confirmed empirically on their own full nightlies rather than by reading the code: both come out at 1.00x against ground truth at every turn count, including tasks running 3, 4 and 6 turns.

[github-actions]

Claude finished @bai-uipath's task in 2m 4s —— View job

---

Code Review

I'll review this PR that fixes the token reporting issue in multi-turn Codex tasks.

Todo List:

• Read .github/code_review.md for review guidelines
• Read CLAUDE.md for project conventions
• Review the full changed files for context
• Run git diff to see exact changes
• Analyze the token accounting fix for correctness
• Verify test coverage and quality
• Check for missing edge cases or side effects
• Provide comprehensive review feedback

Starting review now...