fix(antigravity): poll for backgrounded work instead of grading it incomplete

[joeysbase]

Summary

AntigravityAgent.communicate() drained the SDK's step stream once and finalized the turn the instant it went idle. When Gemini backgrounds a long-running tool call (e.g. a slow solve) and pauses intending to check back later, the SDK's receive_steps() exhausts anyway — it never blocks waiting for future work — so coder-eval was grading the task with the background job still in flight.

The turn now polls (sleep + re-drain) while an orphaned tool call is still ACTIVE, bounded by _MAX_BACKGROUND_POLLS (120 cycles) and the pre-existing turn watchdog. A normal turn (which always closes its tool calls before the stream exhausts) takes this branch zero times — no added latency.

Hardened across three rounds of independent review before being validated end-to-end against the real Gemini API:

• The orphan signal allowlists ACTIVE specifically (not "not yet closed"), so a tool stuck on WAITING_FOR_USER/CANCELED/UNKNOWN is never polled forever.
• The fallback synthetic tool-call id (used when the SDK's call.id is falsy) is stable across a step's own ACTIVE→DONE re-emissions and unique across trajectories (a sub-agent trajectory can reuse the same step indices as the main one).
• The poll loop exits promptly once the watchdog has already decided to fire, instead of relying solely on a later exception handler.
• _drain() retries past the SDK's real two-layer generator re-entrancy window after a cooperative stop (confirmed live that contextlib.aclosing() on the outer Conversation.receive_steps() generator does not synchronously close the inner LocalConnection generator that owns the re-entrancy guard) instead of crashing the next turn with RuntimeError.

Validation

Ran energy-unit-commitment end-to-end against the real Gemini API (gemini-3.1-pro-preview) with the fix baked into freshly rebuilt images. Result: status=SUCCESS score=1.000, duration 890.9s.

The run organically reproduced the original bug scenario and two extra edge cases:

• The model backgrounded a MIP solve and said it would wait for it — the poll loop correctly waited ~4.5 minutes across 52 cycles with zero crashes, then picked up the real completion the moment it arrived.
• A second sequential background job in the same turn was also caught correctly.
• Two tool calls from an earlier, abandoned attempt never received a terminal status from the SDK at all (a genuinely unresolvable case) — the poll loop ran them out to the _MAX_BACKGROUND_POLLS cap and finalized cleanly instead of hanging or crashing.

No Concurrent receive_steps() errors, no AgentCrashError, no TurnTimeoutError — all failure modes the review rounds specifically hardened against.

Test plan

• make verify — all pass except one confirmed pre-existing, unrelated failure (test_effective_model_prefers_config_then_default, caused by a local .env override, reproduces identically on unmodified main)
• make lint — 332/332 custom architectural rules pass
• tests/test_antigravity_agent.py — 52/52 new + existing tests pass, including regression tests that were verified to fail without each corresponding fix
• Real end-to-end validation against the Gemini API (see above)

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com

[github-actions]

🔍 Reviewing PR #111

Working on review...

Todo List:

• Read code review guidelines (.github/code_review.md)
• Read project conventions (CLAUDE.md)
• Examine the full diff
• Review antigravity_agent.py changes
• Review test coverage
• Check cross-file consistency
• Provide structured feedback

View job run

[github-actions]

Claude Code is working…

I'll analyze this and get back to you.

View job run

[github-actions]

Claude finished @joeysbase's task in 2m 2s —— View job

---

Code Review in Progress

Todo List:

• Read .github/code_review.md for review guidelines
• Read CLAUDE.md for project conventions
• Run git diff origin/main...HEAD to see full diff
• Read each changed file in full context
• Perform cross-file consistency checks
• Analyze what's missing
• Provide structured review feedback

---

[bai-uipath]

Right fix at the right layer. One ask before merge: derive the hardcoded poll cap from the turn deadline. Everything below that is fix-if-you-agree.

• The cap is meant to be the safety valve, and it can never open. It exists so a never-resolving orphan still ends in a graded task: give up polling, force-close as UNRESOLVED, finalize COMPLETED. But it is 600s and experiments/default.yaml sets turn_timeout: 300, so the watchdog always wins the race and that path never executes. The real exit is TurnTimeoutError, which propagates out before check_all_async and lands the task as ERROR with zero criteria evaluated.
• For one input class that is a strict regression. A tool call left ACTIVE with no real background job behind it used to finalize immediately and get graded on whatever the agent wrote. Now it burns 300s and gets graded on nothing, scoring 0 on work that may well have been finished.
• That input class is already observed. The validation run saw two tool calls from an abandoned attempt that never got a terminal status, and the orphan signal cannot tell those apart from a live background job by design.
• Root cause is the unit. Every other deadline in the agents layer comes from run_limits; this one is a second clock counted in cycles, meaning 600s only by accident of the 5s interval, and it cannot see the timeout it is competing with.
• Fix: bound the loop at ~0.8 of timeout, keep the cycle cap only for timeout is None. Net deletion of both constants and the margin arithmetic defending them. The "an in-loop deadline races the watchdog" objection holds for a check against the same value, not a fraction of it, where the loop is meant to win.

Rest is optional:

• Drop the three harness-candidates.md entries. All three conclude "Not promoted", which is a decline filed in a promote-when-picked-up queue, and the one durable fact is already in a code comment and two regression tests.
• Minor: the drain's except RuntimeError also wraps step processing, so unrelated errors get five silent retries; budget exhaustion emits only a log line, so nobody can measure how often this fires; the poll tests patch the real asyncio.sleep instead of a module seam.

Allowlisting ACTIVE is the right call, and the zero-cost fast path is well covered. Fix what you agree with, otherwise lgtm.

[uipreliga]

Review: coder_eval — pr:111 (3 files) axis:1,2,3,4,5,6,7,8

Scope: pr:111 (3 files) axis:1,2,3,4,5,6,7,8 · branch fix/antigravity-wait-for-wakeup (PR #111 head) · 3abe902 · 2026-08-13T02:19Z · workflow variant

Change class: complex — introduces a bounded poll/re-drain control-flow loop around the SDK step stream, new orphaned-tool-call state tracking, synthetic tool-call id derivation, and generator re-entrancy retry handling; correctness requires reasoning about concurrency, watchdog interaction, and turn finalization.

Architecture, security, and error handling are clean (10/10 each) and the PR's new poll loop, orphan predicate, and cid fallback are fully covered — but one real harness risk remains: the post-idle poll budget (600 s) is not derived from the effective turn_timeout (300 s baseline), so a never-closing ACTIVE tool call now raises TurnTimeoutError → FinalStatus.ERROR with criteria never graded where main produced a scored result, and it sits alongside soft debt (stale communicate() contract docs, an untested _drain retry-replay path, zero lifecycle-teardown coverage, and a communicate() that grew D27 → E33); bottom line: ship after the poll-budget fix, schedule the rest.

Summary

Axis	Score	🔴	🟠	🟡	🔵	Top Issue
1. Code Quality & Style	9.2 / 10	0	0	1	3	Poll loop inlined into communicate() pushes it from CC 27 (D) to 33 (E) (and _handle_tool_call past the CC-20 bar) despite the PR's own _drain() extraction
2. Type Safety	9.9 / 10	0	0	0	1	Test helper _tc annotates tid: str but the four new call sites pass None, contradicting the SDK's `ToolCall.id: str
3. Test Health	8.8 / 10	0	0	2	2	_drain's new bounded-retry loop has no test for a non-re-entrancy or mid-stream RuntimeError; the exhaustion branch is covered only incidentally
4. Security	10 / 10	0	0	0	0	—
5. Architecture & Design	10 / 10	0	0	0	0	—
6. Error Handling & Resilience	10 / 10	0	0	0	0	—
7. API Surface & Maintainability	9.3 / 10	0	0	1	2	communicate() docstring (antigravity_agent.py:520-522) still states the pre-fix "iterate until the turn goes idle" contract, and no user-facing doc covers the new post-idle poll
8. Evaluation Harness Quality	9 / 10	0	1	0	0	Post-idle poll budget (120 x 5 s = 600 s) is not derived from turn_timeout: a never-closing ACTIVE tool call burns the whole 300 s turn timeout and lands FinalStatus.ERROR with criteria never graded (poll-cap branch unreachable)

Overall Score: 9.5 / 10 · Weakest Axis: Test Health at 8.8 / 10
Totals: 🔴 0 · 🟠 1 · 🟡 4 · 🔵 8 across 8 axes.

Blockers

1. [Axis 8] Post-idle poll budget (120 x 5 s = 600 s) is not derived from turn_timeout: a never-closing ACTIVE tool call burns the whole 300 s turn timeout and lands FinalStatus.ERROR with criteria never graded (poll-cap branch unreachable) (src/coder_eval/agents/antigravity_agent.py:604) — The graceful exit at line 620 (if poll_count >= _MAX_BACKGROUND_POLLS and state.has_orphaned_tool_call(): → warn and finalize, turn is still graded) is unreachable under the shipped limits: _MAX_BACKGROUND_POLLS = 120 (line 140) x _BACKGROUND_POLL_INTERVAL_SECONDS = 5.0 (line 112) = 600s of polling, while experiments/default.yaml:31 sets turn_timeout: 300 (layer-1 baseline for every run) and tasks/agents/antigravity_hello_world.yaml sets its own turn_timeout: 300 / task_timeout: 360. The ThreadedWatchdog therefore always wins first: state.timeout_hit flips at 300s, _finalize_and_raise_timeout raises TurnTimeoutError, which has NO dedicated handler in run() (orchestrator.py:528 only catches TaskTimeoutError) and lands in orchestrator.py:566 except Exception: → orchestrator.py:568 self.result.final_status = FinalStatus.ERROR. check_all_async (orchestrator.py:1559) is never reached, so success_criteria_results is empty and score is 0 — and models/enums.py:37 FinalStatus.ERROR: "error" puts the row in the harness-error bucket, not the "failed" bucket that FinalStatus.TIMEOUT (models/enums.py:42) uses. On main the same trajectory finalized immediately and produced a scored FAILURE (or SUCCESS). The PR's own validation confirms the trigger is common, not pathological: "Two tool calls from an earlier, abandoned attempt never received a terminal status from the SDK at all ... the poll loop ran them out to the _MAX_BACKGROUND_POLLS cap" — 600s of polling, which only produced SUCCESS score=1.000 because that task's turn budget exceeded 600s. Blast radius: this is the production run path for antigravity tasks driven by the external coder-eval-uipath / eval-runner repo (the validation task energy-unit-commitment is not in this repo's tasks/), and the PR body states no nightly impact; task.json's shape is unchanged, but the final_status value distribution shifts from FAILURE to ERROR, which that repo's dashboards bucket as harness breakage. Fix: derive the poll deadline from the effective turn timeout (stop polling at timeout - margin so the loop always exits through line 620's graceful finalize), or set _MAX_BACKGROUND_POLLS * _BACKGROUND_POLL_INTERVAL_SECONDS below the default turn_timeout of 300s. Add a test that a never-resolving orphan with timeout=300.0 finalizes and is graded rather than raising TurnTimeoutError.

Non-blocking, but please consider before merge

1. [Axis 1] Poll loop inlined into communicate() pushes it from CC 27 (D) to 33 (E) (and _handle_tool_call past the CC-20 bar) despite the PR's own _drain() extraction (src/coder_eval/agents/antigravity_agent.py:504) — Verified with radon against both revisions (git show origin/main:...>/tmp/base_ag.py, git show pr-111:...>/tmp/pr111_ag.py; uv run radon cc -s):

block	main	PR HEAD	delta
AntigravityAgent.communicate	D (27) @ base:409	E (33) @ pr:504	+6
_AntigravityTurnState._handle_tool_call	C (19) @ base:674	D (21) @ pr:819	+2
module average	—	B / 6.44	—

Explicit answer to the routed question: this PR did NOT create communicate's >20 crossing — it was already D(27) on main; the PR deepened it by +6, making it the single most complex block in the module and now more complex than either sibling agent's equivalent (ClaudeCodeAgent.communicate D(30), _CodexTurnState._flush_message E(34) — repo-wide radon cc -n C src/coder_eval/). This PR DID create _handle_tool_call's crossing (19 → 21) by adding the call_index parameter plus the branch step_key = f"{trajectory_id}:{step.step_index}" if trajectory_id else str(step.step_index) (pr:836) and self._tool_last_status[cid] = sstatus (pr:838).

Filed 🟡 Medium, not 🟠: the Axis-1 🟠 anchor scopes CC>20 to the named hot modules (orchestrator, checker, sandbox), and agents/antigravity_agent.py is not one; per "when torn between two levels, pick the lower one".

The fix is mechanical and matches what the PR already did once. The author correctly extracted _drain() (pr:451) so the drain shape lives in one place — but then inlined the whole new poll phase (pr:577-623, of which 600-618 is the loop) directly into communicate, whose try block now carries send → initial drain → poll loop → budget warning → cooperative-cancel, under a ThreadedWatchdog context, wrapped by two nested exception ladders. Extract the block at pr:600-623 into a sibling of _drain, e.g.:

async def _poll_for_backgrounded_work(self, conversation, state, should_stop) -> None:
    poll_count = 0
    while (not state.stopped_early_hit and not state.timeout_hit
           and state.has_orphaned_tool_call() and poll_count < _MAX_BACKGROUND_POLLS):
        ...

and let communicate read await self._poll_for_backgrounded_work(conversation, state, should_stop). That also gives the 23-line rationale comment at pr:577-599 a docstring to live in (see the separate prose finding) and drops poll_count (pr:568) out of communicate's local state.
2. [Axis 3] _drain's new bounded-retry loop has no test for a non-re-entrancy or mid-stream RuntimeError; the exhaustion branch is covered only incidentally (tests/test_antigravity_agent.py:1041) — antigravity_agent.py:495 except RuntimeError: catches EVERY RuntimeError out of the try body — including one raised after steps were already fed to state.process_step(step) (:489) — and then re-calls conversation.receive_steps() from scratch up to _RECEIVE_STEPS_REENTRY_RETRIES times. The only test of the retry, test_communicate_recovers_from_transient_reentrancy_after_cooperative_stop:1041, raises before any step is yielded and asserts a single line (:1076-1077 tr = await agent.communicate("do it again"); assert tr.agent_output == "second turn"). I instrumented it: receive_steps_call_count goes 1 -> 4, i.e. 2 of the 5 retries are consumed — nothing pins that headroom. The exhaustion raise at :497 is reached only incidentally by the PRE-EXISTING test_communicate_crash_sets_pending_partial_turn:469-480 (raise RuntimeError("kaboom")), which was written before the retry existed and asserts nothing about it.
Fix: add (a) a test where the fake yields two usage-bearing steps and THEN raises RuntimeError, asserting the retried drain does not duplicate tr.commands or double tr.token_usage; and (b) a test that a permanently-raising receive_steps() surfaces AgentCrashError after exactly _RECEIVE_STEPS_REENTRY_RETRIES calls (monkeypatch the constant low, as the cap test already does at :852), plus an assertion in the reentrancy test that receive_steps_call_count > 2 so it proves a retry actually happened.
3. [Axis 3] AntigravityAgent lifecycle teardown (stop/kill/kill_sync/_teardown/_conversation_or_none/get_environment_info) has zero test coverage, and _teardown() suppresses harness teardown failures with no log (src/coder_eval/agents/antigravity_agent.py:677) — Coverage of this module is 84.66% (370 stmts, 48 miss). Mapping the missing lines: 282/285/302->300/312 (_resolve_skills_paths error branches), 367-368/404-406/418-420 (start()'s SDK-import and thinking-level paths), 634/637/644/656-666 (outer exception handlers), 911/922/949/977/996 — all PRE-EXISTING. The single NEW-in-this-PR gap is 485->exit, an unreachable for-loop-exit branch in _drain (every iteration returns at :494 or raises at :497), i.e. a coverage artifact, not a real gap — the PR's new poll loop (:575-622), has_orphaned_tool_call (:951-968), and the cid fallback (:835-836) are all fully covered, which is genuine credit.
The largest uncovered contiguous block is pre-existing and is the Agent-ABC teardown surface: 679-680 (stop() -> await self._teardown() / self._mark_stopped()), 684-688 (kill()), 698 (kill_sync()), 708-713 (_conversation_or_none), 717-722 (_teardown). The sibling agent has this covered (tests/test_codex_agent.py:1517 test_kill_sync_interrupts_turn_and_closes, :1826 test_kill_sync_cleans_login_home); tests/test_timeout_orchestrator.py:216 only asserts on a MagicMock, so nothing verifies AntigravityAgent's kill_sync (:690) stays synchronous and non-raising for the watchdog's non-asyncio thread.
Fix: add the Codex-equivalent trio to tests/test_antigravity_agent.py — stop() closes the exit stack and leaves get_state() == AgentState.STOPPED; kill() calls conversation.cancel() then tears down; kill_sync() is a plain def, returns None, and does not raise when _sdk_agent is None.
4. [Axis 7] communicate() docstring (antigravity_agent.py:520-522) still states the pre-fix "iterate until the turn goes idle" contract, and no user-facing doc covers the new post-idle poll (src/coder_eval/agents/antigravity_agent.py:520) — The docstring at lines 520-522 still reads: "Drives one logical turn: conversation.send(prompt) then iterate receive_steps() until the turn goes idle, mapping the Gemini step stream onto the standardized event protocol." That is now false — the turn continues past idle via the poll loop at lines 600-618 whenever state.has_orphaned_tool_call() is true. The PR also changes no docs: docs/agents/ANTIGRAVITY.md (the user-facing agent guide, whose "Known limitations" section at lines 168-186 lists exactly four items of this shape) says nothing about backgrounded work, the poll, or its interaction with run_limits. Two user-visible consequences go unexplained anywhere: (a) a turn can now sit for minutes emitting no events (the poll only logs at DEBUG, line 607), so a run looks hung; (b) both shipped example tasks pin turn_timeout: 300 (tasks/agents/antigravity_hello_world.yaml and ..._docker.yaml), which is below the 600 s poll budget — so on an orphaned-tool turn the watchdog now fires where the turn previously finalized immediately, changing the terminal outcome for identical agent output. Fix: update the docstring to state that the turn polls past stream-idle while a tool call is ACTIVE, and add a "Known limitations" entry to docs/agents/ANTIGRAVITY.md naming the bound and telling users to size run_limits.turn_timeout/task_timeout for backgrounded jobs.

Nits

1. [Axis 1] Rationale prose dominates the diff (~66% of added lines are comments/docstrings), including a 23-line inline block in communicate() restating what the named constants already carry (src/coder_eval/agents/antigravity_agent.py:101) — Counted, not estimated: git diff origin/main...pr-111 -- src/coder_eval/agents/antigravity_agent.py | grep -c '^+' = 190 added lines; | grep -cE '^\+\s*#' = 82 # lines; plus ~43 docstring lines (pr:457-483 and pr:952-967). Concrete ratios at PR HEAD:

• pr:101-140 — 35 comment lines for 3 constant assignments (_BACKGROUND_POLL_INTERVAL_SECONDS = 5.0 at 112, _RECEIVE_STEPS_REENTRY_RETRIES = 5 at 120, _MAX_BACKGROUND_POLLS = 120 at 140).
• pr:577-599 — 23 consecutive comment lines immediately above a 24-line loop (600-623).
• pr:451-501 — _drain: 27-line docstring (457-483) over a 17-line body (485-501).
• pr:951-968 — has_orphaned_tool_call: 16-line docstring (952-967) over a one-line body: return any(cid not in self._closed_tools and s == _STATUS_ACTIVE for cid, s in self._tool_last_status.items()).

The SDK facts here are genuinely non-obvious and worth recording (wait_for_wakeup() is a stub; receive_steps() is two nested generators). The problem is the argumentative register — the comments litigate against a hypothetical reviewer rather than explain the code: pr:110-111 "Not user-configurable: a tuning constant, not a feature."; pr:118-119 "this constant carries a 2.5x margin, not a separately-tuned budget."; pr:129-139 "Deliberately NOT 'break after N consecutive empty polls' instead: ..." (an 11-line rebuttal of a design not taken); pr:964-967 "The not in _closed_tools guard is layered on top (not a substitute) purely as a monotonicity backstop...".

Same pattern in the in-scope test file, where shipped docstrings cite a review thread a future reader cannot see: tests/test_antigravity_agent.py:553 "(Phase-2-review finding)", :555 and :714 "(final-review finding)", :758 and :1046 "(round-3 review finding)", :1087 "per round-3 review".

Recommendation: keep the ~8 lines that state observed SDK behavior next to the code that depends on it; move the design-alternatives-rejected and review-history material to the PR description and .claude/harness-candidates.md (which this PR already updates for exactly that purpose), and strip (round-3 review finding) / (final-review finding) / (Phase-2-review finding) from the seven test docstrings — the assertion itself is the durable record.
2. [Axis 1] New _tool_last_status is a fifth parallel cid-keyed collection typed dict[str, Any], leaving the ACTIVE comparison unchecked where a set of active ids would express the predicate (src/coder_eval/agents/antigravity_agent.py:785) — _AntigravityTurnState.__init__ now carries five collections keyed on the same cid: _seen_tools (pr:775), _closed_tools (pr:776), _open_tools (pr:777), _tool_input_keys (pr:781), and the new self._tool_last_status: dict[str, Any] = {} (pr:785). It is written on every call at pr:838 (self._tool_last_status[cid] = sstatus), never pruned (closed tools' entries live for the whole turn), stores the raw status as Any, and has exactly one consumer, which then needs a two-condition scan plus a documented "backstop" clause (pr:968):

return any(cid not in self._closed_tools and s == _STATUS_ACTIVE for cid, s in self._tool_last_status.items())

The state the poll loop actually needs is a single membership question. Track it directly instead:

self._active_tools: set[str] = set()          # in __init__
...
(self._active_tools.add if sstatus == _STATUS_ACTIVE else self._active_tools.discard)(cid)   # in _handle_tool_call
...
def has_orphaned_tool_call(self) -> bool:
    return bool(self._active_tools)

This drops the Any, removes the not in _closed_tools backstop (a DONE/ERROR re-emission discards the id by construction, so closure is automatically authoritative), removes the per-check O(n) dict scan, and makes the allowlist-on-ACTIVE semantics the shape of the data rather than a 16-line docstring. It also keeps the container count from growing: _seen_tools is already fully redundant with _open_tools (both are populated only at pr:823/832 and never removed from) — pre-existing, not this PR's doing, but the direction of travel is worth reversing rather than extending.
3. [Axis 1] .claude/harness-candidates.md entries describe fixes that differ from the code this same PR ships (.claude/harness-candidates.md:327) — Both new notes under ## From the coder-eval-code-review of fix/antigravity-wait-for-wakeup (2026-08-12) (line 318) under-describe the shipped fix, so the deferred-harness log is already stale on arrival:

1. Line 327 says the fallback id was "Fixed by deriving the fallback from (step.step_index, call_index) instead". The shipped code at antigravity_agent.py:836-837 also keys on trajectory_id:

   trajectory_id = getattr(step, "trajectory_id", "") or ""
   step_key = f"{trajectory_id}:{step.step_index}" if trajectory_id else str(step.step_index)
   cid = call.id or f"{raw_name}_{step_key}_{call_index}"

   That component is not incidental — this PR adds a dedicated test for it (tests/test_antigravity_agent.py:751 test_id_less_tool_calls_in_different_trajectories_do_not_collide, asserting {"run_command_1_0", "run_command_subagent-42:1_0"}), because step_index alone collides across a sub-agent trajectory. As written, the note documents the intermediate fix that the same PR then superseded.

2. Line 343 says "Fixed by adding and not state.timeout_hit to the condition." — that is half the shipped fix. The loop also gained a mid-body early exit at antigravity_agent.py:609-614 (if state.timeout_hit: ... break) whose whole purpose is to skip the re-drain after the sleep, and which has its own test (tests/test_antigravity_agent.py:900 test_communicate_poll_loop_exits_promptly_once_watchdog_flag_lands, asserting receive_steps_call_count == 2). Omitting it makes the note's "not promoted to a CExxx rule" rationale rest on an incomplete description of the shape.

Update both sentences to match pr HEAD (mention trajectory_id in #1; mention the post-sleep break in #2).
4. [Axis 2] Test helper _tc annotates tid: str but the four new call sites pass None, contradicting the SDK's ToolCall.id: str | None (tests/test_antigravity_agent.py:233) — Line 233 is def _tc(name: str, tid: str, args: dict) -> SimpleNamespace:, but this PR adds four call sites that pass None for tid — lines 724, 736, 765 and 773 (e.g. line 724: tool_calls=[_tc("run_command", None, {"command_line": "sleep 12 && echo done"})]). Passing None is the correct test behavior: the wheel confirms ToolCall.id: str | None = None (google/antigravity/types.py:454), which is exactly the optionality the new synthetic-cid fallback at antigravity_agent.py:834 (cid = call.id or f"{raw_name}_{step_key}_{call_index}") exists to handle. The annotation is what is wrong, and because pyright excludes tests/ nothing flags it, so the helper's signature now actively misdocuments the SDK contract these tests were written to pin. Fix: def _tc(name: str, tid: str | None, args: dict[str, Any]) -> SimpleNamespace:.
5. [Axis 3] _WatchdogFiresLater.captured_on_timeout is class-level mutable state shared by two tests and never reset (tests/test_antigravity_agent.py:886) — 886: captured_on_timeout: Callable[[], None] | None = None is written from __init__ (889: _WatchdogFiresLater.captured_on_timeout = on_timeout) by both test_communicate_poll_loop_exits_promptly_once_watchdog_flag_lands:898 and test_communicate_poll_budget_exhausted_finalizes_via_existing_timeout_path:1080, and is never cleared. It therefore outlives each test holding a live closure over a finished turn's _AntigravityTurnState. Today both readers instantiate the class first so the value is fresh, but a future test that reads captured_on_timeout without a preceding communicate() would silently flip timeout_hit on a dead turn state and pass/fail for the wrong reason. Fix: reset it in an autouse fixture (or return the callback via a per-test list/monkeypatch.setattr on an instance) rather than parking it on the class.
6. [Axis 3] Two fast-path tests replace the global asyncio.sleep with a raising stub, so any incidental sleep fails them with a misleading message (tests/test_antigravity_agent.py:595) — antigravity_agent.py does a plain import asyncio (:21), so antigravity_agent.asyncio is sys.modules["asyncio"] — 595: monkeypatch.setattr(antigravity_agent.asyncio, "sleep", _sleep_should_not_be_called) (and the identical :626) swaps the stdlib asyncio.sleep process-wide for the test, with a body that raises AssertionError("asyncio.sleep must not be called on the no-orphan fast path"). Patching global asyncio.sleep is an established convention here (tests/test_orchestrator.py:1785, tests/test_retry_logic_comprehensive.py:63), but those install a no-op; a RAISING stub means _drain's own re-entrancy yield (antigravity_agent.py:502 await asyncio.sleep(0)) — or any sleep in an awaited library frame — fails the test with a message that misdescribes the cause, and the stub stops proving "the poll loop was not entered". The real contract is already asserted one line later (:597 assert conv.receive_steps_call_count == 1). Fix: make the stub a no-op recorder and assert sleep_calls == [], or scope the patch to the poll interval by asserting no call carries _BACKGROUND_POLL_INTERVAL_SECONDS.
7. [Axis 7] "Orphaned tool call" now means two different things inside _AntigravityTurnState (src/coder_eval/agents/antigravity_agent.py:951) — has_orphaned_tool_call() (line 951) returns any(cid not in self._closed_tools and s == _STATUS_ACTIVE for cid, s in self._tool_last_status.items()) (line 968) — an ACTIVE-only allowlist — while finalize() 20 lines later documents itself as "Close orphaned tools, flush leftover blocks, emit TurnEnd + AgentEnd" (line 971) and implements the broader sense: for cid, tel in self._open_tools.items(): if cid in self._closed_tools: continue (lines 981-983), i.e. any not-yet-closed tool regardless of status. So a WAITING_FOR_USER tool is "orphaned" to finalize() but deliberately not to has_orphaned_tool_call(). The (excellent) docstring at lines 956-966 explains the allowlist rationale, but the name is what the poll-loop call site reads (line 603). Rename the predicate to what it actually tests — e.g. has_active_background_tool_call() — or add a one-line cross-reference in finalize()'s docstring noting the two senses are intentionally different.
8. [Axis 7] .claude/harness-candidates.md: three "Not promoted" decisions are filed as open - [ ] TODOs, against the file's own resolved-item convention and the rubric's one-line deferral format (.claude/harness-candidates.md:318) — All three new entries under "## From the coder-eval-code-review of fix/antigravity-wait-for-wakeup (2026-08-12)" (lines 318-362) are explicit decisions not to add a guard — "Not promoted to a CExxx rule: this is the only id-fallback-driving-control-flow site in the codebase today" (line 328), "Not promoted: ThreadedWatchdog + a bespoke poll loop … no second instance exists to generalize a rule from" (line 344), "Not promoted: detecting "a test double is missing a delegation layer the source has" is a semantic match against third-party source" (line 359) — yet each is written as an unchecked - [ ] item in a file whose header says "Deferred lint/test guardrails … Promote to a CExxx rule (or a test) when picked up" (lines 3-4). The file already has a convention for items that are no longer pending: - [x] ~~**taskCarriesRepoTag …**~~ **RESOLVED in PR #94 review round 2.** (line 156). Every future reader of this queue will re-read ~45 lines to re-derive "nothing to do here". Note also that .claude/shared/review-rubric.md § Harness loop prescribes one line per deferral (- [ ] <rule> — <why nothing guards it today> — caught in <context>) and says a non-gap should be skipped rather than recorded. Either mark these - [x] under a "Considered and declined" heading, or compress each to one line.

What's Missing

Tests:

• 🟡 🟡 Token/reconciliation invariant is never asserted on a multi-drain turn. The CLAUDE.md invariant ("summing the four token buckets across TurnRecord.messages equals token_usage exactly") is pinned only in the pre-existing single-drain test (tests/test_antigravity_agent.py:400-413); every new polled test (:646 poll-and-resume, :787 two sequential background jobs) asserts only result_status / agent_output / receive_steps_call_count. The poll path now folds usage_metadata from N separate receive_steps() cycles through process_step → _flush_generation, so a re-drain that re-emits a usage-bearing DONE step would double-count tokens and cost in task.json with the whole suite green. Add assert tr.token_usage.output_tokens == … plus the per-bucket message-sum assertions to test_communicate_polls_and_resumes_after_orphaned_tool_closes. (trigger: tests/test_antigravity_agent.py)
• 🟠 🟠 No orchestrator-level test that a polled turn under a realistic turn_timeout is still graded. Every new poll test drives communicate() directly with timeout=None or a monkeypatched watchdog; none exercises the shipped configuration (experiments/default.yaml:31 turn_timeout: 300 vs. the 600 s poll budget), where a never-closing ACTIVE tool now raises TurnTimeoutError → FinalStatus.ERROR with check_all_async never reached. A test at the Orchestrator level (or an EvaluationResult-shaped assertion) is what would have caught the un-derived budget before merge. (trigger: tests/test_antigravity_agent.py) (restates: Axis 8: Post-idle poll budget (120 x 5 s = 600 s) is not derived from turn_timeout)
• 🟡 🟡 _drain's new bounded-retry loop ships with one happy-path test and no mid-stream / exhaustion coverage. antigravity_agent.py:495 catches every RuntimeError out of the drain — including one raised after steps were fed to state.process_step — and restarts receive_steps() from scratch, yet no test raises mid-stream (would prove replay does not duplicate agent_output / token totals) and none pins the _RECEIVE_STEPS_REENTRY_RETRIES bound. (trigger: src/coder_eval/agents/antigravity_agent.py) (restates: Axis 3: _drain's new bounded-retry loop has no test for a non-re-entrancy or mid-stream RuntimeError)

Downstream consumers:

• 🟡 🟡 EarlyStopWatcher's UNRESOLVED-orphan carve-out was written for the pre-fix world and was not revisited. orchestration/early_stop.py:456-461 deliberately records but never counts or evaluates on an UNRESOLVED ToolEndEvent, because a backgrounded antigravity tool ALWAYS force-closed that way at finalize(). Polling now resolves those same calls into real ToolEndEvents, so on an armed antigravity task a backgrounded run_command now increments _tool_call_index (the decide_within step budget) and runs _evaluate_impl — it can fire a live pass-stop or fail-stop where it structurally could not before. Neither early_stop.py nor tests/test_early_stop.py was touched, and no test covers an armed criterion deciding on a polled tool result. (trigger: src/coder_eval/agents/antigravity_agent.py)
• 🔵 🔵 Turn duration now silently absorbs idle poll time, and no consumer was given a way to separate it. finalize() reports duration_seconds = time.monotonic() - self.turn_start_time (antigravity_agent.py:1024), which now includes up to 600 s of asyncio.sleep waiting on a background job. That value feeds reports.py:302's avg-turn-duration column and the evalboard duration aggregations (evalboard/lib/trends.ts:167, overview.ts:883), so antigravity turn-latency series get a step change with background wait indistinguishable from agent compute. No poll_seconds / background-wait field was added to TurnRecord to let the reports split them. (trigger: src/coder_eval/agents/antigravity_agent.py)

Display & mapping dicts:

• 🟡 🟡 The hand-mirrored StepStatus constants got a new load-bearing consumer but no parity guard. _STATUS_ACTIVE/_DONE/_ERROR (antigravity_agent.py:187-189) mirror google.antigravity.types.StepStatus as plain strings, and the poll loop's ACTIVE-only allowlist now depends on both the spelling AND the closed-world assumption that no other member denotes in-flight background work — WAITING_FOR_USER / CANCELED / UNKNOWN are enumerated only in prose (:956-966) and re-hardcoded as string literals in the test (tests/test_antigravity_agent.py:569). Nothing ties the mirror to the pinned SDK enum (google-antigravity==0.1.7, pyproject.toml:107) the way evalboard/lib/__tests__/pricing-parity.test.ts does for the pricing table, so a dependabot bump that renames ACTIVE or adds a PENDING/BACKGROUND member silently reverts this entire fix to the fast path with the suite green. Add an importorskip-guarded parity test asserting the three constants are members of StepStatus and that the full member set is exactly the six known values. (trigger: src/coder_eval/agents/antigravity_agent.py)

Daily/nightly:

• 🟡 🟡 Unstated baseline shift for the external eval-runner: backgrounded tool calls flip from "unknown" to "success". Before this PR a backgrounded run_command was force-closed by finalize() as result_status="unknown" (antigravity_agent.py:981-983, asserted at tests/test_antigravity_agent.py:641); with polling it now resolves to "success". command_executed gates exactly on that value (criteria/command_executed.py:173 if criterion.require_success and cmd.result_status != "success"), and commands_efficiency / skill_triggered read the same trajectory — so task pass/fail, suite P/R/F1, and the evalboard regression tags shift on precisely the antigravity tasks this fix targets. That is the intended effect, but the PR never states the expected baseline movement for the nightly consumed by the external coder-eval-uipath / eval-runner repo. (trigger: src/coder_eval/agents/antigravity_agent.py)
• 🔵 🔵 Nightly wall-clock / throughput impact of the poll is unstated. A turn that previously finalized the instant the stream went idle can now hold a run_batch worker for the full run_limits.turn_timeout (or 600 s where none is configured) on every orphaned-tool turn — the PR's own validation notes a task that "ran them out to the _MAX_BACKGROUND_POLLS cap". Neither the PR body nor docs/agents/ANTIGRAVITY.md states the resulting per-cycle duration/cost change or advises sizing task_timeout for parallel nightly batches. (trigger: src/coder_eval/agents/antigravity_agent.py)

Parallel paths:

• 🟡 🟡 The user-facing agent guide is the parallel surface of the changed docstring and was not updated. docs/agents/ANTIGRAVITY.md is untouched: its "Known limitations" list (lines 169-183, four items of exactly this shape) says nothing about the post-idle poll, its 600 s bound, the minutes-long event-silent window it creates (the loop logs only at DEBUG, antigravity_agent.py:607), or the need to size run_limits.turn_timeout/task_timeout above the background job — while both shipped example tasks pin turn_timeout: 300, below the poll budget. (trigger: src/coder_eval/agents/antigravity_agent.py) (restates: Axis 7: communicate() docstring still states the pre-fix "iterate until the turn goes idle" contract)
• 🔵 🔵 The cross-agent cooperative-stop seam file was not extended for the PR's second should_stop site. tests/test_early_stop.py is the SSOT for the "stop cuts the trajectory at the deciding step" contract across Claude / Codex / Antigravity (TestAntigravityCooperativeStopSeam:3127), but the new stop check inside the poll loop (antigravity_agent.py:615) is covered only agent-locally (tests/test_antigravity_agent.py:945), and that file's fake steps (_ag_step, _CountingConversation) carry no tool calls, so the seam suite never reaches the poll path at all. (trigger: src/coder_eval/agents/antigravity_agent.py)

Harness & Lint Improvements

Static checks (lint / type):

• [ce-lint] CE035 — an agent's internal sleep-loop budget must be derived from the effective turn timeout (or provably fit inside the shipped baseline). New whole-tree @pytest.mark.lint class in tests/test_custom_lint.py (same wiring as CE026/CE034 — it reads YAML as well as Python AST, so it is not a BaseRule in tests/lint/runner.py). For every While/For node in src/coder_eval/agents/** whose body contains await asyncio.sleep(<name>): either (a) the loop test references a deadline/timeout-derived name (a parameter, not a module constant), or (b) interval_constant * bound_constant (both resolved from module-level ast.Assign literals) is strictly less than turn_timeout in experiments/default.yaml — otherwise the loop's own cap branch is unreachable under the layer-1 baseline and the watchdog always wins first. Measured baseline cost on main: zero violations — the only existing agent sleep-loop is codex_agent.py:1787 (attempts: int = 20 x asyncio.sleep(0.1) = 2 s << 300 s), which passes. On PR HEAD it fires on antigravity_agent.py:600 (_MAX_BACKGROUND_POLLS = 120 x _BACKGROUND_POLL_INTERVAL_SECONDS = 5.0 = 600 s vs. experiments/default.yaml:31 turn_timeout: 300). Prevents: The single 🟠 High finding (antigravity_agent.py:604 — post-idle poll budget not derived from turn_timeout; the graceful cap branch at :620 is dead code under the shipped 300 s baseline, so a never-closing ACTIVE tool call lands FinalStatus.ERROR with criteria never graded instead of a scored FAILURE). Also pre-empts the same shape in any future agent.
• [ruff] Enable C901 with [tool.ruff.lint.mccabe] max-complexity = 18 (add "C90" to select in pyproject.toml). This is the exact precedent already established for PLR0915/PLR0912 (max-statements = 80, max-branches = 25, with the comment "gates NEW growth past these bounds; a god-function must carry a visible # noqa debt marker") — complexity is the one dimension of that trio not currently gated. Measured: PR HEAD's AntigravityAgent.communicate is mccabe 19 > 18 (fires); origin/main's same function is 17 (passes), so the threshold gates exactly this PR's growth. Baseline cost on main = 5 functions needing a # noqa: C901 debt marker: claude_code_agent.py:881 communicate (21), codex_agent.py:957 _setup_skills (19), isolation/docker_runner.py:1070 _build_argv (29), orchestrator.py:1798 _simulation_dialog_loop (22), reports_experiment.py:589 generate_variant_report (24). The finding's own prototype extraction (_poll_for_backgrounded_work) drops the function back under the bar (radon E33 -> D24). Prevents: 🟡 Medium: communicate() pushed from CC 27 (D) to 33 (E) and _handle_tool_call from 19 to 21, despite the PR's own _drain() extraction. Would have forced the extraction at authoring time rather than at review time.
• [pyright] Type-check tests/ under a second, narrow pyright project. Add pyrightconfig.tests.json with include: ["tests"], typeCheckingMode: "off", and only reportArgumentType/reportCallIssue set to error; wire it as make typecheck-tests and add it to make verify + the CI typecheck job. Verified against PR HEAD: it reports Argument of type "None" cannot be assigned to parameter "tid" of type "str" in function "_tc" at tests/test_antigravity_agent.py 724, 736, 765, 773 — precisely the finding. Adoption must be incremental, not big-bang: measured today the suite has 675 errors across 82 of 240 test files (652 reportArgumentType, 23 reportCallIssue), so ship it with those 82 files listed in exclude as an explicit shrinking baseline (a file drops off the list when it is cleaned), so every new or rewritten test file lands clean. tests/ is currently in pyright's exclude list, which is exactly why nothing flagged this. Prevents: 🔵 Low (Axis 2): test helper _tc(tid: str) annotated non-Optional while four new call sites pass None, misdocumenting the SDK's real ToolCall.id: str | None contract that the new synthetic-cid fallback exists to handle. Would also catch the whole class of test doubles whose signatures drift from the source types they stand in for.
• [ce-lint] CE036 — kill_sync must be synchronous and await-free on every Agent subclass. New BaseRule in tests/lint/rules/ce036_kill_sync_is_sync.py, wired into ALL_RULES in tests/lint/runner.py. For any class whose bases include Agent (or a known agent base) that defines kill_sync: the node must be ast.FunctionDef (not AsyncFunctionDef), must contain no Await/AsyncWith/AsyncFor, and must not call asyncio.run / get_event_loop().run_until_complete. Zero violations on main (all four agents already comply) — it pins the contract, which is currently pinned nowhere. This is the statically-reachable half of the teardown-coverage finding; the behavioural half (it must also not raise) goes to the harness bucket. Prevents: 🟡 Medium (Axis 3/6): AntigravityAgent's entire lifecycle surface (stop/kill/kill_sync/_teardown/_conversation_or_none/get_environment_info, lines 677-722) has zero test coverage, and kill_sync is invoked from the watchdog's non-asyncio timer thread — nothing today fails if it is changed to await.
• [ce-lint] CE037 — agent tuning constants must appear in that agent's user-facing guide. New doc-parity @pytest.mark.lint class, shaped exactly like CE027 (doc/env-var parity) and CE030 (doc/schema parity). For each src/coder_eval/agents/<name>_agent.py, every module-level constant matching _*(SECONDS|INTERVAL|TIMEOUT|POLLS|RETRIES|MAX_*) must be named verbatim in docs/agents/<NAME>.md, or carry an # EXEMPT: <reason> comment. Measured baseline cost on main: zero — no agent module currently defines such a constant. On PR HEAD it fires on all three new constants (_BACKGROUND_POLL_INTERVAL_SECONDS, _RECEIVE_STEPS_REENTRY_RETRIES, _MAX_BACKGROUND_POLLS), forcing the 'Known limitations' entry the finding asks for. The complementary docstring-staleness half is not statically reachable (whether prose is now false is semantic) — see the harness bucket. Prevents: 🟡 Medium (Axis 7): the new 600 s post-idle poll is documented nowhere user-facing — docs/agents/ANTIGRAVITY.md is untouched, so users get no guidance that a turn can now sit for minutes emitting only DEBUG logs, nor that both shipped example tasks pin turn_timeout: 300 below the poll budget.
• [ce-lint] CE038 — cap consecutive full-line comment runs at 20 in src/coder_eval/ (line-based, no AST needed; # noqa: CE038 as the debt marker, matching the PLR0915 convention). Measured: longest run on main is 22 (orchestration/early_stop.py:643) and the next is 20 (codex_agent.py:1282), so a > 20 bar means exactly one pre-existing noqa. PR HEAD has a 23-line run at antigravity_agent.py:577 (immediately above a 24-line loop) plus an 18-line run at :122 — against a pre-PR maximum of 9 in that same file. The rule does not judge whether a comment is useful; it forces prose past a threshold into a docstring or the PR description, which is exactly the finding's recommendation. Prevents: 🔵 Low (Axis 1/7): ~66% of the diff's added lines are comments/docstrings, including the 23-line inline block at :577-599 restating what the named constants already carry, an 11-line rebuttal of a design not taken, and a 27-line docstring over a 17-line body.
• [ce-lint] CE039 — no review-round provenance in shipped source or test prose. Regex rule over src/ and tests/ docstrings/comments forbidding round-\d+ review, final-review finding, Phase-\d+-review, \(.*review finding\). Cheap (line-based), zero baseline cost to verify before landing. Would flag tests/test_antigravity_agent.py 553, 555, 714, 758, 1046, 1087. The rationale is durable: the assertion is the record; a reader six months from now cannot see 'round 3' of a closed PR thread. Prevents: 🔵 Low (Axis 1/7): seven test docstrings shipped citing an invisible review thread ('(Phase-2-review finding)', '(final-review finding)', '(round-3 review finding)', 'per round-3 review').
• [ce-lint] CE040 — .claude/harness-candidates.md checkbox convention. Doc-surface @pytest.mark.lint rule: any entry whose body contains Not promoted or RESOLVED must not be an open - [ ] item (the file already establishes the closed form at line 156: - [x] ~~**...**~~ **RESOLVED in PR #94...**), and every open - [ ] entry must be a single line matching the deferral shape prescribed in .claude/shared/review-rubric.md § Harness loop (- [ ] <rule> — <why nothing guards it today> — caught in <context>). Measured: 21 open - [ ] entries on main to audit once; PR HEAD adds three multi-paragraph 'Not promoted' decisions filed as open TODOs (lines 318-362). Prevents: 🔵 Low (Axis 7): three explicit decisions not to add a guard are filed as open work items, so every future reader of the queue re-reads ~45 lines to re-derive 'nothing to do here'.
• [ce-lint] CE041 — do not monkeypatch stdlib asyncio.sleep through a module alias in tests. AST rule: forbid monkeypatch.setattr(<anything>.asyncio, "sleep", ...) (and setattr(asyncio, "sleep", ...)); direct callers to a shared recording_sleep conftest fixture that records durations and returns immediately. Because antigravity_agent.py:21 does a plain import asyncio, antigravity_agent.asyncio is sys.modules['asyncio'] — the patch is process-wide for the test, and a raising stub then turns any incidental sleep (e.g. _drain's own await asyncio.sleep(0) re-entrancy yield at :502) into a failure whose message misdescribes the cause. Baseline: 2 sites in the PR (tests/test_antigravity_agent.py:595, :626) plus the existing no-op patches at tests/test_orchestrator.py:1785 and tests/test_retry_logic_comprehensive.py:63, which the fixture subsumes. Prevents: 🔵 Low (Axis 3): two fast-path tests replace global asyncio.sleep with an AssertionError-raising stub, so any incidental sleep fails them with a misleading message — and the stub stops proving the thing it was written to prove (the real contract is already asserted one line later by receive_steps_call_count == 1).
• [ce-lint] CE042 — a test double must not assign to its own class attribute from an instance method. AST rule over tests/: inside a method of class C, an Assign whose target is Attribute(value=Name('C')) is a violation (use an instance attribute, a per-test list, or monkeypatch.setattr on the instance). Catches the general shape of leaked cross-test state, which is otherwise invisible until an unrelated test starts failing. Prevents: 🔵 Low (Axis 3): _WatchdogFiresLater.captured_on_timeout (tests/test_antigravity_agent.py:886) is written from __init__ (:889) by two tests and never reset, so it outlives each test holding a live closure over a finished turn's _AntigravityTurnState.
• [ce-lint] CE043 — no Any in annotated instance-attribute declarations under src/coder_eval/agents/**. AST rule: an AnnAssign whose target is self.<x> and whose annotation subtree contains Any is a violation unless it carries # noqa: CE043. Rationale: agent modules are the boundary where third-party SDK objects enter, so an Any state field is exactly where a wrong-typed value survives to a consumer. Note the honest cost/benefit: 12 pre-existing sites would need noqa markers (7 in codex_agent.py, 4 in claude_code_agent.py, 1 new in antigravity_agent.py), and the rule only reaches the typing half of the finding — the design half (five parallel cid-keyed collections where a set[str] of active ids expresses the predicate) is not statically detectable and stays a review judgment. Note also that pyright cannot do this natively: standard pyright has no reportExplicitAny (that is basedpyright), and ruff's ANN401 covers parameters/returns only. Prevents: 🔵 Low (Axis 1/2/5): the new self._tool_last_status: dict[str, Any] is a fifth parallel cid-keyed collection, stores the raw SDK status unchecked, is never pruned, and needs a 16-line docstring plus a two-condition scan for what is a single membership question.

Harness improvements (not statically reachable):

• Shared agent-lifecycle conformance suite, parameterized over AgentRegistry. Add tests/test_agent_contract.py that enumerates registered agent types and runs one common body against each: stop() closes the exit stack and leaves get_state() == AgentState.STOPPED; kill() cancels the conversation then tears down; kill_sync() returns None and does not raise when the SDK handle is None; a mid-turn crash leaves a crashed=True pending_turn that discard_pending_turn() clears. Today this exists only for Codex (tests/test_codex_agent.py:1517, :1826); tests/test_timeout_orchestrator.py:216 asserts kill_sync only against a MagicMock. A registry-driven suite means the next agent (in-tree or plugin) inherits the coverage instead of re-litigating it. Why not static: CE036 can prove kill_sync is a plain def, but 'does not raise when there is no live conversation' and 'the exit stack is actually closed' are runtime states — they need an instantiated agent with a fake SDK conversation and an assertion on post-call state. Prevents: The 🟡 Medium teardown-coverage gap: AntigravityAgent lines 677-722 (measured 86.13% module coverage, 43 missed statements against the whole suite; the missed set 679-680, 684-688, 698, 702, 708-713, 717-722 is byte-identical file-scoped and suite-scoped).
• Add a behavioural budget case to that same conformance suite: a never-resolving backgrounded tool call must produce a graded turn, not a harness error. With the shipped constant ratio preserved and the wall clock scaled (poll interval monkeypatched low, real ThreadedWatchdog), assert that communicate(timeout=T) returns a TurnRecord and the run reaches check_all_async — i.e. the terminal final_status lands in the "failed" bucket, never FinalStatus.ERROR. Pair it with an orchestrator-level guard that TurnTimeoutError maps to FinalStatus.TIMEOUT (category "failed"), since orchestrator.py:528 catches only TaskTimeoutError and everything else falls to :566 except Exception -> :568 FinalStatus.ERROR, which downstream dashboards bucket as harness breakage. Why not static: CE035 pins the constants' arithmetic relationship; it cannot observe the terminal final_status a real turn produces. Reproducing this needs the watchdog's timer thread, a fake step stream, and the full orchestrator exception ladder — the reviewer had to execute a scaled repro to confirm it. Prevents: The 🟠 High poll-budget finding, and the broader Axis-8 blocker class it belongs to (a change that alters final_status for identical agent output: FAILURE on main -> ERROR on the PR, with criteria never evaluated).
• Turn on branch coverage and add a per-package floor for src/coder_eval/agents/**. Set branch = true under [tool.coverage.run] and add a second, higher --cov-fail-under (or a coverage report --include='src/coder_eval/agents/*' --fail-under=90 step) in make verify and .github/workflows/pr-checks.yml. The global 80% floor (Makefile:60, pr-checks.yml:153) is why a module sitting at 86% with its entire lifecycle surface uncovered passes CI unnoticed. Branch mode additionally surfaces partial-branch artifacts like 485->exit as first-class, which is how the reviewer had to distinguish 'unreachable by construction' from 'untested'. Why not static: Coverage is a runtime measurement by definition; no AST rule can know which lines a test suite executes. Prevents: Both 🟡 Medium Test Health findings — the untested _drain retry loop and the zero-coverage teardown block — neither of which dents the global 80% number.
• Pin bounded-retry caps with an explicit call-count assertion, and give the suite a helper for it. Add a small CallCounter / assert_attempts(n) test util plus the convention (documented next to the CE035/CE036 entries) that any _*_RETRIES / _MAX_* constant ships with a test that monkeypatches it low and asserts the exact number of underlying calls. Concretely for this PR: (a) a mid-stream RuntimeError case — the fake yields two usage-bearing steps then raises — asserting the retried drain does not double tr.token_usage, duplicate tr.agent_output parts, or inflate the assistant-message count (note: tr.commands is not the right target; _handle_tool_call already dedupes on _seen_tools/_closed_tools, whereas _output_parts and total_usage accumulate unconditionally on every process_step); and (b) an assertion that a permanently-raising receive_steps() surfaces AgentCrashError after exactly _RECEIVE_STEPS_REENTRY_RETRIES calls. The existing reentrancy test consumes 2 of 5 retries and still passes with the constant at 3. Why not static: A lint rule can see that a retry constant exists and even that its name appears somewhere in tests/, but not whether any test actually drives the loop to exhaustion or replays a partially-consumed stream — that requires executing the loop and counting. Prevents: The 🟡 Medium _drain retry-coverage finding (no test for a mid-stream RuntimeError; the exhaustion branch is reached only incidentally by a pre-existing test that asserts nothing about it).
• Run one CI job with randomized test order (pytest-randomly, or a shuffled-seed job), reporting the seed on failure so a discovered order-dependence is reproducible. Why not static: CE042 catches the direct ClassName.attr = ... shape; order dependence introduced indirectly (module-level singletons, a fixture that mutates a shared registry, a cached SDK handle) is only observable by executing the suite in a different order. Prevents: The 🔵 Low class-level-mutable-test-state finding, and the general class of tests that pass only because a sibling test ran first.
• Generate .claude/harness-candidates.md entries mechanically from the settled review-finding JSON (title, file:line anchor, verdict, one-line 'why nothing guards it today'), instead of hand-summarizing them in prose. The review workflow already has the structured findings in hand at the point the file is written; hand-transcription is the only reason the note can diverge from the code. Why not static: No checker can compare an English summary of a fix against the fix itself — that is exactly the semantic-judgment boundary. The fix is to delete the transcription step rather than to police its output. Prevents: The 🔵 Low finding that both new harness-candidate entries describe intermediate versions of the fix this same PR ships (the note omits the trajectory_id component of the cid fallback, which has its own dedicated test, and omits the post-sleep break at :609-614, which also has its own test) — i.e. the deferred-guardrail log is already stale on arrival.
• Add an advisory 'changed body, unchanged docstring' report to the PR review workflow. For each function in the diff whose body changed by more than N lines while its docstring is byte-identical, list it in the review output as a docstring-staleness candidate (advisory, not blocking). Why not static: Whether a docstring has become false is semantic — a heuristic can flag the candidate but cannot decide it, so this belongs in the review workflow's output rather than in make verify. Prevents: The 🟡 Medium Axis-7 finding: communicate()'s docstring still states the pre-fix 'iterate receive_steps() until the turn goes idle' contract, which the new poll loop directly contradicts. (The user-facing-doc half of that finding is covered statically by CE037.)

Top 5 Priority Actions

1. Derive the post-idle poll deadline from the effective turn timeout (stop at timeout - margin) or drop _MAX_BACKGROUND_POLLS * _BACKGROUND_POLL_INTERVAL_SECONDS below 300 s — src/coder_eval/agents/antigravity_agent.py:604 (constants :112, :140) — so an orphaned ACTIVE tool call exits through the graceful cap branch and is graded instead of raising TurnTimeoutError into orchestrator.py:566 → FinalStatus.ERROR with empty success_criteria_results, and add a test that a never-resolving orphan at timeout=300.0 finalizes and is scored.
2. Pin _drain's bounded-retry semantics at src/coder_eval/agents/antigravity_agent.py:485-497 with a test that raises RuntimeError after steps were already fed to state.process_step (:489) and asserts tr.token_usage / tr.agent_output / assistant-message count are not doubled by the replay — silent double-counting can trip run_limits.max_total_tokens/max_usd and flip a run to TOKEN_BUDGET_EXCEEDED for identical agent output — plus a test that a permanently-raising receive_steps() surfaces AgentCrashError after exactly _RECEIVE_STEPS_REENTRY_RETRIES calls (tests/test_antigravity_agent.py:1041 currently proves neither).
3. Correct the now-false turn contract: rewrite the communicate() docstring at src/coder_eval/agents/antigravity_agent.py:520-522 (it still says "iterate until the turn goes idle") and add a "Known limitations" entry to docs/agents/ANTIGRAVITY.md:169-183 naming the poll bound, the minutes-long event-silent window (DEBUG-only logging at :607), and the need to size run_limits.turn_timeout/task_timeout above it — the two shipped example tasks pin turn_timeout: 300, below the 600 s budget.
4. Close the AntigravityAgent lifecycle-teardown coverage hole at src/coder_eval/agents/antigravity_agent.py:677-722 (stop/kill/kill_sync/get_environment_info/_conversation_or_none/_teardown, uncovered at 86.13% suite-scoped) by mirroring tests/test_codex_agent.py:1517 and :1826, since kill_sync is invoked from the watchdog's non-asyncio thread and nothing pins it as a plain non-raising def, and log rather than silently swallow the failure in _teardown.
5. Take the mechanical complexity and state cleanups: extract the poll block at src/coder_eval/agents/antigravity_agent.py:600-623 into a _poll_for_backgrounded_work sibling of _drain (prototyped: communicate E33 → D24, below its pre-PR D27), replace the fifth parallel cid-keyed _tool_last_status: dict[str, Any] (:785) with an _active_tools: set[str] so has_orphaned_tool_call (:951) becomes bool(self._active_tools), and fix the two .claude/harness-candidates.md notes at :327 and :343 that describe intermediate fixes this same PR superseded.

---

Stats: 0 🔴 · 1 🟠 · 4 🟡 · 8 🔵 across 8 axes reviewed.

[joeysbase]

Fixed in 59552e8 — the poll loop's exit bound now derives from 0.8 * timeout (the actual value passed to communicate()), falling back to the cycle-based cap only when timeout is None. This wins the race against the ThreadedWatchdog's cutoff by design (a strictly earlier, non-identical deadline) rather than by accident, so it also holds for any non-default turn_timeout a task sets.

Added test_communicate_finalizes_gracefully_under_a_realistic_turn_timeout: a never-resolving orphan under the framework's real default turn_timeout: 300 now finalizes through the graceful path (force-closed as unresolved, turn still graded) instead of crashing as TurnTimeoutError. Confirmed it fails without the fix and passes with it.

Also fixed both .claude/harness-candidates.md entries uipreliga flagged as already-stale (missing the trajectory_id component and the mid-loop break), and added a new entry noting uipreliga's proposed generic lint rule (CE035) as a deferred-but-worth-revisiting candidate, since it would catch this class of bug in any agent, not just this one.

Thank you both — this was a real, well-diagnosed blocker and I'd rather have caught it here than in production.