Skip to content

feat(profiling): triage --axis compile — classify compile drift, close the arc #221

Description

@Jammy2211

Overview

Phase 3 of 3 of the compile-axis arc (phase 1 #218/#219; phase 2 autolens_profiling#103,
PR #220 + autolens_profiling#104).

Phases 1–2 make compile drift visible; this makes it actionable, answering the
question that decides what anyone does next: who owns this?

Plan

  • Add --axis compile to triage, classifying each ingest finding.
  • Compare a suspect warm row against its own cold scale — the yardstick that makes
    cache-regression a measurement rather than a guess.
  • Route library findings to bug/ via intake; never debug the library in the profiling
    repo (the agent's existing boundary, unchanged).
  • Close the arc in AGENTS.md: compile-time profiling moves out of Future modes,
    and Boundaries records that release-validation script cost stayed with hygiene.

Classifications

classification signal actionable
cache-regression warm compile returned to its own cold scale yes — config/stack, never the library
autotune-regression GPU compile up ≥10× with no cold-scale match yes — check the flag reaches XLA
library-regression growth on an unchanged key, no other explanation yes — bug/ via intake
host-load measuring host's 1m load average was high no — re-measure idle first
expected-recompile key differs from a pin only by jax_version no — one recompile is by design
new-machine / new-precision / new-cell key is simply unpinned no — pin it

Acceptance

  • Every ingest finding classified into exactly one outcome, with evidence.
  • A synthetic warm-reverting-to-cold row classifies as cache-regression
    verified against a copy of the real workspace, so the cold reference is real
    data: a warm vag row moved 1.622 s → its own 34.592 s cold cost and classified
    correctly with that evidence quoted.
  • A jax_version bump classifies as expected-recompile, not a regression.
  • Nothing written to autolens_profiling (asserted by mtime comparison).
  • AGENTS.md Modes + Boundaries updated.

Note on the prompt's original table

Two of the five categories the prompt specified — drift from a jax_version bump, and
drift from changed measurement conditions — cannot reach triage as drift by
construction
, because both are different comparability keys and ingest reports them
as unpinned rather than drifted. That's the design working rather than a gap. They're
still classified so nothing vanishes, but they're never regressions. host-load was
added in their place, which the prompt didn't list but the data supports — it's what
host_state was added for in phase 2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions