Skip to content

Audit period strips by reviewed template, not an exact spelling list - #309

Merged
MaxGhenis merged 6 commits into
codex/thesis-ledger-factsfrom
fix/catalog-period-invariant
Oct 13, 2026
Merged

MaxGhenis merged 6 commits into
codex/thesis-ledger-factsfrom
fix/catalog-period-invariant

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

test_committed_catalog_is_current_and_valid pinned the exact list of stripped period spellings in the committed catalog. Every Thesis resolver append that records the next weekly claims print adds a spelling (week_2026-08-29, week_2026-09-05, …), so every append failed the required "Arch checks" status. The resolver merges through the API with an admin token (it waits only for "Append gate"), so the red check was ignored:

What the list protected

The builder can't tell a real period label from a statute, cohort, or edition label that happens to spell the row's own period. stripped_segments exists so a curator can catch that (build_series_catalog.py module docstring; classify_segments). The exact list forced a human look at every new spelling. It also failed when a spelling disappeared.

It had a gap: only spellings were pinned, not concepts. A new concept starting to strip an already-listed spelling (for example 2026_07) passed unseen.

The invariant now checked

build_series_catalog.stripped_segment_review_problems(stripped_segments, strips, reviewed) audits every strip. A strip is keyed by (canonical concept, identifier field, strip shape, spelling):

  • the identifier field is measure.concept or source_record_id;
  • the strip shape (strip_shape) is that field's family pattern with each {P} written {P:<template>}, for example abs.cpi.all_groups.yoy.{P:9999_99}.first_print. period_spelling_template turns digits into 9 and month words into <month>, and keeps separators and literal words. So a shape fixes every kept segment and where each stripped segment sits. Two identifiers share a shape exactly when they have the same kept segments, stripped slots in the same positions, and the same template in each slot. They can differ only in what the template normalizes: digits, including those inside an after_<qualifier>_ slot, and month words, with full and abbreviated names counting as one spelling.

The reviewed map is tests/fixtures/series_catalog/reviewed_stripped_segments.json, keyed concept → field → shape → spelling → classify_segments kinds. It holds the catalog at 55bbf3d that #244 reviewed, plus the 2026_08 ABS CPI decision (see notes).

A strip that is not in the reviewed map passes only when both of these hold:

  1. its concept already has a reviewed strip in the same field and the same shape, so it is the next week or month written the same way, in the same place;
  2. every current observation behind it spells its own declared period directly (kind derived, never overlap).

Everything else is still reported:

Event Old exact list Now
next weekly/monthly print, same concept, field and shape fail (every append) pass
new concept strips for the first time (new series, or a label that spells its row's period) fail fail
existing concept strips a new template (e.g. week_ending_… where it used week_…) fail fail
reviewed template in a new field or position, or beside a new kept segment (e.g. abs.cpi.2026_09.all_groups.yoy, ….revised) fail only if the spelling was new fail
overlap strip (identifier finer than the declared period, the June-13 defect) fail only if the spelling was new fail
reviewed strip starts being reached as an overlap pass fail
new concept strips an already-listed spelling pass fail
reviewed strip disappears from its field and shape fail (spelling only) fail
non-period spelling, spelling that fills no slot of its shape, catalog–plan disagreement n/a fail

build_catalog exposes the strips as plan["strips"]. The catalog bytes do not change: --check is green, the byte-regeneration assertion passes, and no catalog keys are added, so Thesis's frozen CATALOG_TOP_LEVEL_KEYS/CATALOG_ROW_KEYS are unaffected. Literal {P:...} segments are now reserved alongside {P} (no current ledger or docket-seed identifier uses one), so a shape can never mistake a kept segment for a stripped one.

Why field and shape, not just concept and template (review of 0852fcd)

The first version of this PR keyed decisions on (concept, spelling template). The independent review executed a copy of the 2026-08 ABS CPI first print (ledger line 231), moved to 2026-09, with its period also spelled inside measure.concept:

row["measure"]["concept"] = "abs.cpi.2026_09.all_groups.yoy"
row["source_record_id"] = "abs.cpi.all_groups.yoy.2026_09.first_print"

The builder folds it into the reviewed abs.cpi.all_groups.yoy identity with no mint, under a new family pattern abs.cpi.{P}.all_groups.yoy. The template-only audit passed it, so an unreviewed cohort or edition slot inside the concept borrowed the approval of the source_record_id period suffix. Now the source_record_id strip passes (reviewed shape), and the measure.concept strip is reported:

'2026_09' on 'abs.cpi.all_groups.yoy': first strip in measure.concept shape 'abs.cpi.{P:9999_99}.all_groups.yoy' for this concept — confirm it is a period label, not a statute, cohort, or edition label, and add it to the reviewed map

test_a_reviewed_template_in_a_new_concept_position_needs_review is that exact regression on the real ledger.

week_2026-06-13 vs week_2026_06_13

Not a naming bug. Both come from the one 2026-06-13 initial-claims row (ledger line 44):

  • its source_record_id is us.dol.initial_claims.sa.week_2026-06-13 (ISO hyphens);
  • its measure.concept is us.dol.initial_claims.sa.week_2026_06_13 (underscores).

The builder strips period tokens from both fields and records each distinct spelling. Both strip as overlap because the row declares month 2026-06, the cadence defect already in docs/catalog-curation-backlog.md ("Initial claims cadence metadata"). They are now two reviewed strips, one in each field's shape. The fixture notes and the backlog document this, and test_reviewed_stripped_segments_fixture_is_wellformed asserts it.

Evidence on real data

The fixture was regenerated from 55bbf3d's inputs with this builder. It covers exactly the 176 (spelling, concept) pairs the previous fixture listed, with the same kinds, now as 354 strips in 297 (concept, field, shape) decisions across 120 concepts.

Each merged append since 55bbf3d and each refused proposal head, built with this branch's builder (before the #344 merge, which changes no strip) and audited against the #244 map (the fixture without the 2026-10-12 ABS CPI decision):

Head Unreviewed strips Problems
#283 78308f8 (refused) 10 0
#297 7917dbe (refused) 10 0
#298 7bc1595 (refused) 10 0
598f394 (merged, 8 prints) 11 0
#289 177c461 (refused) 11 1: '2026_08' on 'abs.labour.unemployment_rate': first strip in source_record_id shape 'abs.labour.unemployment_rate.{P:9999_99}.first_print'…
2ed9bae (merged) 13 1: '2026_08' on 'abs.cpi.all_groups.yoy': first strip in source_record_id shape 'abs.cpi.all_groups.yoy.{P:9999_99}.first_print'…
d7e45bc (merged; the base before #344) 15 1: the same ABS CPI strip

Scoping by field and shape added no false alarms on any routine append. The two remaining reports are real curation events:

  • Record 8 first-print observation(s) via resolve_pending.py #289: the docket placeholder abs.labour.unemployment_rate took its first print beside an already-observed abs.labour.unemployment_rate.australia, which looks like a concept-spelling duplicate.
  • 2ed9bae: the ABS CPI series switched from 2026-06 to 2026_08 in its source_record_id. 0852fcd reviewed that strip (a period label) and recorded it with a note. Against the full fixture, 2ed9bae and d7e45bc have no problems.

Invariants (tested)

Property tests use Hypothesis, derandomized so this required check stays deterministic:

  • T1: a spelling's template depends only on how it is written, never on the period it names.
  • T2: different spelling families get different templates. Full and abbreviated month names share one template on purpose, because may is both.
  • T3: every generated family spells its own period directly.
  • T4: an identifier's strip shape depends only on its kept segments, where its period sits, and how it is written. It never depends on the period named, and a different position or template gives a different shape.
  • A1, routine continuation passes: end to end through the real builder, appending further periods of observed series written as they already are never produces a problem. A non-vacuity assertion checks that the new strips really are unreviewed.
  • A2, soundness: end to end, the audit always names, by spelling and concept:
    • a new concept;
    • a new template;
    • a new shape: the period spelled inside measure.concept, which folds into the reviewed identity, or beside a new kept segment of source_record_id;
    • an overlap strip, whether it is a fresh spelling or a reviewed one that gains a kind.
  • A3, persistence: a reviewed strip missing from the plan is reported as gone.
  • A4, monotone in review: reviewing more live strips never adds problems, and reviewing every live strip leaves none.
  • A5: the output is sorted, has no duplicates, and does not depend on input order.
  • A6, exactness: a reference model read straight off the rule names the same (spelling, concept) pairs as the audit, in both directions. Its reviewed maps include reduced kinds and stray reviews.

Other tests:

  • Differential: test_strips_match_an_independent_replay re-derives every strip from the ledger with classify_segments and period_spelling_template alone: its field, its shape, and its kinds. It resolves each observation to its catalog row through that row's concept and aliases, and requires plan["strips"] and the catalog's stripped_segments to agree exactly.
  • Real ledger, next period: for every eligible current observation (223: its source_record_id strips a period and all its strips are direct spellings), append the next period written the same way, month words kept full or abbreviated as written. The test asserts all 223 are appended (the first version skipped the 66 month-name prints). They land on the existing identities with no mints, supersedes or drops. They add ≥50 new strips, all in shapes their concepts already strip in, and none needs review.
  • Examples: test_strip_audit_flags_a_reviewed_template_in_a_new_shape covers four cases, each folding into the reviewed identity:
    • the period inside measure.concept;
    • the very shape string of the reviewed source_record_id strips, but in measure.concept;
    • a new kept segment (.revised);
    • a new source_record_id position.
  • Corruption self-test: test_committed_audit_fails_on_corrupted_catalogs derives its unprinted week from initial-claims prints and asserts only that the initial-claims pair is absent (the review's minor finding). It runs once with a valid continued-claims print of that same week, the case that broke the old global-absence assertion.
  • Mutation check (run at ab6d9b9): 14 mutants of the new code. Each one is killed by at least one test:
    • keying review by concept and template only (the old rule);
    • ignoring the field;
    • shapes without templates;
    • dropping the slot check;
    • reporting "gone" by pair;
    • stripping source_record_id only;
    • reserving {P} only;
    • dropping each of the gained-kind, overlap, catalog/plan and non-period checks;
    • unsorted output;
    • swapped field labels;
    • kinds merged across shapes.

Scope and coordination

  • Fold seven entity-drift lineages into their economy/aggregate siblings #344 (entity folds) merged first (65822f4); this branch merges it in f2a6d76. The Hypothesis pyproject.toml/uv.lock hunks were identical. build_catalog keeps both plan keys (entity_folds_applied, strips). The committed test keeps this audit in place of Fold seven entity-drift lineages into their economy/aggregate siblings #344's one-time re-pin of the exact list. _committed_build(), the ABS CPI regression and the next-period test pass bsc.ENTITY_FOLDS. The replay resolves a row that an active fold absorbs at the fold's survivor entity: geography and absorbed entity match, and the row's concept is one of the declared absorbed_names, read straight from the seed. The folds change no strip, so the reviewed fixture is unchanged.

  • Unchanged pins: the row-count pin (== 221 since Fold seven entity-drift lineages into their economy/aggregate siblings #344) and test_identity_uuid_map_matches_reviewed_anchor fire on identity events (mints, enrichments, supersedes). suspect_segments == ["LNU02374597"] fires on those and also on any new date-like kept segment (for example a source_record_id ending .edition_2026), with no identity event. The repo documents all of these as needing curation.

  • CI steps run locally (receipt 0.6.2, Python 3.14, Hypothesis 6.169, ruff 0.14.10):

    • build_series_catalog.py --check;
    • pytest tests/test_build_series_catalog.py (222 passed at ab6d9b9);
    • the module's doctests;
    • --verify-registry-append-only against the base registry;
    • ruff check on both changed Python files.

    I did not run the Arch-surface pytest list, the DB build or the wheel steps locally. No module outside this test imports the builder or reads the plan, and CI runs them.

  • ledger/ is untouched.

axiom: n/a: catalog test and tooling change, no policy logic.

🤖 Generated with Claude Code

test_committed_catalog_is_current_and_valid pinned the exact list of
stripped period spellings in the committed catalog. Every resolver append
that recorded the next weekly claims print added a spelling
(week_2026-08-29, week_2026-09-05, ...) and failed the required "Arch
checks" status, so proposals merged past it with the resolver's admin
token (#240 on 2026-09-03, 598f394 on 2026-09-29) and the check stopped
meaning anything.

The list stood for one question: has a curator decided that this strip is
a period label and not a statute, cohort, or edition label that happens
to spell its row's period? The test now asks that per (spelling, concept)
pair:

- build_series_catalog.stripped_segment_review_problems audits the
  catalog's stripped_segments against a reviewed map
  (tests/fixtures/series_catalog/reviewed_stripped_segments.json, the
  catalog at 55bbf3d that #244 reviewed, with each strip's kinds).
- A strip outside the map passes only if its concept already has a
  reviewed strip of the same period_spelling_template (the same way of
  writing a period) and every observation behind it spells its own
  declared period directly (kind derived).
- New concepts, new templates, unreviewed overlap strips, reviewed strips
  that gain the overlap kind, reviewed strips that disappear, non-period
  spellings, and catalog/plan disagreements all still fail. A concept
  that starts stripping an already-listed spelling used to pass unseen;
  it now fails.

build_catalog exposes each strip's classify_segments kinds as
plan["stripped_kinds"]; catalog bytes are unchanged (--check green).

week_2026-06-13 and week_2026_06_13 are not a naming bug: both come from
the one 2026-06-13 initial-claims row (ledger line 44), whose
source_record_id uses ISO hyphens and whose measure.concept uses
underscores. Both strip as overlap because that row declares a month (the
cadence defect in docs/catalog-curation-backlog.md). Documented in the
fixture notes and the backlog, and asserted by the fixture test.

Hypothesis joins the dev extras with the same pyproject and uv.lock change
as #306.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Adopting this PR to land it. Its authoring session can't be found in the session index, and the PR has had no activity since 2026-09-30.

Why it matters now: the ledger branch's required "Arch checks" job has been red since 598f394 (the stale stripped-spelling list in test_committed_catalog_is_current_and_valid), and branch protection requires it, strict. That blocks the Chronicle entity-fold PR for Max's ruling d353 (fold seven entity-drift lineage pairs; thesis side is ThesisInstitute/thesis#342). This PR is the durable fix.

Plan: bring the branch up to date with codex/thesis-ledger-facts (two resolver appends landed since, 2ed9bae and d7e45bc), let CI rerun, get an independent GPT-6.1 Sol review at the updated head, and merge only if CI is green and the review agrees. #306 also re-pins the same list and will need a rebase after this lands; I am not touching it.

Session: Resolve duplicate Chronicle lineages for three BLS concepts (local_c7ccacf7-d314-449b-b837-d0cd5835d3e0).

MaxGhenis and others added 2 commits October 12, 2026 08:48
…rinted

Two resolver appends landed after this branch was written.

- 2ed9bae recorded abs.cpi.all_groups.yoy for 2026-08 as
  abs.cpi.all_groups.yoy.2026_08.first_print. The segment is the row's own
  period with an underscore; the concept's June print used a hyphen, so the
  audit reported the concept's first 9999_99 strip. Reviewed and recorded.
- d7e45bc printed week_2026-09-19 for us.dol.initial_claims.sa, the week the
  corruption self-test used as unprinted, so two of its cases stopped
  corrupting anything. The test now derives a week one year past the latest
  printed week and asserts it is absent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Update from the adopting session. I am not landing this PR.

The independent GPT-6.1 Sol review at 0852fcd requested changes (full report: ~/chief-of-staff/state/chronicle-lineage/reviews/pr309-review.md on Max's machine). Summary:

  1. Major. Strips from measure.concept and source_record_id are merged without field or position, and continuation needs only the canonical concept and spelling template. The reviewer executed a crafted row (concept abs.cpi.2026_09.all_groups.yoy, record id abs.cpi.all_groups.yoy.2026_09.first_print) that minted a new family pattern abs.cpi.{P}.all_groups.yoy and passed the audit. The exact list would have rejected it. Fix: scope reviewed decisions to identifier field and full family pattern.
  2. Minor. The corruption self-test's derived unprinted week asserts global absence but only looks at initial-claims weeks, so another concept's valid continuation could fail it. Assert the (spelling, us.dol.initial_claims.sa) pair instead.
  3. CI at this head was not verified by the reviewer (no network).

#344 (Max's ruling d353) no longer depends on this PR. It re-pins the existing list as routine maintenance, the way #244 did, and carries the same Hypothesis dev-dependency hunks as this PR, so the two merge cleanly in either order. A follow-up session to address the findings above is proposed to Max.

The independent review of 0852fcd executed a copy of the 2026-08 ABS CPI
first print moved to 2026-09 with the period also spelled inside
measure.concept (abs.cpi.2026_09.all_groups.yoy). The builder folds it into
the reviewed abs.cpi.all_groups.yoy identity under a new family pattern,
abs.cpi.{P}.all_groups.yoy, and the template-only audit passed it: the
measure.concept slot borrowed the approval of the source_record_id period
suffix.

A reviewed decision now covers one concept, one identifier field
(measure.concept or source_record_id) and one strip_shape: the field's
family pattern with each {P} written {P:<period_spelling_template>}, so it
fixes every kept segment, where each stripped segment sits and how it is
spelled. Only a calendar substitution inside a reviewed shape passes
without curation; a new field, position, template or kept segment is
reported. The build plan's stripped_kinds becomes strips (concept -> field
-> shape -> spelling -> kinds); catalog bytes are unchanged. Literal
{P:...} segments join {P} as reserved, so a shape never mistakes a kept
segment for a stripped one.

The reviewed fixture is regenerated in that form from 55bbf3d's inputs
(the catalog #244 reviewed) plus the 2026_08 ABS CPI decision; it covers
exactly the same 176 (spelling, concept) pairs with the same kinds.

The corruption self-test now derives its unprinted week from initial
claims and asserts only that the initial-claims pair is absent, and runs
once with a continued-claims print of that same week, so another concept's
valid continuation can no longer break it.

New tests: the review's exact regression on the real ledger; field,
position and kept-segment examples; a new_shape novelty in the A2
property; T4 (shape = identifier up to calendar values); A6, a reference
model of the rule compared with the audit in both directions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 2 commits October 12, 2026 19:38
…period-invariant

Conflicts: build_catalog keeps both plan keys (entity_folds_applied from
#344, strips from this branch); test_committed_catalog_is_current_and_valid
keeps the reviewed-strip audit in place of #344's one-time re-pin of the
exact spelling list. The committed catalog now declares
entity_folds_sha256, so _committed_build, the ABS CPI regression and the
next-period test pass bsc.ENTITY_FOLDS, and the independent replay resolves
a row an active fold absorbs (fold geography and absorbed entity, concept
among the declared absorbed_names, read from the seed) at the survivor
entity. The folds change no strip: the reviewed fixture is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the independent review of 5fa0c34 (APPROVE, one MINOR, three NITs):

- MINOR: the real-ledger next-period test skipped the 66 observations
  whose period is a month name, because the successor has both a full and
  an abbreviated spelling of the same template. _respell now keeps each
  month word full or abbreviated as written ("may" is both, so its
  successor is written in full), and the test asserts it appends every
  eligible observation (223 of 223), not ">= 100".
- The curation backlog no longer says a re-perioded June-13 row makes its
  strips disappear: re-spelling or retracting the row does; re-perioding
  it to its week keeps the strips and reports gained kinds.
- period_spelling_template, strip_shape, the module docstring, the audit
  docstring, the backlog and the fixture comment now say shapes agree up
  to the digits and month words the template normalizes (qualifier digits
  and full/abbreviated month names included), not "only calendar values".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 8c4a0f8 into codex/thesis-ledger-facts Oct 13, 2026
8 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged at head ab6d9b9f (merge commit 8c4a0f86) on these gates:

  • gh pr checks exited 0 on the latest runs: Append gate, Arch checks and Trusted base append gate all passed, and so did Arch bundle drift.
  • mergeable was MERGEABLE (merge state CLEAN), the PR was not a draft, and no changes were requested.
  • Independent reviews by GPT-6.1 Sol (Subfleet review --tier hard):
    • 0852fcd6: REQUEST_CHANGES. One MAJOR: a reviewed template could authorize a new identifier shape. One MINOR: the corruption self-test's unprinted week.
    • 5fa0c34a: APPROVE. One MINOR (month-name coverage in the next-period test) and three documentation NITs.
    • ab6d9b9f (final head, after merging Fold seven entity-drift lineages into their economy/aggregate siblings #344): APPROVE. One PR-body NIT, now fixed.
  • Locally at ab6d9b9f:
    • tests/test_build_series_catalog.py: 222 passed;
    • --check: current, 221 series;
    • doctests and ruff check clean;
    • registry append-only against the base;
    • all 14 mutants killed.
  • ledger/ is byte-identical to the base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant