[MaxGhenis]

Stacked on #402. Merge #402 first; GitHub then retargets this PR to main.

Every fact in FPB's June 2026 outlook annex carries a table-wide source concept, such as fpb.economic_outlook_2026_2031.t07.published_cell. A consumer that treats the observed concept as series identity then takes different rows of one table for one series.

Microcosm's roll-forward check (_rolled_forward_series_key, ledger_targets.py:3484-3505) does exactly that. A Microcosm triage accepted a 2025 total-population numerator over a 2024 working-age-population denominator as one series, with a ratio of 1.59, inside its bound. Microcosm's hub ruled this a fix for Belgium's Chronicle adapter, not a shared change (B3 triage, claim 5).

What changes

• packages/fpb/economic_outlook_2026_2031_june_2026/source_package.yaml: each of the 1,000 record sets now names its table and row, in Statbel's style: fpb.economic_outlook_2026_2031.t07.total_population. That gives 100 concepts, one per published row, each spanning 2022–2031.
  • The package has no generator, so a one-off script rewrote the 1,000 source_concept lines.
  • A YAML reload matched every record set to the original except for that field.
  • Row ids are unique within each table (checked).
• tests/test_source_fpb_row_source_concept.py: 11 tests.
• tests/test_source_be_period_coverage.py: the digest test restores the table-wide concept before hashing, so it still proves that Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's coverage moved no key.
• tests/test_belgium_targets.py: the per-table count now reads the concept's table prefix.

Invariants (tested at build years 2023 and 2026)

• Every fact's source concept is fpb.economic_outlook_2026_2031.<sheet>.<row id>. The sheet comes from the package's own record set, and the row id equals the layout's source_row_id.
• Each pinned row id names its publisher row. The test pins 21 (sheet, row id) pairs to the header in column A of DATA_FOR_MLT_FR.xlsx: all ten T07 rows and a sample from every other table. Each pair's expected_row_header and every fact's groupby_value_label equal that header. The package guard already refuses a fact whose header cell differs (chronicle/sources/specs.py:260-267, :541-547), so the pin ties the id, and with it the concept, to the publisher's text.
• Each concept is one series: exactly one record-set spec, one canonical concept and one unit, and one fact for each year from 2022 to 2031. The tables hold 4, 3, 10, 21, 7 and 55 rows.
• No two rows share a concept, and every aggregate key is unique. Each row has one observed-measure key and one alignment key of its own.
• Semantic keys and values don't move. Their digest equals the one at 2c7e835.
• Every key that hashes the observed concept moves, and one digest pins all four. The observed-measure key hashes source_concept (chronicle/consumer_contract.py:129-138, :797-804). The aggregate key (:178) and the alignment key (:247) nest the observed-measure key, and the legacy key hashes the whole measure (chronicle/core.py:638).

Evidence

• Tests at 0c1c6ee: the new file, the period-coverage file, the Eurostat FX file and test_belgium_targets.py::test_fpb_annex_has_requested_table_counts_and_vintage_boundary give 39 passed.

• Belgium suite: test_belgium_targets.py -k "fpb or bfp" gives 3 passed. The one failure is bfp-economic-outlook-2026-06, missing from the sparse checkout ("Source package not found").

• Ablations at 0c1c6ee, over the new file and the period-coverage file:

  • Reverting the YAML to Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's head fails 7 of the new tests.
  • Giving the 2025 total-population row the working-age concept fails the same 7.
  • In both cases the semantic and coverage tests pass.
  • Swapping value_id and source_concept between T07 rows 4 and 5 in all ten years, a mislabel consistent with itself (the review's A2), fails the header test at both build years. It also fails both digests in the new file and Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's coverage digest: 8 failures in all.

• Before and after, over all 1,000 facts at build year 2026, 7 fields differ, each in every fact:

  • aggregate_fact_key and legacy_fact_key;
  • observed_measure.source_concept and observed_measure_key;
  • concept_alignment.source_concept and concept_alignment.concept_alignment_key;
  • layout.record_set_spec_hash.

  The semantic key, source_series_key, the value, lineage and period_coverage don't move. The review found the same 7 at build years 2023 and 2026.

• Validation: validate_source_package passes at 2023 and 2026, with 0 errors.

• Ruff 0.14.10: clean.

• The local run used a sparse checkout, so the full suite runs in CI.

Microcosm's read

Microcosm's roll-forward check keys a series on the observed source concept. It reaches the canonical concept only when a fact has no source concept (_source_measure_concept, ledger_targets.py). Run through _rolled_forward_series_key as it stands on Microcosm main c3b9b736, the 1,000 FPB rows give:

Chronicle	Series keys	Keys that mix canonical concepts
#402 alone (827504ba)	9	6
With this PR (5926dfc1)	100	0

With this PR, each of the 100 keys spans 2022–2031 with no repeated year. The YAML hasn't changed since either head. So #402 and this PR should land back to back: #402 alone gives Microcosm both the coverage it needs and series keys that mix rows.

Downstream

Microcosm-BE pins FPB's aggregate keys and source concepts in its B3 reference documents under microcosm/build/be/ (target_references.json, holdout_references.json, reference_inventory.json and others). None of them pins the observed-measure key, the alignment key or the spec hash. Those documents regenerate when Microcosm repins its feed to the next BE consumer artifact. That one rebuild picks up both #402 and this PR. A release holds the host's disk until about 11/05.

Judges

ledger-source-fidelity and ledger-boundary: an independent review at 5926dfc passed both (review). A delta review of 5926dfc..0c1c6ee follows.

🤖 Generated with Claude Code

[MaxGhenis]

Review of #403 at 5926dfc: APPROVE
ledger-source-fidelity: PASS, ledger-boundary: PASS

The head is still 5926dfc1 and the PR is open. I reviewed git diff 827504ba..5926dfc1 after a uv sync --locked --all-extras checkout. The diff changes no file under chronicle/. All 2,000 changed lines in the YAML are source_concept: lines (1,000 removed, 1,000 added). CI is green on this head: Chronicle checks, surface and bundle.

Findings, ranked

1. The PR body understates which derived fields move (non-blocking)

• Where: the PR body ("only source_concept, aggregate_fact_key and legacy_fact_key differ") and tests/test_source_fpb_row_source_concept.py:34 (ROW_KEY_DIGEST).
• What I ran: I dumped consumer_fact_rows plus period_coverage for all 1,000 facts at build years 2023 and 2026, once in a worktree at 827504ba and once at 5926dfc1. I then diffed every leaf field, matched by source_record_id.
• Result: 7 fields differ, each in 1,000 of 1,000 facts:
  • aggregate_fact_key
  • legacy_fact_key
  • observed_measure.source_concept
  • observed_measure_key
  • concept_alignment.source_concept
  • concept_alignment.concept_alignment_key
  • layout.record_set_spec_hash
• What doesn't change: semantic_fact_key, source_series_key, value, lineage cell keys and period_coverage.
• Why it's non-blocking: all four extra fields are derived from the concept, so they are expected to move.
• Why it still matters: Microcosm-BE pins FPB keys in its B3 reference documents. Any document that pins observed_measure_key, concept_alignment_key or record_set_spec_hash will move on repin too, not just the aggregate and legacy keys.
• Fix:
  • Correct the "Evidence" bullet in the body.
  • Optionally add observed_measure_key and concept_alignment_key to the ROW_KEY_DIGEST rows, so every moved key is pinned.

2. test_every_fact_names_its_table_and_row checks the package against itself (non-blocking)

• Where: tests/test_source_fpb_row_source_concept.py:58-66.
• The gap: the test asserts source_concept == PREFIX.<sheet>.<groupby_value_id>. Both sides come from the package YAML, so the test proves the concept is consistent with the package's own row id. It doesn't prove the row id names the publisher row.
• Ablation A2: I swapped both value_id and source_concept between T07 total_population (row 4) and working_age_population (row 5) for all 10 years, leaving expected_row_header alone. That is a consistent mislabel.
  • This test passed.
  • The mislabel was caught only by the two pinned digests (test_row_concepts_move_no_semantic_key_or_value, test_row_concepts_pin_aggregate_and_legacy_keys) and by Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's coverage digest: 6 failures in all.
• Why it's non-blocking: the digests do catch it. They just don't say why.
• Fix: add an assertion that ties the concept's row id to publisher text. For example, pin a {(sheet, row_id): expected_row_header} table for the 10 T07 rows, plus a sample from T11 and T24. Or assert that fact.layout carries the header from the workbook's column A.

3. I couldn't read Microcosm's code (non-blocking, unverified)

This session's GitHub scope is policyengine/chronicle only. The call failed:

gh api "repos/PolicyEngine/microcosm/contents/packages/microcosm-build/src/microcosm/build/ledger_targets.py?ref=ba34140449e2cf3ce32a110054e9c7295bc00b04" -H "Accept: application/vnd.github.raw"
gh: GitHub access to this repository is not enabled for this session. Use add_repo to request access. ... (HTTP 403)

I didn't widen the scope, so I couldn't read _rolled_forward_series_key or _source_measure_concept. Instead, I tested the property they need on the Chronicle side.

I grouped consumer_fact_rows by observed_measure.source_concept at both build years:

Commit	Series	Series that mix canonical rows	Year coverage
827504ba	6	6	—
5926dfc1	100	0	each series covers exactly 2022–2031

So under the PR, two different FPB rows give different observed concepts, and one row gives one concept across 2022–2031. This holds if Microcosm keys series on the observed source concept and nothing coarser. Someone with Microcosm access should confirm that against its code.

Verified and correct

• Fidelity against publisher bytes, all 1,000 record sets:
  • I opened db/data/fpb/economic_outlook_2026_2031_june_2026/DATA_FOR_MLT_FR.xlsx with openpyxl myself (I didn't rely on the package guard).
  • For every record set, the column-A cell at row_number equals expected_row_header.
  • The new concept equals fpb.economic_outlook_2026_2031.<sheet_name.lower()>.<value_id>.
  • The canonical concept ends in the same value_id.
  • Each of the 100 concepts maps to exactly one (sheet, row_number, header, spec).
  • Result: 0 problems.
• All 10 T07 rows checked by hand:
  • total_population → r4 "I. Population totale (II+III)"
  • working_age_population → r5 "I.bis Population d'âge actif (15-64 ans)"
  • domestic_employment → r9 "IV. Emploi intérieur"
  • private and public employees and self-employed → r10, r12, r11
  • bfp_unemployment → r14 "VI. Chômage (concept BFP)"
  • registered jobseekers → r15
  • older non-jobseekers → r17
  • bfp_unemployment_rate → r18 "Taux de chômage (VI)/(III)"
• All 21 T11 rows and a sample of T24 checked by hand. Examples: t11.interest_paid → r14 "Intérêts payés (-)"; t24.social_security_cash_private_sector_pensions_total → r10 "a. Pensions du secteur privé". All match.
• Row grain is more faithful than …published_cell. The old label named no publisher series: one table-wide label stood for 4–55 distinct rows.
• The form follows Statbel's convention: <publisher>.<release>.<series>, for example statbel.population_structure.population and statbel.fiscal_income.total_net_taxable_income.
• Identity: semantic keys, values and period_coverage are identical at 827504ba and 5926dfc1, at build years 2023 and 2026 (see finding 1 for the derived fields that do move).
• Chronicle readers:
  • concept_alignments (chronicle/database.py:271-290, :812-866) uses (source_concept, canonical_concept, relation, legal_vintage, period) as its primary key. Before, the table-wide concept fanned out to 100 canonical concepts; now each pair is 1:1. Either way the rows are unique, so no primary-key collision appears and none disappears.
  • _observed_concept_key (chronicle/bundle.py:1389) only drives counts.by_observed_concept. That count now gets 100 buckets instead of 6, which is the intended change.
  • Nothing else in chronicle/ reads the FPB concept, and no other file refers to published_cell except this PR's test helper.
• The period-coverage test restoration is sound (tests/test_source_be_period_coverage.py:123-137):
  • It rewrites only measure.source_concept, and only on FPB-prefixed facts, so any other key input that moved would still change Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's digest. That digest still matches, which also shows the spec-hash change feeds no key.
  • Rows → table is many-to-one, so the restoration alone could hide a row swap. But this PR's SEMANTIC_DIGEST and ROW_KEY_DIGEST pin that independently, and ablation A2 tripped them.
• Ablations, run in a separate worktree:
  • A1: I relabelled one 2025 cell, t07.total_population → t07.working_age_population. 7 new tests fail; the semantic and coverage tests pass.
  • A2: the consistent swap described in finding 2. 6 tests fail.
  • A3: I reverted the YAML to 827504ba. The same 7 tests fail.
  • These match the PR body's ablation claims.
• Focused tests: I ran tests/test_source_fpb_row_source_concept.py, tests/test_source_be_period_coverage.py and test_belgium_targets.py::test_fpb_annex_has_requested_table_counts_and_vintage_boundary: 24 passed.
• Validation and lint: validate_source_package reports 0 errors at 2023 and 2026. Ruff check and format are clean on the three test files.
• Boundary: this is a pure relabelling of the publisher row. No code changes, and no reconciliation, aging, imputation, support-aware activation, solver construction or PolicyEngine-computed values.
  • The imputed_social_contributions hits in the diff are FPB's own row labels ("Cotisations sociales imputées"), not Chronicle imputation.
  • assertion (observation for 2022–2025, source_projection for 2026–2031) and period_coverage are unchanged.

Not verified

• Microcosm's _rolled_forward_series_key and _source_measure_concept code (finding 3).
• The full test suite locally. CI ran it and passed.

---

Generated by Claude Code

[MaxGhenis]

Review of #403 at 0c1c6ee: APPROVE
ledger-source-fidelity: PASS, ledger-boundary: PASS

The head is still 0c1c6ee6, and the PR is open. I reviewed 5926dfc1..0c1c6ee6. It touches only tests/test_source_be_period_coverage.py and tests/test_source_fpb_row_source_concept.py. The package YAML, the workbook and chronicle/ are unchanged, so the 5926dfc fidelity and boundary findings still hold.

When I posted this, CI on 0c1c6ee6 was still running: Chronicle bundle and Chronicle surface showed in_progress. The owner's landing condition still needs it green.

Findings, ranked

1. The header pin can't tell apart T24 rows that share a header (non-blocking)

• Where: tests/test_source_fpb_row_source_concept.py, the ROW_HEADERS entry ("t24", "federal_government_in_kind_health_care"), and test_row_ids_name_the_publisher_rows.
• The cause: in T24, 8 column-A header texts are each shared by 2–4 rows. For example, "1. Prestations de soins de santé" heads the federal, social-security and communities-and-regions health-care rows. I found this by grouping expected_row_header by sheet over all 100 rows.
• Ablation A4: I swapped value_id and source_concept between federal_government_in_kind_health_care and social_security_in_kind_health_care in all ten years.
  • test_row_ids_name_the_publisher_rows passed at both build years.
  • The swap was caught only by the semantic digest, the row-key digest and Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's coverage digest: 6 failures in all.
• Why it's non-blocking: the digests still catch it.
• Fix: pin (sheet, row_id) → (row_number, header), or pin the section header (B. Prestations en nature under Pouvoir fédéral, and so on). That way the pin ties rows whose headers repeat to their cells.

2. groupby_value_label comes from the package (non-blocking, wording only)

• Where: chronicle/sources/specs.py:332 sets groupby_value_label=row.label.
• The issue: the label comes from the YAML row label, not from the workbook.
• Why it's non-blocking: the second assertion in the header test checks that label agrees with expected_row_header. Only the package guard ties the header to the cell, and the test comment says so.
• Fix: none needed.

Verified

• A2 rerun on 0c1c6ee6: I swapped value_id and source_concept between T07 rows 4 and 5 in all ten years. 8 tests fail:

  • test_row_ids_name_the_publisher_rows at 2023 and 2026, the new catch;
  • test_row_concepts_move_no_semantic_key_or_value at 2023 and 2026;
  • test_row_concepts_pin_every_key_they_move at 2023 and 2026;
  • test_period_coverage_moved_no_fact_key_or_value[fpb-…] at 2023 and 2026.

  This matches the body's claim.

• All 21 pinned headers: I read column A of DATA_FOR_MLT_FR.xlsx with openpyxl, at the row_number each row's record sets use. All 21 match exactly:

  • T01 A12, T06 A4;
  • T07 A4, A5, A9, A10, A11, A12, A14, A15, A17 and A18;
  • T11 A5, A12, A14 and A19;
  • T17 A8;
  • T24 A4, A10, A22 and A40.

  The pins cover all ten T07 rows and every table.

• Extended digest: I recomputed it myself at 2023 and 2026. sha256 of the sorted [source_record_id, aggregate_fact_key, legacy_fact_key, observed_measure_key, concept_alignment_key] rows is d9795cd8… at both years, equal to ROW_KEY_DIGEST. The old three-field digest is still dc0d8c71…. Both key sets have 100 unique values, one per row, as the new count assertions claim.

• Merge commit 3f8ee1b: its change to the coverage test is the same, line for line, as Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's 827504ba..03ee1e37. origin/be/fx-fpb-period-coverage is now 03ee1e37.

• Declare period coverage on Belgium's Eurostat FX and FPB outlook facts #402's tighter note pins pass with the row-grain concepts: "Column headed {year} in …" and "a year in/before the FPB's 2026-2031 projection period".

• Body claims Chronicle can settle, all confirmed:

  • 39 passed over the new file, the coverage file, the Eurostat FX file and the T-counts test. I reran it: 39 passed.
  • 7 fields move in every fact. This matches my 5926dfc diff; the delta changes no YAML.
  • The YAML hasn't changed since either head. 5926dfc1..0c1c6ee6 and 827504ba..03ee1e37 touch only test files.
  • The module docstring now names every key that moves, plus the spec hash.
  • Ruff check and format are clean.

• Boundary: the delta adds tests only. It adds no reconciliation, aging, imputation, activation or solver construction.

Not checked

These body claims need Microcosm, which this session can't read (scope is policyengine/chronicle):

• "Microcosm's read": 9 series keys with 6 mixing canonical concepts at 827504ba, and 100 and 0 at 5926dfc1, run through _rolled_forward_series_key at Microcosm c3b9b736. The Chronicle side is consistent with this: 6 observed concepts before the PR, and 100 after, each spanning 2022–2031. I can't account for the 9 against my 6 without Microcosm's key fields.
• "Downstream": that no B3 reference document pins the observed-measure key, the alignment key or the spec hash.

---

Generated by Claude Code

[MaxGhenis]

The Belgium session (local session 0a5932a8) retargeted this to main after #402 merged, then merged it as 1a3f2c41, a merge commit, pinned to the reviewed head 0c1c6ee6 with --match-head-commit. At that head, gh pr checks exited 0, GitHub reported the PR mergeable and clean against main, and no review requested changes.

Independent cloud reviewers approved it twice: comment 6594859598 at 5926dfc1, and comment 6595176959 on the delta to 0c1c6ee6, which touches only tests/test_source_be_period_coverage.py and tests/test_source_fpb_row_source_concept.py.