[MaxGhenis]

Fixes #2759

What and why

The counted-once checker (policyengine_uk/utils/aggregate_impact/guard.py) raises each component by 1 and compares each head's response with a threshold. NaN compares false with every threshold, so on #2758 four checks passed a NaN response:

Check	#2758's rule	+inf	−inf	NaN
Own head	abs(change - expected) > slack	fails	fails	passes
Declared-reader containment	perturbed & (abs(slope - expected) <= allowance)	fails only where the check added something	same	passes
Spending rises	change > slack	fails	passes	passes
Model total	abs(model - heads) > slack	fails unless the heads are infinite too	same	passes

A declared double count whose container turned NaN also passed (present compared NaN with the slack). #2758 already fails a non-finite undeclared tax movement (_moves).

When two heads turned NaN, the check stopped without a report. It measures a failing head again while it holds the other heads at their values after the first point. When one of those values was NaN, policyengine-core 3.33.12 refused it, and the check stopped. Holder.set_input calls _raise_if_input_contains_nan (policyengine_core/holders/holder.py, lines 610 and 667), which raises on NaN and accepts infinity:

ValueError: Unable to set value for variable "tax_c", as the input contains NaN values.

This PR changes eight things:

• Every check fails a response that is not finite. Own head, declared-reader containment (at one point and at two), spending rises and the model total each fail a response that is +inf, −inf or NaN, wherever it occurs. So do both held measurements (still_breaks, breaks_on_slope). The new _contains and _rises helpers follow Detect undeclared tax movement at any baseline level and count non-finite responses as movement #2758's _moves, which this PR reuses unchanged. A non-finite response now appears in the component's responses.

• Every problem says whether it rests on a response that is not finite. Each problem carries not_finite, which the check sets when a measurement behind it (the first reading, or a held measurement that judged the household) saw such a response in its households. One helper, problem_text, writes the text of every problem and every mediated response. When a household with weight saw the response (not_finite_weighted), it adds the same clause for every check, and a containment no longer concludes that the head contains the component. Finite messages keep their wording:

  Check	Finite	A response behind it is not finite
  Own head	tax_b does not move by exactly what was added to tax_b	tax_b does not move by exactly what was added to tax_b, or is not finite,
  Undeclared tax movement	tax_a moves when tax_b does	tax_a moves, or is not finite, when tax_b does
  Declared reader	tax_a rises one for one with tax_b, so it contains it,	tax_a rises one for one with tax_b, or is not finite,
  Spending rises	ben_d rises when ben_c does	ben_d rises, or is not finite, when ben_c does
  Model total	the model's gov_tax moves by other than its heads when tax_b does	the model's gov_tax moves by other than its heads, or is not finite, when tax_b does

  measure and the check sum with numpy, and NaN times a weight of zero stays NaN, so a NaN in a household with no weight still sets not_finite. When only such households saw it, the message keeps its finite text for the households it counts and says where the NaN sits:

  gov_tax: tax_a moves when tax_b does in 11 households, and is not finite in households with no weight
  

• The second held measurement judges household by household. It holds every other head at its value after the first point. In each unit where one of those heads is not finite (in any month, for a monthly head), it holds that head at its baseline value instead, and it judges no household the unit belongs to. Benefit units and people map to households through map_result, the mapping that sums every head to households. In those households a problem keeps every household it had. A head that may respond, whose slope was neither 0 nor 1 and which the baseline hold did not show rising one for one, fails there as unconfirmed. Every other household gets both measurements.

  When tax_h contains tax_b behind tax_g, a passport on tax_b, only the second held measurement shows its slope of 1. With ben_c, a benefit no tax head reads, NaN in one of twelve households, tax_b fails in all twelve (test_a_gated_containment_fails_wherever_a_nan_elsewhere_leaves_it):

  gov_tax: tax_h rises one for one with tax_b, so it contains it, in 11 households
  gov_tax: tax_h responds to tax_b in fewer than 10 households where the check could not hold ben_c, which is not finite there once tax_b rises, so it could not confirm whether tax_h contains tax_b
  

  A head that responds only through another head fails only where the check could not hold ben_c, and stays mediated in the other eleven households (test_one_bad_record_fails_a_mediated_head_only_in_its_household):

  gov_tax: tax_a moves when tax_b does in fewer than 10 households (in fewer than 10 of them the check could not hold ben_c, which is not finite there once tax_b rises, so it could not show that tax_a responds only through other heads)
  

  For each head the check measured again, the component's result names the heads the check could not hold (heads_not_held, with heads_held_skipped) and counts the households where it could not hold them (households_not_held). Each problem names and counts them for the households that rest on the measurement the check could not make there. For the issue's case, counted_once reports tax_b as failing with gov_tax: tax_a moves, or is not finite, when tax_b does … and gov_tax: tax_c moves, or is not finite, when tax_b does …, then checks tax_c.

• Mediated responses and passing components say what the check could not see. A mediated response carries not_finite and not_finite_weighted for its first reading, and render writes it with problem_text:

  - `tax_b`: `tax_a` responds only through other heads, or is not finite, in fewer than 10 households (other heads that moved there: `tax_c`; gov_tax; not a failure)
  

  A component can pass while the check could not hold a head, when that head broke no rule and showed no slope other than 0 in the households the check left unjudged. Its detail then names the head in the same words as the comment (not_held_note):

  in fewer than 10 households the check could not hold ben_c, which is not finite there once tax_b rises, so it measured tax_a again there only with the other heads at their baseline values
  

  record_guard_checks, which measure now calls to record the counted-once result, puts that detail on the passing counted_once:<component> check through a new note argument to Recorder.check. Before, a passing check kept no detail.

• The comment shows each head the check could not hold. render says, in the guard section's summary, how the second measurement treats a head that is not finite, and adds the note above for each head the check measured again without holding every other head. publishable_guard keeps the head names and the flags, which hold no figures, and drops a count of households below 10 (households_not_held), as it already does for the counts and shares of problems.

• Double counts. A declared double count whose container is not finite now fails with not_finite: True, and every double count carries the flag. Its detail says the check cannot show that the container still contains the component; it does not tell the reader to remove the declaration. publishable_guard keeps that detail for a cell of fewer than 10 units, since it holds no figures. The comment renderer says the container is NOT FINITE when the component rises, rather than NO LONGER contained.

• The comparison says when the base could not confirm a failure. Repairing an unrelated NaN can move a reader from unconfirmed on the base to contains on the head, so a containment both sides share reads as new. compare still fails the job, because a new containment reads the same, and it now lists those scopes in the check's base_unconfirmed and ends the failure with a note that names each one: "On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it." render counts the check apart from those this change broke:

  **Checks:** 0 broken by this change, 1 failing that the base could not confirm, 0 failing on both sides, 0 fixed.
  

  The note applies only when every new scope confirms a containment the base failed as unconfirmed. A new failure beside one, a failure of another kind, and a head that stops confirming a containment the base found all count as broken. A comparison with no such failure prints the wording it printed before.

  I ran counted_once on the test file's fake simulation, with ben_c NaN in every household on the base and finite on the head, and passed its result through record_guard_checks, publishable_guard and compare, which lists this failure:

  Raising tax_b by £1 raises its own head by £1, moves no other tax head and raises no other benefit (counted_once:tax_b): gov_tax: tax_h rises one for one with tax_b, so it contains it, in 10 households. On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it.
  

• parse_partition rejects a colon in a name. record_guard_checks writes each counted-once failure under the scope total:head:kind, and compare splits that scope at its colons to name the containment in the note. A colon in a total or head name would let two different pairs write the same scope: total gov:tax with head tax_h, and total gov with head tax:tax_h, both write gov:tax:tax_h:contains, and the note would then read the base's pair off the head's scope. So structural_problems now reports, by name, a colon in every name the report writes into a check id or a scope: totals, head ids, head variables, declared double counts and overlap guard ids. parse_partition raises PartitionError on that report, and the checked-in fiscal_partition.yaml, the CLI's --partition and --fallback-partition, and compute_budget_metrics's default all load through it. The checked-in partition holds no colon and still loads.

Invariants

1. A non-finite response fails unless both held measurements clear it, and the result says so either way. For every check, and in the first reading and both held measurements, every household where the judged response is +inf, −inf or NaN falls in the failing households, unless both held measurements clear a first reading there. That reading then stays mediated with not_finite set. The check sets not_finite on the problem that covers a failing household, and adds the clause when a household with weight saw the response. Tests: test_a_non_finite_response_fails_every_check (7 checks × 3 values), test_a_non_finite_held_measurement_breaks_every_rule (3 kinds × 3 values), test_every_check_adds_one_clause_when_a_response_is_not_finite (7 checks × 4 values), test_a_response_not_finite_only_in_households_with_no_weight_says_so, with Detect undeclared tax movement at any baseline level and count non-finite responses as movement #2758's test_a_non_finite_response_is_movement_on_every_path.
2. The flags say what the measurements saw. For every problem, not_finite holds exactly when a measurement behind it saw a non-finite response. Its heads_not_held names the heads that stopped the second held measurement when the baseline hold alone would have cleared it or the check could not confirm a reader, and stays empty otherwise. The check fails a reader as unconfirmed wherever the oracle finds one, and nowhere else. For each head the check measured again, each component lists the heads whose value after the first point is not finite, and no others. A mediated response's not_finite matches its first reading, and the check finds no response mediated where it could not make the second measurement. Test: test_every_problem_says_whether_a_measurement_behind_it_was_not_finite, which recomputes every measurement on the fake simulation.
3. The check judges each household as it would alone. Households do not interact in the model. In each household of a model of two or three, the check finds the same problems, flags and mediated responses as in that household alone. The property covers components that are zero in every household or in none: where a component is zero in some households, the check perturbs only the others, though a zero household alone would get the £1. For each head it measured again, it leaves unjudged the households where the oracle, run on each household alone, finds a held head not finite, and no others; the component names those heads and counts those households. Test: test_the_check_judges_each_household_as_it_would_alone.
4. A NaN in one household changes no verdict in another. When a benefit no head reads turns NaN in one household, every other household keeps the problems, mediated responses and unjudged heads it had. Every head in the property belongs to a household. A benefit unit or a person maps to each household its members live in, so a unit that turns NaN leaves unjudged every household where one of its members lives, and the property's other households are those where none does. Tests: test_a_nan_in_one_household_changes_no_verdict_in_another, test_a_benefit_unit_not_finite_leaves_only_its_household_unjudged, test_a_person_not_finite_leaves_only_their_household_unjudged_on_core and test_a_benefit_unit_in_two_households_leaves_both_unjudged.
5. Finite results stay the same. On finite responses, every household mask, message, screen, response and coefficient equals what Detect undeclared tax movement at any baseline level and count non-finite responses as movement #2758's rules give, and the new fields keep their defaults: not_finite and not_finite_weighted false, heads_not_held and households_not_held empty, heads_held_skipped false and no unconfirmed problem. Tests: test_finite_responses_keep_every_verdict and the property in invariant 2, plus the scratch differential below.
6. Held checks only narrow, and the second measurement decides nothing where it could not judge. A held measurement never adds a household outside those it receives. In a household the second measurement could not judge, only the baseline hold decides, whatever slope the second measurement read there. A unit counts as not finite when any of its months is. Tests: test_the_held_checks_stay_within_the_households_they_are_given, test_the_second_held_measurement_judges_only_the_households_it_could_hold, test_a_unit_is_not_finite_when_any_month_is.
7. The check never stops on a response that turns not finite, and names every head that turns. Tax heads may turn not finite directly or through another head, declared or not. Either way counted_once returns a row for every component, names the heads that turned and no finite head, passes the other components, and recomputes the baseline to the same values. Tests: test_the_check_names_every_head_that_turns_not_finite_and_goes_on, and on policyengine-core test_two_heads_that_turn_nan_on_core_fail_by_name_and_the_check_goes_on (yearly and monthly) and test_a_head_not_finite_in_one_month_is_not_held.
8. The comparison excuses no failure. The job fails either way, and the report moves a failure out of "broken by this change" only when every new scope confirms a containment the base failed as unconfirmed. Tests: test_the_report_excuses_only_failures_the_base_could_not_confirm and test_repairing_an_unrelated_nan_reports_no_tax_broken_by_the_change.
9. No partition with a colon in a scoped name loads. parse_partition raises PartitionError for a colon in a total, head id, head variable, double count or overlap guard id, and for either side of the collision above. Tests: test_a_name_with_a_colon_is_rejected (5 kinds of name), test_neither_side_of_a_colon_collision_loads (base and head) and test_the_small_partition_loads.

The tests also pin four intended results:

• A non-finite response fails a reader even in a household where the check added nothing.
• A head that turns NaN only through another head stays a failure where a held measurement also fails to clear it, and its message says the check could not show that route. Where both held measurements clear the reading, it stays mediated with not_finite set.
• A reader fails as unconfirmed in a household the second measurement could not judge, though no measurement showed it rising one for one there.
• A problem whose non-finite households all have no weight keeps not_finite.

Limits

• The second measurement reads nothing where it could not hold a head. There the check reads the slope from no hold but the baseline one, so a containment that only the second measurement would show fails as unconfirmed, not as a containment. With one household, the gated tax_h above fails that way.
• compute_budget_metrics doesn't check a caller's own Partition. It takes a Partition object as given, so a caller who builds one with the dataclass, without parse_partition, gets no structural check. No code in the package builds one that way.
• Non-finite baselines stay out of scope. The counted-once check raises or misreports when a head's baseline value is NaN #2773 tracks them. As on main and Detect undeclared tax movement at any baseline level and count non-finite responses as movement #2758, a head that is NaN at baseline raises Core's ValueError when the check perturbs it or holds it at its baseline value. It also fails components it ignores (its change is NaN − NaN), and it makes the reset check report a changed baseline, because np.array_equal treats NaN as differing from NaN.

Test plan

• For policy changes: not applicable (CI tooling; no policy, parameter or variable changes).
• make format: I ran the equivalent with uvx ruff@0.17.2 format --check and ruff check on the repository. I also ran ruff check --select F811,F821 policyengine_uk and git diff --check; all pass.
• Changelog fragment changelog.d/2759.fixed.md

I ran each file in policyengine_uk/tests/aggregate_impact/ alone at cca0b1ddf, with -n 4 -o xfail_strict=true and without survey data. The childcare provenance file ran whole, in one run of about two minutes. No test failed.

File	Result
test_aggregate_impact_guard.py	184 passed
test_aggregate_impact_measure.py	46 passed
test_aggregate_impact_compare.py	142 passed
test_aggregate_impact_childcare_provenance.py	42 passed
test_aggregate_impact_data_mode.py	4 passed
test_aggregate_impact_dataset.py	13 passed
test_aggregate_impact_partition.py	19 passed
test_aggregate_impact_reforms.py	20 passed
test_aggregate_impact_withheld_reasons.py	25 passed
test_aggregate_impact_certified.py	not run: its 9 cases need the survey dataset
policyengine_uk/tests/code_health	3,091 passed

This PR adds 140 test cases: 121 in the guard file (184 against 63 on #2758), 2 in the measure file (46 against 44), 9 in the compare file (142 against 133) and 8 in the partition file (19 against 11). The compare file's renderer test checks the summary's new wording.

I didn't rerun the property draws, the 1,500-model differential or the earlier mutations at cca0b1ddf, so the numbers in those three paragraphs come from the commits they name.

The properties reach the cases they pin. One run of each at 8df6cc6 drew these models:

• The single-household property drew 121 models with 470 problems: 322 not finite, 20 resting on a household the second measurement could not judge, and 15 readers the check could not confirm. It also drew 4 mediated responses, all with a first reading that is not finite.
• The household-by-household property drew 62 models of two or three households. In 54 the check left some household unjudged for a head, and in 26 it left a head unjudged in some households and judged in others. For 12 problems, some but not all of their households went unjudged, and one component passed although the check could not hold a head.
• In the one-household-NaN property, the NaN changed what the check found in that household in 24 of 40 models, and in no other household in any of them.

Differential against #2758, over finite models. A scratch script, not committed, ran counted_once from this PR (8df6cc6) and from #2758 (a122f23) on 1,500 random finite models on the test file's fake simulation. The models mixed tax and benefit heads, readers declared or not, steps, floors, zero cells, model totals that count a head twice, and live or stale declared double counts. Every new field held its default, and every other part of every result matched, value for value: statuses, coefficients, model coefficients, responses, every problem with its message, household count and share, every mediated record, the double counts and the reset flag. The models produced 3,316 failing components and 4,192 problems (1,813 moves, 1,304 model totals, 602 rises, 473 containments), 229 mediated records and 942 double counts, 443 of them failing.

Mutations. I applied each of 47 single-line mutations to a scratch copy and ran the guard file with the five measure and compare test cases that read the guard (189 cases). I ran the first 24 at 982dcd2, whose code differs from 8df6cc6 only in docstrings, and the other 23 at 8df6cc6. I applied 8 more at df99770 and ran the seven new compare cases, and one at 89bd11f with the four compare cases on the note. At cca0b1d, I put back whether {total} contains {head} and ran the compare file, and dropped the colon check and ran the partition file. Every mutation failed at least one test case:

Mutation	Failing test cases
_rises drops its non-finite term	9
_contains drops its non-finite term	7
the second hold keeps a value that is not finite instead of the baseline value	25
the own-head check drops its non-finite term	2
the model-total check drops its non-finite term	10
a declared double count ignores a container that is not finite	1
a household the second measurement could not judge clears a problem	11
a reader fails a non-finite response only where the check added something	9
publishable_guard rewrites the not-finite double count's detail	1
the report drops non-finite responses	3
a confirmed reader ignores non-finite held measurements	1
a confirmed reader that joins a problem drops that problem's non-finite households	1
no unit of a monthly head counts as not finite	3
a unit of a monthly head counts as not finite only when every month is	2
the second measurement judges the households it could not hold a head in	13
problem_text never adds the clause	73
a kept problem ignores non-finite held measurements	5
no problem lists the heads the check could not hold	10
every problem of a measured head lists every head the check could not hold	8
the component's result omits the heads the check could not hold	25
undeclared movement never sets not_finite	10
the own head never sets not_finite	6
spending rises never set not_finite	6
a declared reader's containment never sets not_finite	22
the model total never sets not_finite	20
the comment leaves out the notes on heads the check could not hold	3
_held_nan ignores the baseline hold's second point	1
_held_nan ignores the second measurement	1
heads_held_skipped stays false	14
a double count's not_finite stays false	3
a confirmed reader that joins a problem drops that problem's unheld heads	1
the second measurement holds a head that is not finite at its baseline value in every unit	1
every household counts as judged	26
a unit's mask stands in for its households, with no mapping	3
a mediated response never sets not_finite	3
a passing component's detail leaves out the note	1
the passing check drops the note	1
a reader the check could not confirm passes	16
a reader the holds did not confirm fails as unconfirmed in every household, judged or not	38
the clause appears for a NaN only in households with no weight	2
the message leaves out the households with no weight	2
households_not_held counts households with no weight	2
publishable_guard keeps a problem's small households_not_held	1
publishable_guard keeps a component's small households_not_held	1
a problem lists every head the check could not hold, wherever it could not hold it	8
a mediated response's not_finite_weighted ignores the weights	1
the comment drops a mediated response's not_finite	2
any new failure counts as one the base could not confirm	5
one such failure excuses the new failures beside it	2
a new failure of another kind counts as one the base could not confirm	2
no failure counts as one the base could not confirm	3
the failure text leaves out the note	1
the note drops the containments' names	3
the note names the total and head, not the head and component	3
parse_partition accepts a colon in a name	7
such a failure stops failing the job	3
the header counts such a failure as broken	3
the header always prints the new count	2

#1963 (marriage-allowance-election) adds lines at the same spots in measure.py and its test file, so whichever merges second keeps both sides. #1963's component_guard_diagnostic lists the problem kinds it prints, so whichever merges second also adds unconfirmed to that list. The branch includes main at 52b30809f. In render.py, where #2772 and this branch each added an import on the same line, the merge keeps both. git merge-tree merges the head cleanly with main at 1d4d020aa and with #2854 at 9ee51800e; on the merge with #2854, the compare (142), guard (184) and guard baseline (48) files pass. With #1963 at 69c919f6, it conflicts only in measure.py and test_aggregate_impact_measure.py, as above, and in three files that conflict between #1963 and main alone.

Reproducer: policyengine_uk/tests/aggregate_impact/test_aggregate_impact_guard.py::test_two_heads_that_turn_nan_on_core_fail_by_name_and_the_check_goes_on

axiom: n/a: CI tooling.

🤖 Generated with Claude Code

[github-actions]

Aggregate impact on certified data: pass

Base 1d4d020aa1 against head 9d8b637547, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

• In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed

Caseloads

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed

Standard reform costings (change in gov_balance)

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise Class 1 main employee NICs rate to 10%	0	0	0	0	0	suppressed	suppressed
Raise additional rate by 3pp	0	0	0	0	0	suppressed	suppressed

The report suppresses a reform's costing and target change in these years, for the reasons below (privacy.py):

• Raise Class 1 main employee NICs rate to 10%, 2030-31 to 2031-32 (base and head): the reform changes income_tax in fewer than 10 records.
• Raise additional rate by 3pp, 2030-31 to 2031-32 (base and head): the reform changes universal_credit in fewer than 10 records.

Standard reforms: change in each reform's target

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise Class 1 main employee NICs rate to 10%: change in national_insurance	0	0	0	0	0	suppressed	suppressed
Raise additional rate by 3pp: change in income_tax	0	0	0	0	0	suppressed	suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles.

Checks failing on both base and head (1)

• VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-53,888,684 in total; 2026-27: 76 units below zero, a weighted £-55,777,260 in total; 2027-28: 76 units below zero, a weighted £-57,727,917 in total

Counted-once check

The check raised each of 92 components by £1 per unit in 2026-27 and read every head of its totals household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A response that is not finite breaks every rule. The check measures a head that breaks a rule twice more: once holding every other head at its baseline value, and once holding every other head at its value after the first £1 while the component rises again. The head fails where either measurement still shows the response. The check cannot hold a head at a value that is not finite. In each unit where another head is not finite after the first £1, it holds that head at its baseline value, and the second measurement judges no household the unit belongs to. There a head fails wherever it first broke a rule, or wherever the check could not confirm whether its response contains the component. 0 failed and the check skipped 0.

• widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
• widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)

Provenance

Side	Commit	Runtime	Environment hash	Partition hash	Time
base	main 1d4d020aa1	uk 2.172.1 (files 27900fb6fb5f), core 3.34.1, py 3.13.18 aarch64	f421d4d66ebc	7cb61324a46b	2228.46s
head	#2850 merged into main 9d8b637547	uk 2.172.1 (files 4552b3831750), core 3.34.1, py 3.13.18 aarch64	f421d4d66ebc	7cb61324a46b	2207.1s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Final charge on cash interest in non-cash ISAs	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Lifetime ISA withdrawal charge	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Income tax	£294.46bn	£315.23bn	£335.82bn	£351.28bn	£366.96bn	£385.36bn	£400.00bn
National Insurance (employee and self-employed)	£51.70bn	£54.14bn	£56.13bn	£57.73bn	£59.76bn	£61.64bn	£63.51bn
National Insurance (employer)	£148.71bn	£155.37bn	£160.88bn	£165.09bn	£172.06bn	£177.36bn	£182.57bn
Employer NI change borne by capital	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Employer NI change borne by consumers	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Capital gains tax	£12.60bn	£13.15bn	£13.60bn	£14.11bn	£14.67bn	£15.24bn	£15.80bn
Student loan repayments	£7.82bn	£8.13bn	£8.39bn	£8.58bn	£8.79bn	£9.04bn	£9.29bn
VAT excluding private school fees	£249.37bn	£256.87bn	£266.33bn	£272.70bn	£278.97bn	£285.24bn	£291.59bn
Change in VAT from its baseline	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
VAT on private school fees	£1.95bn	£2.01bn	£2.09bn	£2.14bn	£2.20bn	£2.26bn	£2.31bn
Change in private school VAT from its baseline	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Fuel duties	£24.71bn	£24.13bn	£26.00bn	£26.83bn	£26.51bn	£25.78bn	£24.63bn
Alcohol duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Tobacco duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Vehicle excise duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Betting and gaming duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
TV licence fee	£4.26bn	£4.40bn	£4.41bn	£4.42bn	£4.43bn	£4.44bn	£4.45bn
Council tax	£49.21bn	£52.16bn	£54.89bn	£57.95bn	£61.16bn	£64.47bn	£67.91bn
High value council tax surcharge	£0.00bn	£0.00bn	£0.00bn	£0.48bn	£0.49bn	£0.50bn	£0.51bn
Northern Ireland domestic rates	£0.55bn	£0.58bn	£0.60bn	£0.63bn	£0.66bn	£0.69bn	£0.72bn
Business rates	£34.68bn	£36.20bn	£38.00bn	£39.46bn	£42.27bn	£43.25bn	£44.29bn
Stamp duty land tax on the household's own purchases	£6.88bn	£7.32bn	£7.66bn	£8.06bn	£8.47bn	£8.91bn	£9.35bn
Stamp duty land tax paid by companies	£4.72bn	£4.57bn	£5.29bn	£6.03bn	£6.37bn	£6.92bn	£7.45bn
Land transaction tax (Wales)	£0.07bn	£0.08bn	£0.08bn	£0.09bn	£0.10bn	£0.10bn	£0.11bn
Land and buildings transaction tax (Scotland)	£0.31bn	£0.32bn	£0.33bn	£0.35bn	£0.36bn	£0.38bn	£0.40bn
Unspecified tax change borne by capital	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Unspecified tax change borne by consumers	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Wealth tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Wealth tax excluding the main residence	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Land value tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Carbon tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Spending by benefit and service: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Lifetime ISA bonus	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
State Pension	£121.68bn	£125.92bn	£126.72bn	£129.49bn	£132.99bn	£136.66bn	£140.42bn
Pension Credit	£6.66bn	£7.03bn	£7.29bn	£7.48bn	£7.60bn	£7.84bn	£8.03bn
Winter Fuel Payment	£1.86bn	£1.85bn	£1.79bn	£1.77bn	£1.78bn	£1.78bn	£1.78bn
Pension Age Winter Heating Payment (Scotland)	£0.19bn	£0.20bn	£0.20bn	£0.21bn	£0.21bn	£0.22bn	£0.22bn
Universal Credit	£75.48bn	£80.52bn	£82.95bn	£84.96bn	£86.11bn	£87.61bn	£88.21bn
Housing Benefit	£8.56bn	£8.89bn	£9.23bn	£9.46bn	£9.69bn	£9.99bn	£10.27bn
Council Tax Reduction	£2.07bn	£2.20bn	£2.31bn	£2.45bn	£2.58bn	£2.72bn	£2.86bn
Employment and Support Allowance (income-related)	£0.79bn	£0.82bn	£0.84bn	£0.86bn	£0.88bn	£0.90bn	£0.92bn
Employment and Support Allowance (contribution-based)	£5.20bn	£5.37bn	£5.56bn	£5.69bn	£5.82bn	£5.95bn	£6.09bn
Jobseeker's Allowance (income-based)	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Jobseeker's Allowance (contribution-based)	£0.29bn	£0.30bn	£0.31bn	£0.31bn	£0.32bn	£0.33bn	£0.34bn
Income Support	£0.11bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Working Tax Credit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Child Tax Credit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Incapacity Benefit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Personal Independence Payment	£26.95bn	£28.03bn	£29.05bn	£30.09bn	£30.75bn	£31.51bn	£32.21bn
Disability Living Allowance	£7.49bn	£7.79bn	£8.07bn	£8.36bn	£8.55bn	£8.76bn	£8.95bn
Attendance Allowance	£8.68bn	£9.03bn	£9.36bn	£9.69bn	£9.90bn	£10.15bn	£10.38bn
Carer's Allowance	£4.48bn	£4.66bn	£4.83bn	£5.00bn	£5.11bn	£5.27bn	£5.42bn
Carer Support Payment (Scotland)	£0.31bn	£0.32bn	£0.33bn	£0.34bn	£0.35bn	£0.36bn	£0.37bn
Carer's Allowance Supplement (Scotland)	£0.00bn	£0.04bn	£0.04bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn
Severe Disablement Allowance	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn
Industrial Injuries Disablement Benefit	£0.50bn	£0.52bn	£0.54bn	£0.55bn	£0.56bn	£0.58bn	£0.59bn
Armed Forces Compensation Scheme	£0.61bn	£0.63bn	£0.65bn	£0.66bn	£0.68bn	£0.69bn	£0.71bn
War Widow's or Widower's Pension	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Child Benefit	£16.19bn	£16.82bn	£17.39bn	£18.00bn	£18.39bn	£18.84bn	£19.25bn
Scottish Child Payment	£0.49bn	£0.52bn	£0.53bn	£0.55bn	£0.56bn	£0.57bn	£0.56bn
Two Child Limit Payment (Scotland, counterfactual)	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Healthy Start vouchers	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn
Maternity Allowance	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Statutory Maternity Pay	£3.30bn	£3.41bn	£3.54bn	£3.62bn	£3.70bn	£3.78bn	£3.87bn
Statutory Paternity Pay	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Statutory Sick Pay	£1.12bn	£1.16bn	£1.20bn	£1.23bn	£1.26bn	£1.29bn	£1.32bn
Sure Start Maternity Grant	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn
Bereavement Support Payment	£0.67bn	£0.69bn	£0.71bn	£0.73bn	£0.75bn	£0.76bn	£0.78bn
Widowed Parent's Allowance	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Tax-Free Childcare	£0.56bn	£0.57bn	£0.57bn	£0.58bn	£0.60bn	£0.61bn	£0.59bn
Extended childcare entitlement	£2.99bn	£3.50bn	£3.59bn	£3.71bn	£3.78bn	£3.86bn	£3.93bn
Universal childcare entitlement	£2.06bn	£2.22bn	£2.29bn	£2.37bn	£2.43bn	£2.48bn	£2.53bn
Targeted childcare entitlement	£0.50bn	£0.51bn	£0.52bn	£0.54bn	£0.55bn	£0.57bn	£0.56bn
Childcare Offer for Wales	£0.26bn	£0.28bn	£0.28bn	£0.29bn	£0.30bn	£0.31bn	£0.32bn
Care to Learn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Childcare Grant	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Parents' Learning Allowance	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn
Adult Dependants' Grant	£0.06bn	£0.06bn	£0.04bn	£0.04bn	£0.04bn	£0.04bn	£0.04bn
Travel Grant	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed
16 to 19 Bursary Fund	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
NHS spending	£203.89bn	£204.30bn	£204.56bn	£204.91bn	£205.34bn	£205.85bn	£206.35bn
State education spending	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn
Transport subsidy spending	£25.05bn	£25.10bn	£25.14bn	£25.18bn	£25.23bn	£25.30bn	£25.36bn
Cost-of-living support payment	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Energy Bills Rebate	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Household Energy Payment	£0.00bn	£2.11bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Energy Price Guarantee subsidy	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Basic income	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Disability basic income	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Other public spending budget change	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Caseloads: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Lifetime ISA bonus	0k	0k	0k	0k	0k	0k	0k
State Pension	11,264k	11,145k	10,819k	10,700k	10,723k	10,750k	10,776k
Pension Credit	1,486k	1,480k	1,472k	1,452k	1,450k	1,455k	1,464k
Winter Fuel Payment	7,844k	7,783k	7,473k	7,375k	7,391k	7,409k	7,427k
Pension Age Winter Heating Payment (Scotland)	803k	802k	800k	797k	799k	801k	802k
Universal Credit	6,342k	6,347k	6,336k	6,297k	6,292k	6,271k	6,223k
Housing Benefit	1,249k	1,245k	1,244k	1,227k	1,227k	1,229k	1,232k
Council Tax Reduction	3,176k	3,195k	3,200k	3,214k	3,224k	3,235k	3,242k
Employment and Support Allowance (income-related)	71k	71k	71k	71k	71k	71k	71k
Employment and Support Allowance (contribution-based)	719k	720k	721k	722k	724k	726k	728k
Jobseeker's Allowance (income-based)	0k	0k	0k	0k	0k	0k	0k
Jobseeker's Allowance (contribution-based)	67k	67k	67k	67k	67k	68k	68k
Income Support	25k	0k	0k	0k	0k	0k	0k
Working Tax Credit	0k	0k	0k	0k	0k	0k	0k
Child Tax Credit	0k	0k	0k	0k	0k	0k	0k
Incapacity Benefit	0k	0k	0k	0k	0k	0k	0k
Personal Independence Payment	3,567k	3,575k	3,579k	3,585k	3,593k	3,602k	3,611k
Disability Living Allowance	1,291k	1,294k	1,296k	1,298k	1,300k	1,304k	1,307k
Attendance Allowance	1,695k	1,698k	1,700k	1,703k	1,707k	1,711k	1,715k
Carer's Allowance	1,034k	1,037k	1,038k	1,040k	1,042k	1,060k	1,062k
Carer Support Payment (Scotland)	71k	71k	71k	72k	72k	72k	72k
Carer's Allowance Supplement (Scotland)	0k	71k	71k	72k	72k	72k	72k
Severe Disablement Allowance	4k	4k	4k	4k	4k	4k	4k
Industrial Injuries Disablement Benefit	137k	137k	137k	138k	138k	138k	139k
Armed Forces Compensation Scheme	93k	93k	94k	94k	94k	94k	94k
War Widow's or Widower's Pension	0k	0k	0k	0k	0k	0k	0k
Child Benefit	8,123k	8,120k	8,120k	8,130k	8,145k	8,161k	8,178k
Scottish Child Payment	351k	352k	352k	353k	350k	348k	337k
Two Child Limit Payment (Scotland, counterfactual)	0k	0k	0k	0k	0k	0k	0k
Healthy Start vouchers	200k	200k	200k	201k	201k	201k	202k
Maternity Allowance	0k	0k	0k	0k	0k	0k	0k
Statutory Maternity Pay	333k	333k	334k	334k	335k	336k	337k
Statutory Paternity Pay	0k	0k	0k	0k	0k	0k	0k
Statutory Sick Pay	181k	182k	182k	182k	182k	183k	183k
Sure Start Maternity Grant	26k	26k	26k	26k	26k	26k	26k
Bereavement Support Payment	98k	98k	98k	98k	98k	99k	99k
Widowed Parent's Allowance	0k	0k	0k	0k	0k	0k	0k
Tax-Free Childcare	1,043k	1,027k	1,014k	1,009k	1,011k	1,009k	993k
Extended childcare entitlement	795k	800k	796k	797k	798k	798k	792k
Universal childcare entitlement	1,071k	1,072k	1,072k	1,073k	1,076k	1,079k	1,074k
Targeted childcare entitlement	165k	156k	154k	154k	155k	155k	150k
Childcare Offer for Wales	39k	39k	39k	39k	39k	39k	39k
Care to Learn	0k	0k	0k	0k	0k	0k	0k
Childcare Grant	0k	0k	0k	0k	0k	0k	0k
Parents' Learning Allowance	98k	98k	98k	99k	99k	99k	99k
Adult Dependants' Grant	18k	18k	12k	12k	12k	12k	12k
Travel Grant	0k	0k	0k	0k	0k	0k	0k
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed
16 to 19 Bursary Fund	0k	0k	0k	0k	0k	0k	0k
Cost-of-living support payment	0k	0k	0k	0k	0k	0k	0k
Energy Bills Rebate	0k	0k	0k	0k	0k	0k	0k
Household Energy Payment	0k	21,065k	0k	0k	0k	0k	0k
Energy Price Guarantee subsidy	0k	0k	0k	0k	0k	0k	0k
Basic income	0k	0k	0k	0k	0k	0k	0k
Disability basic income	0k	0k	0k	0k	0k	0k	0k

Government balance, each component counted once: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Government receipts, each component once	£892.00bn	£934.66bn	£980.50bn	£1,015.93bn	£1,054.24bn	£1,091.56bn	£1,124.89bn
Government spending	£529.38bn †	£545.66bn †	£550.77bn †	£559.51bn †	£567.20bn †	£575.95bn †	£583.60bn †
Government balance, each receipt once	£362.62bn †	£389.00bn †	£429.74bn †	£456.42bn †	£487.04bn †	£515.61bn †	£541.29bn †
Benefits in household_benefits but not in gov_spending	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Model totals: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
gov_tax	£892.00bn	£934.66bn	£980.50bn	£1,015.93bn	£1,054.24bn	£1,091.56bn	£1,124.89bn
household_tax	£491.97bn	£520.41bn	£551.21bn	£576.01bn	£601.00bn	£626.70bn	£648.42bn
pre_budget_change_household_tax	£484.15bn	£512.28bn	£542.82bn	£566.95bn	£591.72bn	£617.16bn	£638.61bn
gov_spending	£529.38bn †	£545.66bn †	£550.77bn †	£559.51bn †	£567.20bn †	£575.95bn †	£583.60bn †
household_benefits	£529.38bn †	£545.66bn †	£550.77bn †	£559.51bn †	£567.20bn †	£575.95bn †	£583.60bn †
gov_balance	£362.62bn †	£389.00bn †	£429.74bn †	£456.42bn †	£487.04bn †	£515.61bn †	£541.29bn †

Household income totals: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Household market income	£1,637.96bn	£1,702.77bn	£1,757.35bn	£1,806.41bn	£1,855.97bn	£1,912.43bn	£1,970.44bn
Household net income	£1,636.69bn †	£1,687.82bn †	£1,715.45bn †	£1,747.49bn †	£1,778.69bn †	£1,817.02bn †	£1,859.69bn †
Household net income (HBAI definition)	£1,449.57bn	£1,499.83bn	£1,530.86bn	£1,565.17bn	£1,597.89bn	£1,635.60bn	£1,676.61bn

Population: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
People	69,245k	69,383k	69,473k	69,592k	69,738k	69,912k	70,080k
Benefit units	35,110k	35,181k	35,226k	35,286k	35,360k	35,449k	35,534k
Households	31,046k	31,108k	31,149k	31,201k	31,267k	31,345k	31,420k

Poverty rates: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Absolute poverty, before housing costs: everyone	16.3%	15.5%	16.4%	16.3%	16.5%	16.5%	16.6%
Absolute poverty, before housing costs: children	24.7%	22.1%	23.3%	22.9%	23.1%	23.2%	23.5%
Absolute poverty, before housing costs: working-age adults	14.1%	13.8%	14.7%	14.7%	14.8%	14.8%	15.0%
Absolute poverty, before housing costs: pensioners	13.7%	13.4%	14.0%	13.8%	13.9%	13.9%	13.9%
Absolute poverty, after housing costs: everyone	21.6%	21.0%	22.0%	22.6%	22.9%	23.1%	23.4%
Absolute poverty, after housing costs: children	33.3%	31.3%	32.3%	32.5%	32.9%	33.4%	34.2%
Absolute poverty, after housing costs: working-age adults	19.9%	19.8%	20.8%	21.6%	21.9%	22.0%	22.3%
Absolute poverty, after housing costs: pensioners	12.7%	12.4%	13.2%	13.2%	13.3%	13.4%	13.6%
Relative poverty, before housing costs: everyone	17.0%	16.0%	16.4%	16.3%	16.4%	16.3%	16.6%
Relative poverty, before housing costs: children	25.9%	22.9%	23.3%	22.9%	23.0%	22.9%	23.5%
Relative poverty, before housing costs: working-age adults	14.5%	14.1%	14.6%	14.7%	14.7%	14.7%	15.0%
Relative poverty, before housing costs: pensioners	14.5%	14.0%	13.9%	13.8%	13.8%	13.9%	13.9%
Relative poverty, after housing costs: everyone	21.4%	21.0%	21.3%	21.3%	21.3%	21.5%	21.7%
Relative poverty, after housing costs: children	33.1%	31.1%	31.2%	31.0%	31.1%	31.2%	31.8%
Relative poverty, after housing costs: working-age adults	19.9%	19.8%	20.2%	20.4%	20.5%	20.6%	20.8%
Relative poverty, after housing costs: pensioners	12.5%	12.4%	12.2%	11.7%	11.7%	11.8%	12.0%

People in poverty: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Absolute poverty, before housing costs: everyone	11,278k	10,779k	11,402k	11,365k	11,476k	11,513k	11,651k
Absolute poverty, before housing costs: children	3,688k	3,314k	3,496k	3,445k	3,480k	3,502k	3,549k
Absolute poverty, before housing costs: working-age adults	5,965k	5,893k	6,308k	6,362k	6,420k	6,430k	6,521k
Absolute poverty, before housing costs: pensioners	1,624k	1,572k	1,598k	1,558k	1,576k	1,580k	1,581k
Absolute poverty, after housing costs: everyone	14,944k	14,600k	15,310k	15,721k	15,937k	16,141k	16,426k
Absolute poverty, after housing costs: children	4,975k	4,682k	4,837k	4,888k	4,945k	5,040k	5,179k
Absolute poverty, after housing costs: working-age adults	8,463k	8,461k	8,971k	9,344k	9,491k	9,584k	9,706k
Absolute poverty, after housing costs: pensioners	1,506k	1,457k	1,502k	1,490k	1,501k	1,517k	1,542k
Relative poverty, before housing costs: everyone	11,750k	11,084k	11,388k	11,364k	11,416k	11,415k	11,651k
Relative poverty, before housing costs: children	3,872k	3,423k	3,496k	3,445k	3,466k	3,454k	3,549k
Relative poverty, before housing costs: working-age adults	6,159k	6,013k	6,301k	6,362k	6,389k	6,389k	6,521k
Relative poverty, before housing costs: pensioners	1,720k	1,648k	1,590k	1,557k	1,562k	1,572k	1,581k
Relative poverty, after housing costs: everyone	14,849k	14,552k	14,776k	14,809k	14,874k	15,014k	15,234k
Relative poverty, after housing costs: children	4,942k	4,660k	4,673k	4,663k	4,679k	4,707k	4,806k
Relative poverty, after housing costs: working-age adults	8,426k	8,440k	8,711k	8,823k	8,873k	8,967k	9,062k
Relative poverty, after housing costs: pensioners	1,482k	1,452k	1,392k	1,323k	1,322k	1,339k	1,365k

Inequality: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Gini coefficient, equivalised HBAI income before housing costs	0.3080	0.3052	0.3058	0.3058	0.3063	0.3069	0.3085

Mean household net income in each side's own deciles: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Decile 1: mean household_net_income per household	£17,681 †	£17,997 †	£17,915 †	£18,152 †	£18,307 †	£18,531 †	£18,717 †
Decile 2: mean household_net_income per household	£27,939 †	£29,023 †	£28,711 †	£29,013 †	£29,276 †	£29,691 †	£30,150 †
Decile 3: mean household_net_income per household	£34,320 †	£35,485 †	£36,073 †	£36,611 †	£37,197 †	£37,737 †	£38,187 †
Decile 4: mean household_net_income per household	£38,933 †	£40,687 †	£41,723 †	£42,727 †	£43,334 †	£44,132 †	£44,432 †
Decile 5: mean household_net_income per household	£43,239 †	£45,567 †	£46,014 †	£46,959 †	£47,712 †	£48,584 †	£50,282 †
Decile 6: mean household_net_income per household	£47,927 †	£49,210 †	£50,892 †	£51,939 †	£52,945 †	£53,942 †	£54,680 †
Decile 7: mean household_net_income per household	£55,662 †	£56,571 †	£57,442 †	£57,670 †	£58,729 †	£59,657 †	£61,082 †
Decile 8: mean household_net_income per household	£68,789 †	£70,803 †	£71,858 †	£73,466 †	£73,880 †	£75,320 †	£77,178 †
Decile 9: mean household_net_income per household	£78,890 †	£81,013 †	£80,861 †	£82,646 †	£85,464 †	£86,946 †	£89,509 †
Decile 10: mean household_net_income per household	£117,005 †	£120,339 †	£124,742 †	£126,632 †	£125,971 †	£128,600 †	£130,938 †

Standard reform costings (change in gov_balance): head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise basic rate by 1pp	£6.41bn	£6.66bn	£6.63bn	£6.85bn	£7.03bn	£7.25bn	£7.49bn
Raise higher rate by 1pp	£3.61bn	£3.87bn	£4.00bn	£4.22bn	£4.44bn	£4.71bn	£4.83bn
Raise personal allowance by ~800GBP/year	£-3.80bn	£-3.92bn	£-3.98bn	£-4.04bn	£-4.10bn	£-4.23bn	£-1.95bn
Raise child benefit by 25GBP/week per additional child	£-1.99bn	£-1.75bn	£-1.55bn	£-1.38bn	£-1.30bn	£-1.18bn	£-1.07bn
Reduce Universal Credit taper rate to 20%	£-20.77bn	£-21.32bn	£-21.75bn	£-21.99bn	£-22.34bn	£-22.75bn	£-22.94bn
Raise Class 1 main employee NICs rate to 10%	£10.89bn	£11.30bn	£11.71bn	£12.01bn	£12.31bn	suppressed	suppressed
Raise VAT standard rate by 2pp	£24.89bn	£25.71bn	£26.63bn	£27.24bn	£27.87bn	£28.50bn	£29.14bn
Raise additional rate by 3pp	£2.90bn	£3.15bn	£3.22bn	£3.41bn	£3.61bn	suppressed	suppressed
Raise the Universal Credit standard allowance by 10%	£-3.06bn	£-3.18bn	£-3.26bn	£-3.35bn	£-3.44bn	£-3.49bn	£-3.50bn
Raise the State Pension by 10%	£-7.78bn	£-8.01bn	£-8.24bn	£-8.47bn	£-8.82bn	£-8.55bn	£-8.89bn

Standard reforms: change in each reform's target: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise basic rate by 1pp: change in income_tax	£6.55bn	£6.86bn	£6.79bn	£7.00bn	£7.20bn	£7.44bn	£7.66bn
Raise higher rate by 1pp: change in income_tax	£3.61bn	£3.92bn	£4.01bn	£4.23bn	£4.45bn	£4.72bn	£4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax	£-3.93bn	£-4.07bn	£-4.13bn	£-4.18bn	£-4.22bn	£-4.40bn	£-1.75bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit	£2.33bn	£2.14bn	£1.94bn	£1.76bn	£1.65bn	£1.52bn	£1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit	£20.74bn	£21.31bn	£21.78bn	£22.04bn	£22.37bn	£22.78bn	£22.99bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance	£11.15bn	£11.63bn	£12.00bn	£12.29bn	£12.61bn	suppressed	suppressed
Raise VAT standard rate by 2pp: change in vat	£24.69bn	£25.51bn	£26.42bn	£27.03bn	£27.65bn	£28.28bn	£28.91bn
Raise additional rate by 3pp: change in income_tax	£2.90bn	£3.15bn	£3.22bn	£3.42bn	£3.62bn	suppressed	suppressed
Raise the Universal Credit standard allowance by 10%: change in universal_credit	£3.06bn	£3.19bn	£3.28bn	£3.37bn	£3.43bn	£3.49bn	£3.51bn
Raise the State Pension by 10%: change in state_pension	£10.68bn	£11.17bn	£11.28bn	£11.60bn	£12.03bn	£12.47bn	£12.97bn

[MaxGhenis]

Ready for hub review

Verdict: REQUEST_CHANGES at ea04c31983fb0763045c97c743d32dc1ad1dd88f

Delta reviewed: df997704d..ea04c3198 (89bd11fec, the merge ea8d87b8a, ea04c3198).

Findings

P2-1. The new note names the wrong containment

policyengine_uk/utils/aggregate_impact/compare.py:117-124 (_unconfirmed_note) writes whether {total} contains {head}. The check is about whether the head contains the component, not whether the total contains the head. Every total contains its heads by construction (the partition puts them there). The guard's own unconfirmed message says could not confirm whether {head} contains {component} (guard.py:465-467). The compare test's docstring says the same thing (test_aggregate_impact_compare.py:1600: "The base could not confirm whether tax_h contained tax_b"). So the note contradicts the detail printed just before it.

Real-engine output at the head, from the review's measurement helper (counted_once on FakeSim, then record_guard_checks, publishable_guard and compare), with a NaN in ben_c on the base:

Raising tax_b by £1 raises its own head by £1, moves no other tax head and raises no other benefit (counted_once:tax_b): gov_tax: tax_h rises one for one with tax_b, so it contains it, in 10 households. The check could not confirm whether gov_tax contains tax_h on the base, so this change may not have introduced it.

When the heads sit in the spending total, it reads could not confirm whether gov_spending contains tax_h. A reader who looks for what the base left open is pointed at a relation that cannot fail. The job still fails in every case, so this is P2, not P1. But the note is the whole point of 89bd11fec, and the PR body quotes it.

The tests pin the wrong wording, so a correct note would fail them (test_aggregate_impact_compare.py:1610, :1639, :1649).

Suggested replacement (compare.py:111-124, plus the call at :640):

def _unconfirmed_note(check_id: str, scopes: list) -> str:
    """The failure text's note on the scopes of ``_unconfirmed_on_base``. It
    names each containment, as the text shows only the check's first three
    details."""
    if not scopes:
        return ""
    component = check_id.split(":", 1)[-1]
    containments = " or ".join(
        f"whether {head} contains {component} ({total})"
        for total, head, _ in (scope.rsplit(":", 2) for scope in scopes)
    )
    return (
        f". On the base, the check could not confirm {containments}, so this "
        f"change may not have introduced {'it' if len(scopes) == 1 else 'them'}."
    )

            + _unconfirmed_note(check["id"], check["base_unconfirmed"])

That gives On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it. Update the three assertions to match. In the PR body's section "The comparison says when the base could not confirm a failure", replace

"The check could not confirm whether gov_tax contains tax_h on the base, so this change may not have introduced it."

with

"On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it."

P3-1. Test table leaves out a file the merge brought in

The body says "I ran each file in policyengine_uk/tests/aggregate_impact/ alone", but its table has no row for test_aggregate_impact_withheld_reasons.py, which arrived from main (#2772) in ea8d87b8a. At the head it collects 25 cases, and all 25 pass in my run (below). Add the row:

| `test_aggregate_impact_withheld_reasons.py` | 25 passed |

Also add a sentence saying that you reran the files at the head after the merge. As written, the last rerun the body names is at 89bd11fec, which predates the merge and the test rewrite.

P3-2. The colon collision (round 3's P3-1) is still open, and the new note now prints the base's identity

The author left this open on three grounds. I checked each one:

1. "Model variables are Python class names." Partly true. Ordinary model variables are class statements, but policyengine-core's TaxBenefitSystem.add_variable accepts a class that type() builds with a colon in its name. The review's test_real_colon_collision.py still passes at the head, with heads gov:tax/tax_h (base) and gov/tax:tax_h (head). The scope also uses partition names (total, and the guard's head), and partition.structural_problems does not reject a colon.
2. "The partition holds no colons." True. policyengine_uk/data/fiscal_partition.yaml names 111 distinct totals, head ids, head variables and head totals. None holds a colon, and all are Python identifiers. report_spec.yaml names no totals or heads (its top-level keys are schema_version, horizon, guard_year, poverty, inequality, deciles, flags, identity_relative_tolerance, identity_absolute_tolerance_gbp, counted_once_tolerance, comment), and none of its keys or values holds a colon.
3. "The job still fails." True. The witness has status == "fail".

Paths into a scope: a reform can't put a colon there. counted_once runs on the baseline simulation with the partition (measure.py:1729), and reforms.py never touches the partition. A dataset can't either, because H5 columns do not name heads. The partition comes from each side's checked-in fiscal_partition.yaml, or from the CLI's --partition / --fallback-partition (__main__.py:82-95; the workflow passes the head's own file). So only a commit that edits the partition (and adds a matching variable) or an operator's CLI argument can do it.

New since round 3: the note reads the base's identity off the head's scope. The witness's failure now ends The check could not confirm whether gov:tax contains tax_h on the base, while the detail just before it names gov: tax:tax_h. This is still P3, and the job fails either way. A one-line guard in structural_problems that rejects a total or head name containing : would close it.

Checked and sound

• Round 3's P3-2 is closed, apart from P2-1's wording. I wrote compare_r2.py and render_r2.py from git show 8df6cc61c:…. I ran the witness with PYTHONPATH set to the head, -n 0 -o xfail_strict=true and both env vars unset. test_truncated_failure_note_refers_only_to_confirmed_base_failures now fails at assert "tax_new" not in failure, because the note names tax_new even though the three displayed details are old. Eight test_new_scopes_always_block… soft cases fail only on the old literal "could not confirm this on the base". The other 47 cases pass, including the 48-case byte-for-byte check against 8df6cc61c, so finite comparisons print what they printed before. (I copied the witness file's comparison tests. I did not copy its measurement-based tests; I ran those inputs separately, below.)

• Adversarial inputs (probe.py, probe2.py). The job fails (status == "fail") in every case:

  • Five new containments: the note names all five (… or whether gov_tax contains tax_h …), uses "them", and the header counts 1 "failing that the base could not confirm".
  • A containment that a displayed detail names: the note names it again. This is harmless.
  • Total or head renamed between base and head, a new moves beside it, and strict mode with a shared contains: in each case there is no note, and the header counts 1 broken.
  • A shared moves failure plus unconfirmed→contains: the note names only the containment, as intended.
  • Strict mode on a new containment: the note appears and the job fails.
  • Small cells: sizes 1 and 10, tax and spending totals, strict on and off. The note holds names only, the details say "fewer than 10 households" at size 1, and every case fails.

• The test rewrite. I loaded the test function from ea8d87b8a and from ea04c3198 and wrapped compare to capture its arguments. The two versions pass byte-identical (base, head, options) (JSON, 3,186 chars), and their assert lines are identical.

• Reverting 89bd11fec's compare.py change fails 3 cases: test_the_failure_names_a_containment_its_details_leave_out, …names_every_containment… and test_a_failure_the_base_could_not_confirm_blocks_and_says_so.

• Mutants (scratch worktree, compare file, -n 4). All were killed:

  • A, revert to "this": 3 failed.
  • B, {containments} replaced by "this": 3 failed.
  • C, the note names only the first scope: 1 failed.
  • D (mine), the total dropped (whether it contains {head}): 3 failed.
  • E (mine), the pronoun always "it": 1 failed.

  The body's "the note drops the containments' names | 3" matches A, B and D.

• The in-place write scan. scan_source returns no findings on all 7 .py files the PR changes, and 0 package-wide on both the head and main 1d4d020aa. At ea8d87b8a it flags the compare test at lines 1628-1629, as the commit message says. Its call matching treats any name("literal", …) result as a cached array (_cached_call, test_cached_arrays_not_written_in_place.py:55-71), so no other current test trips it. But any future test that writes by subscript into a helper's result (check(...), result(...)) will trip it. The round-3 witness's base_check["details"] = … would be flagged if it sat in the package.

• The merge. git show --remerge-diff ea8d87b8a touches only render.py and keeps both from .guard import not_held_note, problem_text and from .privacy import withheld_text. From a122f2355 (Detect undeclared tax movement at any baseline level and count non-finite responses as movement #2758) to 52b30809f, main (Record why the aggregate impact report withholds a reform year #2772) changes api.py, compare.py (a withheld_because field on metric rows), measure.py (reform_withheld becomes withheld_reasons), privacy.py, render.py and README, and adds test_aggregate_impact_withheld_reasons.py. That changes what the reform costing rows publish, but not the counted-once check, record_guard_checks, _unconfirmed_on_base or check classification. main changed no existing assertion in the compare, guard or measure test files. main has since moved to 1d4d020aa with no aggregate_impact change.

• Counts at the head, uv run pytest … -n 4, env vars unset:

  • All tests/aggregate_impact/ files except childcare provenance and certified, plus both code-health files: 584 passed.
  • Per file: compare 142, guard 184, measure 46, data_mode 4, dataset 13, partition 11, reforms 20, withheld_reasons 25, test_changelog_fragments.py 103, test_cached_arrays_not_written_in_place.py 36.
  • On main, compare 133, guard 63 and measure 44, so the body's "132 test cases: 121 + 2 + 9" holds.

• Fix evidence. I ran .github/check_fix_pr.py from main, as pr_fix_evidence.yaml does, on the live title, body and labels and the --name-status -z diff from merge-base 52b30809f. Result: PASS … Changed executable tests: 3.

• CI at the head. All 20 check runs completed with success: Aggregate impact, Changelog fragments, Changes, Documentation build, Fix evidence, Install + smoke-import (py3.11–3.14), Lint, Release build, Test / Python 1–7/7, Test / Result and Test / YAML.

• Merges (git merge-tree --write-tree --name-only):

  • Against origin/main (1d4d020aa): clean.
  • Against Name heads that are NaN at baseline in the counted-once check #2854 (9ee51800e): clean.
  • Against Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963 (69c919f6): conflicts in measure.py and test_aggregate_impact_measure.py. Both are add/add at the same spot: this branch's record_guard_checks meets Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963's component_guard_diagnostic, and this branch's not-finite double-count test meets Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963's certified-diagnostic test. The other three conflicts (income_tax_subtractions.yaml, uses_property_allowance.py, property_finance_cost_relief.py) also occur between Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963 and main alone. Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963's kinds = {"own_head", "contains", "moves", "rises", "model_total"} (measure.py:1232 on Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963) lacks unconfirmed, as the body says.

• Writing. The _unconfirmed_note docstring and the new test docstring describe current behaviour with no fix history. The commit messages are clear. Apart from P2-1 and P3-1, the delta-touched body text matches the code.

What I couldn't check

• The childcare provenance and certified files: I did not run them (by scope, and the certified file needs survey data).
• The body's property draws, the 1,500-model differential and the 47 earlier mutations predate this delta. I did not rerun them.
• The round-3 reviewer's out-r3.md and run_test.py sit on another machine. I used the witness text from the brief instead.
• No legislation is involved.

---

Generated by Claude Code

[MaxGhenis]

Picking this up for the UK hub. Reply here if you're mid-change and I'll stop.

---

Generated by Claude Code

[MaxGhenis]

PR_TITLE.txt

Fail an infinite or NaN response on every counted-once check

---

Generated by Claude Code

[MaxGhenis]

PR_BODY.md

Fixes #2759

## What and why

The counted-once checker (`policyengine_uk/utils/aggregate_impact/guard.py`) raises each component by 1 and compares each head's response with a threshold. NaN compares false with every threshold, so on #2758 four checks passed a NaN response:

| Check | #2758's rule | +inf | −inf | NaN |
| --- | --- | --- | --- | --- |
| Own head | `abs(change - expected) > slack` | fails | fails | **passes** |
| Declared-reader containment | `perturbed & (abs(slope - expected) <= allowance)` | fails only where the check added something | same | **passes** |
| Spending rises | `change > slack` | fails | **passes** | **passes** |
| Model total | `abs(model - heads) > slack` | fails unless the heads are infinite too | same | **passes** |

A declared double count whose container turned NaN also passed (`present` compared NaN with the slack). #2758 already fails a non-finite undeclared tax movement (`_moves`).

When two heads turned NaN, the check stopped without a report. It measures a failing head again while it holds the other heads at their values after the first point. When one of those values was NaN, policyengine-core 3.33.12 refused it, and the check stopped. `Holder.set_input` calls `_raise_if_input_contains_nan` (`policyengine_core/holders/holder.py`, lines 610 and 667), which raises on NaN and accepts infinity:

```
ValueError: Unable to set value for variable "tax_c", as the input contains NaN values.
```

This PR changes eight things:

- **Every check fails a response that is not finite.** Own head, declared-reader containment (at one point and at two), spending rises and the model total each fail a response that is +inf, −inf or NaN, wherever it occurs. So do both held measurements (`still_breaks`, `breaks_on_slope`). The new `_contains` and `_rises` helpers follow #2758's `_moves`, which this PR reuses unchanged. A non-finite response now appears in the component's `responses`.
- **Every problem says whether it rests on a response that is not finite.** Each problem carries `not_finite`, which the check sets when a measurement behind it (the first reading, or a held measurement that judged the household) saw such a response in its households. One helper, `problem_text`, writes the text of every problem and every mediated response. When a household with weight saw the response (`not_finite_weighted`), it adds the same clause for every check, and a containment no longer concludes that the head contains the component. Finite messages keep their wording:

  | Check | Finite | A response behind it is not finite |
  | --- | --- | --- |
  | Own head | `tax_b does not move by exactly what was added to tax_b` | `tax_b does not move by exactly what was added to tax_b, or is not finite,` |
  | Undeclared tax movement | `tax_a moves when tax_b does` | `tax_a moves, or is not finite, when tax_b does` |
  | Declared reader | `tax_a rises one for one with tax_b, so it contains it,` | `tax_a rises one for one with tax_b, or is not finite,` |
  | Spending rises | `ben_d rises when ben_c does` | `ben_d rises, or is not finite, when ben_c does` |
  | Model total | `the model's gov_tax moves by other than its heads when tax_b does` | `the model's gov_tax moves by other than its heads, or is not finite, when tax_b does` |

  `measure` and the check sum with numpy, and NaN times a weight of zero stays NaN, so a NaN in a household with no weight still sets `not_finite`. When only such households saw it, the message keeps its finite text for the households it counts and says where the NaN sits:

  ```
  gov_tax: tax_a moves when tax_b does in 11 households, and is not finite in households with no weight
  ```

- **The second held measurement judges household by household.** It holds every other head at its value after the first point. In each unit where one of those heads is not finite (in any month, for a monthly head), it holds that head at its baseline value instead, and it judges no household the unit belongs to. Benefit units and people map to households through `map_result`, the mapping that sums every head to households. In those households a problem keeps every household it had. A head that may respond, whose slope was neither 0 nor 1 and which the baseline hold did not show rising one for one, fails there as `unconfirmed`. Every other household gets both measurements.

  When `tax_h` contains `tax_b` behind `tax_g`, a passport on `tax_b`, only the second held measurement shows its slope of 1. With `ben_c`, a benefit no tax head reads, NaN in one of twelve households, `tax_b` fails in all twelve (`test_a_gated_containment_fails_wherever_a_nan_elsewhere_leaves_it`):

  ```
  gov_tax: tax_h rises one for one with tax_b, so it contains it, in 11 households
  gov_tax: tax_h responds to tax_b in fewer than 10 households where the check could not hold ben_c, which is not finite there once tax_b rises, so it could not confirm whether tax_h contains tax_b
  ```

  A head that responds only through another head fails only where the check could not hold `ben_c`, and stays mediated in the other eleven households (`test_one_bad_record_fails_a_mediated_head_only_in_its_household`):

  ```
  gov_tax: tax_a moves when tax_b does in fewer than 10 households (in fewer than 10 of them the check could not hold ben_c, which is not finite there once tax_b rises, so it could not show that tax_a responds only through other heads)
  ```

  For each head the check measured again, the component's result names the heads the check could not hold (`heads_not_held`, with `heads_held_skipped`) and counts the households where it could not hold them (`households_not_held`). Each problem names and counts them for the households that rest on the measurement the check could not make there. For the issue's case, `counted_once` reports `tax_b` as failing with `gov_tax: tax_a moves, or is not finite, when tax_b does …` and `gov_tax: tax_c moves, or is not finite, when tax_b does …`, then checks `tax_c`.
- **Mediated responses and passing components say what the check could not see.** A mediated response carries `not_finite` and `not_finite_weighted` for its first reading, and `render` writes it with `problem_text`:

  ```
  - `tax_b`: `tax_a` responds only through other heads, or is not finite, in fewer than 10 households (other heads that moved there: `tax_c`; gov_tax; not a failure)
  ```

  A component can pass while the check could not hold a head, when that head broke no rule and showed no slope other than 0 in the households the check left unjudged. Its detail then names the head in the same words as the comment (`not_held_note`):

  ```
  in fewer than 10 households the check could not hold ben_c, which is not finite there once tax_b rises, so it measured tax_a again there only with the other heads at their baseline values
  ```

  `record_guard_checks`, which `measure` now calls to record the counted-once result, puts that detail on the passing `counted_once:<component>` check through a new `note` argument to `Recorder.check`. Before, a passing check kept no detail.
- **The comment shows each head the check could not hold.** `render` says, in the guard section's summary, how the second measurement treats a head that is not finite, and adds the note above for each head the check measured again without holding every other head. `publishable_guard` keeps the head names and the flags, which hold no figures, and drops a count of households below 10 (`households_not_held`), as it already does for the counts and shares of problems.
- **Double counts.** A declared double count whose container is not finite now fails with `not_finite: True`, and every double count carries the flag. Its detail says the check cannot show that the container still contains the component; it does not tell the reader to remove the declaration. `publishable_guard` keeps that detail for a cell of fewer than 10 units, since it holds no figures. The comment renderer says the container is NOT FINITE when the component rises, rather than NO LONGER contained.
- **The comparison says when the base could not confirm a failure.** Repairing an unrelated NaN can move a reader from `unconfirmed` on the base to `contains` on the head, so a containment both sides share reads as new. `compare` still fails the job, because a new containment reads the same, and it now lists those scopes in the check's `base_unconfirmed` and ends the failure with a note that names each one: "On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it." `render` counts the check apart from those this change broke:

  ```
  **Checks:** 0 broken by this change, 1 failing that the base could not confirm, 0 failing on both sides, 0 fixed.
  ```

  The note applies only when every new scope confirms a containment the base failed as `unconfirmed`. A new failure beside one, a failure of another kind, and a head that stops confirming a containment the base found all count as broken. A comparison with no such failure prints the wording it printed before.

  With `ben_c` NaN in every household on the base and finite on the head, `counted_once` on the test file's fake simulation, then `record_guard_checks`, `publishable_guard` and `compare`, print:

  ```
  Raising tax_b by £1 raises its own head by £1, moves no other tax head and raises no other benefit (counted_once:tax_b): gov_tax: tax_h rises one for one with tax_b, so it contains it, in 10 households. On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it.
  ```
- **The partition rejects a colon in a name.** `record_guard_checks` writes each counted-once failure under the scope `total:head:kind`, and `compare` splits that scope at its colons to name the containment in the note. A colon in a total or head name would let two different pairs write the same scope: total `gov:tax` with head `tax_h`, and total `gov` with head `tax:tax_h`, both write `gov:tax:tax_h:contains`, and the note would then read the base's pair off the head's scope. So `structural_problems` now reports, by name, a colon in every name the report writes into a check id or a scope: totals, head ids, head variables, declared double counts and overlap guard ids. `parse_partition` raises `PartitionError` on that report, and the checked-in `fiscal_partition.yaml`, the CLI's `--partition` and `--fallback-partition`, and `compute_budget_metrics`'s default all load through it. The checked-in partition holds no colon and still loads.

## Invariants

1. **A non-finite response fails unless both held measurements clear it, and the result says so either way.** For every check, and in the first reading and both held measurements, every household where the judged response is +inf, −inf or NaN falls in the failing households, unless both held measurements clear a first reading there. That reading then stays `mediated` with `not_finite` set. The check sets `not_finite` on the problem that covers a failing household, and adds the clause when a household with weight saw the response. Tests: `test_a_non_finite_response_fails_every_check` (7 checks × 3 values), `test_a_non_finite_held_measurement_breaks_every_rule` (3 kinds × 3 values), `test_every_check_adds_one_clause_when_a_response_is_not_finite` (7 checks × 4 values), `test_a_response_not_finite_only_in_households_with_no_weight_says_so`, with #2758's `test_a_non_finite_response_is_movement_on_every_path`.
2. **The flags say what the measurements saw.** For every problem, `not_finite` holds exactly when a measurement behind it saw a non-finite response. Its `heads_not_held` names the heads that stopped the second held measurement when the baseline hold alone would have cleared it or the check could not confirm a reader, and stays empty otherwise. The check fails a reader as `unconfirmed` wherever the oracle finds one, and nowhere else. For each head the check measured again, each component lists the heads whose value after the first point is not finite, and no others. A mediated response's `not_finite` matches its first reading, and the check finds no response mediated where it could not make the second measurement. Test: `test_every_problem_says_whether_a_measurement_behind_it_was_not_finite`, which recomputes every measurement on the fake simulation.
3. **The check judges each household as it would alone.** Households do not interact in the model. In each household of a model of two or three, the check finds the same problems, flags and mediated responses as in that household alone. The property covers components that are zero in every household or in none: where a component is zero in some households, the check perturbs only the others, though a zero household alone would get the £1. For each head it measured again, it leaves unjudged the households where the oracle, run on each household alone, finds a held head not finite, and no others; the component names those heads and counts those households. Test: `test_the_check_judges_each_household_as_it_would_alone`.
4. **A NaN in one household changes no verdict in another.** When a benefit no head reads turns NaN in one household, every other household keeps the problems, mediated responses and unjudged heads it had. Every head in the property belongs to a household. A benefit unit or a person maps to each household its members live in, so a unit that turns NaN leaves unjudged every household where one of its members lives, and the property's other households are those where none does. Tests: `test_a_nan_in_one_household_changes_no_verdict_in_another`, `test_a_benefit_unit_not_finite_leaves_only_its_household_unjudged`, `test_a_person_not_finite_leaves_only_their_household_unjudged_on_core` and `test_a_benefit_unit_in_two_households_leaves_both_unjudged`.
5. **Finite results stay the same.** On finite responses, every household mask, message, screen, response and coefficient equals what #2758's rules give, and the new fields keep their defaults: `not_finite` and `not_finite_weighted` false, `heads_not_held` and `households_not_held` empty, `heads_held_skipped` false and no `unconfirmed` problem. Tests: `test_finite_responses_keep_every_verdict` and the property in invariant 2, plus the scratch differential below.
6. **Held checks only narrow, and the second measurement decides nothing where it could not judge.** A held measurement never adds a household outside those it receives. In a household the second measurement could not judge, only the baseline hold decides, whatever slope the second measurement read there. A unit counts as not finite when any of its months is. Tests: `test_the_held_checks_stay_within_the_households_they_are_given`, `test_the_second_held_measurement_judges_only_the_households_it_could_hold`, `test_a_unit_is_not_finite_when_any_month_is`.
7. **The check never stops on a response that turns not finite, and names every head that turns.** Tax heads may turn not finite directly or through another head, declared or not. Either way `counted_once` returns a row for every component, names the heads that turned and no finite head, passes the other components, and recomputes the baseline to the same values. Tests: `test_the_check_names_every_head_that_turns_not_finite_and_goes_on`, and on policyengine-core `test_two_heads_that_turn_nan_on_core_fail_by_name_and_the_check_goes_on` (yearly and monthly) and `test_a_head_not_finite_in_one_month_is_not_held`.
8. **The comparison excuses no failure.** The job fails either way, and the report moves a failure out of "broken by this change" only when every new scope confirms a containment the base failed as `unconfirmed`. Tests: `test_the_report_excuses_only_failures_the_base_could_not_confirm` and `test_repairing_an_unrelated_nan_reports_no_tax_broken_by_the_change`.
9. **No partition with a colon in a scoped name loads.** `parse_partition` raises `PartitionError` for a colon in a total, head id, head variable, double count or overlap guard id, and for either side of the collision above. Tests: `test_a_name_with_a_colon_is_rejected` (5 kinds of name), `test_neither_side_of_a_colon_collision_loads` (base and head) and `test_the_small_partition_loads`.

The tests also pin four intended results:

- A non-finite response fails a reader even in a household where the check added nothing.
- A head that turns NaN only through another head stays a failure where a held measurement also fails to clear it, and its message says the check could not show that route. Where both held measurements clear the reading, it stays `mediated` with `not_finite` set.
- A reader fails as `unconfirmed` in a household the second measurement could not judge, though no measurement showed it rising one for one there.
- A problem whose non-finite households all have no weight keeps `not_finite`.

## Limits

- **The second measurement reads nothing where it could not hold a head.** There the check reads the slope from no hold but the baseline one, so a containment that only the second measurement would show fails as `unconfirmed`, not as a containment. With one household, the gated `tax_h` above fails that way.
- **A caller's own `Partition` skips the colon check.** `compute_budget_metrics` accepts a `Partition` object, and a caller who builds one with the dataclass, not `parse_partition`, gets no structural check at all. No code in the package does that.
- **Non-finite baselines stay out of scope.** #2773 tracks them. As on main and #2758, a head that is NaN at baseline raises Core's `ValueError` when the check perturbs it or holds it at its baseline value. It also fails components it ignores (its change is NaN − NaN), and it makes the reset check report a changed baseline, because `np.array_equal` treats NaN as differing from NaN.

## Test plan

- [x] For policy changes: not applicable (CI tooling; no policy, parameter or variable changes).
- [x] `make format`: I ran the equivalent with `uvx ruff@0.17.2 format --check` and `ruff check` on the repository. I also ran `ruff check --select F811,F821 policyengine_uk` and `git diff --check`; all pass.
- [x] Changelog fragment `changelog.d/2759.fixed.md`

I ran each file in `policyengine_uk/tests/aggregate_impact/` alone at cca0b1ddff41b9f1ae3b55e44333b799cf14087a, the head after the merge of main, with `-n 4 -o xfail_strict=true` and without survey data. The childcare provenance file ran whole, in one run of about two minutes. No test failed.

| File | Result |
| --- | --- |
| `test_aggregate_impact_guard.py` | 184 passed |
| `test_aggregate_impact_measure.py` | 46 passed |
| `test_aggregate_impact_compare.py` | 142 passed |
| `test_aggregate_impact_childcare_provenance.py` | 42 passed |
| `test_aggregate_impact_data_mode.py` | 4 passed |
| `test_aggregate_impact_dataset.py` | 13 passed |
| `test_aggregate_impact_partition.py` | 19 passed |
| `test_aggregate_impact_reforms.py` | 20 passed |
| `test_aggregate_impact_withheld_reasons.py` | 25 passed |
| `test_aggregate_impact_certified.py` | not run: its 9 cases need the survey dataset |
| `policyengine_uk/tests/code_health` | 3,091 passed |

This PR adds 140 test cases: 121 in the guard file (184 against 63 on #2758), 2 in the measure file (46 against 44), 9 in the compare file (142 against 133) and 8 in the partition file (19 against 11). The compare file's renderer test checks the summary's new wording.

I did not rerun the property draws, the 1,500-model differential or the mutations below at the final head, except the two mutations at cca0b1ddf. The numbers in those three paragraphs come from the commits they name.

**The properties reach the cases they pin.** One run of each at 8df6cc61c drew these models:

- The single-household property drew 121 models with 470 problems: 322 not finite, 20 resting on a household the second measurement could not judge, and 15 readers the check could not confirm. It also drew 4 mediated responses, all with a first reading that is not finite.
- The household-by-household property drew 62 models of two or three households. In 54 the check left some household unjudged for a head, and in 26 it left a head unjudged in some households and judged in others. For 12 problems, some but not all of their households went unjudged, and one component passed although the check could not hold a head.
- In the one-household-NaN property, the NaN changed what the check found in that household in 24 of 40 models, and in no other household in any of them.

**Differential against #2758, over finite models.** A scratch script, not committed, ran `counted_once` from this PR (8df6cc61c) and from #2758 (a122f2355) on 1,500 random finite models on the test file's fake simulation. The models mixed tax and benefit heads, readers declared or not, steps, floors, zero cells, model totals that count a head twice, and live or stale declared double counts. Every new field held its default, and every other part of every result matched, value for value: statuses, coefficients, model coefficients, responses, every problem with its message, household count and share, every mediated record, the double counts and the reset flag. The models produced 3,316 failing components and 4,192 problems (1,813 moves, 1,304 model totals, 602 rises, 473 containments), 229 mediated records and 942 double counts, 443 of them failing.

**Mutations.** I applied each of 47 single-line mutations to a scratch copy and ran the guard file with the five measure and compare test cases that read the guard (189 cases). I ran the first 24 at 982dcd21b, whose code differs from 8df6cc61c only in docstrings, and the other 23 at 8df6cc61c. I applied 8 more at df997704d and ran the seven new compare cases, and one at 89bd11fec with the four compare cases on the note. At cca0b1ddf, I put back `whether {total} contains {head}` and ran the compare file, and dropped the colon check and ran the partition file. Every mutation failed at least one test case:

| Mutation | Failing test cases |
| --- | --- |
| `_rises` drops its non-finite term | 9 |
| `_contains` drops its non-finite term | 7 |
| the second hold keeps a value that is not finite instead of the baseline value | 25 |
| the own-head check drops its non-finite term | 2 |
| the model-total check drops its non-finite term | 10 |
| a declared double count ignores a container that is not finite | 1 |
| a household the second measurement could not judge clears a problem | 11 |
| a reader fails a non-finite response only where the check added something | 9 |
| `publishable_guard` rewrites the not-finite double count's detail | 1 |
| the report drops non-finite responses | 3 |
| a confirmed reader ignores non-finite held measurements | 1 |
| a confirmed reader that joins a problem drops that problem's non-finite households | 1 |
| no unit of a monthly head counts as not finite | 3 |
| a unit of a monthly head counts as not finite only when every month is | 2 |
| the second measurement judges the households it could not hold a head in | 13 |
| `problem_text` never adds the clause | 73 |
| a kept problem ignores non-finite held measurements | 5 |
| no problem lists the heads the check could not hold | 10 |
| every problem of a measured head lists every head the check could not hold | 8 |
| the component's result omits the heads the check could not hold | 25 |
| undeclared movement never sets `not_finite` | 10 |
| the own head never sets `not_finite` | 6 |
| spending rises never set `not_finite` | 6 |
| a declared reader's containment never sets `not_finite` | 22 |
| the model total never sets `not_finite` | 20 |
| the comment leaves out the notes on heads the check could not hold | 3 |
| `_held_nan` ignores the baseline hold's second point | 1 |
| `_held_nan` ignores the second measurement | 1 |
| `heads_held_skipped` stays false | 14 |
| a double count's `not_finite` stays false | 3 |
| a confirmed reader that joins a problem drops that problem's unheld heads | 1 |
| the second measurement holds a head that is not finite at its baseline value in every unit | 1 |
| every household counts as judged | 26 |
| a unit's mask stands in for its households, with no mapping | 3 |
| a mediated response never sets `not_finite` | 3 |
| a passing component's detail leaves out the note | 1 |
| the passing check drops the note | 1 |
| a reader the check could not confirm passes | 16 |
| a reader the holds did not confirm fails as `unconfirmed` in every household, judged or not | 38 |
| the clause appears for a NaN only in households with no weight | 2 |
| the message leaves out the households with no weight | 2 |
| `households_not_held` counts households with no weight | 2 |
| `publishable_guard` keeps a problem's small `households_not_held` | 1 |
| `publishable_guard` keeps a component's small `households_not_held` | 1 |
| a problem lists every head the check could not hold, wherever it could not hold it | 8 |
| a mediated response's `not_finite_weighted` ignores the weights | 1 |
| the comment drops a mediated response's `not_finite` | 2 |
| any new failure counts as one the base could not confirm | 5 |
| one such failure excuses the new failures beside it | 2 |
| a new failure of another kind counts as one the base could not confirm | 2 |
| no failure counts as one the base could not confirm | 3 |
| the failure text leaves out the note | 1 |
| the note drops the containments' names | 3 |
| the note names the total and head, not the head and component | 3 |
| the partition accepts a colon in a name | 7 |
| such a failure stops failing the job | 3 |
| the header counts such a failure as broken | 3 |
| the header always prints the new count | 2 |

#1963 (`marriage-allowance-election`) adds lines at the same spots in `measure.py` and its test file, so whichever merges second keeps both sides. #1963's `component_guard_diagnostic` lists the problem kinds it prints, so whichever merges second also adds `unconfirmed` to that list. The branch includes main at `52b30809f`. In `render.py`, where #2772 and this branch each added an import on the same line, the merge keeps both. `git merge-tree` merges the head cleanly with main at `1d4d020aa` and with #2854 at `9ee51800e`; on that merge, the compare (142), guard (184) and guard baseline (48) files pass. With #1963 at `69c919f6`, it conflicts only in `measure.py` and `test_aggregate_impact_measure.py`, as above, and in three files that conflict between #1963 and main alone.

Reproducer: policyengine_uk/tests/aggregate_impact/test_aggregate_impact_guard.py::test_two_heads_that_turn_nan_on_core_fail_by_name_and_the_check_goes_on

axiom: n/a: CI tooling.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---

Generated by Claude Code

[MaxGhenis]

Ready for hub review

Head: cca0b1ddff41b9f1ae3b55e44333b799cf14087a on claude/note-naming-colon-validation-62frrw, two commits on ea04c31983fb0763045c97c743d32dc1ad1dd88f (effb0165 code and tests, cca0b1dd changelog and formatting). No pull request opened; fix-2759-non-finite-responses untouched. The branch head of #2850 sat at the brief's base, pushed at 12:00 UTC, and no comment said anyone was mid-change, so I went ahead.

Findings

P2-1, the note names the wrong containment: resolved. I applied the review's replacement as written: _unconfirmed_note(check_id, scopes) at policyengine_uk/utils/aggregate_impact/compare.py:111-125 and its call at :641. I updated the three assertions (test_aggregate_impact_compare.py:1610, :1639, :1649). At :1638 the test splits the failure at ". On the base" now, not ". The check", because the new note itself holds "tax_h contains tax_b". No docstring or comment quoted the old note; the docstring at :1600 already describes the head containing the component, so I left it.

The real path (counted_once on FakeSim with the guard file's gated model, 10 households, ben_c NaN in all of them on the base and finite on the head; then record_guard_checks, publishable_guard and compare) prints, with status fail:

Raising tax_b by £1 raises its own head by £1, moves no other tax head and raises no other benefit (counted_once:tax_b): gov_tax: tax_h rises one for one with tax_b, so it contains it, in 10 households. On the base, the check could not confirm whether tax_h contains tax_b (gov_tax), so this change may not have introduced it.

Mutation (scratch worktree, whether {total} contains {head} restored, compare file, -n 4): killed, 3 failed: test_a_failure_the_base_could_not_confirm_blocks_and_says_so, test_the_failure_names_a_containment_its_details_leave_out, test_the_failure_names_every_containment_the_base_could_not_confirm.

P3-1, missing test row: resolved in the body: test_aggregate_impact_withheld_reasons.py | 25 passed. The test-plan paragraph now names cca0b1ddf as the head where every file last ran.

P3-2, colon collision: resolved. _colon_problems (policyengine_uk/utils/aggregate_impact/partition.py:394-419), which structural_problems calls first (:423), reports "<kind> '<name>' contains a colon, …" for each of:

• total names (scope total:head:kind, check id identity:{total});
• head ids (finite:, sign:, inert: check ids);
• head variables (scope's head part, counted_once:{variable});
• double-count variables (counted_once:{variable}, double_count_present:{variable});
• overlap guard ids (overlap_guard:{id}), which I found in measure_year, beyond the brief's list.

contained_in must already equal a head variable, and reads, guard heads and dropped_when must name existing heads or components, so the check covers them through those names. Closure scopes hold free problem text, and the scope's third part (kind) comes from the guard, not the partition.

Tests (test_aggregate_impact_partition.py:224-292): _small_partition, test_the_small_partition_loads, test_a_name_with_a_colon_is_rejected (5 cases, one per kind of name) and test_neither_side_of_a_colon_collision_loads (base: total gov:tax, head tax_h; head: total gov, head tax:tax_h; both raise PartitionError).

Mutation (scratch worktree, problems = [] in place of the colon call, partition file): killed, 7 failed: all 5 test_a_name_with_a_colon_is_rejected cases and both test_neither_side_of_a_colon_collision_loads cases.

Loading: fiscal_partition.yaml still loads (test_shipped_partition_is_found_and_loads, test_structural_checks_pass_on_the_shipped_file), and every test file that builds partitions passes (all go through parse_partition). Production paths that build a Partition: parse_partition (partition.py) only; load_partition calls it, and __main__.py:97 (--partition, --fallback-partition, default) and api.py:164 (compute_budget_metrics's default) call load_partition. One path skips it: compute_budget_metrics(partition=…) takes a caller's Partition object as is, so a caller who builds the dataclass directly gets no structural check. Nothing in the package does that; I named it under Limits in the body and left the code alone.

Body and changelog. Replaced the quoted note; added the real-path output, an eighth "changes" bullet on the colon check and why, invariant 9 and a Limits entry. changelog.d/2759.fixed.md gains "and rejects a fiscal partition that puts a colon in a name it writes into a check id or scope". Counts: 140 added test cases (8 more in the partition file: 19 against 11), partition row 19, code health row, two new mutation rows, merge notes. The test plan says I did not rerun the property draws, the 1,500-model differential or the earlier mutations (47 + 8 + 1). Title unchanged.

Tests at cca0b1ddf (-n 4 -o xfail_strict=true, no survey data, env vars unset)

File	Result
guard	184 passed
measure	46 passed
compare	142 passed
childcare provenance	42 passed (one whole run, 122 s)
data_mode	4 passed
dataset	13 passed
partition	19 passed
reforms	20 passed
withheld_reasons	25 passed
certified	not run (needs survey data)
policyengine_uk/tests/code_health -n 4	3,091 passed

After the formatting change to the compare test file I reran it with test_cached_arrays_not_written_in_place.py: 178 passed (142 + 36).

• .github/check_fix_pr.py (from main, --name-status --no-renames -z diff from the merge base, new title and body, no labels): PASS … Changed executable tests: 4.
• uvx ruff@0.17.2 format --check .: 2,174 files already formatted. ruff check . and ruff check --select F811,F821 policyengine_uk: all checks passed. git diff --check ea04c3198..HEAD: clean.
• git merge-tree --write-tree --name-only: origin/main (1d4d020aa) clean; Name heads that are NaN at baseline in the counted-once check #2854 (9ee51800e) clean; Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963 (69c919f6) conflicts in measure.py, test_aggregate_impact_measure.py, income_tax_subtractions.yaml, uses_property_allowance.py and property_finance_cost_relief.py. The last three also conflict between Model Marriage Allowance as the statutory s.55B transfer and Step 6 reduction #1963 and main alone. No conflict beyond the review's.
• On the merge with Name heads that are NaN at baseline in the counted-once check #2854: compare 142, guard 184, guard_baseline 48 passed. (Name heads that are NaN at baseline in the counted-once check #2854's other changed test path, counted_once_base_reference.py, holds helpers, not tests.)

Judgment calls

• I took the review's replacement verbatim, including the total in brackets after each containment.
• I added overlap guard ids to the colon check, since they reach a check id; the brief's list said "at least".
• I ran with -n 4 as the brief says; the body's test plan now says -n 4 instead of -n 0.
• The code-health row covers the whole directory (3,091), not only the changelog file.
• I posted the pickup comment despite the base push at 12:00 UTC, as that push is the brief's own base.

---

Generated by Claude Code

[juaristi22]

Reviewed cca0b1ddff41b9f1ae3b55e44333b799cf14087a. COMPLETE — no findings or material evidence gaps.

The counted-once paths explicitly detect non-finite responses, retain failures for affected households and keep judging finite households. Base-unconfirmed containment failures still block the comparison, with qualified reporting. The colon checks cover the parser fields used in check IDs and scopes.

Validation: all 391 focused engineering tests passed on the head's locked Core 3.34.1 environment, including real-Core yearly/monthly NaN cases, mixed entity mappings, finite controls, restoration and guard→record→compare→render regressions. Existing non-finite baselines and direct construction that bypasses parser validation remain the explicitly disclosed scope limits.

Exact-head CI is green. Approval is deferred because the PR remains a draft; the completed review can be retained when readiness and current-head CI are checked later.

[juaristi22]

Approval-only follow-up at unchanged cca0b1ddff41b9f1ae3b55e44333b799cf14087a: COMPLETE, no findings or material gaps.

The completed engineering review and all 391 prior passing checks remain valid after exact-head/body/artifact/runtime validation. No tests were rerun. The previously disclosed baseline-NaN scope limit and separate #2854 integration remain distinct from this assessment.

The PR is now ready; applicable exact-head required checks and activity were refreshed before approval.