Run the tests that need the dataset in their own CI shard by MaxGhenis · Pull Request #2893 · PolicyEngine/policyengine-uk

MaxGhenis · 2026-11-06T13:18:10Z

The tests that carry the microsimulation marker now run in an eighth CI shard with one worker, so only one simulation of the whole dataset runs at a time. The other seven shards run none of them.

These tests set the memory peaks on the 16 GB runners. In merge queue run 42212587066, the five shards that built the fixture in test_aggregate_impact_certified.py peaked at 14.2 to 15.3 GB, and the two without it at 10.9 and 12.6 GB.

#2887, which merges 28 pull requests, leaves less room, since its runners hold about 11 GB in the first minute of tests. With data tests on all seven shards, it lost eight of its 14 shard jobs at 15.4 to 15.7 GB within minutes (Test, Release test). Shard 6/7 of the first then lost its runner at 15.6 GB 51 minutes in, with test_latest_data_smoke.py running.

An earlier head of this PR moved only the certified file to one shard. On #2887's merge that shard still lost its runner at 15.6 GB, about seven minutes in (job 127936972309), so this head moves every data test.

The table lists 33 minutes for these tests, including four entries that each measured the same fixture, against the job's 90-minute limit. I removed their 22 entries from shard_durations.toml, which re-deals the rest: 463 of the remaining 551 listed tests change shard, and the seven shards' listed loads stay within 20 seconds of each other. Unlisted tests keep their shard. The queue requires Test / Result and Lint, which don't name shards, so the rename to K/8 needs no ruleset change.

Invariants, tested in test_ci_test_shards.py:

With two or more shards, the last runs the tests with the marker and no others, and shard_of over one shard fewer places the rest, so every collected test runs in one shard and only one.
One shard runs every test.
The workflow numbers its shards from 1 to N and gives the last one worker.
A real pytest run over three shards, with and without xdist, puts the sample data test alone in the last shard's outcome log, and check_test_shards.py passes the run.

Seven mutations of this rule each fail one of these tests.

Reproducer: policyengine_uk/tests/code_health/test_ci_test_shards.py::test_the_last_shard_runs_exactly_the_tests_that_need_the_dataset

🤖 Generated with Claude Code

The module fixture in test_aggregate_impact_certified.py builds its own simulations, and each shard that holds one of the file's tests pays for them. The duration table deals the file's long entries apart, so main spreads its nine tests over five shards. Deal and hash the file as one unit, so one shard pays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

juaristi22

Review at 94a43ce01ed17357b179fc657e1966fe0e853a98: COMPLETE, no findings or material evidence gaps.

The placement-unit change and its collection/deal/hash integration preserve the tested partition, shared-file co-location and load-bound contracts. All 55 focused sharding/CI-health tests passed in one campaign. No model calculations, certified aggregate fixtures or private data were run.

This validation covers shard placement and the associated collection behavior. Historical memory figures and expected memory savings remain attributed to the author's CI evidence; they were not independently measured here.

The PR remains Draft with CI pending, so this is a completed review comment, not an approval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The certified aggregate tests on one shard of seven still lost that shard's runner at 15.6 GB on a large merge, and the latest-data smoke tests lost another. The tests that carry the microsimulation marker now run in an eighth shard with one worker, and the other seven run none of them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-11-06T14:48:40Z

Aggregate impact on certified data: pass

Base e0502003df against head c4278de660, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed

Caseloads

Measure	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target.

Checks failing on both base and head (1)

VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-53,888,684 in total; 2026-27: 76 units below zero, a weighted £-55,777,260 in total; 2027-28: 76 units below zero, a weighted £-57,727,917 in total

Counted-once check

The check raised each of 92 components by £1 per unit in 2026-27 and read every head of its totals household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A response that is not finite breaks every rule. The check measures a head that breaks a rule twice more: once holding every other head at its baseline value, and once holding every other head at its value after the first £1 while the component rises again. The head fails where either measurement still shows the response. The check cannot hold a head at a value that is not finite. In each unit where another head is not finite after the first £1, it holds that head at its baseline value, and the second measurement judges no household the unit belongs to. There a head fails wherever it first broke a rule, or wherever the check could not confirm whether its response contains the component. 0 failed and the check skipped 0.

widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)

Provenance

Side	Commit	Runtime	Environment hash	Partition hash	Time
base	main `e0502003df`	uk 2.176.1 (files `280db20b85c9`), core 3.34.1, py 3.13.18 aarch64	`3d7fa2d68fc7`	`3bcfb3473960`	2547.22s
head	#2893 merged into main `c4278de660`	uk 2.176.1 (files `280db20b85c9`), core 3.34.1, py 3.13.18 aarch64	`3d7fa2d68fc7`	`3bcfb3473960`	2523.06s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Final charge on cash interest in non-cash ISAs	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Lifetime ISA withdrawal charge	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Income tax	£294.46bn	£315.23bn	£335.82bn	£351.28bn	£366.96bn	£385.36bn	£399.99bn
National Insurance (employee and self-employed)	£51.70bn	£54.14bn	£56.13bn	£57.73bn	£59.76bn	£61.64bn	£63.51bn
National Insurance (employer)	£148.71bn	£155.37bn	£160.88bn	£165.09bn	£172.06bn	£177.36bn	£182.57bn
Employer NI change borne by capital	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Employer NI change borne by consumers	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Capital gains tax	£12.60bn	£13.15bn	£13.60bn	£14.11bn	£14.67bn	£15.24bn	£15.80bn
Student loan repayments	£7.82bn	£8.13bn	£8.39bn	£8.58bn	£8.79bn	£9.04bn	£9.29bn
VAT excluding private school fees	£249.37bn	£256.87bn	£266.33bn	£272.70bn	£278.97bn	£285.24bn	£291.59bn
Change in VAT from its baseline	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
VAT on private school fees	£1.95bn	£2.01bn	£2.09bn	£2.14bn	£2.20bn	£2.26bn	£2.31bn
Change in private school VAT from its baseline	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Fuel duties	£24.71bn	£24.13bn	£26.00bn	£26.83bn	£26.51bn	£25.78bn	£24.63bn
Alcohol duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Tobacco duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Vehicle excise duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Betting and gaming duties	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
TV licence fee	£4.26bn	£4.40bn	£4.41bn	£4.42bn	£4.43bn	£4.44bn	£4.45bn
Council tax	£49.21bn	£52.16bn	£54.89bn	£57.95bn	£61.16bn	£64.47bn	£67.91bn
High value council tax surcharge	£0.00bn	£0.00bn	£0.00bn	£0.48bn	£0.49bn	£0.50bn	£0.51bn
Northern Ireland domestic rates	£0.55bn	£0.58bn	£0.60bn	£0.63bn	£0.66bn	£0.69bn	£0.72bn
Business rates	£34.68bn	£36.20bn	£38.00bn	£39.46bn	£42.27bn	£43.25bn	£44.29bn
Stamp duty land tax on the household's own purchases	£6.88bn	£7.32bn	£7.66bn	£8.06bn	£8.47bn	£8.91bn	£9.35bn
Stamp duty land tax paid by companies	£4.72bn	£4.57bn	£5.29bn	£6.03bn	£6.37bn	£6.92bn	£7.45bn
Land transaction tax (Wales)	£0.07bn	£0.08bn	£0.08bn	£0.09bn	£0.10bn	£0.10bn	£0.11bn
Land and buildings transaction tax (Scotland)	£0.31bn	£0.32bn	£0.33bn	£0.35bn	£0.36bn	£0.38bn	£0.40bn
Unspecified tax change borne by capital	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Unspecified tax change borne by consumers	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Wealth tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Wealth tax excluding the main residence	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Land value tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Carbon tax	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Spending by benefit and service: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Lifetime ISA bonus	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
State Pension	£121.68bn	£125.92bn	£126.72bn	£129.48bn	£132.98bn	£136.65bn	£140.41bn
Pension Credit	£6.21bn	£6.46bn	£6.68bn	£6.82bn	£6.89bn	£7.07bn	£7.22bn
Winter Fuel Payment	£1.86bn	£1.85bn	£1.79bn	£1.77bn	£1.78bn	£1.78bn	£1.78bn
Pension Age Winter Heating Payment (Scotland)	£0.19bn	£0.20bn	£0.20bn	£0.21bn	£0.21bn	£0.22bn	£0.22bn
Universal Credit	£75.48bn	£81.56bn	£84.02bn	£86.07bn	£87.22bn	£88.76bn	£89.38bn
Housing Benefit	£8.56bn	£8.51bn	£8.83bn	£9.05bn	£9.28bn	£9.56bn	£9.83bn
Council Tax Reduction	£2.18bn	£2.31bn	£2.43bn	£2.57bn	£2.71bn	£2.86bn	£3.01bn
Employment and Support Allowance (income-related)	£0.79bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Employment and Support Allowance (contribution-based)	£5.20bn	£5.37bn	£5.56bn	£5.69bn	£5.82bn	£5.95bn	£6.09bn
Jobseeker's Allowance (income-based)	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Jobseeker's Allowance (contribution-based)	£0.29bn	£0.30bn	£0.31bn	£0.31bn	£0.32bn	£0.33bn	£0.34bn
Income Support	£0.11bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Working Tax Credit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Child Tax Credit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Incapacity Benefit	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Personal Independence Payment	£26.95bn	£28.03bn	£29.05bn	£30.09bn	£30.75bn	£31.51bn	£32.21bn
Disability Living Allowance	£7.49bn	£7.79bn	£8.07bn	£8.36bn	£8.55bn	£8.76bn	£8.95bn
Attendance Allowance	£8.68bn	£9.03bn	£9.36bn	£9.69bn	£9.90bn	£10.15bn	£10.38bn
Carer's Allowance	£4.48bn	£4.66bn	£4.83bn	£5.00bn	£5.11bn	£5.27bn	£5.42bn
Carer Support Payment (Scotland)	£0.31bn	£0.32bn	£0.33bn	£0.34bn	£0.35bn	£0.36bn	£0.37bn
Carer's Allowance Supplement (Scotland)	£0.00bn	£0.04bn	£0.04bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn
Severe Disablement Allowance	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn	£0.03bn
Industrial Injuries Disablement Benefit	£0.50bn	£0.52bn	£0.54bn	£0.55bn	£0.56bn	£0.58bn	£0.59bn
Armed Forces Compensation Scheme	£0.61bn	£0.63bn	£0.65bn	£0.66bn	£0.68bn	£0.69bn	£0.71bn
War Widow's or Widower's Pension	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Child Benefit	£16.19bn	£16.82bn	£17.39bn	£18.00bn	£18.39bn	£18.84bn	£19.25bn
Scottish Child Payment	£0.49bn	£0.52bn	£0.53bn	£0.55bn	£0.56bn	£0.57bn	£0.56bn
Two Child Limit Payment (Scotland, counterfactual)	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Healthy Start vouchers	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn	£0.05bn
Maternity Allowance	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Statutory Maternity Pay	£3.30bn	£3.41bn	£3.54bn	£3.62bn	£3.70bn	£3.78bn	£3.87bn
Statutory Paternity Pay	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Statutory Sick Pay	£1.12bn	£1.16bn	£1.20bn	£1.23bn	£1.26bn	£1.29bn	£1.32bn
Sure Start Maternity Grant	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn	£0.01bn
Bereavement Support Payment	£0.67bn	£0.69bn	£0.71bn	£0.73bn	£0.75bn	£0.76bn	£0.78bn
Widowed Parent's Allowance	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Tax-Free Childcare	£0.56bn	£0.57bn	£0.57bn	£0.58bn	£0.60bn	£0.61bn	£0.59bn
Extended childcare entitlement	£2.99bn	£3.50bn	£3.59bn	£3.71bn	£3.78bn	£3.86bn	£3.93bn
Universal childcare entitlement	£2.06bn	£2.22bn	£2.29bn	£2.37bn	£2.43bn	£2.48bn	£2.53bn
Targeted childcare entitlement	£0.50bn	£0.51bn	£0.52bn	£0.54bn	£0.55bn	£0.57bn	£0.56bn
Childcare Offer for Wales	£0.26bn	£0.28bn	£0.28bn	£0.29bn	£0.30bn	£0.31bn	£0.32bn
Care to Learn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Childcare Grant	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Parents' Learning Allowance	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn	£0.20bn
Adult Dependants' Grant	£0.06bn	£0.06bn	£0.04bn	£0.04bn	£0.04bn	£0.04bn	£0.04bn
Travel Grant	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed
16 to 19 Bursary Fund	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
NHS spending	£203.89bn	£204.30bn	£204.56bn	£204.91bn	£205.34bn	£205.85bn	£206.35bn
State education spending	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn	£0.02bn
Transport subsidy spending	£25.05bn	£25.10bn	£25.14bn	£25.18bn	£25.23bn	£25.30bn	£25.36bn
Cost-of-living support payment	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Energy Bills Rebate	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Household Energy Payment	£0.00bn	£2.11bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Energy Price Guarantee subsidy	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Basic income	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Disability basic income	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn
Other public spending budget change	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Caseloads: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Lifetime ISA bonus	0k	0k	0k	0k	0k	0k	0k
State Pension	11,264k	11,145k	10,819k	10,700k	10,723k	10,750k	10,776k
Pension Credit	1,489k	1,484k	1,476k	1,454k	1,452k	1,457k	1,466k
Winter Fuel Payment	7,844k	7,783k	7,473k	7,375k	7,391k	7,409k	7,427k
Pension Age Winter Heating Payment (Scotland)	803k	802k	800k	797k	799k	801k	802k
Universal Credit	6,342k	6,418k	6,408k	6,369k	6,363k	6,343k	6,295k
Housing Benefit	1,249k	1,182k	1,181k	1,164k	1,164k	1,166k	1,168k
Council Tax Reduction	3,319k	3,336k	3,344k	3,357k	3,369k	3,381k	3,388k
Employment and Support Allowance (income-related)	71k	0k	0k	0k	0k	0k	0k
Employment and Support Allowance (contribution-based)	719k	720k	721k	722k	724k	726k	728k
Jobseeker's Allowance (income-based)	0k	0k	0k	0k	0k	0k	0k
Jobseeker's Allowance (contribution-based)	67k	67k	67k	67k	67k	68k	68k
Income Support	25k	0k	0k	0k	0k	0k	0k
Working Tax Credit	0k	0k	0k	0k	0k	0k	0k
Child Tax Credit	0k	0k	0k	0k	0k	0k	0k
Incapacity Benefit	0k	0k	0k	0k	0k	0k	0k
Personal Independence Payment	3,567k	3,575k	3,579k	3,585k	3,593k	3,602k	3,611k
Disability Living Allowance	1,291k	1,294k	1,296k	1,298k	1,300k	1,304k	1,307k
Attendance Allowance	1,695k	1,698k	1,700k	1,703k	1,707k	1,711k	1,715k
Carer's Allowance	1,034k	1,037k	1,038k	1,040k	1,042k	1,060k	1,062k
Carer Support Payment (Scotland)	71k	71k	71k	72k	72k	72k	72k
Carer's Allowance Supplement (Scotland)	0k	71k	71k	72k	72k	72k	72k
Severe Disablement Allowance	4k	4k	4k	4k	4k	4k	4k
Industrial Injuries Disablement Benefit	137k	137k	137k	138k	138k	138k	139k
Armed Forces Compensation Scheme	93k	93k	94k	94k	94k	94k	94k
War Widow's or Widower's Pension	0k	0k	0k	0k	0k	0k	0k
Child Benefit	8,123k	8,120k	8,120k	8,130k	8,145k	8,161k	8,178k
Scottish Child Payment	351k	352k	352k	353k	350k	348k	337k
Two Child Limit Payment (Scotland, counterfactual)	0k	0k	0k	0k	0k	0k	0k
Healthy Start vouchers	200k	200k	200k	201k	201k	201k	202k
Maternity Allowance	0k	0k	0k	0k	0k	0k	0k
Statutory Maternity Pay	333k	333k	334k	334k	335k	336k	337k
Statutory Paternity Pay	0k	0k	0k	0k	0k	0k	0k
Statutory Sick Pay	181k	182k	182k	182k	182k	183k	183k
Sure Start Maternity Grant	26k	26k	26k	26k	26k	26k	26k
Bereavement Support Payment	98k	98k	98k	98k	98k	99k	99k
Widowed Parent's Allowance	0k	0k	0k	0k	0k	0k	0k
Tax-Free Childcare	1,043k	1,027k	1,014k	1,009k	1,011k	1,009k	993k
Extended childcare entitlement	795k	800k	796k	797k	798k	798k	792k
Universal childcare entitlement	1,071k	1,072k	1,072k	1,073k	1,076k	1,079k	1,074k
Targeted childcare entitlement	165k	156k	154k	154k	155k	155k	150k
Childcare Offer for Wales	39k	39k	39k	39k	39k	39k	39k
Care to Learn	0k	0k	0k	0k	0k	0k	0k
Childcare Grant	0k	0k	0k	0k	0k	0k	0k
Parents' Learning Allowance	98k	98k	98k	99k	99k	99k	99k
Adult Dependants' Grant	18k	18k	12k	12k	12k	12k	12k
Travel Grant	0k	0k	0k	0k	0k	0k	0k
Disabled Students' Allowance	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed	suppressed
16 to 19 Bursary Fund	0k	0k	0k	0k	0k	0k	0k
Cost-of-living support payment	0k	0k	0k	0k	0k	0k	0k
Energy Bills Rebate	0k	0k	0k	0k	0k	0k	0k
Household Energy Payment	0k	21,065k	0k	0k	0k	0k	0k
Energy Price Guarantee subsidy	0k	0k	0k	0k	0k	0k	0k
Basic income	0k	0k	0k	0k	0k	0k	0k
Disability basic income	0k	0k	0k	0k	0k	0k	0k

Government balance, each component counted once: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Government receipts, each component once	£892.00bn	£934.66bn	£980.50bn	£1,015.93bn	£1,054.24bn	£1,091.56bn	£1,124.89bn
Government spending	£529.04bn †	£545.05bn †	£550.11bn †	£558.81bn †	£566.42bn †	£575.13bn †	£582.72bn †
Government balance, each receipt once	£362.96bn †	£389.61bn †	£430.39bn †	£457.12bn †	£487.81bn †	£516.43bn †	£542.17bn †
Benefits in household_benefits but not in gov_spending	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn	£0.00bn

Model totals: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
gov_tax	£892.00bn	£934.66bn	£980.50bn	£1,015.93bn	£1,054.24bn	£1,091.56bn	£1,124.89bn
household_tax	£491.97bn	£520.41bn	£551.21bn	£576.00bn	£601.00bn	£626.69bn	£648.42bn
pre_budget_change_household_tax	£484.15bn	£512.28bn	£542.82bn	£566.95bn	£591.72bn	£617.16bn	£638.61bn
gov_spending	£529.04bn †	£545.05bn †	£550.11bn †	£558.81bn †	£566.42bn †	£575.13bn †	£582.72bn †
household_benefits	£529.04bn †	£545.05bn †	£550.11bn †	£558.81bn †	£566.42bn †	£575.13bn †	£582.72bn †
gov_balance	£362.96bn †	£389.61bn †	£430.39bn †	£457.12bn †	£487.81bn †	£516.43bn †	£542.17bn †

Household income totals: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Household market income	£1,637.96bn	£1,702.77bn	£1,757.35bn	£1,806.41bn	£1,855.97bn	£1,912.43bn	£1,970.44bn
Household net income	£1,636.35bn †	£1,687.21bn †	£1,714.80bn †	£1,746.79bn †	£1,777.92bn †	£1,816.20bn †	£1,858.81bn †
Household net income (HBAI definition)	£1,449.23bn	£1,499.22bn	£1,530.21bn	£1,564.47bn	£1,597.11bn	£1,634.77bn	£1,675.74bn

Population: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
People	69,245k	69,383k	69,473k	69,592k	69,738k	69,912k	70,080k
Benefit units	35,110k	35,181k	35,226k	35,286k	35,360k	35,449k	35,534k
Households	31,046k	31,108k	31,149k	31,201k	31,267k	31,345k	31,420k

Poverty rates: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Absolute poverty, before housing costs: everyone	16.3%	15.5%	16.4%	16.3%	16.4%	16.5%	16.6%
Absolute poverty, before housing costs: children	24.7%	22.1%	23.3%	22.9%	23.1%	23.2%	23.4%
Absolute poverty, before housing costs: working-age adults	14.1%	13.8%	14.6%	14.7%	14.8%	14.8%	14.9%
Absolute poverty, before housing costs: pensioners	13.7%	13.4%	14.0%	13.8%	14.0%	14.0%	13.9%
Absolute poverty, after housing costs: everyone	21.5%	21.0%	22.0%	22.6%	22.8%	23.1%	23.4%
Absolute poverty, after housing costs: children	33.2%	31.2%	32.1%	32.5%	32.8%	33.3%	34.2%
Absolute poverty, after housing costs: working-age adults	19.9%	19.8%	20.8%	21.6%	21.9%	22.0%	22.3%
Absolute poverty, after housing costs: pensioners	12.7%	12.4%	13.2%	13.2%	13.3%	13.4%	13.6%
Relative poverty, before housing costs: everyone	17.0%	16.0%	16.4%	16.3%	16.4%	16.3%	16.6%
Relative poverty, before housing costs: children	25.9%	22.8%	23.3%	22.9%	23.0%	22.9%	23.4%
Relative poverty, before housing costs: working-age adults	14.5%	14.1%	14.6%	14.7%	14.7%	14.7%	14.9%
Relative poverty, before housing costs: pensioners	14.5%	14.1%	14.0%	13.8%	13.8%	13.9%	13.9%
Relative poverty, after housing costs: everyone	21.4%	21.0%	21.2%	21.3%	21.3%	21.5%	21.7%
Relative poverty, after housing costs: children	33.0%	31.1%	31.0%	31.0%	31.1%	31.2%	31.8%
Relative poverty, after housing costs: working-age adults	19.8%	19.8%	20.2%	20.4%	20.5%	20.6%	20.8%
Relative poverty, after housing costs: pensioners	12.5%	12.4%	12.2%	11.8%	11.7%	11.8%	12.1%

People in poverty: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Absolute poverty, before housing costs: everyone	11,271k	10,769k	11,393k	11,359k	11,469k	11,507k	11,644k
Absolute poverty, before housing costs: children	3,685k	3,308k	3,490k	3,440k	3,476k	3,498k	3,544k
Absolute poverty, before housing costs: working-age adults	5,962k	5,886k	6,302k	6,357k	6,414k	6,425k	6,515k
Absolute poverty, before housing costs: pensioners	1,624k	1,575k	1,601k	1,562k	1,579k	1,585k	1,584k
Absolute poverty, after housing costs: everyone	14,904k	14,586k	15,282k	15,707k	15,924k	16,128k	16,418k
Absolute poverty, after housing costs: children	4,955k	4,675k	4,820k	4,878k	4,935k	5,030k	5,169k
Absolute poverty, after housing costs: working-age adults	8,443k	8,450k	8,956k	9,336k	9,484k	9,577k	9,703k
Absolute poverty, after housing costs: pensioners	1,506k	1,461k	1,506k	1,494k	1,506k	1,522k	1,546k
Relative poverty, before housing costs: everyone	11,745k	11,082k	11,379k	11,350k	11,408k	11,409k	11,644k
Relative poverty, before housing costs: children	3,870k	3,420k	3,490k	3,438k	3,461k	3,449k	3,544k
Relative poverty, before housing costs: working-age adults	6,155k	6,010k	6,295k	6,352k	6,383k	6,384k	6,515k
Relative poverty, before housing costs: pensioners	1,719k	1,651k	1,594k	1,560k	1,564k	1,576k	1,584k
Relative poverty, after housing costs: everyone	14,835k	14,539k	14,727k	14,798k	14,879k	15,019k	15,237k
Relative poverty, after housing costs: children	4,931k	4,654k	4,652k	4,654k	4,676k	4,705k	4,803k
Relative poverty, after housing costs: working-age adults	8,414k	8,429k	8,682k	8,817k	8,878k	8,972k	9,065k
Relative poverty, after housing costs: pensioners	1,490k	1,456k	1,393k	1,327k	1,325k	1,342k	1,369k

Inequality: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Gini coefficient, equivalised HBAI income before housing costs	0.3078	0.3050	0.3056	0.3055	0.3060	0.3066	0.3082

Mean household net income in each side's own deciles: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Decile 1: mean household_net_income per household	£17,745 †	£17,989 †	£17,921 †	£18,173 †	£18,313 †	£18,534 †	£18,738 †
Decile 2: mean household_net_income per household	£27,982 †	£29,039 †	£28,756 †	£29,017 †	£29,325 †	£29,743 †	£30,188 †
Decile 3: mean household_net_income per household	£34,259 †	£35,407 †	£36,023 †	£36,500 †	£36,969 †	£37,666 †	£38,076 †
Decile 4: mean household_net_income per household	£38,941 †	£40,729 †	£41,727 †	£42,778 †	£43,455 †	£44,144 †	£44,488 †
Decile 5: mean household_net_income per household	£43,234 †	£45,559 †	£46,019 †	£46,917 †	£47,683 †	£48,556 †	£50,244 †
Decile 6: mean household_net_income per household	£47,933 †	£49,241 †	£50,934 †	£51,985 †	£52,964 †	£53,977 †	£54,715 †
Decile 7: mean household_net_income per household	£55,659 †	£56,577 †	£57,445 †	£57,649 †	£58,750 †	£59,668 †	£61,096 †
Decile 8: mean household_net_income per household	£68,786 †	£70,806 †	£71,868 †	£73,516 †	£73,890 †	£75,326 †	£77,181 †
Decile 9: mean household_net_income per household	£78,889 †	£81,012 †	£80,859 †	£82,645 †	£85,463 †	£86,945 †	£89,508 †
Decile 10: mean household_net_income per household	£116,861 †	£120,160 †	£124,545 †	£126,419 †	£125,752 †	£128,367 †	£130,693 †

Standard reform costings (change in gov_balance): head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise basic rate by 1pp	£6.41bn	£6.66bn	£6.63bn	£6.85bn	£7.03bn	£7.25bn	£7.49bn
Raise higher rate by 1pp	£3.61bn	£3.87bn	£4.00bn	£4.22bn	£4.44bn	£4.71bn	£4.83bn
Raise personal allowance by ~800GBP/year	£-3.81bn	£-3.93bn	£-3.99bn	£-4.05bn	£-4.11bn	£-4.24bn	£-1.95bn
Raise child benefit by 25GBP/week per additional child	£-1.99bn	£-1.75bn	£-1.55bn	£-1.38bn	£-1.30bn	£-1.18bn	£-1.07bn
Reduce Universal Credit taper rate to 20%	£-20.77bn	£-21.32bn	£-21.75bn	£-22.00bn	£-22.35bn	£-22.75bn	£-22.94bn
Raise Class 1 main employee NICs rate to 10%	£10.89bn	£11.30bn	£11.71bn	£12.01bn	£12.31bn	£12.62bn	£13.00bn
Raise VAT standard rate by 2pp	£24.89bn	£25.71bn	£26.63bn	£27.24bn	£27.87bn	£28.50bn	£29.14bn
Raise additional rate by 3pp	£2.90bn	£3.15bn	£3.22bn	£3.41bn	£3.61bn	£3.85bn	£4.14bn
Raise the Universal Credit standard allowance by 10%	£-3.06bn	£-3.23bn	£-3.30bn	£-3.39bn	£-3.48bn	£-3.53bn	£-3.55bn
Raise the State Pension by 10%	£-7.76bn	£-7.99bn	£-8.22bn	£-8.44bn	£-8.79bn	£-8.52bn	£-8.86bn

Standard reforms: change in each reform's target: head levels

Measure (head)	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31	2031-32
Raise basic rate by 1pp: change in income_tax	£6.55bn	£6.86bn	£6.79bn	£7.00bn	£7.20bn	£7.44bn	£7.66bn
Raise higher rate by 1pp: change in income_tax	£3.61bn	£3.92bn	£4.01bn	£4.23bn	£4.45bn	£4.72bn	£4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax	£-3.93bn	£-4.07bn	£-4.13bn	£-4.18bn	£-4.22bn	£-4.40bn	£-1.75bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit	£2.33bn	£2.14bn	£1.94bn	£1.76bn	£1.65bn	£1.52bn	£1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit	£20.74bn	£21.31bn	£21.79bn	£22.05bn	£22.37bn	£22.78bn	£22.99bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance	£11.15bn	£11.63bn	£12.00bn	£12.29bn	£12.61bn	£12.94bn	£13.31bn
Raise VAT standard rate by 2pp: change in vat	£24.69bn	£25.51bn	£26.42bn	£27.03bn	£27.65bn	£28.28bn	£28.91bn
Raise additional rate by 3pp: change in income_tax	£2.90bn	£3.15bn	£3.22bn	£3.42bn	£3.62bn	£3.86bn	£4.12bn
Raise the Universal Credit standard allowance by 10%: change in universal_credit	£3.06bn	£3.22bn	£3.32bn	£3.41bn	£3.47bn	£3.53bn	£3.55bn
Raise the State Pension by 10%: change in state_pension	£10.68bn	£11.17bn	£11.28bn	£11.60bn	£12.03bn	£12.46bn	£12.96bn

juaristi22

Incremental review at 595fd3fdf55ba252ed9c6c995e27174e56df52c3: COMPLETE, no findings or material evidence gaps.

The revised design routes microsimulation-marked tests to the last of eight CI shards with one worker, while the other seven shards partition the remaining tests. The single-shard fallback, marker inheritance, partition/selection and outcome-log contracts, workflow worker settings, and duration-table changes were checked together.

146 focused tests passed, with no failures or skips, across CI-sharding, runner-variable and aggregate-workflow health checks. All 22 removed duration entries bind to actual marked tests. The earlier 55 passes describe the superseded design and are not counted as validation of this revision.

No private data, certified aggregate fixtures or model calculations were run. The validation establishes routing/worker contracts; memory figures and performance expectations remain attributed to the author's hosted evidence.

The PR remains Draft with CI pending, so this is a completed review comment, not an approval. The passing code review can be retained while readiness and exact-head CI are checked later.

juaristi22 reviewed Nov 6, 2026

View reviewed changes

MaxGhenis added a commit that referenced this pull request Nov 6, 2026

Merge #2893

c0a0d11

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

MaxGhenis mentioned this pull request Nov 6, 2026

Run CI on 28 pull requests merged together (not for merging) #2887

Draft

MaxGhenis changed the title ~~Run the certified aggregate tests in one CI shard~~ Run the tests that need the dataset in their own CI shard Nov 6, 2026

juaristi22 reviewed Nov 6, 2026

View reviewed changes

Navigation Menu

Repository navigation

Run the tests that need the dataset in their own CI shard - #2893

Run the tests that need the dataset in their own CI shard#2893
MaxGhenis wants to merge 2 commits into
mainfrom
certified-aggregate-one-shard

MaxGhenis commented Nov 6, 2026 •

edited

Loading

Uh oh!

juaristi22 left a comment

Uh oh!

github-actions Bot commented Nov 6, 2026 •

edited

Loading

Uh oh!

juaristi22 left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

MaxGhenis commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

juaristi22 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Aggregate impact on certified data: pass

Notes

Moves flagged for review

Spending by benefit and service

Caseloads

Household net income by decile

Uh oh!

juaristi22 left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MaxGhenis commented Nov 6, 2026 •

edited

Loading

github-actions Bot commented Nov 6, 2026 •

edited

Loading