Skip to content

Guard against random draws in the engine - #2873

Open
MaxGhenis wants to merge 2 commits into
mainfrom
claude/deterministic-engine-guard-o2b5an
Open

MaxGhenis wants to merge 2 commits into
mainfrom
claude/deterministic-engine-guard-o2b5an

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Addresses #2874
axiom: n/a: code-health test only

Why

policyengine-core bans randomness in variable formulas, so the data build has to draw each stochastic input and store it (PolicyEngine/policyengine-core#500). Core's own check looks for numpy and stdlib randomness in formulas, but it misses the splitmix64 hash of entity ids in policyengine_uk/utils/stochastic.py and the seeded numpy draws outside formulas. This PR adds policyengine_uk/tests/code_health/test_no_engine_time_randomness.py, which fails on any new draw and lists today's draws as a ratchet. Later PRs move each draw into the data build and lower the ratchet. This PR changes no model code and no output.

Where the guard comes from

The file ports the scanner of PolicyEngine/policyengine-us#10447 at 221f2fe44ff265b8911810bcc440b3586fa6fc51 (policyengine_us/tests/code_health/test_no_engine_time_randomness.py, git blob 15b6826c0ebecbec84c6b51f128e5f76226e0e47). The scanner, from the constants down to package_scan, and every generic detector test stay as they were. The port changes only:

  • the package name, policyengine_us to policyengine_uk;
  • RATCHET, RATCHET_CEILING, EXEMPT and OFFLINE_MODULES (empty: the scan finds no offline tool that needs one, so test_model_never_imports_offline_modules passes trivially until a module joins it);
  • the tests that named pe-us modules: test_model_api_passes_on_cores_numpy_and_its_own_helpers now checks a pe-uk variable module and add_for_members, which policyengine_uk/model_api.py passes on; test_splitmix64_hash_and_its_uprating_caller_are_draw_helpers replaces the pe-us start-month test; the offline-import tests check an example name of their own, policyengine_uk.tools.offline, since pe-uk has no tools package;
  • the docstring's and comments' pe-us examples, its sentence on how the dataset build draws inputs, and a new limit on the clock;
  • a module-scoped autouse fixture that clears package_scan's cache when the module's tests finish, as test_test_module_caches_released.py requires of every test module;
  • the four pytest.skip calls for cases that need newer syntax. _on_this_python filters those cases as pytest collects them, and two tests whose cases all need Python 3.12 sit under if sys.version_info >= (3, 12), so no Python collects a case it cannot parse and no run reports a skip. Every case stays.

The lists

EXEMPT holds 21 digest occurrences in 9 files. Each computes a SHA-256 for provenance, cache keys, content checks or an equality check, and none decides who gets something or meets a threshold:

File Scopes
build_metadata.py get_data_build_fingerprint
data/capital_gains_projection.py _fingerprint
data/dataset_sources.py _cached_dataset_path
data/economic_assumptions.py uprating_input_fingerprint
utils/aggregate_impact/__main__.py _sha256
utils/aggregate_impact/dataset.py sha256_file
utils/aggregate_impact/measure.py _digest_tool_files, definitions_digest, model_fingerprint, runtime_identity
utils/aggregate_impact/partition.py load_partition
utils/build_lha_published_rates.py fetch

each with its module's hashlib import.

RATCHET holds 21 occurrences in 10 files, so RATCHET_CEILING = 21:

  • utils/stochastic.py: splitmix64_uniform, the three splitmix64 constants and three xorshift steps;
  • uc_deduction_random_draw.py, uc_deduction_type_random_draw.py, warm_home_discount_random_draw.py and months_since_last_birthday.py: one formula call each to splitmix64_uniform;
  • data/economic_assumptions.py: uprate_student_loan_plans hashes person_id and compares the draw with the take-up rate; apply_single_year_uprating, apply_uprating, extend_single_year_dataset and Simulation._extend_dataset_with_cgt_provenance in simulation.py reach it;
  • dynamics/participation.py: a seeded numpy RandomState in apply_participation_responses;
  • scenarios/uc_reform.py: a seeded numpy Generator in add_universal_credit_reform, and the module-level Scenario that passes it on;
  • scenarios/pip_reform.py: np.random.seed and np.random.random in modify_simulation, and the module-level Scenario that passes it on.

The scan over main at 47a2817e2d57e6acbc08b76e09fde89fc27b81fa reads 1,534 modules, cuts no lookup at the depth limit and finds no other draw pattern. Without EXEMPT, the scan reports 82 occurrences in 21 files: the 21 in RATCHET, the 21 digests, and 40 draw helper calls that reach the digests.

Invariants

  • The guard counts every occurrence, per file, enclosing scope and kind.
  • An occurrence beyond its entry fails test_package_draws_nothing_outside_the_allowlists, including a second call in a scope the lists already name and a new function in a listed file.
  • An entry that allows more than the scan finds fails test_allowlist_entries_still_match, and test_ratchet_only_shrinks fails when RATCHET exceeds RATCHET_CEILING. So RATCHET can't grow unless a change also raises RATCHET_CEILING, which the file says never to do.

The guard reports recognised patterns, so it works as a ratchet, not a proof that no hand-written draw exists. It inherits every limit the docstring lists: it does not follow eval or exec, imports of a computed name, values reached through objects or containers, or methods as draw helpers. For example, Simulation calls universal_credit_july_2025_reform.simulation_modifier(self) through an object, which the scan does not follow. The guard does not scan policyengine-core, so it cannot show that nothing downstream reads numpy's global stream. It also looks for draws and hashes, not reads of the clock: four real income variables read inflation_adjustment, which prices to the year datetime.datetime.now() returns (#2875).

Interaction with open work

The file says never to raise RATCHET_CEILING, so none of these PRs can add a RATCHET entry. The guard fails each of them until the PR changes or, for #2862, adds an EXEMPT entry:

Impact

Impact: none; a code-health test only.

Tests

On commit 0835944512f8632f878168bbe1d2eceba951500d:

  • uv run pytest policyengine_uk/tests/code_health/test_no_engine_time_randomness.py -n 4 (Python 3.13.18, locked environment): 382 passed.
  • The guard alone with only pytest 9.2.0, --noconftest, and a plugin that makes _pytest.python.Package.setup a no-op so pytest doesn't import policyengine_uk/__init__.py: Python 3.11.18, 369 passed; 3.12.3, 380 passed; 3.13.18, 382 passed; 3.14.6, 382 passed. None skipped.
  • The guard from the built wheel, installed as .github/actions/test-environment installs it (Python 3.13.18): 382 passed; the scan reads the same 1,534 modules with the same counts.
  • uv run pytest policyengine_uk/tests/code_health -n 4 (Python 3.13.18): 3,544 passed.
  • ruff format --check, ruff check, ruff check --select F811,F821 policyengine_uk and git diff --check pass.

This head differs from that commit only in comments, the docstring, two test rows from the pe-us guard and the changelog fragment, and CI runs the whole guard on it.

🤖 Generated with Claude Code

Port policyengine-us's scanner for engine-time randomness and hash draws
as a code-health test. EXEMPT lists the package's SHA-256 digests, which
fingerprint, key caches or check content; RATCHET lists today's draws:
the splitmix64 helper, the formulas and uprating that call it, and the
seeded numpy draws in participation and two scenarios. Cases that need
newer syntax are filtered at collection, so no Python reports a skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6H1FBndAKeuXdSGuft
@MaxGhenis

MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Head: 0835944512f8632f878168bbe1d2eceba951500d on claude/deterministic-engine-guard-o2b5an (the harness pins this branch name), from main at 47a2817e2d57e6acbc08b76e09fde89fc27b81fa. main had not moved when I opened the PR.

Pre-flight

Comparison with the reference (after formatting)

diff -u shows 13 hunks. Each falls under a listed item, except one:

  1. The docstring (lines 2–15): I dropped "the months a reported annual amount falls in" (a pe-us example) and changed "and tools" to "scenarios, simulation code, … and utilities". Item 4.
  2. Line 61, policyengine_uk.model_api import np as lib. Item 1. pe-uk's model_api star-imports core's, so the example holds.
  3. The lists. Item 2.
  4. Outside the items: _release_package_scan, a module-scoped autouse fixture that calls package_scan.cache_clear() after its yield (just below package_scan). pe-uk's test_test_module_caches_released.py::test_every_test_module_cache_is_released failed without it: every test module must clear its module-level functools caches. The cost is that a worker may rebuild the scan (about 8 s) if it leaves this module and comes back to it.
  5. _on_this_python, plus the re-wrapped parameter lists, the type-alias test and the type-parameter test. Item 5. diff -w shows only the wrapper call, the removed skips and the two if sys.version_info >= (3, 12): blocks.
  6. The mcdonald_rivet comment. pe-uk has no package whose __init__ rebinds a subpackage's name (I checked every __init__.py), so I dropped the parenthetical rather than invent an example.
  7. test_model_api_passes_on_cores_numpy_and_its_own_helpers now uses policyengine_uk.variables.gov.dwp.tax_credits_current_year_income (it star-imports model_api and calls add_for_members) and checks for policyengine_uk.utils.benefit_unit.add_for_members.
  8. test_splitmix64_hash_and_its_uprating_caller_are_draw_helpers replaces the start-month test. It checks the three names the brief gives.
  9. The offline tests: _is_offline(name, offline=OFFLINE_MODULES), plus EXAMPLE_OFFLINE_MODULES = {"policyengine_uk.tools.offline"}, and the import-form cases rewritten from taxsim to offline. OFFLINE_MODULES = set(), so test_model_never_imports_offline_modules passes on an empty set. I kept it so the check is ready if a later change adds an offline module.

The scanner from the constants down to package_scan is unchanged, apart from the package name.

Lists, reconciled

My run on main at 47a2817e: 1,534 modules, no lookup cut at the depth limit. It reports exactly the brief's 21 EXEMPT occurrences in 9 files and 21 RATCHET occurrences in 10 files, entry by entry, so RATCHET_CEILING = 21. The scan finds nothing else.

I read every function behind the digests. All are SHA-256 used for provenance, a cache key or a content check, and none decides who gets something or meets a threshold. The two that the code compares are:

  • uprating_input_fingerprint (data/economic_assumptions.py:302): simulation.py:1101 and :1168 compare it with != to decide whether a cached projection still applies.
  • fetch (utils/build_lha_published_rates.py:102) and verify_file (utils/aggregate_impact/dataset.py:131): both compare a digest with an expected value and raise if they differ.

The ratchet draws I read:

  • utils/stochastic.py:19–22: three constants and three xorshift steps.
  • The four formulas:
    • uc_deduction_random_draw.py:28 (salt 0)
    • uc_deduction_type_random_draw.py:27 (salt 1)
    • warm_home_discount_random_draw.py:29 (household_id, salt 2)
    • months_since_last_birthday.py:45 (person_id, salt 2)
  • data/economic_assumptions.py:929, salt 3, with draws < prob. Its callers are at lines 519, 389 and 363, and at simulation.py:951.
  • dynamics/participation.py:599: np.random.RandomState.
  • scenarios/uc_reform.py:311: default_rng(43), compared with new_claimant_share. The module-level Scenario is at line 358.
  • scenarios/pip_reform.py:7–8: np.random.seed(42) and np.random.random. The Scenario is at line 23.

One possible difference from the brief: without EXEMPT, my run reports 82 occurrences in 21 files, not 80. That count covers every occurrence, including the 21 that RATCHET covers; 61 occurrences in 13 files go beyond RATCHET. I couldn't reproduce 80 under any count.

Observations, not acted on:

  • warm_home_discount_random_draw and months_since_last_birthday both use salt 2. They hash different ids (household_id and person_id).
  • simulation.py:416 calls universal_credit_july_2025_reform.simulation_modifier(self). That is a call through an object, which the scan doesn't follow. The PR body names it as an example of the limits.

Star imports in a possible cycle

The scanner's own graph shows 9 of 1,432 star imports (1,418 of them from package modules) in a possible cycle:

  • 3 importers of model_api
  • entities
  • the system imports in tax_benefit_system, microsimulation and simulation
  • reforms.policyengine.disable_simulated_benefits
  • reforms.conservatives.household_based_hitc

policyengine_uk/__init__.py imports system while it runs. The lazy imports in modules that run during that import put them in a cycle:

  • data/economic_assumptions.py:237, 375, 660: from policyengine_uk.system import system inside functions.
  • simulation.py:416, 488 and utils/scenario.py:224, 234, 338, 739: scenarios/scenarios.uc_reform inside functions.
  • reforms/reforms.py: code there that may run later reaches reforms.policyengine.

Tests (all on 0835944)

  • uv sync --frozen --extra dev, then uv run pytest policyengine_uk/tests/code_health/test_no_engine_time_randomness.py -n 4 (Python 3.13.18): 382 passed.

  • Only pytest 9.2.0, --noconftest -p no:cacheprovider:

    Python Collected Passed Skipped Wall time Peak RSS
    3.11.18 369 369 0 18.8 s 56 MB
    3.12.3 380 380 0 22.7 s 59 MB
    3.13.18 382 382 0 20.3 s 59 MB
    3.14.6 382 382 0 18.8 s 65 MB

    3.11 leaves out the 13 cases that need 3.12 or 3.13 syntax; 3.12 leaves out the 2 that need 3.13.

    Caveat: the brief's bare uvx … pytest <file> --noconftest gives collection errors on every Python. pytest 9 sets up the policyengine_uk Package node above the file, which imports policyengine_uk/__init__.py, and that needs policyengine-core. I ran instead through a scratch plugin (-p skip_package_init) that turns _pytest.python.Package.setup into a no-op. The plugin is not committed. The guard itself imports only the standard library and pytest.

  • The guard file in the locked environment, serial, with conftest: 28.8 s wall, 316 MB peak RSS. Most of that memory comes from the conftest importing the model. The longest single call is the first package_scan (about 7 s; the scan alone takes about 7.8 s for 1,534 modules on 3.13).

  • The wheel: uv build --wheel (sha256 48570c87…), locked environment with --no-install-project, then uv pip install --no-deps the wheel, run from an empty directory against the installed tests directory: 382 passed. The scan reads the same 1,534 modules with identical counts and lines.

  • uv run pytest policyengine_uk/tests/code_health -n 4: 3,544 passed. Before I added the fixture, it gave 1 failed (test_every_test_module_cache_is_released) and 3,543 passed.

  • ruff format --check ., ruff check ., ruff check --select F811,F821 policyengine_uk (all ruff 0.17.2) and git diff --check pass.

Mutation checks (scratch copies; all killed)

  1. import random added to warm_home_discount_random_draw.py: killed. stochastic import (1 found, 0 allowed).
  2. A second splitmix64_uniform call in warm_home_discount_random_draw.formula: killed. draw helper call (2 found, 1 allowed).
  3. A new new_draws in data/economic_assumptions.py that calls splitmix64_uniform: killed. new_draws: draw helper call (1 found, 0 allowed).
  4. One ratchet occurrence removed (the module-level Scenario in pip_reform.py replaced): killed as stale by test_allowlist_entries_still_match.
  5. The xorshift entry raised from 3 to 4, which takes RATCHET to 22 against a ceiling of 21: killed by test_ratchet_only_shrinks and as stale.
  6. The _fingerprint EXEMPT entry removed: killed. The hash use and 7 draw helper calls spread through capital_gains_projection.py, simulation.py and tax_benefit_system.py.

Open work, checked

I ran the guard over each PR's head:

None has merged.

Judgment calls

  • No changelog fragment. check_changelog_fragments.py only warns when a PR adds none, a fragment would trigger a release for a tests-only change, and recent tests-only commits on main (abab81f8b, c78f88649, 8199bc05e, c9397c23e) add none. github-prs.md says every PR needs one, so the hub may want changelog.d/deterministic-engine-guard.changed.md.
  • I added the cache-release fixture (hunk 4 above) to meet pe-uk's own code-health rule.
  • The PR body has no Fixes line, because no issue exists for this work.

CI on 0835944 (checked 05:57Z)

All seven Test / Python shards, Test / YAML and Test / Result pass, so the strict outcome check found no new skip, xfail or XPASS. Changelog fragments, Changes, Documentation build, Fix evidence, Lint, the four install and smoke-import jobs and Release build also pass. Aggregate impact was still running at 05:57Z, after 73 minutes. No push was needed.

Skipped

Nothing needs private data. I didn't run the full suite; CI covers it.


Generated by Claude Code

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Aggregate impact on certified data: pass

Base 47a2817e2d against head 4b78867754, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

  • In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Caseloads

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target.

Checks failing on both base and head (1)
  • VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-53,888,684 in total; 2026-27: 76 units below zero, a weighted £-55,777,260 in total; 2027-28: 76 units below zero, a weighted £-57,727,917 in total
Counted-once check

Each of 92 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.

  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)
Provenance
Side Commit Runtime Environment hash Partition hash Time
base main 47a2817e2d uk 2.175.2 (files 97879027d30c), core 3.34.1, py 3.13.18 aarch64 f8b95f5110a2 3bcfb3473960 2477.07s
head #2873 merged into main 4b78867754 uk 2.175.2 (files 97879027d30c), core 3.34.1, py 3.13.18 aarch64 f8b95f5110a2 3bcfb3473960 2498.6s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Final charge on cash interest in non-cash ISAs £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Lifetime ISA withdrawal charge £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Income tax £294.46bn £315.23bn £335.82bn £351.28bn £366.96bn £385.36bn £399.99bn
National Insurance (employee and self-employed) £51.70bn £54.14bn £56.13bn £57.73bn £59.76bn £61.64bn £63.51bn
National Insurance (employer) £148.71bn £155.37bn £160.88bn £165.09bn £172.06bn £177.36bn £182.57bn
Employer NI change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Employer NI change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Capital gains tax £12.60bn £13.15bn £13.60bn £14.11bn £14.67bn £15.24bn £15.80bn
Student loan repayments £7.82bn £8.13bn £8.39bn £8.58bn £8.79bn £9.04bn £9.29bn
VAT excluding private school fees £249.37bn £256.87bn £266.33bn £272.70bn £278.97bn £285.24bn £291.59bn
Change in VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
VAT on private school fees £1.95bn £2.01bn £2.09bn £2.14bn £2.20bn £2.26bn £2.31bn
Change in private school VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Fuel duties £24.71bn £24.13bn £26.00bn £26.83bn £26.51bn £25.78bn £24.63bn
Alcohol duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tobacco duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Vehicle excise duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Betting and gaming duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
TV licence fee £4.26bn £4.40bn £4.41bn £4.42bn £4.43bn £4.44bn £4.45bn
Council tax £49.21bn £52.16bn £54.89bn £57.95bn £61.16bn £64.47bn £67.91bn
High value council tax surcharge £0.00bn £0.00bn £0.00bn £0.48bn £0.49bn £0.50bn £0.51bn
Northern Ireland domestic rates £0.55bn £0.58bn £0.60bn £0.63bn £0.66bn £0.69bn £0.72bn
Business rates £34.68bn £36.20bn £38.00bn £39.46bn £42.27bn £43.25bn £44.29bn
Stamp duty land tax on the household's own purchases £6.88bn £7.32bn £7.66bn £8.06bn £8.47bn £8.91bn £9.35bn
Stamp duty land tax paid by companies £4.72bn £4.57bn £5.29bn £6.03bn £6.37bn £6.92bn £7.45bn
Land transaction tax (Wales) £0.07bn £0.08bn £0.08bn £0.09bn £0.10bn £0.10bn £0.11bn
Land and buildings transaction tax (Scotland) £0.31bn £0.32bn £0.33bn £0.35bn £0.36bn £0.38bn £0.40bn
Unspecified tax change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Unspecified tax change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax excluding the main residence £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Land value tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Carbon tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Spending by benefit and service: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
State Pension £121.68bn £125.92bn £126.72bn £129.48bn £132.98bn £136.65bn £140.41bn
Pension Credit £6.21bn £6.46bn £6.68bn £6.82bn £6.89bn £7.07bn £7.22bn
Winter Fuel Payment £1.86bn £1.85bn £1.79bn £1.77bn £1.78bn £1.78bn £1.78bn
Pension Age Winter Heating Payment (Scotland) £0.19bn £0.20bn £0.20bn £0.21bn £0.21bn £0.22bn £0.22bn
Universal Credit £75.48bn £80.52bn £82.95bn £84.96bn £86.11bn £87.61bn £88.21bn
Housing Benefit £8.56bn £8.89bn £9.23bn £9.46bn £9.69bn £9.99bn £10.27bn
Council Tax Reduction £2.07bn £2.20bn £2.32bn £2.45bn £2.59bn £2.73bn £2.87bn
Employment and Support Allowance (income-related) £0.79bn £0.82bn £0.84bn £0.86bn £0.88bn £0.90bn £0.92bn
Employment and Support Allowance (contribution-based) £5.20bn £5.37bn £5.56bn £5.69bn £5.82bn £5.95bn £6.09bn
Jobseeker's Allowance (income-based) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Jobseeker's Allowance (contribution-based) £0.29bn £0.30bn £0.31bn £0.31bn £0.32bn £0.33bn £0.34bn
Income Support £0.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Working Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Incapacity Benefit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Personal Independence Payment £26.95bn £28.03bn £29.05bn £30.09bn £30.75bn £31.51bn £32.21bn
Disability Living Allowance £7.49bn £7.79bn £8.07bn £8.36bn £8.55bn £8.76bn £8.95bn
Attendance Allowance £8.68bn £9.03bn £9.36bn £9.69bn £9.90bn £10.15bn £10.38bn
Carer's Allowance £4.48bn £4.66bn £4.83bn £5.00bn £5.11bn £5.27bn £5.42bn
Carer Support Payment (Scotland) £0.31bn £0.32bn £0.33bn £0.34bn £0.35bn £0.36bn £0.37bn
Carer's Allowance Supplement (Scotland) £0.00bn £0.04bn £0.04bn £0.05bn £0.05bn £0.05bn £0.05bn
Severe Disablement Allowance £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn
Industrial Injuries Disablement Benefit £0.50bn £0.52bn £0.54bn £0.55bn £0.56bn £0.58bn £0.59bn
Armed Forces Compensation Scheme £0.61bn £0.63bn £0.65bn £0.66bn £0.68bn £0.69bn £0.71bn
War Widow's or Widower's Pension £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Benefit £16.19bn £16.82bn £17.39bn £18.00bn £18.39bn £18.84bn £19.25bn
Scottish Child Payment £0.49bn £0.52bn £0.53bn £0.55bn £0.56bn £0.57bn £0.56bn
Two Child Limit Payment (Scotland, counterfactual) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Healthy Start vouchers £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn
Maternity Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Maternity Pay £3.30bn £3.41bn £3.54bn £3.62bn £3.70bn £3.78bn £3.87bn
Statutory Paternity Pay £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Sick Pay £1.12bn £1.16bn £1.20bn £1.23bn £1.26bn £1.29bn £1.32bn
Sure Start Maternity Grant £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn
Bereavement Support Payment £0.67bn £0.69bn £0.71bn £0.73bn £0.75bn £0.76bn £0.78bn
Widowed Parent's Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tax-Free Childcare £0.56bn £0.57bn £0.57bn £0.58bn £0.60bn £0.61bn £0.59bn
Extended childcare entitlement £2.99bn £3.50bn £3.59bn £3.71bn £3.78bn £3.86bn £3.93bn
Universal childcare entitlement £2.06bn £2.22bn £2.29bn £2.37bn £2.43bn £2.48bn £2.53bn
Targeted childcare entitlement £0.50bn £0.51bn £0.52bn £0.54bn £0.55bn £0.57bn £0.56bn
Childcare Offer for Wales £0.26bn £0.28bn £0.28bn £0.29bn £0.30bn £0.31bn £0.32bn
Care to Learn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Childcare Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Parents' Learning Allowance £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn
Adult Dependants' Grant £0.06bn £0.06bn £0.04bn £0.04bn £0.04bn £0.04bn £0.04bn
Travel Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
NHS spending £203.89bn £204.30bn £204.56bn £204.91bn £205.34bn £205.85bn £206.35bn
State education spending £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn
Transport subsidy spending £25.05bn £25.10bn £25.14bn £25.18bn £25.23bn £25.30bn £25.36bn
Cost-of-living support payment £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Bills Rebate £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Household Energy Payment £0.00bn £2.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Price Guarantee subsidy £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disability basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Other public spending budget change £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Caseloads: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus 0k 0k 0k 0k 0k 0k 0k
State Pension 11,264k 11,145k 10,819k 10,700k 10,723k 10,750k 10,776k
Pension Credit 1,489k 1,484k 1,476k 1,454k 1,452k 1,457k 1,466k
Winter Fuel Payment 7,844k 7,783k 7,473k 7,375k 7,391k 7,409k 7,427k
Pension Age Winter Heating Payment (Scotland) 803k 802k 800k 797k 799k 801k 802k
Universal Credit 6,342k 6,347k 6,336k 6,297k 6,292k 6,271k 6,223k
Housing Benefit 1,249k 1,245k 1,244k 1,227k 1,227k 1,229k 1,232k
Council Tax Reduction 3,180k 3,200k 3,206k 3,219k 3,234k 3,245k 3,252k
Employment and Support Allowance (income-related) 71k 71k 71k 71k 71k 71k 71k
Employment and Support Allowance (contribution-based) 719k 720k 721k 722k 724k 726k 728k
Jobseeker's Allowance (income-based) 0k 0k 0k 0k 0k 0k 0k
Jobseeker's Allowance (contribution-based) 67k 67k 67k 67k 67k 68k 68k
Income Support 25k 0k 0k 0k 0k 0k 0k
Working Tax Credit 0k 0k 0k 0k 0k 0k 0k
Child Tax Credit 0k 0k 0k 0k 0k 0k 0k
Incapacity Benefit 0k 0k 0k 0k 0k 0k 0k
Personal Independence Payment 3,567k 3,575k 3,579k 3,585k 3,593k 3,602k 3,611k
Disability Living Allowance 1,291k 1,294k 1,296k 1,298k 1,300k 1,304k 1,307k
Attendance Allowance 1,695k 1,698k 1,700k 1,703k 1,707k 1,711k 1,715k
Carer's Allowance 1,034k 1,037k 1,038k 1,040k 1,042k 1,060k 1,062k
Carer Support Payment (Scotland) 71k 71k 71k 72k 72k 72k 72k
Carer's Allowance Supplement (Scotland) 0k 71k 71k 72k 72k 72k 72k
Severe Disablement Allowance 4k 4k 4k 4k 4k 4k 4k
Industrial Injuries Disablement Benefit 137k 137k 137k 138k 138k 138k 139k
Armed Forces Compensation Scheme 93k 93k 94k 94k 94k 94k 94k
War Widow's or Widower's Pension 0k 0k 0k 0k 0k 0k 0k
Child Benefit 8,123k 8,120k 8,120k 8,130k 8,145k 8,161k 8,178k
Scottish Child Payment 351k 352k 352k 353k 350k 348k 337k
Two Child Limit Payment (Scotland, counterfactual) 0k 0k 0k 0k 0k 0k 0k
Healthy Start vouchers 200k 200k 200k 201k 201k 201k 202k
Maternity Allowance 0k 0k 0k 0k 0k 0k 0k
Statutory Maternity Pay 333k 333k 334k 334k 335k 336k 337k
Statutory Paternity Pay 0k 0k 0k 0k 0k 0k 0k
Statutory Sick Pay 181k 182k 182k 182k 182k 183k 183k
Sure Start Maternity Grant 26k 26k 26k 26k 26k 26k 26k
Bereavement Support Payment 98k 98k 98k 98k 98k 99k 99k
Widowed Parent's Allowance 0k 0k 0k 0k 0k 0k 0k
Tax-Free Childcare 1,043k 1,027k 1,014k 1,009k 1,011k 1,009k 993k
Extended childcare entitlement 795k 800k 796k 797k 798k 798k 792k
Universal childcare entitlement 1,071k 1,072k 1,072k 1,073k 1,076k 1,079k 1,074k
Targeted childcare entitlement 165k 156k 154k 154k 155k 155k 150k
Childcare Offer for Wales 39k 39k 39k 39k 39k 39k 39k
Care to Learn 0k 0k 0k 0k 0k 0k 0k
Childcare Grant 0k 0k 0k 0k 0k 0k 0k
Parents' Learning Allowance 98k 98k 98k 99k 99k 99k 99k
Adult Dependants' Grant 18k 18k 12k 12k 12k 12k 12k
Travel Grant 0k 0k 0k 0k 0k 0k 0k
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund 0k 0k 0k 0k 0k 0k 0k
Cost-of-living support payment 0k 0k 0k 0k 0k 0k 0k
Energy Bills Rebate 0k 0k 0k 0k 0k 0k 0k
Household Energy Payment 0k 21,065k 0k 0k 0k 0k 0k
Energy Price Guarantee subsidy 0k 0k 0k 0k 0k 0k 0k
Basic income 0k 0k 0k 0k 0k 0k 0k
Disability basic income 0k 0k 0k 0k 0k 0k 0k
Government balance, each component counted once: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Government receipts, each component once £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.89bn
Government spending £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
Government balance, each receipt once £363.07bn † £389.57bn † £430.34bn † £457.08bn † £487.75bn † £516.37bn † £542.11bn †
Benefits in household_benefits but not in gov_spending £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Model totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
gov_tax £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.89bn
household_tax £491.97bn £520.41bn £551.21bn £576.00bn £601.00bn £626.69bn £648.42bn
pre_budget_change_household_tax £484.15bn £512.28bn £542.82bn £566.95bn £591.72bn £617.16bn £638.61bn
gov_spending £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
household_benefits £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
gov_balance £363.07bn † £389.57bn † £430.34bn † £457.08bn † £487.75bn † £516.37bn † £542.11bn †
Household income totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Household market income £1,637.96bn £1,702.77bn £1,757.35bn £1,806.41bn £1,855.97bn £1,912.43bn £1,970.44bn
Household net income £1,636.24bn † £1,687.25bn † £1,714.85bn † £1,746.83bn † £1,777.98bn † £1,816.26bn † £1,858.87bn †
Household net income (HBAI definition) £1,449.13bn £1,499.27bn £1,530.25bn £1,564.51bn £1,597.18bn £1,634.83bn £1,675.80bn
Population: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
People 69,245k 69,383k 69,473k 69,592k 69,738k 69,912k 70,080k
Benefit units 35,110k 35,181k 35,226k 35,286k 35,360k 35,449k 35,534k
Households 31,046k 31,108k 31,149k 31,201k 31,267k 31,345k 31,420k
Poverty rates: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 16.3% 15.5% 16.4% 16.3% 16.5% 16.5% 16.6%
Absolute poverty, before housing costs: children 24.7% 22.1% 23.3% 22.9% 23.1% 23.2% 23.5%
Absolute poverty, before housing costs: working-age adults 14.1% 13.8% 14.7% 14.7% 14.8% 14.8% 15.0%
Absolute poverty, before housing costs: pensioners 13.7% 13.4% 14.0% 13.8% 13.9% 14.0% 13.9%
Absolute poverty, after housing costs: everyone 21.6% 21.0% 22.0% 22.6% 22.9% 23.1% 23.4%
Absolute poverty, after housing costs: children 33.3% 31.3% 32.3% 32.5% 32.9% 33.4% 34.2%
Absolute poverty, after housing costs: working-age adults 19.9% 19.8% 20.8% 21.6% 21.9% 22.0% 22.3%
Absolute poverty, after housing costs: pensioners 12.7% 12.4% 13.1% 13.2% 13.3% 13.4% 13.6%
Relative poverty, before housing costs: everyone 17.0% 16.0% 16.4% 16.3% 16.4% 16.3% 16.6%
Relative poverty, before housing costs: children 25.9% 22.9% 23.3% 22.9% 23.0% 22.9% 23.5%
Relative poverty, before housing costs: working-age adults 14.5% 14.1% 14.6% 14.7% 14.7% 14.7% 15.0%
Relative poverty, before housing costs: pensioners 14.5% 14.0% 13.9% 13.8% 13.8% 13.9% 13.9%
Relative poverty, after housing costs: everyone 21.5% 21.0% 21.3% 21.3% 21.3% 21.5% 21.7%
Relative poverty, after housing costs: children 33.1% 31.1% 31.2% 31.0% 31.1% 31.2% 31.8%
Relative poverty, after housing costs: working-age adults 19.9% 19.8% 20.2% 20.4% 20.5% 20.6% 20.8%
Relative poverty, after housing costs: pensioners 12.5% 12.4% 12.2% 11.7% 11.7% 11.8% 12.0%
People in poverty: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 11,277k 10,778k 11,402k 11,365k 11,475k 11,514k 11,650k
Absolute poverty, before housing costs: children 3,688k 3,314k 3,496k 3,445k 3,480k 3,502k 3,549k
Absolute poverty, before housing costs: working-age adults 5,965k 5,893k 6,308k 6,362k 6,420k 6,430k 6,521k
Absolute poverty, before housing costs: pensioners 1,624k 1,571k 1,597k 1,558k 1,575k 1,581k 1,580k
Absolute poverty, after housing costs: everyone 14,944k 14,599k 15,310k 15,721k 15,937k 16,140k 16,426k
Absolute poverty, after housing costs: children 4,975k 4,682k 4,837k 4,888k 4,945k 5,040k 5,179k
Absolute poverty, after housing costs: working-age adults 8,463k 8,461k 8,971k 9,344k 9,491k 9,584k 9,706k
Absolute poverty, after housing costs: pensioners 1,506k 1,456k 1,501k 1,489k 1,501k 1,517k 1,541k
Relative poverty, before housing costs: everyone 11,750k 11,084k 11,387k 11,364k 11,416k 11,416k 11,654k
Relative poverty, before housing costs: children 3,872k 3,423k 3,496k 3,445k 3,466k 3,454k 3,551k
Relative poverty, before housing costs: working-age adults 6,159k 6,013k 6,301k 6,362k 6,389k 6,389k 6,523k
Relative poverty, before housing costs: pensioners 1,719k 1,648k 1,590k 1,558k 1,561k 1,573k 1,580k
Relative poverty, after housing costs: everyone 14,864k 14,552k 14,777k 14,809k 14,872k 15,012k 15,232k
Relative poverty, after housing costs: children 4,944k 4,660k 4,673k 4,663k 4,679k 4,707k 4,806k
Relative poverty, after housing costs: working-age adults 8,431k 8,440k 8,711k 8,823k 8,873k 8,967k 9,062k
Relative poverty, after housing costs: pensioners 1,489k 1,452k 1,393k 1,323k 1,321k 1,338k 1,364k
Inequality: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Gini coefficient, equivalised HBAI income before housing costs 0.3078 0.3050 0.3056 0.3055 0.3060 0.3066 0.3082
Mean household net income in each side's own deciles: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Decile 1: mean household_net_income per household £17,681 † £17,998 † £17,916 † £18,152 † £18,308 † £18,533 † £18,718 †
Decile 2: mean household_net_income per household £27,942 † £29,026 † £28,715 † £29,017 † £29,280 † £29,698 † £30,156 †
Decile 3: mean household_net_income per household £34,316 † £35,481 † £36,069 † £36,606 † £37,195 † £37,732 † £38,182 †
Decile 4: mean household_net_income per household £38,934 † £40,684 † £41,726 † £42,728 † £43,331 † £44,130 † £44,432 †
Decile 5: mean household_net_income per household £43,235 † £45,581 † £46,012 † £46,961 † £47,713 † £48,725 † £50,296 †
Decile 6: mean household_net_income per household £47,931 † £49,202 † £50,891 † £51,936 † £52,936 † £53,820 † £54,663 †
Decile 7: mean household_net_income per household £55,659 † £56,569 † £57,437 † £57,642 † £58,733 † £59,661 † £61,087 †
Decile 8: mean household_net_income per household £68,784 † £70,802 † £71,864 † £73,512 † £73,886 † £75,319 † £77,177 †
Decile 9: mean household_net_income per household £78,889 † £81,012 † £80,859 † £82,645 † £85,463 † £86,945 † £89,508 †
Decile 10: mean household_net_income per household £116,861 † £120,161 † £124,545 † £126,420 † £125,753 † £128,368 † £130,693 †
Standard reform costings (change in gov_balance): head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp £6.41bn £6.66bn £6.63bn £6.85bn £7.03bn £7.25bn £7.49bn
Raise higher rate by 1pp £3.61bn £3.87bn £4.00bn £4.22bn £4.44bn £4.71bn £4.83bn
Raise personal allowance by ~800GBP/year £-3.81bn £-3.93bn £-3.99bn £-4.05bn £-4.11bn £-4.24bn £-1.95bn
Raise child benefit by 25GBP/week per additional child £-1.99bn £-1.75bn £-1.55bn £-1.38bn £-1.30bn £-1.18bn £-1.07bn
Reduce Universal Credit taper rate to 20% £-20.77bn £-21.32bn £-21.75bn £-22.00bn £-22.35bn £-22.75bn £-22.94bn
Raise Class 1 main employee NICs rate to 10% £10.89bn £11.30bn £11.71bn £12.01bn £12.31bn £12.62bn £13.00bn
Raise VAT standard rate by 2pp £24.89bn £25.71bn £26.63bn £27.24bn £27.87bn £28.50bn £29.14bn
Raise additional rate by 3pp £2.90bn £3.15bn £3.22bn £3.41bn £3.61bn £3.85bn £4.14bn
Raise the Universal Credit standard allowance by 10% £-3.06bn £-3.18bn £-3.26bn £-3.35bn £-3.44bn £-3.49bn £-3.50bn
Raise the State Pension by 10% £-7.76bn £-7.99bn £-8.22bn £-8.45bn £-8.80bn £-8.53bn £-8.87bn
Standard reforms: change in each reform's target: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp: change in income_tax £6.55bn £6.86bn £6.79bn £7.00bn £7.20bn £7.44bn £7.66bn
Raise higher rate by 1pp: change in income_tax £3.61bn £3.92bn £4.01bn £4.23bn £4.45bn £4.72bn £4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax £-3.93bn £-4.07bn £-4.13bn £-4.18bn £-4.22bn £-4.40bn £-1.75bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit £2.33bn £2.14bn £1.94bn £1.76bn £1.65bn £1.52bn £1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit £20.74bn £21.31bn £21.79bn £22.05bn £22.37bn £22.78bn £22.99bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance £11.15bn £11.63bn £12.00bn £12.29bn £12.61bn £12.94bn £13.31bn
Raise VAT standard rate by 2pp: change in vat £24.69bn £25.51bn £26.42bn £27.03bn £27.65bn £28.28bn £28.91bn
Raise additional rate by 3pp: change in income_tax £2.90bn £3.15bn £3.22bn £3.42bn £3.62bn £3.86bn £4.12bn
Raise the Universal Credit standard allowance by 10%: change in universal_credit £3.06bn £3.19bn £3.28bn £3.37bn £3.43bn £3.49bn £3.51bn
Raise the State Pension by 10%: change in state_pension £10.68bn £11.17bn £11.28bn £11.60bn £12.03bn £12.46bn £12.96bn

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Verdict: REQUEST_CHANGES at 0835944512f8632f878168bbe1d2eceba951500d

The port is faithful, every test passes on every Python, and the lists match the scan entry by entry. Three things need to change before merge: a model output that differs between runs and that neither the lists nor the stated limits cover, a missing changelog fragment whose cited precedent doesn't hold, and advice to four open PRs that the ratchet's own ceiling rules out. None is P1: CI is green and no model output changes.

All runs below used a detached scratch worktree of the head, with HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset.

Findings

P2-1. Formulas read the clock, and neither the lists nor the stated limits say so

policyengine_uk/variables/household/income/inflation_adjustment.py:16 and :38 call cpi(datetime.datetime.now().strftime("%Y-01-01")) inside the formulas of inflation_adjustment and inflation_adjustment_ahc. Four formulas multiply by inflation_adjustment: real_household_net_income, real_household_net_income_ahc and real_hbai_household_net_income (hbai_household_net_income.py:132-134), and real_hbai_household_net_income_ahc (hbai_household_net_income_ahc.py:44-46). The same inputs therefore give different outputs in 2026 and 2027, which breaks core#500's rule that a formula is a pure function of its inputs.

$ grep -rnE "datetime\.now|date\.today|time\.time" --include=*.py policyengine_uk | grep -v tests/
variables/household/income/inflation_adjustment.py:16:        now_cpi = cpi(datetime.datetime.now().strftime("%Y-01-01"))
variables/household/income/inflation_adjustment.py:38:        now_cpi_ahc = cpi(datetime.datetime.now().strftime("%Y-01-01"))
(plus labels in three variable files, and timings in utils/aggregate_impact/measure.py, which don't reach an output)

A mutation confirms the guard can't see this. A new formula returning datetime.datetime.now().microsecond survived: all 20 package tests passed. The guard looks for draws, so it isn't wrong to miss this. But the body says the scan "finds nothing else", and the docstring's list of limits doesn't mention the clock, so later PRs that rely on the guard will take the engine to be deterministic apart from RATCHET.

Replacement text:

  • Body, ## The lists: replace "cuts no lookup at the depth limit and finds nothing else." with "cuts no lookup at the depth limit and finds no other draw pattern."
  • Body, the Limits: paragraph, add at the end: "The guard looks for draws and hashes, not for reads of the clock: inflation_adjustment and inflation_adjustment_ahc (variables/household/income/inflation_adjustment.py:16 and :38) price to the year datetime.datetime.now() returns, so real_household_net_income, real_household_net_income_ahc, real_hbai_household_net_income and real_hbai_household_net_income_ahc change with the year a simulation runs."
  • Docstring, after "or methods as draw helpers.", add: "It looks for draws, not for other reads that differ between runs, such as the clock: inflation_adjustment reads the year from datetime.datetime.now()." (This is a pe-uk example, so it falls under the brief's item 4.)

P2-2. The PR needs a changelog fragment, and the hand-off's precedent says the opposite of what it shows

docs/engineering/skills/github-prs.md:20 reads: "Every PR must include a Towncrier changelog fragment under changelog.d/, except for a dependency-only update". The hand-off cites four tests-only commits without a fragment. Each one is an intermediate commit of a PR whose merge did add a fragment, and one of those PRs is itself tests-only:

$ for c in abab81f8b c78f88649 8199bc05e c9397c23e; do m=$(git log --merges --ancestry-path --format=%h $c..47a2817e2 | tail -1); git log -1 --format="$c -> %h %s" $m; git diff --stat $m^1 $m -- changelog.d | head -1; done
abab81f8b -> ed3342c5 Merge pull request #2589 ...   changelog.d/isa-current-law.fixed.md | 1 +
c78f88649 -> f153a60f Merge pull request #2598 ...   changelog.d/2597.changed.md | 1 +
8199bc05e -> 7e7b1452 Merge pull request #2555 ...   changelog.d/remove-core-reform-api-tests.changed.md | 1 +
c9397c23e -> 7e7b1452 (same PR)
$ git diff --stat 7e7b1452^1 7e7b1452   # #2555: three test files and the fragment, nothing else

.github/check_changelog_fragments.py only warns when a PR adds no fragment, which is why Changelog fragments passed. A changed fragment merged to main starts .github/workflows/versioning.yaml (on.push.paths: changelog.d/**). That workflow bumps the patch version (bump_version.py: changed is neither in MAJOR_TYPES nor MINOR_TYPES), builds the wheel, runs the suite from it (this guard ships in the wheel's tests/) and publishes. It does not trigger Fix evidence, which looks for fixed fragments only.

Add changelog.d/deterministic-engine-guard.changed.md:

The test suite fails on any new random draw, sampling call or hash of entity ids in the package, and lists today's draws as a ratchet that only falls.

P2-3. Three of the four open PRs can't "carry their own entry"

The body says: "The guard fails each of these PRs until it carries its own entry or changes." test_ratchet_only_shrinks fails if RATCHET exceeds RATCHET_CEILING = 21, and the docstring says never to raise the ceiling. So #2618, #2265 and #2308 can't add a RATCHET entry, and EXEMPT is for digests that decide nothing. Only #2862 can pass with an entry.

I merged the guard's head into each spot-checked head (git merge-tree --write-tree, clean) and ran the package tests:

#2618 (47ee0515b + guard): 1 failed, 19 passed
  household_wealth_decile.py:8,9,10 in _fixed_hash: hash constant (3 found, 0 allowed)
  household_wealth_decile.py:11,9,10 in _fixed_hash: xorshift (3 found, 0 allowed)
  household_wealth_decile.py:35 in household_wealth_decile.formula: draw helper call (1 found, 0 allowed)
  (no stale entries once main's warm_home_discount_random_draw.py is present)
#2862 (3886de2d1 + guard): 1 failed, 19 passed; 9 occurrences in simulation.py and utils/prepared_baseline.py
#2862 + EXEMPT {"utils/prepared_baseline.py": {("<module>", "hash library"): 1, ("_core_fingerprint", "hash use"): 1}}: 20 passed

_fixed_hash in #2618 orders households with equal wealth, so it decides which decile a tied household falls in: a draw by the brief's test. _core_fingerprint in #2862 identifies policyengine-core's code and decides nothing. The failure message ("Draw in the dataset build and store the draw as an input variable") tells the authors of #2618, #2265 and #2308 what to do. It misleads #2862's author, who sees nine "draw helper call" lines from a provenance digest and no mention of EXEMPT.

Replace the section's opening sentence and the #2862 bullet with:

`test_ratchet_only_shrinks` keeps `RATCHET` at or below `RATCHET_CEILING`, so none of these PRs can add a `RATCHET` entry. The guard fails each of them until it changes, or, for #2862, until it adds an `EXEMPT` entry:
...
- #2862 adds a SHA-256 digest in `_core_fingerprint` in `utils/prepared_baseline.py`, which identifies policyengine-core's code and decides nothing. An `EXEMPT` entry for `utils/prepared_baseline.py`, `("<module>", "hash library"): 1` and `("_core_fingerprint", "hash use"): 1`, clears all nine occurrences it reports.

and add to the #2618 bullet: "Its hash orders tied households and so decides their decile; #2618 must order ties without a hash, for example by household_id itself."

P3-1. The "82" sentence says the spread reaches 82 occurrences; it reaches 40

The scan without EXEMPT gives 82 occurrences in 21 files, which matches the body and the hand-off. The brief's 80 matches no count I could build. Those 82 are the 21 RATCHET occurrences, the 21 digests and 40 draw helper calls that reach the digests.

$ uv run python scan.py <guard>   # analyse + draw_helpers(analyses, {}) + count_findings
without EXEMPT: occurrences 82 files 21
beyond RATCHET: 61 in 13 files     # 21 digests + 40 draw helper calls

Body replacement: "Without EXEMPT, the scan reports 82 occurrences in 21 files: the 21 in RATCHET, the 21 digests, and 40 draw helper calls that reach the digests. So a stale exemption fails loudly."

P3-2. "So RATCHET and RATCHET_CEILING only fall" claims more than the tests enforce

test_ratchet_only_shrinks compares two numbers in the same file, so a change that raises both passes. Only review stops it. Replacement: "and test_ratchet_only_shrinks fails when RATCHET exceeds RATCHET_CEILING. So RATCHET can't grow unless a change also raises RATCHET_CEILING, which the file says never to do."

P3-3. The UC rebalancing comment misses that every simulation runs it

simulation.py:416-418 imports universal_credit_july_2025_reform and calls its simulation_modifier(self) in Simulation.__init__, and parameters/gov/dwp/universal_credit/rebalancing/active.yaml reads true from 2025. So current law draws default_rng(43) in every simulation, not only in a scenario someone opts into. The body's limits name the call through an object, but the RATCHET comment at test line 222 reads like an optional scenario. Replacement:

    # UC rebalancing: a seeded numpy Generator draws one value per benefit
    # unit and compares it with the new claimant share. Simulation.__init__
    # applies it to every simulation through the Scenario object, a call the
    # scan does not follow; the Scenario at module level passes on the
    # modifier.

P3-4. splitmix64_uniform has four constants, not three

utils/stochastic.py:18-21 uses four constants: 0x632BE59BD9B4E019, which spaces the salts, and the three splitmix64 ones. The scan recognises three (HASH_CONSTANTS). Test lines 173-175 and the body say "its three mixing constants". Replacement for the comment: "The splitmix64 finaliser that maps entity ids to uniform draws on [0, 1): the three splitmix64 constants the scan recognises and three xorshift steps." In the body: "the three splitmix64 constants and three xorshift steps".

P3-5. Docstring lines 5-6 and 12: wrapping and voice

The edits left two over-long lines (97 and 85 characters, against 79 in the rest of the docstring), and "are drawn" is passive. Replacement for lines 4-9:

(policyengine-core#500), and seeding does not exempt a draw. The dataset
build draws stochastic inputs such as take-up flags and stores them as input
variables; the model compares them with its rules. policyengine-core#519
checks formulas for numpy and stdlib randomness when variables register, but
a hand-written integer hash of entity ids passes that check, so this scan
looks for the hash as well.

and rewrap lines 11-13 so that "A name resolves to" starts the next line. "finalizer" (line 173) should read "finaliser".

P3-6. A filtered case changes its node id on Python 3.11

pytest gives some cases index ids (body13), and filtering shifts the index:

$ comm -23 ids_3.11.txt ids_3.13.txt
...::test_a_class_body_that_binds_a_name_reads_it_from_the_module[body11-True-11]
# on 3.13 the same case is [body13-True-11]; [body11-True-12] there is a 3.12-only case

CI runs only Python 3.13 (.github/actions/test-environment: python-version: "3.13", and the collection, shards and check_test_shards.py all run on one interpreter). So ids stay stable across shards and Test / Result is unaffected. If ids should also match across Pythons, pass ids= built from each case's text. Optional.

P3-7. The body's pytest-only runs can't be reproduced as written

"The guard alone with only pytest 9.2.0" omits that a bare pytest <file> --noconftest fails at collection: pytest 9 sets up the policyengine_uk package node and imports policyengine_uk/__init__.py. Replacement: "The guard alone with only pytest 9.2.0, --noconftest, and a plugin that makes _pytest.python.Package.setup a no-op so pytest doesn't import policyengine_uk/__init__.py: …". Also, test_model_never_imports_offline_modules can't fail while OFFLINE_MODULES is empty. The hand-off says so and the body doesn't. Add "(so test_model_never_imports_offline_modules passes trivially until a module is listed)" after "OFFLINE_MODULES (empty: …)".

Checked and sound

  • Reference. Fetched from raw.githubusercontent.com at f36d09d3…: git hash-object gives b8a87d1bd151266469a3b8ac253fd86140499d6d.
  • Port. I compared the reference, with policyengine_us→policyengine_uk substituted, against the head: diff -u gives 11 hunks and diff -u -w gives 13. Every hunk falls under the brief's items 1-5 or is _release_package_scan. From the constants to package_scan, the only change is the package name. _is_offline(name, offline=OFFLINE_MODULES) binds the same set object, so the default changes nothing.
  • Version gating. _on_this_python (sys.version_info >= (3, case[minor])) and the two if sys.version_info >= (3, 12): blocks cover every case that had a skip. 3.11 collects 369, 3.12 collects 380, and 3.13 and 3.14 collect 382 each. 3.11 leaves out 13 cases and 3.12 leaves out 2, and every left-out case needs newer syntax. Every case runs on every Python that parses it.
  • Fixture. No test in the file changes EXEMPT, RATCHET or a package file, and a rebuild reads the same files, so clearing can't produce a wrong scan. With an instrumented copy, pytest policyengine_uk/tests/code_health -n 4 built the scan once. In CI the six tests that call package_scan hash (conftest.shard_of) to shards 1, 3, 4, 6, 6 and 7. Each of those shards builds once, and shard 6 may build a second time (about 7 s) under --dist worksteal.
  • Lists. I reran the scan on Python 3.13.18: 1,534 modules, 7.4 s, 0 cut lookups, 42 occurrences in 18 files, no violations and no stale entries. It reports exactly the 21 EXEMPT occurrences in 9 files and the 21 RATCHET occurrences in 10 files, so RATCHET_CEILING = 21 holds. I read every function behind an EXEMPT entry: get_data_build_fingerprint, _fingerprint, _cached_dataset_path, uprating_input_fingerprint (compared with != at simulation.py:1101 and :1168 to reuse an uprated dataset), _sha256, sha256_file/verify_file, _digest_tool_files, definitions_digest, model_fingerprint, runtime_identity, load_partition and fetch. Each serves provenance, a cache key or a content check, and none decides who gets something or meets a threshold. I also read each RATCHET site, and the comments match the code apart from P3-3 and P3-4.
  • Other sources (step 4). I searched for np.random, numpy.random, random., default_rng, RandomState, hash(, hashlib, crc32, adler32, uuid, time.time, datetime.now, date.today, os.urandom, secrets, shuffle, .sample(, choice(, PYTHONHASHSEED and os.environ, plus iteration over set displays. Apart from P2-1, no hit outside the lists reaches an output:
    • os.environ selects the default dataset URL (simulation.py:219), an input, and filters git's environment (build_metadata.py:205).
    • The set iteration in utils/dated_reforms.py:673 and :703 feeds all-or-nothing checks, so order doesn't matter.
    • time.time only times the aggregate impact tool.
  • Tests on the head. uv sync --frozen --extra dev, then uv run pytest <guard> -n 4 (Python 3.13.18): 382 passed in 21.4 s, 35 s wall including startup. With only pytest 9.2.0 and the Package.setup no-op plugin: 3.11.18 gave 369 passed (17.6 s), 3.12.3 gave 380 (21.2 s), 3.13.18 gave 382 (18.0 s) and 3.14.6 gave 382 (18.0 s). No run skipped or xfailed a test.
  • Wheel. uv build --wheel reproduced the hand-off's sha256 48570c87…. I installed it as test-environment does (--no-install-project, then uv pip install --no-deps) and ran from an empty directory: 382 passed. The scan read the same 1,534 modules with the same counts.
  • Code health and lint. uv run pytest policyengine_uk/tests/code_health -n 4: 3,544 passed. ruff@0.17.2 format --check ., check . and check --select F811,F821 policyengine_uk all pass, and so does git diff --check 47a2817e2..HEAD.
  • Mutations. I ran each on the scratch worktree, then restored it.
    • The hand-off's six were all killed: import random, a second splitmix64_uniform call, new_draws, the pip_reform Scenario removed (stale), xorshift raised to 4 (ceiling and stale), and the _fingerprint entry removed.
    • Mine, killed: import numpy.random as r, from numpy import random as r, splitmix64_uniform as mix re-exported through a new utils/mixer.py and star-imported by a formula, 0x9E3779B97F4A7C15 hand-written in a new formula, hash(person_id) % 100, np.random.default_rng through model_api's np, a module-level draw, a class-attribute draw, a draw in a comprehension, frame.sample(5), a staticmethod helper called through its class, and H.r = np.random (caught at the read).
    • Mine, survived: a Knuth multiplicative hash (ids * 2654435761) & 0xFFFFFFFF and a sin-fract hash np.sin(x * 12.9898) * 43758.5453 of an id. Both fall within the stated limit ("a ratchet over recognised patterns, not a proof"). datetime.datetime.now() in a formula also survived, which is P2-1.
  • Merge and CI. origin/main is still 47a2817e2. git merge-tree --write-tree --name-only origin/main 0835944 gives a clean tree (32ac6a18e…, exit 0). The head has 20 check runs, all completed/success: Test / Python 1/7-7/7, Test / YAML, Test / Result, Lint, Changelog fragments, Fix evidence, Changes, Documentation build, Aggregate impact, Release build records the commit, and Install + smoke-import on 3.11-3.14.
  • Body numbers. The 382/369/380/382/382, the wheel's 382 and 1,534 modules, 3,544, the 21 and 21 and 82 in 21 files, and every entry in the EXEMPT table match my runs. The title and the axiom: and Impact: lines are as the brief requires.

What I couldn't check


Generated by Claude Code

The guard's docstring now says it looks for draws and not for the clock
(#2875). The comments on the splitmix64 and Universal Credit entries say
which constants the scan recognises, which entries reach the hash and how
Simulation.__init__ applies the modifier. Two test rows follow
PolicyEngine/policyengine-us#10447 at 221f2fe44, and a changelog fragment
records the guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 6f4d3313060987611bd00da46d3f392ffda1013e — COMPLETE; no confirmed findings or material evidence gaps.

All 389 focused checks passed (384 randomness-guard checks and five cache-release checks). The manual review covered alias/import resolution, scope handling, exclusions, model inventory and ratchet behavior, and compared the earlier-head author review with the fixes adopted in this head. Conclusions apply to the guard’s documented static-analysis scope and exemptions.

The PR remains draft; this is a review comment, not an approval.

@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 11:22
@MaxGhenis
MaxGhenis requested a review from juaristi22 November 5, 2026 11:22

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review status: COMPLETE — approved at 6f4d3313060987611bd00da46d3f392ffda1013e. No findings or material evidence gaps remain.

The head and description are unchanged from the completed review. I retained the verified 389 prior test passes (384 randomness-guard cases and five module-cache checks), together with the completed resolver/import-graph and author-feedback assessment; no code review or tests were repeated for the readiness change.

The PR is now ready for review. Fresh checks pass at this exact head, including the required Lint and Test / Result contexts. Historical benchmarks and the guard's documented limits retain their original qualifications.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants