Repository navigation
Conversation
Port policyengine-us's scanner for engine-time randomness and hash draws as a code-health test. EXEMPT lists the package's SHA-256 digests, which fingerprint, key caches or check content; RATCHET lists today's draws: the splitmix64 helper, the formulas and uprating that call it, and the seeded numpy draws in participation and two scenarios. Cases that need newer syntax are filtered at collection, so no Python reports a skip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6H1FBndAKeuXdSGuft
The guard's docstring now says it looks for draws and not for the clock (#2875). The comments on the splitmix64 and Universal Credit entries say which constants the scan recognises, which entries reach the hash and how Simulation.__init__ applies the modifier. Two test rows follow PolicyEngine/policyengine-us#10447 at 221f2fe44, and a changelog fragment records the guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e reads it Define uc_new_claimant_health_element_draw, pip_phase_in_draw and would_claim_student_loan as yearly inputs with no formula, and register them with the supplied-input helpers. supplied_input_periods, and with it supplied_by, latest_supplied_input_period and supplied_input, now count a recorded key only when its branch is the one core reads for the period and the tier core reads there still holds the input. latest_supplied_input_period ranks with core's _latest_input_key. set_model_value refuses the three new columns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
input_source(population, name, period) names the supplied input policyengine-core reads for a registered draw, returns None where none reaches the period, and raises for anything else. In a simulation built from data, Simulation.calculate applies it before core, drops the request's fast-cache entry, and checks core's answer against the source after. calculate_add, calculate_divide and calculate_output refuse a draw, and each counts as a running calculation. set_input refuses a new draw while a calculation runs or a derived value of the draw is stored, and checks would_claim_student_loan's raw values before core casts them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
…cenario Each consumer asks input_source whether a stored input reaches the year. The UC rebalancing modifier compares uc_new_claimant_health_element_draw, promoted to float64, with the new-claimant share; the PIP scenario does the same with pip_phase_in_draw and the phase-in share; student loan uprating uses the stored would_claim_student_loan decision after checking its raw values. Without the column, each runs its existing seeded draw, inline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
…e core versions The core Dataset route refuses a dataset carrying a stored column whose person, benefit unit or household identifiers do not survive the int32 cast, and one whose stored columns or exempting outcomes are supplied for a later period but not the first, or whose draws change between periods, before it exports anything. A code-health test fails when uv.lock or the installed policyengine-core holds a version outside the qualified set, and another fails on any package read of a stored draw that skips calculate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
…eachable branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
…umers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6HUE9hj2BPEjEXLL68
Aggregate impact on certified data: passBase Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in Notes
Moves flagged for reviewNone. Spending by benefit and service
Caseloads
Household net income by decileNo household's net income changes in any year. No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target. Checks failing on both base and head (1)
Counted-once checkEach of 92 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.
Provenance
Dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
juaristi22
left a comment
There was a problem hiding this comment.
Review status: COMPLETE at 5a784a7e926d4bc5b1471770ae67d4f96f139140. No findings or material evidence gaps.
Independent code/test and policy/source review covered the stored UC health and PIP draws, student-loan take-up, supplied-input precedence, provenance/cache/lifecycle checks and Core Dataset routing. The change introduces allocation inputs rather than new statutory rules.
Validation: 202/202 focused repository tests passed on the lock-matched Core 3.34.1 environment, plus one synthetic final-output/order check. The stored UC draw changes LCWRA, final Universal Credit and household net income by the same £2,550.48; reversing calculation order gives identical results. The unchanged #2873 randomness guard was checked byte-for-byte against its prior reviewed head; its earlier 384 passes were reused, not rerun or added to the 202 count. All 20 hosted checks currently pass at this exact head.
Equivalence remains conditional on the supplied reference decisions and covered thresholds. This review does not certify producer calibration, private datasets, population effects, arbitrary-share float64 equivalence or the author's full dual-Core-version/mutation totals. The disclosed shadowed-input/carry-ranking changes and seeded fallback boundary remain explicit.
Approval withheld: this PR is still a draft and its description requires #2873 to land first. The completed review can be reused when readiness, parent integration and exact-head checks are reassessed.
Addresses #2874
axiom: n/a: microsimulation-only: stored inputs for engine draws, no rule changes
This PR contains #2873's two commits (the engine-randomness guard) and lands after #2873.
What and why
Three new yearly inputs, with no formula:
uc_new_claimant_health_element_draw(benefit unit, float, default 1.0): a uniform in [0, 1) that the dataset build stores. In each Universal Credit rebalancing year with no status for the year,add_universal_credit_reformgives a benefit unit the new-claimant LCWRA rate when the draw falls belownew_claimant_share.pip_phase_in_draw(person, float, default 1.0): the PIP phase-in scenario removes a person's PIP in a year from 2025 to 2029 when the draw falls below that year's share.would_claim_student_loan(person, bool, default False): the stored take-up decision thatuprate_student_loan_plansuses for new borrowers.Each consumer uses the stored input when the dataset supplies it and runs its own seeded draw, unchanged and inline, when it does not. The UC modifier and the PIP scenario ask
input_sourcewhether a supplied draw reaches the year, read it throughSimulation.calculate, and compare it with the share in float64. Student loan uprating reads the column from the projected table after checking its raw values. No published dataset carries these columns yet, so outputs on today's data change only where the tightened helper below changes an answer.Reading a stored draw
utils/stored_draws.pyregisters the two draws with their entity, value type and definition period.input_source(population, name, period)returns the period, branch, tier and sequence number of the supplied input policyengine-core reads, None where no supplied input reaches the period, and raises otherwise. It changes nothing in storage and returns no array. It refuses:year:2026:2,month:2026-06:12,year:2025-01-15);defined_for,calculate_output,adds,subtractsorrequires_computation_after; a neutralised or abolished draw; core's macro cache; carry turned off where the read needs it;In a simulation built from data,
Simulation.calculateruns the check after its input-lifecycle and stale-record cleanups, drops the request's fast-cache entry, and for a source-backed draw calls core withmap_to=None, checks that core's answer equals the source's array (same dtype and shape) and that the source has not changed, then maps it.calculate_add,calculate_divideandcalculate_outputrefuse a registered draw. A context variable counts running pe-uk calculations on any simulation, andset_inputrefuses a new draw while one runs or while the holder stores a derived value of the draw. Situations keep core's path.Supplied-input helpers
SUPPLIED_INPUT_VARIABLESnow holds the three new columns.set_model_valuerefuses them. A recorded key now counts only when its branch is the one core reads for the period and the tier core reads there (memory before disk) still holds the input.latest_supplied_input_periodranks with core's_latest_input_key.What this changes for existing callers (
supplied_input,supplied_input_periods,supplied_by,latest_supplied_input_period, used bycurrent_education, the maintenance loan, PIP, trading allowance, Class 4 loss and housing benefit formulas):test_a_value_cached_over_an_input_in_its_slot_is_not_supplied);test_a_child_branch_value_over_a_parent_input_is_not_supplied_on_the_child);test_an_input_on_disk_counts_only_when_memory_does_not_hide_it);test_two_periods_starting_on_one_day_rank_by_core_order).Calculation itself never caches a value over a supplied input, so these cases arise only from low-level holder writes or branch caches; the required test files below pass unchanged.
The core Dataset route
Before it exports,
Simulation.build_from_datasetrefuses a dataset that carries any stored column (the UC deduction draws,months_since_last_birthday,warm_home_discount_random_drawand the three new columns) when any person, benefit unit or household identifier fails to survive the int32 cast, naming the entity and identifiers. It also refuses a dataset whose stored columns,attends_private_school_random_draw,uc_receives_new_claimant_health_elementorattends_private_schoolit supplies for some period but not the first, or whose draws change between periods.Datasets the route now refuses: core-format datasets carrying one of those columns with identifiers beyond int32, and core-format datasets supplying one of them only for later periods, or with a draw that changes between periods. None of pe-uk's own core-route fixtures carries such a column, so the route refuses none of them.
Core versions
QUALIFIED_CORE_VERSIONSrecords 3.33.3 and 3.34.1.tests/code_health/test_core_version_gate.pyfails whenuv.lockor the installed core holds another version, and says what to rerun before adding one. It readsuv.lockfrom the source tree or the directory the tests run from, so it runs without a skip when CI tests the built wheel.Invariants
input_sourcebefore core and the answer check after; ADD, DIVIDE and output refuse draws; the fast cache never answers one.Test plan
test_supplied_input_tightening.py: registration, ranking (with a Hypothesis property against core's order), the tightened key and theset_model_valuerefusal.test_stored_draw_input_source.py: provenance, requests, definitions, disk tier, dispatch, setter, lifecycle, restore and the differential against core's owncalculate.test_stored_draw_consumers.py: each consumer with and without its column, precision, precedence, validation and properties.test_stored_draw_core_route.py: identities for each entity, supplied periods, and the with/without differential on both routes.code_health/test_core_version_gate.pyandcode_health/test_stored_draw_reads.py(no package read of a draw skipscalculate).On core 3.34.1 (the lock) and 3.33.3 (
uv run --frozen --with 'policyengine-core==3.33.3', which printed 3.33.3), the 17 required existing files, these six files and the five other core-Dataset-route files gave 1069 passed, 0 failed on each.policyengine_uk/tests/code_healthgave 3556 passed. Mutations of each refusal, of each consumer's float64 promotion, of the fast-cache eviction, of theset_model_valuerefusal, of both core-route checks, of #6's validation and of its loader each fail a test.ruff format --check,ruff checkandgit diff --checkpass.What later stages finish
Stage 2 adds protected storage and sealed populations, with a storage check inside
input_source. Phase 2 fails closed: it removes the seeded fallbacks, registers the other required draws and outcomes, and addsmonths_since_last_birthday's answer check and inclusive lookup.Generated by Claude Code