Skip to content

Grow alcohol and tobacco quantities with OBR clearances per head - #2829

Queued
MaxGhenis wants to merge 32 commits into
mainfrom
fix-2596-excise-quantity-indices
Queued

MaxGhenis wants to merge 32 commits into
mainfrom
fix-2596-excise-quantity-indices

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #2596

What and why

Alcohol and tobacco duty read eleven household quantity inputs, and none of them had an uprating index. Dataset projection therefore kept the base-year quantities in every later year, and the model charged each year's duty rates on them. In its October 2026 EFO the OBR forecasts 2031-32 cigarette clearances 47.2% below its March 2026 EFO's 2024-25 outturn. Household weights also grow by population, 1.54% over those years. A flat input's weighted total would therefore end 92.4% above the forecast clearances for cigarettes. The other lines would end 79.9% above (hand-rolled tobacco), 62.7% (other tobacco products), 12.2% (beer and cider), 10.3% (wine) and 8.3% (spirits).

This PR builds on #2761 (merged 2 Nov) and adds six per-head clearance series, one for each product group the OBR forecasts:

Inputs Series (gov.economic_assumptions.yoy_growth.obr.per_capita.) OBR line
cigarettes_per_week tobacco_clearances.cigarettes Table 4.11, "Cigarette clearances (million sticks)"
hand_rolling_tobacco_grams_per_week tobacco_clearances.hand_rolling_tobacco Table 4.11, "Hand rolled tobacco clearances"
cigars_grams_per_week, other_tobacco_grams_per_week, heated_tobacco_grams_per_week tobacco_clearances.other_tobacco_products Table 4.11, "Other tobacco products"
beer_litres, cider_litres, sparkling_cider_litres alcohol_clearances.beer_and_cider Table 4.12, "Beer and cider"
wine_litres, other_fermented_litres alcohol_clearances.wine Table 4.12, "Wine" (includes other fermented products)
spirits_litres alcohol_clearances.spirits Table 4.12, "Spirits" (includes spirit-based ready-to-drinks)
  • Importer. import_obr_forecasts.py adds ClearanceSeriesSpec and six CLEARANCE_SERIES_SPECS to RECEIPTS_SERIES_SPECS, so the run that refreshes the receipts series rewrites these too. A clearance spec takes no rates and no outturn of its own.
    • It finds each line by table title and row label, and links the outturn year to the previous EFO's outturn, as it does for the receipts series.

    • It interprets no words in the clearance tables. CLEARANCE_TABLE_TEXTS lists each text, other than titles, numbers and fiscal years, in the March 2025, March 2026 and October 2026 tables: 7 for the tobacco clearances and 9 for the alcohol clearance forecasts. It compares a cell's text in lower case and single spaces, without a footnote number after a label's last letter, as find_receipts_line reads row labels.

    • It reads the whole text of each cell. The reader takes shared and inline strings through one helper, _string_text, which joins a string's own <t> and the <t> of each formatting run, so a cell whose text sits in several runs reads whole. It leaves out the <t> of a phonetic run (<rPh>). read_sheet_rows also keeps each row's number on the worksheet (SheetRow.number). No cell in the three receipts releases changes value: they hold no inline strings and no phonetic runs, and the shared-string reader already joined every run.

    • Before it reads a line's values, it compares the text cells of the line's table with the list, in every column, from the table's title through its configured end: the row above a title CLEARANCE_TABLE_ENDS lists, or the sheet's last row. It skips the title cell, numbers and fiscal years. The OBR follows the alcohol clearances with the alcohol receipts forecasts, so the check leaves out their "£ billion" cell and note. The importer checks no cell above a table's title or below its end, and the economic assumptions README asks whoever runs it to read any note there.

    • In both EFOs it stops before writing anything when:

      • a cell holds any other text, such as a note that states another unit or scale, a line for another product or a unit column's heading. It names the table and each such cell, and asks whoever runs it to read the text, convert the table's figures to the listed units if it states another, and add the text to the list;
      • the table lacks the cell in which the OBR states a line's unit (unit_cell): "Cigarette clearances (million sticks)", "Non-cigarette clearances ('000s kg)" for both tobacco parts, or Table 4.12's "Note: All clearances are in thousands of hectolitres of alcohol.";
      • a text sits beside the years in the fiscal-year row over a clearance line, such as a unit column's heading or a line's label;
      • hand-rolled or other tobacco (part_of_total) does not sit in the OBR's layout (validate_clearance_part): an "of which:" row with no other cell directly above the line and the total's other parts, and directly above that row the non-cigarette total's row, which holds its label, its figures and nothing else. A row that holds the total's label and no figures does not count as the total, and an empty row on either side of "of which:" breaks the layout. The message names the whole layout the importer needs;
      • the sheet repeats the table's number and title. It names the rows, since it would check the text under one heading and could read a line under another;
      • it finds a cell beside a line's label and values, on any receipts or clearance line.

      Without these checks, if a note put the October 2026 cigarette forecasts in ten million sticks, with the figures divided by ten, the importer would read 2026-27 cigarette growth of -91.0% in place of -10.3%. If a second Table 4.12 heading below the alcohol receipts forecasts held the alcohol lines, with the spirits forecasts in ten thousand hectolitres, the importer would read 2026-27 spirits growth of -90.1% in place of -0.8%. If the OBR stated kg for other tobacco beside its line, the importer would read the line in the non-cigarette total's thousand kg, at growth of 999.0 for clearances that did not change.

      A table can keep every text on the list and still put a line under another unit. Two rearranged copies of the real October 2026 Table 4.11 put hand-rolled and other tobacco under the cigarette label: one moves the label beside a second row of years over them, and one moves the cigarette line directly above "of which:". Both halve the two parts' figures. Without the two layout checks, the importer would read 2025-26 growth of -58.6% for hand-rolled tobacco in place of -17.1%, and -56.8% for other tobacco in place of -13.6%. A phonetic run can also spell "4.12 Alcohol receipts forecasts" in a cell with no text of its own. A reader that kept its <t> would end the alcohol clearances at that cell, above a note in ten thousand hectolitres, and the importer would read the same -90.1% spirits growth.

    • The importer writes every series under per_capita, these six and Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761's seven gambling bases, through one per_head_growth, which computes (1 + aggregate growth) / (1 + ons.population growth) - 1. It divides one plus each year's growth before rounding (ReceiptsGrowth.ratios) by one plus that year's ons.population growth, subtracts one and rounds once to six decimals. It reads the population from the yoy_growth.yaml it rewrites. For a gambling base it divides the base's growth, receipts over the rate the model charges, whether the importer links the line to the previous EFO or to its own outturn.

    • After the forecast it holds the clearances flat, as road_fuel_volume holds road fuel volume.

  • Series. The importer writes the series in yoy_growth.yaml from the October 2026 tables over the March 2026 ones, and a test rebuilds the file byte for byte from the real tables.
  • Inputs. Each input declares its group's index (gov.economic_assumptions.indices.obr.per_capita.*). uprating_indices.yaml maps it to the series, so dataset projection to 2032 and Core's uprating after it use the same index.
  • Fixtures. We added sheets 3.11 and 3.12 (March 2025 and March 2026) and 4.11 and 4.12 (October 2026) to the real-table fixtures. Each kept sheet's XML and the shared strings match the originals byte for byte, and we list the originals' hashes in the fixture README.
  • Docs. We document the projection of alcohol and tobacco quantities in a new section of docs/engineering/excise-duties.md, with every source file, sheet, row and hash, and list the series and the refresh step in the economic assumptions README.

Dataset projection keeps the strengths (*_abv), draught shares and cigarette_price_per_pack at their base-year values, since they measure ratios and a price, as the issue says.

Why per head: how projection applies weights

From policyengine_uk/data/economic_assumptions.py:

  • extend_single_year_dataset copies the base year into each later year.
  • apply_single_year_uprating multiplies each column uprating_indices.yaml lists by one plus its series' growth for that year.
  • The file lists household_weight under ons.population.

So, for household i in year t, with population growth p(t) and quantity series g(t):

  • the weight is w(i, t) = w(i, t - 1) × (1 + p(t));
  • the quantity is q(i, t) = q(i, t - 1) × (1 + g(t));
  • the weighted total is Q(t) = (1 + p(t)) × (1 + g(t)) × Q(t - 1).

With g(t) = (1 + c(t)) / (1 + p(t)) - 1, where c(t) is clearance growth, Q(t) = (1 + c(t)) × Q(t - 1), so the total follows the clearances. Growing quantities by c(t) itself would count population growth twice. ons.population grows 1.54% from 2024-25 to 2031-32, so totals grown that way would end 1.54% above the clearances. In the OBR's outturn, cigarette clearances fell 9.97%, from 13,578.2 million sticks in 2024-25 (March 2026 EFO) to 12,224.6 million in 2025-26 (October 2026 EFO). Population grows 0.33% in 2025, so the importer writes 12,224.6 / 13,578.2 / 1.0033 - 1 = -0.102653.

By 2031-32 the indices (2024 = 1) stand at:

Group Index
Cigarettes 0.520
Hand-rolled tobacco 0.556
Other tobacco products 0.615
Beer and cider 0.891
Wine 0.907
Spirits 0.924

Budget measures, and what they mean for reforms

The OBR incorporates the Budget's measures in its October 2026 forecast (EFO paragraph 3.2). In policy Table 1.1 it lists no change to either duty rate. It lists three measures there that change the two duties' receipts, each by less than £13 million a year:

  • measure 19, Small Producer Relief from 1 February 2027: alcohol duty £1.5 million to £9.9 million a year lower;
  • measure 45, HMRC's tax-gap measures: tobacco duty up to £12.7 million and alcohol duty up to £4.8 million a year higher;
  • measure 95, low-value imports from October 2028: up to £0.8 million and £0.4 million a year lower.

The OBR still assumes rising duty rates. It attributes alcohol's receipts growth to "increases to the duty rate that offset an assumed average 1 per cent decline in alcohol consumption in each year" (paragraph 4.43). For tobacco it says "higher duty rates only partly offset falling consumption" (paragraph 4.44). So the OBR forecasts the clearances along its consumption trend, under its own path of rate rises.

The dataset projection reads no duty rate (uprating_input_paths), so when a reform changes alcohol or tobacco duty rates, the model moves receipts through the rates alone and leaves quantities where they were. It applies no consumption response, as in a static costing, and a test checks it.

Projected duty on one constructed household

A constructed household, not survey data, buys 300 litres of 5% beer, 60 of 4.5% cider, 20 of 7% sparkling cider, 80 of 12% wine, 10 of 40% spirits and 15 of 5% other fermented drinks a year. Each week it buys 70 cigarettes at £16 a pack, 25 g of hand-rolling tobacco, 5 g of cigars, 3 g of other tobacco and 4 g of heated tobacco. Its duty from the 2024 inputs, on #2761 at d08fbb6 (quantities flat) and on this branch (quantities per head):

Model year Alcohol duty, flat (£) Alcohol duty, this PR (£) Change Tobacco duty, flat (£) Tobacco duty, this PR (£) Change
2024 809.16 809.16 0.0% 2,435.77 2,435.77 0.0%
2025 828.98 804.23 -3.0% 2,568.19 2,251.62 -12.3%
2026 853.88 818.22 -4.2% 2,785.28 2,215.55 -20.5%
2027 853.88 801.76 -6.1% 2,921.07 2,114.66 -27.6%
2028 853.88 795.21 -6.9% 2,921.07 1,937.37 -33.7%
2029 853.88 789.18 -7.6% 2,921.07 1,779.19 -39.1%
2030 853.88 779.57 -8.7% 2,921.07 1,674.52 -42.7%
2031 853.88 770.34 -9.8% 2,921.07 1,569.66 -46.3%
2032 853.88 768.65 -10.0% 2,921.07 1,566.21 -46.4%

Gap to the OBR's receipts forecast

We put the UK's 2024-25 clearances (March 2026 Tables 3.11 and 3.12) in a second constructed household, also not survey data, at weight 1, so its weighted duty covers the whole UK. We assume:

  • beer and cider all as 4% beer, with no draught relief;
  • wine and other fermented products as 12% wine;
  • spirits at 40%;
  • other tobacco products as other smoking tobacco;
  • cigarettes at the default price.

We compare its duty with the OBR's alcohol receipts by group (accrued, Tables 3.12 and 4.12, rows 16 to 18) and tobacco duties (cash, Tables 3.8 and 4.8, row 27).

Duty 2024-25 2026-27 2031-32 2031-32, quantities flat (#2761 at d08fbb6)
Beer and cider +22.4% +22.9% +6.3% +19.3%
Wine +2.5% +1.1% -12.5% -3.5%
Spirits -0.2% +1.8% -12.0% -4.8%
Alcohol, total +7.6% +7.9% -6.8% +2.8%
Tobacco +1.9% -0.4% -18.0% +54.6%

The model's alcohol duty exceeds the OBR's receipts by 7.6% in 2024-25 and 7.9% in 2026-27, then by 3.5% in 2027-28 and 0.3% in 2028-29. It first falls short in 2029-30, by 2.5%, and by 6.8% in 2031-32. Its tobacco duty exceeds the OBR's 2024-25 receipts by 1.9%, falls short by 0.8% in 2025-26 and 0.4% in 2026-27, and then by more each year, to 18.0% in 2031-32.

  • In 2024-25, charging all beer and cider at the beer rate with no draught relief lifts that line 22.4% above its receipts. On a real dataset the model charges each household's own product mix, such as the one UK: impute alcohol, tobacco and gambling consumption so policyengine-uk's excise duties stop scoring zero microcosm#1359 will supply.

  • After 2026-27 the model holds its current-law rates: alcohol at its 1 February 2026 values, tobacco at its 1 October 2026 values and the default cigarette price at its 2024 average. The OBR assumes rising rates. Quantities move with the clearances in both, so the gap moves with duty per unit cleared:

    Model duty per unit, 2031-32 vs 2024-25 OBR receipts per unit, 2031-32 vs 2024-25
    Alcohol (per hectolitre of alcohol) +5.8%: +5.7% by 2026-27, then flat but for product mix +22.1%
    Tobacco (volume index weighted by 2024-25 duty) +19.2%: +19.0% by 2027-28, after the October 2026 rise, then flat but for product mix +48.2%
  • On Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761, with quantities flat, the model charges later rates on 2024-25 volumes, and tobacco duty ends 54.6% above the forecast.

Invariants

These hold for every input:

  1. Each index stays positive. For every series the importer divides a ratio of positive levels by one plus population growth, so each stays above -100%. Each index starts at 1 in 2024 and stays positive to 2039. The importer refuses a series whose five-decimal index would reach zero, in compute_receipts_growth, in per_head_growth and through Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761's check_index_stays_positive.
  2. Quantity = base × compounded growth. Dataset projection multiplies each 2024 quantity by its series' compounded growth. The property checks 2024–2032 to 1e-12 relative, compares the compounded factor with the five-decimal index to 1e-4 relative, and checks Core against that index to 1e-6 relative. After 2039, Core holds quantity and weight indices fixed, while explicit dataset extension continues the per-head and population growth.
  3. Weighted totals follow the clearances. In every year, (1 + series) × (1 + population growth) equals 1 + aggregate clearance growth to within 5e-7 × (1 + population growth), and equals 1 after the forecast. Weights grow by population alone.
  4. Per-head growth rounds once. per_head_growth divides 1 + aggregate growth before rounding by 1 + population growth, subtracts one, and rounds once. For any growth and population growth, the product (1 + per-head value) × (1 + population growth) matches 1 + aggregate growth within the stated tolerance, 5e-7 × (1 + population growth). A series that holds a duty's base, as Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761's gambling series do, grows per head by that base's growth, not its receipts'.
  5. The importer reads a clearance line only from text someone has read, in the OBR's layout. A property puts one text in a real clearance table of any of the three releases: in an empty cell of any row from the title through the table's end, in any column, or on a new row. For any text outside the table's list, drawn from st.text() or from unit-shaped strings, the importer refuses every line of the table. It names the cell, or finds no line or table where the text took a label's or the title's place. A listed text other than a line's label or a marker, outside the lines, stops every line when it lands in the fiscal-year row, and stops hand-rolled or other tobacco when it lands between that part and the non-cigarette total's row. Anywhere else the importer reads every line as before. An example test, independent of the draws, changes each release's tobacco clearances in five ways: it inserts a row that holds only the non-cigarette total's label directly above "of which:", writes "Cigarette clearances (million sticks)" after the total's figures or beside "of which:", or inserts an empty row directly above or directly below "of which:". In each case the importer refuses both parts with the part check's message and reads cigarettes as before. A second property splits any text of characters from U+0020 to U+D7FF into one to four formatting runs, with or without a phonetic run. The reader returns the runs' whole text and none of the phonetic run's, from an inline string and from a shared string alike. The importer stops a line with any cell beside its label and values, and stops every line of a table whose sheet repeats its heading. A test checks that each list equals the texts in the three releases' tables, so no unread or stale entry survives.
  6. A duty-rate reform moves receipts by the rate alone. Scaling every alcohol and tobacco rate by k in a year from 2027-28 leaves every projected quantity bit-identical and multiplies each duty by k. The uprating fingerprint stays unchanged, because the projection reads no duty rate.
  7. A rerun of the importer changes nothing until population changes. Rerunning it on the committed file with the real October 2026 and March 2026 tables leaves the file byte-identical. Editing ons.population without rerunning it changes none of the 13 per-head series, so the six clearance series and seven gambling bases keep the values the importer wrote for the old population. Raising 2026 population growth from 0.2% to 0.3% then leaves weighted quantities 1.000998 times the clearances from 2026 on, and the byte-for-byte test fails. Rerunning the importer rewrites each per-head series' 2026 value and nothing else.

Test plan

  • For policy changes: YAML tests whose expected values come from an external source. Not applicable: this projects data inputs from an OBR forecast and changes no rule. The new tests pin the OBR's published clearances (receipts Tables 3.11, 3.12, 4.11 and 4.12).
  • make format: run as uvx ruff@0.17.2 format --check . and uvx ruff@0.17.2 check . on the whole repository, uvx ruff@0.17.2 check --select F811,F821 policyengine_uk, and git diff --check, all clean at 0a68b3fcf.
  • Changelog fragment changelog.d/2596.fixed.md

New tests:

  • test_excise_quantity_projection.py: 10 tests, 2 of them Hypothesis property tests (projection and reform).
  • test_import_obr_forecasts.py gains 31 test functions over Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761 (84 at Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761's head 0037137ad, 115 at 0a68b3fcf), including four Hypothesis properties: rendered clearances, per-head rounding, one arbitrary text in a real clearance table, and a text in formatting runs. One parametrized test edits the real October 2026 tables in 33 ways. It changes units and scales, product labels, listed texts, headings, marker and year rows, and cells that should hold numbers. Another writes a note into the real Table 4.11 as two formatting runs, in three ways, each with a first run that would pass alone. A third moves rows of the real Table 4.11 in three ways that keep every text on the list and put the two tobacco parts under the cigarette label. A fourth changes the rows over the two tobacco parts in each of the three releases in five ways (TOTAL_LAYOUTS, 15 cases): a row that holds only "Non-cigarette clearances ('000s kg)" directly above "of which:", "Cigarette clearances (million sticks)" after the total's figures, the same text beside "of which:", and an empty row directly above or directly below "of which:". One test repeats Table 4.12's heading below the alcohol receipts forecasts, and another fills the empty cell above their title with a phonetic run that spells it. We added clearance tables to the synthetic workbooks, and the byte-for-byte test covers the clearance series.

Results (local, uv sync --frozen --extra dev, Hypothesis 6.170.11, -n 4, HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset): We ran every row below at 0a68b3fcf, 8f3f0f396 or 8d9763345, one file at a time. 8f3f0f396 changes only three lines of the economic assumptions README after 8d9763345, and 0a68b3fcf changes only the part check's message in the importer and its tests after 8f3f0f396. Before the total's-figures change, at 7555d402d, the same environment ran test_import_obr_forecasts.py to 192 passed and 1 failed in 113 seconds: test_any_text_no_one_has_read_stops_a_real_clearance_table drew a row that holds only the non-cigarette total's label above "of which:", which the part check accepted. Between 7555d402d and 0a68b3fcf only the importer, its tests, test_excise_quantity_projection.py (a fixture that clears its cache), the economic assumptions README and docs/engineering/excise-duties.md change.

File Head Result
Importer on the real tables, through compute_receipts_growth and render_receipts_growth_yaml 8d9763345 March 2026 over March 2025 reads all 17 series, 13 of them per head; October 2026 over March 2026 reads 17, 13 of them per head, and rewrites yoy_growth.yaml byte for byte (120,408 bytes, SHA-256 846cbe78c74ff9a1c94ce39c5e4b8bf275645b78943d3c2d84dda830099e24a8)
test_import_obr_forecasts.py 0a68b3fcf 208 passed, 0 failed, in one run of 97 seconds
test_excise_quantity_projection.py 0a68b3fcf 10 passed, in 119 seconds
code_health (the whole directory) 0a68b3fcf 3,069 passed, in 231 seconds
code_health/test_changelog_fragments.py 8d9763345 103 passed, with towncrier installed
code_health/test_uprating_declarations.py 8d9763345 8 passed
code_health/test_cached_arrays_not_written_in_place.py 8d9763345 36 passed
test_parameter_descriptions.py 8d9763345 24 passed
test_gambling_duty_base_projection.py 8d9763345 23 passed
test_population_vintage.py 8d9763345 95 passed
test_projection_horizon.py 8d9763345 14 passed
policy/baseline/gov/hmrc/alcohol_duty/alcohol_duty.yaml 8d9763345 15 passed
policy/baseline/gov/hmrc/tobacco_duty/tobacco_duty.yaml 8d9763345 12 passed
policy/baseline/gov/hmrc/excise_integration.yaml 8d9763345 2 passed

We left these files to CI's full suite: test_uprating_inputs_reach_data.py, test_excise_schedules.py, test_energy_dataset_projection.py, test_corporate_receipts_statistics.py, test_long_run_joins.py, test_lagged_series.py, test_dated_reform_routes.py, test_ni_metadata_compat.py, test_rail_usage_uprating.py and test_road_fuel_volume_uprating.py. The repository's checkers pass at 0a68b3fcf: .github/check_changelog_fragments.py and .github/check_fix_pr.py (on this body).

Differential against #2761 at d08fbb6: We projected two constructed datasets in this branch and in #2761: two households at weights 700 and 300 buying every product, through 2074, and the national construct above, to 2032.

  • Stored columns differ only in the eleven quantity inputs, from 2025 to 2074 (3,264 cell values compared).

  • We compared 16 outputs per year to 2032 (288 values). The two duties move, and household_tax, gov_tax, household_net_income and gov_balance move by the duty change, within £0.003. Real household net income, equivalised household net income and net income after housing costs move with net income. Income tax, NI, benefits, market income, HBAI net income, VAT and gambling duty stay identical. Other outputs that read net income move too; we compared only these 16.

  • For the first household alone in 2025:

    Output Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761 This PR
    Real household net income £21,031.73 £21,383.67
    Equivalised household net income £30,443.92 £30,953.36
    Net income after housing costs £13,998.53 £14,339.85
  • Through the dataset, the weighted volume of all 11 inputs follows the aggregate clearances in all 51 years from 2024 to 2074, to a maximum relative error of 2.06414×10⁻⁶ (561 checks). Through Core from a 2032 projection, it does so to 1.40668×10⁻⁵ (209 checks).

  • This branch adds 12 parameters to the tree, the six series and their indices, and changes or removes none. It changes the uprating of the eleven inputs and no other variable.

The differential ran at the branch's earlier heads; after them the branch changes the importer's checks, its tests and docs, and no series, input or parameter.

Mutations (each in its own copy of the branch): We applied each mutation alone and ran the tests in its row. We ran the older importer mutations through a harness that loads the importer and runs its test file without importing the rest of the package; unmutated, the importer passes all 43 clearance tests and all 7 per-head tests there. The clearance mutations ran at 65f3b3f4a. Later commits add the reader's run joining and row numbers, the repeated-heading refusal at the top of validate_clearance_table, the spec's part_of_total and the two layout checks that run after it, and change the spec's error message, comments and docstrings; the walk and the checks those mutations change keep their code. The per-head mutations ran at ade1df1ce and the three projection mutations at f0f3d37e4; no later commit changes the code they mutate or the assertions of the tests they ran. Five mutations of the XLSX reader and the heading check, and the walk mutation, ran at 433b20fb7 through the test file itself. The reader and heading mutations ran on the five tests that cover them, the list test, the real-pair test and the phonetic-run test (8 tests); the walk mutation on the 33 edits and the arbitrary-text property (34 tests). Unmutated, all 41 tests of those selections pass. The seven mutations of the layout checks, the total's row and the phonetic-run join ran at 0a68b3fcf on the whole of test_import_obr_forecasts.py (208 tests). Unmutated, all 208 pass.

Mutation Tests run Failed
Drop the refusal of a repeated heading 8 tests 1: test_a_repeated_clearance_heading_stops_the_run
Read only the first run of an inline string 8 tests 5: the three formatting-run edits, the formatting-run property, and test_a_phonetic_run_cannot_end_a_clearance_table, whose empty first run the mutation reads as no value
Read only the first run of a shared string, and every run of an inline one 8 tests 2: the formatting-run property and test_the_read_texts_are_the_texts_of_the_three_releases, since the real alcohol footnotes keep their numbers ("1", "2") in a run of their own
Name the repeated heading's rows by their place in the list, not their number on the worksheet 8 tests 1: test_a_repeated_clearance_heading_stops_the_run
Join every <t> of a string, a phonetic run's included (the .//main:t join) 208 tests 2: test_a_string_reads_whole_in_any_formatting_runs and test_a_phonetic_run_cannot_end_a_clearance_table
Drop the fiscal-year-row check 208 tests 3: both layouts that put the cigarette label in a row of years, and the arbitrary-text property
Drop the part-of-total check 208 tests 17: the layout that moves the cigarette line above "of which:", all 15 TOTAL_LAYOUTS cases and the arbitrary-text property
Accept a total's row that holds the label and no numbers 208 tests 4: the three label-without-figures cases and the arbitrary-text property
Accept any text after the total's label, not only numbers (drop or not all(NUMBER_RE.fullmatch(text) for text in total[1:])) 208 tests 3: the three text-on-total-row cases
Accept an "of which:" row that holds other cells (of_which[:1] != ["of which:"]) 208 tests 3: the three text-on-of-which-row cases
Drop empty rows from the rows above the parts, so an empty row may sit beside "of which:" 208 tests 6: the empty-row-above-of-which and empty-row-below-of-which cases in all three releases
Once the check meets the fiscal-year row, skip every later cell under a year column 34 tests 3: the edits that put a note in E40 and "nan" in D5, and the arbitrary-text property
Check only each row's first text (the title row's second) 43 clearance tests 8, including the kg cell beside a real line, a unit in the "of which:" row, text where a number belongs and the property
End the table one row early 43 clearance tests 19, including every note added below the real Table 4.11 and the text in Table 4.12's last row
Skip the alcohol table 43 clearance tests 11
Skip the tobacco table 43 clearance tests 29
Accept a listed text with more words 43 clearance tests 19, including both extra-words edits
Skip the title's row 43 clearance tests 2: the title-row edit and the property
Skip the title, marker and year rows 43 clearance tests 4: the three heading-row edits and the property
Drop the unit-cell requirement 43 clearance tests 1: test_a_real_table_without_its_unit_cell_refuses_the_lines_it_describes
End the table at any title 43 clearance tests 1: test_the_alcohol_receipts_table_stays_outside_the_clearance_table; the title-shaped note, added later, fails too
Skip any cell that contains a number, not only one that holds a number 43 clearance tests 16
Skip the check 43 clearance tests 38
Make growth_in_effect return zero before population's first entry 7 per-head tests 1: test_population_growth_before_its_first_entry_is_that_entry
Skip per_head_growth's index check 7 per-head tests 1: test_per_head_growth_refuses_an_index_that_rounds_to_zero
Drop per_head_growth's division by population 7 per-head tests 7
Divide one plus the rounded growth, not the growth before rounding 7 per-head tests 5, including test_real_remote_gaming_base_per_head_rounds_once
Commit clearance series that count population twice 3 projection tests 3
Map cider to the wine series 4 projection tests 3, before the run's 10-minute cap stopped the fourth
Take cigarettes out of the uprating map and drop their index 4 projection tests 2, before the run's 10-minute cap stopped the other two

Overlap:

Reproducer

Reproducer: policyengine_uk/tests/test_excise_quantity_projection.py::test_weighted_quantities_follow_the_clearances

On origin/main (2da212a) it fails at the first projected year: weighted cigarette quantities grow by population (1.0033), where the OBR's clearances fall to 0.9003.

axiom: n/a: this projects data inputs from an OBR forecast and changes no rule.

🤖 Generated with Claude Code

MaxGhenis and others added 30 commits October 31, 2026 06:12
The importer now reads the OBR's tobacco clearances (Table 4.11) and
alcohol clearance forecasts (Table 4.12) and writes six per-head series
under gov.economic_assumptions.yoy_growth.obr.per_capita, one per product
group: clearance growth over one plus ons.population growth, to six
decimals. Dataset projection grows household weights by ons.population,
so weighted quantities grow with the clearances. Each quantity input
declares its group's index and is mapped to its series in
uprating_indices.yaml. The real-table fixtures gain the clearance sheets,
with each kept sheet's XML and the shared strings unchanged.

Fixes #2596.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ties

The excise duties page now maps each quantity input to its OBR line,
shows from the projection code why the series divide clearance growth by
population growth, lists the Budget 2026 measures that reach the two
duties, and compares the model's duty on the OBR's volumes with its
receipts forecast. It records each source file, sheet, row and hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dices

Takes #2738's index check (84b1dbf), which now also covers the
clearance series.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A clearance spec now refuses rates and an outturn of its own, which
per_head_growth cannot use. The tail comment states the formula, which
holds once population growth turns negative in 2055. The differential
test also checks gov_balance. The README's unit rule names receipts
tables only, and its refresh step lists every pin and the wine line's
footnote change. The docs name the October 2024 tobacco rise and say
which hashes are of the original downloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dices

Merges #2761 at d08fbb6. Its growth_ratio, check_index,
population_growth, growth_in_effect and per_head_growth replace this
branch's copies, each defined once. ReceiptsSeriesSpec.per_head now
covers the clearance specs, which sit under per_capita, so the renderer
and summary write them per head through the shared helper. The
clearance tail and comment basis stay.

The importer rebuilds the merged yoy_growth.yaml byte for byte from the
real October 2026 and March 2026 tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… head

A line's row may hold only its label and a value under each year. The
importer used to skip any other cell, so a "kg" cell beside "Other tobacco
products" let the line borrow the thousand kg its non-cigarette total
states, and growth read 999.0. A clearance table's year row holds only
years, and the one cell that states a clearance line's unit must describe
the line: its own label, the label of the total it breaks down (the row
above "of which:"), or a note.

ReceiptsGrowth carries each year's growth before rounding, and
per_head_growth divides that, so a gambling base with its own outturn
(previous_line None) grows per head by its base growth, not by its
receipts or a growth rebuilt from levels and rates. The committed
yoy_growth.yaml is unchanged byte for byte.

The population test now covers all 13 per-head series: after the merge
with #2761 the seven gambling bases move with population too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OBR states each clearance unit, splits Table 4.11's non-cigarette
clearances, incorporates the Budget's measures in its forecast and lists
them in its policy Table 1.1; tables, notes and forecasts no longer act.

On the national construct the model's alcohol duty exceeds the OBR's
receipts by 3.5% in 2027-28 and 0.3% in 2028-29 and first falls short in
2029-30, by 2.5%; tobacco duty falls short from 2025-26. The excise page
said both fell short from 2027-28.

The economic assumptions README says the per-head series keep the old
population's values until the importer reruns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The importer reads every clearance unit the OBR states in a table against a
closed list (sticks, kg, litres or hectolitres of alcohol or of product, at
the scales it knows) and compares units by measure and size. It reads a
unit as a line's in the line's label, its total's label, another line's
label that names its product, or a note that names its product or no
product. It refuses a line when such a cell states another unit or scale,
when a note states two units it cannot tie to products, when a note or a
line's cell holds a quantity word outside the list, or when it finds the
unit only in a note about another product. Each spec names its products.

Tests: both round-2 witnesses refused by message; notes about other
products; respellings that agree; a Hypothesis property over labels and
notes in two releases. Test docstrings drop reviewer names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README, the clearance spec's docstring and two test docstrings read as
growth over one plus population growth; they now state one plus the
growth, over one plus population growth, minus one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The importer no longer reads units or products from a clearance table's
words. CLEARANCE_TABLE_TEXTS holds, for each table, the text of every cell
in the March 2025, March 2026 and October 2026 tables, and the importer
refuses any other text from the title's row to the next table's title
(CLEARANCE_TABLE_ENDS), in every column, naming the table and the cell.
Each line also needs the exact cell that states its unit.

The unit parser, its product aliases and their tests go. New tests cover
each witness class on the real tables, the list against the three
releases, the receipts table below the alcohol clearances, and a property
that inserts any text anywhere in a table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The excise duties guide and the economic assumptions README now say what
the importer does: it interprets no words in the clearance tables, checks
every cell against the texts someone has read, needs each line's unit
cell, and names the table and cell it refuses. The README keeps the check
that each alcohol line covers the same products in both EFOs, and the
changelog states the projection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spec's error says which texts lack the unit cell, the docstrings name
who acts, and the per-head docstrings write the ratio as one plus the
aggregate growth over one plus population growth. The refusal test adds a
note shaped like a table's title, and the property draws two such texts,
since sheet_tables starts a table at one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The XLSX reader returned only the first text run of an inline string, so
the clearance check compared a prefix of a formatted cell with its list.
Inline and shared strings now share one helper that joins every <t>.

validate_clearance_table checked the first block with the table's number
and title while find_receipts_line could take a line from a later block
with the same heading. It now refuses a sheet that repeats the heading and
names the worksheet rows, which read_sheet_rows keeps as SheetRow.number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README and the guide now say the importer compares text cells from
each clearance table's title through its configured end, and checks no
cell above the title or below the end. They add the stop on a repeated
heading and the reading of every formatting run, and split the guide's
sentence on duty per unit cleared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Docstrings only: their ASTs match the previous commit's once each
docstring drops out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The closed list of clearance texts does not settle a line's unit: two
rearranged copies of the real October 2026 Table 4.11 keep every text on
the list and still put hand-rolled and other tobacco under the cigarette
label's million sticks. The importer again refuses any text beside the
years in the fiscal-year row over a clearance line, and requires each part
of a total (part_of_total) to sit under an "of which:" row directly below
the total's label, with only the total's other parts between
(validate_clearance_part).

_string_text now joins a string's own <t> and each run's <t>, so the <t>
of a phonetic run (<rPh>) no longer joins a cell's text. An empty cell
whose phonetic run spelt the alcohol receipts title could otherwise end
the clearance table above an unread note.

Tests: the two layouts and a second year row as refusal tests, the year
row and a part's structure in the arbitrary-text property, phonetic runs
in the formatting-run property, and a phonetic end title on the real
Table 4.12. README and guide list both checks; four docstrings reflowed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comments, docstrings, the README step and the guide only: the importer
and its tests parse to the same syntax trees as at 433b20f with
docstrings left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
validate_clearance_part now needs the total's row to hold its label and
one or more numbers after it, and nothing else, so a row that holds only
the label no longer counts as the total. The refusal names the total's
label as receipts_label gives it. A parametrized test inserts such a row
into each real release's tobacco clearances and checks the refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6L5XVCEvFyRi9rsktq
test_import_obr_forecasts.py and test_excise_quantity_projection.py each
gain a module-scoped autouse fixture that clears real_clearance_rows and
current_law after the module's tests, as test_test_module_caches_released
requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6L5XVCEvFyRi9rsktq
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6L5XVCEvFyRi9rsktq
Replace the total's-label test with one parametrized over five layouts
and the three releases: a label row without figures, a text after the
total's figures, a text beside "of which:" and an empty row above or
below it. Make the part check's message describe the whole layout it
needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N2tyHRTCRBaS4PzLNYo6tN
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Aggregate impact on certified data: pass

Base 9dd44e142c against head 7e0d7af800, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

  • In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Caseloads

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles; standard reform costings (change in gov_balance); standard reforms: change in each reform's target.

Checks failing on both base and head (1)
  • VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-53,888,684 in total; 2026-27: 76 units below zero, a weighted £-55,777,260 in total; 2027-28: 76 units below zero, a weighted £-57,727,917 in total
Counted-once check

Each of 92 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.

  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)
Provenance
Side Commit Runtime Environment hash Partition hash Time
base main 9dd44e142c uk 2.174.3 (files 2f333d0d4734), core 3.34.1, py 3.13.18 aarch64 dadc93b7ef0c 7cb61324a46b 2346.13s
head #2829 merged into main 7e0d7af800 uk 2.174.3 (files a3fdef6fb6fb), core 3.34.1, py 3.13.18 aarch64 dadc93b7ef0c 7cb61324a46b 2329.61s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Final charge on cash interest in non-cash ISAs £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Lifetime ISA withdrawal charge £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Income tax £294.46bn £315.23bn £335.82bn £351.28bn £366.96bn £385.36bn £399.99bn
National Insurance (employee and self-employed) £51.70bn £54.14bn £56.13bn £57.73bn £59.76bn £61.64bn £63.51bn
National Insurance (employer) £148.71bn £155.37bn £160.88bn £165.09bn £172.06bn £177.36bn £182.57bn
Employer NI change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Employer NI change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Capital gains tax £12.60bn £13.15bn £13.60bn £14.11bn £14.67bn £15.24bn £15.80bn
Student loan repayments £7.82bn £8.13bn £8.39bn £8.58bn £8.79bn £9.04bn £9.29bn
VAT excluding private school fees £249.37bn £256.87bn £266.33bn £272.70bn £278.97bn £285.24bn £291.59bn
Change in VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
VAT on private school fees £1.95bn £2.01bn £2.09bn £2.14bn £2.20bn £2.26bn £2.31bn
Change in private school VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Fuel duties £24.71bn £24.13bn £26.00bn £26.83bn £26.51bn £25.78bn £24.63bn
Alcohol duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tobacco duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Vehicle excise duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Betting and gaming duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
TV licence fee £4.26bn £4.40bn £4.41bn £4.42bn £4.43bn £4.44bn £4.45bn
Council tax £49.21bn £52.16bn £54.89bn £57.95bn £61.16bn £64.47bn £67.91bn
High value council tax surcharge £0.00bn £0.00bn £0.00bn £0.48bn £0.49bn £0.50bn £0.51bn
Northern Ireland domestic rates £0.55bn £0.58bn £0.60bn £0.63bn £0.66bn £0.69bn £0.72bn
Business rates £34.68bn £36.20bn £38.00bn £39.46bn £42.27bn £43.25bn £44.29bn
Stamp duty land tax on the household's own purchases £6.88bn £7.32bn £7.66bn £8.06bn £8.47bn £8.91bn £9.35bn
Stamp duty land tax paid by companies £4.72bn £4.57bn £5.29bn £6.03bn £6.37bn £6.92bn £7.45bn
Land transaction tax (Wales) £0.07bn £0.08bn £0.08bn £0.09bn £0.10bn £0.10bn £0.11bn
Land and buildings transaction tax (Scotland) £0.31bn £0.32bn £0.33bn £0.35bn £0.36bn £0.38bn £0.40bn
Unspecified tax change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Unspecified tax change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax excluding the main residence £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Land value tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Carbon tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Spending by benefit and service: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
State Pension £121.68bn £125.92bn £126.72bn £129.48bn £132.98bn £136.65bn £140.41bn
Pension Credit £6.21bn £6.46bn £6.68bn £6.82bn £6.89bn £7.07bn £7.22bn
Winter Fuel Payment £1.86bn £1.85bn £1.79bn £1.77bn £1.78bn £1.78bn £1.78bn
Pension Age Winter Heating Payment (Scotland) £0.19bn £0.20bn £0.20bn £0.21bn £0.21bn £0.22bn £0.22bn
Universal Credit £75.48bn £80.52bn £82.95bn £84.96bn £86.11bn £87.61bn £88.21bn
Housing Benefit £8.56bn £8.89bn £9.23bn £9.46bn £9.69bn £9.99bn £10.27bn
Council Tax Reduction £2.07bn £2.20bn £2.32bn £2.45bn £2.59bn £2.73bn £2.87bn
Employment and Support Allowance (income-related) £0.79bn £0.82bn £0.84bn £0.86bn £0.88bn £0.90bn £0.92bn
Employment and Support Allowance (contribution-based) £5.20bn £5.37bn £5.56bn £5.69bn £5.82bn £5.95bn £6.09bn
Jobseeker's Allowance (income-based) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Jobseeker's Allowance (contribution-based) £0.29bn £0.30bn £0.31bn £0.31bn £0.32bn £0.33bn £0.34bn
Income Support £0.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Working Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Incapacity Benefit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Personal Independence Payment £26.95bn £28.03bn £29.05bn £30.09bn £30.75bn £31.51bn £32.21bn
Disability Living Allowance £7.49bn £7.79bn £8.07bn £8.36bn £8.55bn £8.76bn £8.95bn
Attendance Allowance £8.68bn £9.03bn £9.36bn £9.69bn £9.90bn £10.15bn £10.38bn
Carer's Allowance £4.48bn £4.66bn £4.83bn £5.00bn £5.11bn £5.27bn £5.42bn
Carer Support Payment (Scotland) £0.31bn £0.32bn £0.33bn £0.34bn £0.35bn £0.36bn £0.37bn
Carer's Allowance Supplement (Scotland) £0.00bn £0.04bn £0.04bn £0.05bn £0.05bn £0.05bn £0.05bn
Severe Disablement Allowance £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn
Industrial Injuries Disablement Benefit £0.50bn £0.52bn £0.54bn £0.55bn £0.56bn £0.58bn £0.59bn
Armed Forces Compensation Scheme £0.61bn £0.63bn £0.65bn £0.66bn £0.68bn £0.69bn £0.71bn
War Widow's or Widower's Pension £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Benefit £16.19bn £16.82bn £17.39bn £18.00bn £18.39bn £18.84bn £19.25bn
Scottish Child Payment £0.49bn £0.52bn £0.53bn £0.55bn £0.56bn £0.57bn £0.56bn
Two Child Limit Payment (Scotland, counterfactual) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Healthy Start vouchers £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn
Maternity Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Maternity Pay £3.30bn £3.41bn £3.54bn £3.62bn £3.70bn £3.78bn £3.87bn
Statutory Paternity Pay £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Sick Pay £1.12bn £1.16bn £1.20bn £1.23bn £1.26bn £1.29bn £1.32bn
Sure Start Maternity Grant £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn
Bereavement Support Payment £0.67bn £0.69bn £0.71bn £0.73bn £0.75bn £0.76bn £0.78bn
Widowed Parent's Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tax-Free Childcare £0.56bn £0.57bn £0.57bn £0.58bn £0.60bn £0.61bn £0.59bn
Extended childcare entitlement £2.99bn £3.50bn £3.59bn £3.71bn £3.78bn £3.86bn £3.93bn
Universal childcare entitlement £2.06bn £2.22bn £2.29bn £2.37bn £2.43bn £2.48bn £2.53bn
Targeted childcare entitlement £0.50bn £0.51bn £0.52bn £0.54bn £0.55bn £0.57bn £0.56bn
Childcare Offer for Wales £0.26bn £0.28bn £0.28bn £0.29bn £0.30bn £0.31bn £0.32bn
Care to Learn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Childcare Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Parents' Learning Allowance £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn
Adult Dependants' Grant £0.06bn £0.06bn £0.04bn £0.04bn £0.04bn £0.04bn £0.04bn
Travel Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
NHS spending £203.89bn £204.30bn £204.56bn £204.91bn £205.34bn £205.85bn £206.35bn
State education spending £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn
Transport subsidy spending £25.05bn £25.10bn £25.14bn £25.18bn £25.23bn £25.30bn £25.36bn
Cost-of-living support payment £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Bills Rebate £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Household Energy Payment £0.00bn £2.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Price Guarantee subsidy £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disability basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Other public spending budget change £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Caseloads: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus 0k 0k 0k 0k 0k 0k 0k
State Pension 11,264k 11,145k 10,819k 10,700k 10,723k 10,750k 10,776k
Pension Credit 1,489k 1,484k 1,476k 1,454k 1,452k 1,457k 1,466k
Winter Fuel Payment 7,844k 7,783k 7,473k 7,375k 7,391k 7,409k 7,427k
Pension Age Winter Heating Payment (Scotland) 803k 802k 800k 797k 799k 801k 802k
Universal Credit 6,342k 6,347k 6,336k 6,297k 6,292k 6,271k 6,223k
Housing Benefit 1,249k 1,245k 1,244k 1,227k 1,227k 1,229k 1,232k
Council Tax Reduction 3,180k 3,200k 3,206k 3,219k 3,234k 3,245k 3,252k
Employment and Support Allowance (income-related) 71k 71k 71k 71k 71k 71k 71k
Employment and Support Allowance (contribution-based) 719k 720k 721k 722k 724k 726k 728k
Jobseeker's Allowance (income-based) 0k 0k 0k 0k 0k 0k 0k
Jobseeker's Allowance (contribution-based) 67k 67k 67k 67k 67k 68k 68k
Income Support 25k 0k 0k 0k 0k 0k 0k
Working Tax Credit 0k 0k 0k 0k 0k 0k 0k
Child Tax Credit 0k 0k 0k 0k 0k 0k 0k
Incapacity Benefit 0k 0k 0k 0k 0k 0k 0k
Personal Independence Payment 3,567k 3,575k 3,579k 3,585k 3,593k 3,602k 3,611k
Disability Living Allowance 1,291k 1,294k 1,296k 1,298k 1,300k 1,304k 1,307k
Attendance Allowance 1,695k 1,698k 1,700k 1,703k 1,707k 1,711k 1,715k
Carer's Allowance 1,034k 1,037k 1,038k 1,040k 1,042k 1,060k 1,062k
Carer Support Payment (Scotland) 71k 71k 71k 72k 72k 72k 72k
Carer's Allowance Supplement (Scotland) 0k 71k 71k 72k 72k 72k 72k
Severe Disablement Allowance 4k 4k 4k 4k 4k 4k 4k
Industrial Injuries Disablement Benefit 137k 137k 137k 138k 138k 138k 139k
Armed Forces Compensation Scheme 93k 93k 94k 94k 94k 94k 94k
War Widow's or Widower's Pension 0k 0k 0k 0k 0k 0k 0k
Child Benefit 8,123k 8,120k 8,120k 8,130k 8,145k 8,161k 8,178k
Scottish Child Payment 351k 352k 352k 353k 350k 348k 337k
Two Child Limit Payment (Scotland, counterfactual) 0k 0k 0k 0k 0k 0k 0k
Healthy Start vouchers 200k 200k 200k 201k 201k 201k 202k
Maternity Allowance 0k 0k 0k 0k 0k 0k 0k
Statutory Maternity Pay 333k 333k 334k 334k 335k 336k 337k
Statutory Paternity Pay 0k 0k 0k 0k 0k 0k 0k
Statutory Sick Pay 181k 182k 182k 182k 182k 183k 183k
Sure Start Maternity Grant 26k 26k 26k 26k 26k 26k 26k
Bereavement Support Payment 98k 98k 98k 98k 98k 99k 99k
Widowed Parent's Allowance 0k 0k 0k 0k 0k 0k 0k
Tax-Free Childcare 1,043k 1,027k 1,014k 1,009k 1,011k 1,009k 993k
Extended childcare entitlement 795k 800k 796k 797k 798k 798k 792k
Universal childcare entitlement 1,071k 1,072k 1,072k 1,073k 1,076k 1,079k 1,074k
Targeted childcare entitlement 165k 156k 154k 154k 155k 155k 150k
Childcare Offer for Wales 39k 39k 39k 39k 39k 39k 39k
Care to Learn 0k 0k 0k 0k 0k 0k 0k
Childcare Grant 0k 0k 0k 0k 0k 0k 0k
Parents' Learning Allowance 98k 98k 98k 99k 99k 99k 99k
Adult Dependants' Grant 18k 18k 12k 12k 12k 12k 12k
Travel Grant 0k 0k 0k 0k 0k 0k 0k
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund 0k 0k 0k 0k 0k 0k 0k
Cost-of-living support payment 0k 0k 0k 0k 0k 0k 0k
Energy Bills Rebate 0k 0k 0k 0k 0k 0k 0k
Household Energy Payment 0k 21,065k 0k 0k 0k 0k 0k
Energy Price Guarantee subsidy 0k 0k 0k 0k 0k 0k 0k
Basic income 0k 0k 0k 0k 0k 0k 0k
Disability basic income 0k 0k 0k 0k 0k 0k 0k
Government balance, each component counted once: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Government receipts, each component once £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.89bn
Government spending £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
Government balance, each receipt once £363.07bn † £389.57bn † £430.34bn † £457.08bn † £487.75bn † £516.37bn † £542.11bn †
Benefits in household_benefits but not in gov_spending £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Model totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
gov_tax £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.89bn
household_tax £491.97bn £520.41bn £551.21bn £576.00bn £601.00bn £626.69bn £648.42bn
pre_budget_change_household_tax £484.15bn £512.28bn £542.82bn £566.95bn £591.72bn £617.16bn £638.61bn
gov_spending £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
household_benefits £528.93bn † £545.10bn † £550.16bn † £558.85bn † £566.49bn † £575.19bn † £582.78bn †
gov_balance £363.07bn † £389.57bn † £430.34bn † £457.08bn † £487.75bn † £516.37bn † £542.11bn †
Household income totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Household market income £1,637.96bn £1,702.77bn £1,757.35bn £1,806.41bn £1,855.97bn £1,912.43bn £1,970.44bn
Household net income £1,636.24bn † £1,687.25bn † £1,714.85bn † £1,746.83bn † £1,777.98bn † £1,816.26bn † £1,858.87bn †
Household net income (HBAI definition) £1,449.13bn £1,499.27bn £1,530.25bn £1,564.51bn £1,597.18bn £1,634.83bn £1,675.80bn
Population: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
People 69,245k 69,383k 69,473k 69,592k 69,738k 69,912k 70,080k
Benefit units 35,110k 35,181k 35,226k 35,286k 35,360k 35,449k 35,534k
Households 31,046k 31,108k 31,149k 31,201k 31,267k 31,345k 31,420k
Poverty rates: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 16.3% 15.5% 16.4% 16.3% 16.5% 16.5% 16.6%
Absolute poverty, before housing costs: children 24.7% 22.1% 23.3% 22.9% 23.1% 23.2% 23.5%
Absolute poverty, before housing costs: working-age adults 14.1% 13.8% 14.7% 14.7% 14.8% 14.8% 15.0%
Absolute poverty, before housing costs: pensioners 13.7% 13.4% 14.0% 13.8% 13.9% 14.0% 13.9%
Absolute poverty, after housing costs: everyone 21.6% 21.0% 22.0% 22.6% 22.9% 23.1% 23.4%
Absolute poverty, after housing costs: children 33.3% 31.3% 32.3% 32.5% 32.9% 33.4% 34.2%
Absolute poverty, after housing costs: working-age adults 19.9% 19.8% 20.8% 21.6% 21.9% 22.0% 22.3%
Absolute poverty, after housing costs: pensioners 12.7% 12.4% 13.1% 13.2% 13.3% 13.4% 13.6%
Relative poverty, before housing costs: everyone 17.0% 16.0% 16.4% 16.3% 16.4% 16.3% 16.6%
Relative poverty, before housing costs: children 25.9% 22.9% 23.3% 22.9% 23.0% 22.9% 23.5%
Relative poverty, before housing costs: working-age adults 14.5% 14.1% 14.6% 14.7% 14.7% 14.7% 15.0%
Relative poverty, before housing costs: pensioners 14.5% 14.0% 13.9% 13.8% 13.8% 13.9% 13.9%
Relative poverty, after housing costs: everyone 21.5% 21.0% 21.3% 21.3% 21.3% 21.5% 21.7%
Relative poverty, after housing costs: children 33.1% 31.1% 31.2% 31.0% 31.1% 31.2% 31.8%
Relative poverty, after housing costs: working-age adults 19.9% 19.8% 20.2% 20.4% 20.5% 20.6% 20.8%
Relative poverty, after housing costs: pensioners 12.5% 12.4% 12.2% 11.7% 11.7% 11.8% 12.0%
People in poverty: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 11,277k 10,778k 11,402k 11,365k 11,475k 11,514k 11,650k
Absolute poverty, before housing costs: children 3,688k 3,314k 3,496k 3,445k 3,480k 3,502k 3,549k
Absolute poverty, before housing costs: working-age adults 5,965k 5,893k 6,308k 6,362k 6,420k 6,430k 6,521k
Absolute poverty, before housing costs: pensioners 1,624k 1,571k 1,597k 1,558k 1,575k 1,581k 1,580k
Absolute poverty, after housing costs: everyone 14,944k 14,599k 15,310k 15,721k 15,937k 16,140k 16,426k
Absolute poverty, after housing costs: children 4,975k 4,682k 4,837k 4,888k 4,945k 5,040k 5,179k
Absolute poverty, after housing costs: working-age adults 8,463k 8,461k 8,971k 9,344k 9,491k 9,584k 9,706k
Absolute poverty, after housing costs: pensioners 1,506k 1,456k 1,501k 1,489k 1,501k 1,517k 1,541k
Relative poverty, before housing costs: everyone 11,750k 11,084k 11,387k 11,364k 11,416k 11,416k 11,654k
Relative poverty, before housing costs: children 3,872k 3,423k 3,496k 3,445k 3,466k 3,454k 3,551k
Relative poverty, before housing costs: working-age adults 6,159k 6,013k 6,301k 6,362k 6,389k 6,389k 6,523k
Relative poverty, before housing costs: pensioners 1,719k 1,648k 1,590k 1,558k 1,561k 1,573k 1,580k
Relative poverty, after housing costs: everyone 14,864k 14,552k 14,777k 14,809k 14,872k 15,012k 15,232k
Relative poverty, after housing costs: children 4,944k 4,660k 4,673k 4,663k 4,679k 4,707k 4,806k
Relative poverty, after housing costs: working-age adults 8,431k 8,440k 8,711k 8,823k 8,873k 8,967k 9,062k
Relative poverty, after housing costs: pensioners 1,489k 1,452k 1,393k 1,323k 1,321k 1,338k 1,364k
Inequality: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Gini coefficient, equivalised HBAI income before housing costs 0.3078 0.3050 0.3056 0.3055 0.3060 0.3066 0.3082
Mean household net income in each side's own deciles: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Decile 1: mean household_net_income per household £17,681 † £17,998 † £17,916 † £18,152 † £18,308 † £18,533 † £18,718 †
Decile 2: mean household_net_income per household £27,942 † £29,026 † £28,715 † £29,017 † £29,280 † £29,698 † £30,156 †
Decile 3: mean household_net_income per household £34,316 † £35,481 † £36,069 † £36,606 † £37,195 † £37,732 † £38,182 †
Decile 4: mean household_net_income per household £38,934 † £40,684 † £41,726 † £42,728 † £43,331 † £44,130 † £44,432 †
Decile 5: mean household_net_income per household £43,235 † £45,581 † £46,012 † £46,961 † £47,713 † £48,725 † £50,296 †
Decile 6: mean household_net_income per household £47,931 † £49,202 † £50,891 † £51,936 † £52,936 † £53,820 † £54,663 †
Decile 7: mean household_net_income per household £55,659 † £56,569 † £57,437 † £57,642 † £58,733 † £59,661 † £61,087 †
Decile 8: mean household_net_income per household £68,784 † £70,802 † £71,864 † £73,512 † £73,886 † £75,319 † £77,177 †
Decile 9: mean household_net_income per household £78,889 † £81,012 † £80,859 † £82,645 † £85,463 † £86,945 † £89,508 †
Decile 10: mean household_net_income per household £116,861 † £120,161 † £124,545 † £126,420 † £125,753 † £128,368 † £130,693 †
Standard reform costings (change in gov_balance): head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp £6.41bn £6.66bn £6.63bn £6.85bn £7.03bn £7.25bn £7.49bn
Raise higher rate by 1pp £3.61bn £3.87bn £4.00bn £4.22bn £4.44bn £4.71bn £4.83bn
Raise personal allowance by ~800GBP/year £-3.81bn £-3.93bn £-3.99bn £-4.05bn £-4.11bn £-4.24bn £-1.95bn
Raise child benefit by 25GBP/week per additional child £-1.99bn £-1.75bn £-1.55bn £-1.38bn £-1.30bn £-1.18bn £-1.07bn
Reduce Universal Credit taper rate to 20% £-20.77bn £-21.32bn £-21.75bn £-22.00bn £-22.35bn £-22.75bn £-22.94bn
Raise Class 1 main employee NICs rate to 10% £10.89bn £11.30bn £11.71bn £12.01bn £12.31bn £12.62bn £13.00bn
Raise VAT standard rate by 2pp £24.89bn £25.71bn £26.63bn £27.24bn £27.87bn £28.50bn £29.14bn
Raise additional rate by 3pp £2.90bn £3.15bn £3.22bn £3.41bn £3.61bn £3.85bn £4.14bn
Raise the Universal Credit standard allowance by 10% £-3.06bn £-3.18bn £-3.26bn £-3.35bn £-3.44bn £-3.49bn £-3.50bn
Raise the State Pension by 10% £-7.76bn £-7.99bn £-8.22bn £-8.45bn £-8.80bn £-8.53bn £-8.87bn
Standard reforms: change in each reform's target: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp: change in income_tax £6.55bn £6.86bn £6.79bn £7.00bn £7.20bn £7.44bn £7.66bn
Raise higher rate by 1pp: change in income_tax £3.61bn £3.92bn £4.01bn £4.23bn £4.45bn £4.72bn £4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax £-3.93bn £-4.07bn £-4.13bn £-4.18bn £-4.22bn £-4.40bn £-1.75bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit £2.33bn £2.14bn £1.94bn £1.76bn £1.65bn £1.52bn £1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit £20.74bn £21.31bn £21.79bn £22.05bn £22.37bn £22.78bn £22.99bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance £11.15bn £11.63bn £12.00bn £12.29bn £12.61bn £12.94bn £13.31bn
Raise VAT standard rate by 2pp: change in vat £24.69bn £25.51bn £26.42bn £27.03bn £27.65bn £28.28bn £28.91bn
Raise additional rate by 3pp: change in income_tax £2.90bn £3.15bn £3.22bn £3.42bn £3.62bn £3.86bn £4.12bn
Raise the Universal Credit standard allowance by 10%: change in universal_credit £3.06bn £3.19bn £3.28bn £3.37bn £3.43bn £3.49bn £3.51bn
Raise the State Pension by 10%: change in state_pension £10.68bn £11.17bn £11.28bn £11.60bn £12.03bn £12.46bn £12.96bn

@MaxGhenis

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Verdict: APPROVE WITH NITS at 0a68b3fcf37b9e56fb7547edac24fd79bd38bfd7 (branch fix-2596-excise-quantity-indices, delta from 8f3f0f396e9318de59032110aaa16351273f9ac3).

The last round's P2 and all three P3s are closed. The five TOTAL_LAYOUTS cases each add a text or a row and overwrite none, on all three releases. Each of the last round's three mutants now dies on the whole file, with exactly the failures the body names. The new refusal message describes every way validate_clearance_part refuses, and not_under_total matches it word for word. Against main (6c6e229a5, which carries #2761), the PR's diff holds only #2596's change.

I found one optional P3, on two docs that the delta didn't touch. CI on this head is red only because shard 2/6 hit the job's 75-minute timeout with no failing test. That isn't caused by this delta (details below).

P3

P3-1 (optional): two docs describe the layout less fully than the message and the new tests. docs/engineering/excise-duties.md:298–300 and policyengine_uk/parameters/gov/economic_assumptions/README.md:525–528 say the importer stops when a part "does not sit under an "of which:" row directly below the non-cigarette total's row, which holds its label and figures, with only the total's other parts between". The code also refuses a total's row that holds anything else after its figures (import_obr_forecasts.py:1405–1406), and an "of which:" row that holds another cell (:1402). text-on-total-row and text-on-of-which-row now pin both refusals. Neither doc is wrong, since they list what the importer needs and not every way it refuses, so this is optional. If the author takes it, in docs/engineering/excise-duties.md replace

- hand-rolled or other tobacco does not sit under an "of which:" row
  directly below the non-cigarette total's row, which holds its label and
  figures, with only the total's other parts between;

with

- hand-rolled or other tobacco does not sit under an "of which:" row with
  no other cell, directly below the non-cigarette total's row, which holds
  its label, its figures and nothing else, with only the total's other
  parts between;

and make the same two insertions ("with no other cell", "its label, its figures and nothing else") in the README sentence. The README edit sits in the paragraph that conflicts with fix-2721-import-nlw-forecast, so leaving it as it is costs nothing.

Previous round's findings

Finding State Evidence
P2: three refusals untested Closed test_a_part_needs_its_total_directly_above_of_which × TOTAL_LAYOUTS (test_import_obr_forecasts.py:4006–4054), 15 cases; the three mutants are killed (below)
P3-1: message blames the total's row Closed import_obr_forecasts.py:1408–1416 is the review's text; not_under_total (test_import_obr_forecasts.py:3876–3888) matches it
P3-2: test count Closed grep -c '^def test_': 84 at 0037137ad, 115 at 0a68b3fcf; the body says 31 and names the five layouts
P3-3: overlap note Closed The body names main at 6c6e229a5 and 47bda0a7a at this head; both merge cleanly (below)

Checked and sound

I ran everything in a detached worktree at 0a68b3fcf, after uv sync --frozen --extra dev, with HUGGING_FACE_TOKEN and POLICYENGINE_UK_DEFAULT_DATASET unset, on 4 cores. uv run python -c "import hypothesis; print(hypothesis.__version__)" prints 6.170.11.

  • Whole file, unmutated. uv run pytest policyengine_uk/tests/test_import_obr_forecasts.py -n 4 -q -o xfail_strict=true: 208 passed in 121.09 s.

  • The layouts add and overwrite nothing. I printed the rows with real_clearance_rows and receipts_label, as the test does:

    • March 2025 (sheet 3.11) and March 2026 (sheet 3.11): the total's row has keys B–I only, so it has no N key. The "of which:" row has C: None.
    • October 2026 (sheet 4.11): the total's row has N: None, and the "of which:" row has C: None.
    • In each release, the row directly below "of which:" is "Hand rolled tobacco clearances". So text-on-total-row and text-on-of-which-row each fill an empty cell, and the other three insert a row. The March 2025 marker reads "Of which:", and receipts_label lower-cases it, so the test finds it.
  • The last round's three mutants, each alone in the worktree, on the whole file (same command), restored with git checkout after each:

    Mutant Result
    Drop or not all(NUMBER_RE.fullmatch(text) for text in total[1:]) Killed: 3 failed, 205 passed. text-on-total-row in all three releases
    of_which != ["of which:"] → of_which[:1] != ["of which:"] Killed: 3 failed, 205 passed. text-on-of-which-row in all three releases
    above = [t for t in (texts(row) for row in table_rows[:line_index]) if t] (drop empty rows) Killed: 6 failed, 202 passed. empty-row-above-of-which and empty-row-below-of-which in all three releases

    These match the body's rows and counts.

  • Two more body rows rerun, on the same command:

    • Drop or not total[1:]: 4 failed. The three label-without-figures cases and test_any_text_no_one_has_read_stops_a_real_clearance_table.
    • return at the top of validate_clearance_part: 17 failed. cigarette-unit-of-which, all 15 TOTAL_LAYOUTS cases and the same property.

    Both match the body.

  • The message against every refusal (import_obr_forecasts.py:1402–1407, one raise for all):

    • no "of which:" row, or one with another cell: "an "of which:" row with no other cell";
    • an empty row or another row between "of which:" and the parts: "directly above the line and the total's other parts";
    • an empty row above "of which:", a wrong first text, a label with no number, or a non-number after the label: "directly above that row the non-cigarette total's row, which holds its label (…), its figures and nothing else".

    The test copy differs from the importer only in ruff's quotes around ".".join(spec.path), and the 15 exact-match assertions pass. git grep finds no use left of the old message's wording ("lacks the non-cigarette total's label", "needs the label and figures") or of the old test name.

  • The test can fail. Each case asserts the exact message for both parts, and that cigarettes' levels equal the unedited table's. The five mutants above show each layout is the only test that holds its clause, apart from the property's draw for or not total[1:].

  • Body.

  • Diff against main holds only Alcohol and tobacco quantity inputs are not uprated, so later years charge duty on base-year volumes #2596. origin/main is 6c6e229a51fbd804dab2062dac1f94af04c609c2. git merge-base origin/main 0a68b3fcf is 0037137adc62…, Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761's head. The 28 non-merge commits after it are all Alcohol and tobacco quantity inputs are not uprated, so later years charge duty on base-year volumes #2596's. The three merges bring in only fix-2738-gambling-bases commits, all now in main. git diff --stat origin/main...0a68b3fcf lists 23 files: the changelog fragment 2596.fixed.md, the excise docs, uprating_indices.yaml, the economic assumptions README, yoy_growth.yaml, the receipts fixtures and their README, the two test files, the importer and the eleven quantity inputs. None of them belongs to Project the gambling duty bases with the OBR's October 2026 betting and gaming forecast #2761 alone.

  • Merges (git merge-tree --write-tree --name-only 0a68b3fcf <sha>): main 6c6e229a5 is clean (tree 9af40ad3d…), and 47bda0a7a is clean (tree e30e1734d…, as the hand-off says). GitHub reports mergeable: true.

  • Lint. git diff --check 8f3f0f396 0a68b3fcf is clean. uvx ruff@0.17.2 format --check reports "2 files already formatted" and check reports "All checks passed!" on the two changed files.

  • Writing. The new comment above TOTAL_LAYOUTS, the test's docstring and the message use active verbs, say what the code does now and carry no fix history. The comment's empty-cell claim is true (above).

CI on this head (as found, not waited on)

From gh api …/commits/0a68b3fcf…/check-runs:

  • Passed: Lint, Changes, Changelog fragments, Fix evidence, Documentation build, Test documentation builds, Release build records the commit, Aggregate impact, YAML tests, all four smoke-imports, and Python shards 1, 3, 4, 5 and 6.
  • Python 2/6 cancelled, so Test / Result failed and the "Code changes" run (41494566529) shows cancelled.
    • The shard's "Run the tests" step ran from 17:48:40 to 19:04:08Z, and the job sets timeout-minutes: 75 (.github/workflows/test-suite.yaml:93). Its log shows 1,985 PASSED, no FAILED, and the last progress at [ 99%] before ##[error]The operation was canceled.
    • The shard's log names 68 test IDs from test_import_obr_forecasts.py. Locally, the whole file, with the 12 cases this delta adds, ran in 121 s, so those cases can't account for a 75-minute timeout.
    • Shard 2 runs close to the limit on other PRs: fix-2106-allowance-order took 70 min, fix/vat-coverage-a133 74 min, and hb-ctr-non-dep-exemptions was also cancelled at 75 min.
    • A rerun of the shard, or a shorter shard 2, should clear it. This needs no change to this PR.

What I couldn't check

  • Every results-table and mutation row the body attributes to 8d9763345, 8f3f0f396 or earlier heads, the differential and the rebuild hash. This delta changes only an error string and tests, and the byte-for-byte rebuild test is among the 208 that passed.
  • test_excise_quantity_projection.py (10 passed per the hand-off) and the code_health directory (3,069 per the hand-off): I didn't rerun them at this head.
  • .github/check_fix_pr.py on the live body: I didn't run it, but CI's "Fix evidence" check passed on this head.
  • The merge with fix-2721-import-nlw-forecast at 17dbd9067: I didn't recheck it. The body's description of the conflict rests on the hand-off.
  • Whether the shard 2 timeout recurs on a rerun.

Generated by Claude Code

@MaxGhenis
MaxGhenis marked this pull request as ready for review November 3, 2026 10:42

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 92c5fc930df77ef061388a380ac471f374766e39. COMPLETE — no findings or material evidence gaps.

The six current growth series match the independently checked March 2026 and October 2026 OBR workbook cells, units and footnotes. All 42 yearly per-head calculations match the committed values, and the retained XML in both current-source fixtures matches the original workbooks. The population adjustment correctly avoids counting population growth twice.

All 252 selected checks passed, including quantity projection through alcohol/tobacco duty, household tax, government tax/balance and household net income. The importer, reform, horizon and reset checks also passed. Applicable checks are green at this exact head.

The documentation clearly distinguishes OBR's conditional clearance forecasts from a static duty-rate reform and explains the fixed product mix/strength assumptions and the constructed-data limitations. This approval covers the current October-over-March projection; it does not certify the author's national revenue estimates or independently authenticate the older March 2025 compatibility fixture.

@juaristi22
juaristi22 added this pull request to the merge queue Nov 3, 2026
Any commits made after this event will not be merged.
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 4, 2026
main carries #2853, which reads the national living wage forecast from
OBR Table 1.13. In the economic assumptions README, this branch's
clearance text ends step 6, #2853's national living wage step becomes
step 7 and the horizon step step 8. In test_import_obr_forecasts.py,
the merge keeps both branches' imports and fixtures, then main's
national living wage tests and this branch's clearance tests, each block
whole, so each branch's last Hypothesis test keeps its own decorators.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed current head 2df40a383506c309b7de4be9f4cf74ccf6ac7dfe after the merge from main. COMPLETE — no findings or material evidence gaps.

The resolutions preserve the clearance importer alongside main's importer, CLI options, fixtures, tests and documentation. The OBR source basis and clearance parameters are unchanged. All 58 selected repository tests passed, including projected quantities through duties, household tax, government balance and household net income, plus the shared importer/CLI checks.

This retains the forecast, product-mix and population-estimate qualifications in the earlier review. Approval is pending the current head's remaining CI checks; no repeat code review is needed if the reviewed inputs remain unchanged.

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving current head 2df40a383506c309b7de4be9f4cf74ccf6ac7dfe now that its CI has passed. COMPLETE — no open findings or material evidence gaps.

The completed projection/importer/source review stands. The head, description, dependencies and reviewed source evidence are unchanged. The retained 58 passing checks and source/artifact bindings were verified; no tests or calculations were repeated. The forecast, product-mix and population-estimate qualifications from the earlier review remain. Applicable current-head checks pass.

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the updated description at unchanged head 2df40a383506c309b7de4be9f4cf74ccf6ac7dfe. The existing approval stands; no new findings or material evidence gaps.

The engineering notes match the recorded commits: the merged NLW importer and README steps are retained, the test module has 167 distinct test-function names, and the named main/PR overlap checks agree with the revised notes. Those checks verify recorded merge facts, not the behavior of future merges.

The completed code/source assessment and retained 58 passing checks remain valid. No tests, model calculations, private population estimates or author mutation campaigns were repeated or newly certified. Current-head CI remains clear.

@juaristi22
juaristi22 added this pull request to the merge queue Nov 5, 2026
Any commits made after this event will not be merged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Alcohol and tobacco quantity inputs are not uprated, so later years charge duty on base-year volumes

3 participants