Skip to content

UK land value inputs for chronicle#408, on main (re-lands #410) - #415

Open
juaristi22 wants to merge 18 commits into
mainfrom
uk-land-value-inputs-408-main
Open

juaristi22 wants to merge 18 commits into
mainfrom
uk-land-value-inputs-408-main

Conversation

@juaristi22

Copy link
Copy Markdown
Collaborator

This puts #410's code on main. #410 merged into bundle-stream-consumer-rows on 5 November at 16:07 UTC, 86 minutes after #414 had merged that branch into main, so chronicle#408's packages never reached main. #410 has the full description of what they add.

What the branch holds

Tests

Run locally on 9db960f:

Not run locally: the bundle job's two test steps, tests/test_chronicle_uk_national_2025.py and tests/test_chronicle_bundle.py. That job took 3h16m on #410's CI, and it runs again on this PR.

Refs #408. Item 6 waits on #409.

🤖 Generated with Claude Code

juaristi22 and others added 18 commits November 4, 2026 14:09
chronicle#408 item 1: all eight data sheets of MHCLG's 2023 workbook (valuation date
1 October 2023), 10,101 facts, and the generator scripts/build_uk_408_packages.py that writes
the issue's packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 3: tables CTSOP1.0 to 4.0 of the 2026 summary workbook at every
geography they print (222,588 facts in three packages), and the three small-area zips
(CTSOP1.1, 3.1 and 4.1) registered raw-only, in R2 and git-ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 7: domestic certificates lodged, total floor area and habitable rooms
by quarter for England and Wales, each nation, region and local authority (75,051
facts), and dwelling stock density by local authority, 2011 to 2025 (5,406 facts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 4: MHCLG's Tables 6a and 9a for every English billing authority, and
the per-authority rows of the pinned Welsh Table 2 (both years) and SLGFS Tables 2.8 and
2.11. Rows that restate ported facts (Table 6a's England row, QRC4 line 1) are asserted
equal, not ported again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 items 8 and 2, which share a generator section and test module: land
(AN.211) for every sector in both pinned balance sheet workbooks, ONS's 2022 series for
land underlying dwellings, and the Market Value Survey 2024 annex tables the issue
lists. Chapter 4 rows that restate chapter 1 are asserted equal, not ported again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 5: more tables from the pinned Annual Stamp Tax Statistics 2024-25
workbook, both financial years, in hmrc-stamp-tax-statistics-2024-25 (352 to 1,487
facts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 9: preprocessing/config.py, webapp/cells.json and LICENSE of
DanNeidle/lvt_model_2026 at commit fef94f9 (MIT), as a raw-only registration with no
facts, kept in R2 only and git-ignored (config.py is third-party Python source).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The field contracts and caveats of every item in
docs/data-sources/uk-land-value-inputs-408.md, a chronicle#408 section in the PE UK source
checklist (whose Welsh, Scottish and balance sheet rows now count the added rows), and the
changelog fragment. Item 6 waits on chronicle#409.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in the register (#408)

CI's surface job failed one test on 493fe89: the register's static check
(tests/test_dimension_value_register.py) found 396 values that two or more
of this PR's packages emit but chronicle/dimension_values/canonical_labels.json
does not list. Checking the packages' consumer rows with
chronicle.dimension_labels.dimension_label_issues also found values with no
label, which build-bundle refuses ("Every package must label every dimension
and value it emits"):

- Tables 6a and 9a (QRC4): the six billing authority classes;
- the land value estimates: site_density_scenario,
  new_build_price_percentile, agricultural_land_grade,
  office_accommodation_type and retail_location_type;
- the Market Value Survey's chapters 1, 3 and 4: the tenure columns, house
  or flat, occupancy, length of residence (chapter 3), and home value and
  value bands (chapter 4);
- the two VOA crossed packages: Wales' band I, which the register now lists.

scripts/build_uk_408_packages.py now declares those labels from the printed
text:
- each class takes its subtotal row's wording;
- item 1's values take the sheets' headings, and the percentiles, which the
  sheet heads with bare numbers, are labelled "25th percentile" to "75th
  percentile", with "50th percentile (median)";
- a Market Value Survey value takes the row's wording where the package
  prints it as a row, otherwise the column header as printed, otherwise the
  title of the block that sets it.
A value the register labels another way is left to the register, which a
package may not contradict.

The register gains the 396 values by 735e617's method: only the 14 packages
that emit them were built, at 2023. 384 have one wording and take it. Twelve
are worded two ways and take a label in CHOSEN_LABELS:
- "Yes" and "No" for ehs_dependent_children and ehs_household_disability
  and their mhclg. groupby twins. That is the wording most packages use;
  chapter 1's sample-size rows print "Dependent Children - Yes" and the like.
- gbp_1m_to_2m and gbp_2m_or_more, under mvs_market_value_band and its twin.
  Chapter 1 words them "£1,000,000 to £2,000,000" and "£2,000,000 or more",
  chapter 4 "£1m-£2m" and "£2m or more". With one package each, the rule
  does not decide, so chapter 1's wording is proposed for the maintainer's
  review.
The register now holds 8,553 values in 183 dimensions, up from 8,157 in 160.
It still records dece5e9 as its build commit, since only these packages
were rebuilt.

Checked:
- The static check reports no problem.
- All 18 packages this PR adds or changes build with no label issue.
- Seven of the 14 build byte-identical consumer rows before and after, so
  the register relabels nothing there: VOA bands 2025 and 2026, the Welsh
  dwellings package and both collection packages, the 2025 balance sheet and
  the 2022 series.
- The two VOA crossed packages change only by band I's new label.
- No package gains or loses a fact.
- The council tax and Market Value Survey test files pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test

test_the_chosen_labels_are_the_registers_labels pins how many labels
scripts/build_dimension_value_register.py chooses. 03f9689 adds twelve for
the Market Value Survey 2024 packages: eight by the rule, four value bands
for the maintainer. That brings the count from 47 to 59, which is the
surface job's one failure at 03f9689 (1 failed, 5,770 passed). The test's
other check, that each chosen label is the register's label, holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heffield's codes (#408)

Two points from the review of 03f9689:

- Live Table 126 marks 2022 to 2025 '[p]' provisional. Those years'
  record sets now carry shared_filters publication_status: provisional, as
  welshgov-dwelling-stock-by-tenure-2025 does, so a consumer can filter on
  it. The register already labels the value "Provisional". '[r]' (2024)
  stays in the measure header and notes. The fact count is unchanged.
- docs/data-sources/uk-land-value-inputs-408.md gains a table of the codes
  each package carries for Barnsley and Sheffield, E08000016 and E08000019
  or E08000038 and E08000039, including main's council tax levels package.
  The doc says that a join on geography_id drops both authorities unless
  the consumer maps one pair to the other.

Checked: scripts/build_uk_408_packages.py regenerates the package, and
tests/test_source_uk_408_epc.py and tests/test_dimension_value_register.py
pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anch

Both add packages to the same shared files. Keep both checklist sections,
#407's first as main has it, and count the 60 chosen labels: the 47 from
2026-10-26, #407's soc2020_major_group=3 and #408's twelve. The register
merged cleanly and its static check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#414 writes and counts each package's rows as it reads them, so the
bundle no longer holds every row at once. The 16 GB CI runner could not
fit this branch's bundle without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bundle job now fits on the runner (3,578,392 KiB peak with #414's
streaming build) and failed only on the pinned counts. The new pins come
from that run's uploaded reports: 326,495 facts, 12 packages and source
tables, 275 geography keys and 46 period keys. Each change is attributed
per package from the packages' own facts reports, and the attributions
sum to the bundle's changes. Semantic duplicate keys (2,324) and warnings
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nit on #410: Chapter 4 of the Market Value Survey 2024 prints
"£2m or more" in AT4.1 and AT4.2 but "£2m plus" as the column header in
AT4.3 and AT4.4, and the package files both under gbp_2m_or_more. The
comment above the proposed labels now names every printed wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review on #410, round 2, item 3: Chronicle keeps the codes each table
prints, and the consumer maps one vintage to the other. Microcosm's
local-authority crosswalk already lists E08000038 and E08000039 as aliases
of E08000016 and E08000019 (PolicyEngine/microcosm#931, item (e)), so the
doc section now says so instead of pointing at a Chronicle register change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UK land value inputs for chronicle#408: MHCLG land values, VOA's 2026 stock, EPC D3, council tax by authority, the Market Value Survey, land by sector and HMRC stamp tables
#410 merged into bundle-stream-consumer-rows after #414 had merged that
branch into main, so chronicle#408's packages never reached main. This
branch starts from that merge (41c5b37) and brings in main's #413
(chronicle#412's area inputs). The one conflict, .gitignore, keeps both
PRs' raw-only entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 marked this pull request as ready for review November 5, 2026 16:32
@vahid-ahmadi

Copy link
Copy Markdown

Review of 9db960f. I checked that this is #410 as I last reviewed it (round 3 at d2737e6), now on main, with nothing added, dropped or reverted. I compared blob hashes and did not run anything locally.

How I checked

Earlier items: all settled on #410 and carried over unchanged. That covers the provisional years in Live Table 126, the Barnsley and Sheffield table and Microcosm alias note, and the "£2m plus" label.

CI at 9db960f: bundle, surface and checks all pass. "Test merged Chronicle bundles" ran in full and took 1h55m. The whole bundle job took 1h59m, against #410's 3h16m and the 300-minute limit, so the timing watch item can rest for now.

Verdict: this is #410's reviewed content with a clean merge of main. I have nothing further.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants