Repository navigation
UK land value inputs for chronicle#408, on main (re-lands #410) - #415
Open
juaristi22 wants to merge 18 commits into
Open
juaristi22 wants to merge 18 commits into
juaristi22 wants to merge 18 commits into
Conversation
chronicle#408 item 1: all eight data sheets of MHCLG's 2023 workbook (valuation date 1 October 2023), 10,101 facts, and the generator scripts/build_uk_408_packages.py that writes the issue's packages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 3: tables CTSOP1.0 to 4.0 of the 2026 summary workbook at every geography they print (222,588 facts in three packages), and the three small-area zips (CTSOP1.1, 3.1 and 4.1) registered raw-only, in R2 and git-ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 7: domestic certificates lodged, total floor area and habitable rooms by quarter for England and Wales, each nation, region and local authority (75,051 facts), and dwelling stock density by local authority, 2011 to 2025 (5,406 facts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 4: MHCLG's Tables 6a and 9a for every English billing authority, and the per-authority rows of the pinned Welsh Table 2 (both years) and SLGFS Tables 2.8 and 2.11. Rows that restate ported facts (Table 6a's England row, QRC4 line 1) are asserted equal, not ported again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 items 8 and 2, which share a generator section and test module: land (AN.211) for every sector in both pinned balance sheet workbooks, ONS's 2022 series for land underlying dwellings, and the Market Value Survey 2024 annex tables the issue lists. Chapter 4 rows that restate chapter 1 are asserted equal, not ported again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 5: more tables from the pinned Annual Stamp Tax Statistics 2024-25 workbook, both financial years, in hmrc-stamp-tax-statistics-2024-25 (352 to 1,487 facts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chronicle#408 item 9: preprocessing/config.py, webapp/cells.json and LICENSE of DanNeidle/lvt_model_2026 at commit fef94f9 (MIT), as a raw-only registration with no facts, kept in R2 only and git-ignored (config.py is third-party Python source). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The field contracts and caveats of every item in docs/data-sources/uk-land-value-inputs-408.md, a chronicle#408 section in the PE UK source checklist (whose Welsh, Scottish and balance sheet rows now count the added rows), and the changelog fragment. Item 6 waits on chronicle#409. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in the register (#408) CI's surface job failed one test on 493fe89: the register's static check (tests/test_dimension_value_register.py) found 396 values that two or more of this PR's packages emit but chronicle/dimension_values/canonical_labels.json does not list. Checking the packages' consumer rows with chronicle.dimension_labels.dimension_label_issues also found values with no label, which build-bundle refuses ("Every package must label every dimension and value it emits"): - Tables 6a and 9a (QRC4): the six billing authority classes; - the land value estimates: site_density_scenario, new_build_price_percentile, agricultural_land_grade, office_accommodation_type and retail_location_type; - the Market Value Survey's chapters 1, 3 and 4: the tenure columns, house or flat, occupancy, length of residence (chapter 3), and home value and value bands (chapter 4); - the two VOA crossed packages: Wales' band I, which the register now lists. scripts/build_uk_408_packages.py now declares those labels from the printed text: - each class takes its subtotal row's wording; - item 1's values take the sheets' headings, and the percentiles, which the sheet heads with bare numbers, are labelled "25th percentile" to "75th percentile", with "50th percentile (median)"; - a Market Value Survey value takes the row's wording where the package prints it as a row, otherwise the column header as printed, otherwise the title of the block that sets it. A value the register labels another way is left to the register, which a package may not contradict. The register gains the 396 values by 735e617's method: only the 14 packages that emit them were built, at 2023. 384 have one wording and take it. Twelve are worded two ways and take a label in CHOSEN_LABELS: - "Yes" and "No" for ehs_dependent_children and ehs_household_disability and their mhclg. groupby twins. That is the wording most packages use; chapter 1's sample-size rows print "Dependent Children - Yes" and the like. - gbp_1m_to_2m and gbp_2m_or_more, under mvs_market_value_band and its twin. Chapter 1 words them "£1,000,000 to £2,000,000" and "£2,000,000 or more", chapter 4 "£1m-£2m" and "£2m or more". With one package each, the rule does not decide, so chapter 1's wording is proposed for the maintainer's review. The register now holds 8,553 values in 183 dimensions, up from 8,157 in 160. It still records dece5e9 as its build commit, since only these packages were rebuilt. Checked: - The static check reports no problem. - All 18 packages this PR adds or changes build with no label issue. - Seven of the 14 build byte-identical consumer rows before and after, so the register relabels nothing there: VOA bands 2025 and 2026, the Welsh dwellings package and both collection packages, the 2025 balance sheet and the 2022 series. - The two VOA crossed packages change only by band I's new label. - No package gains or loses a fact. - The council tax and Market Value Survey test files pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test test_the_chosen_labels_are_the_registers_labels pins how many labels scripts/build_dimension_value_register.py chooses. 03f9689 adds twelve for the Market Value Survey 2024 packages: eight by the rule, four value bands for the maintainer. That brings the count from 47 to 59, which is the surface job's one failure at 03f9689 (1 failed, 5,770 passed). The test's other check, that each chosen label is the register's label, holds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heffield's codes (#408) Two points from the review of 03f9689: - Live Table 126 marks 2022 to 2025 '[p]' provisional. Those years' record sets now carry shared_filters publication_status: provisional, as welshgov-dwelling-stock-by-tenure-2025 does, so a consumer can filter on it. The register already labels the value "Provisional". '[r]' (2024) stays in the measure header and notes. The fact count is unchanged. - docs/data-sources/uk-land-value-inputs-408.md gains a table of the codes each package carries for Barnsley and Sheffield, E08000016 and E08000019 or E08000038 and E08000039, including main's council tax levels package. The doc says that a join on geography_id drops both authorities unless the consumer maps one pair to the other. Checked: scripts/build_uk_408_packages.py regenerates the package, and tests/test_source_uk_408_epc.py and tests/test_dimension_value_register.py pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…anch Both add packages to the same shared files. Keep both checklist sections, #407's first as main has it, and count the 60 chosen labels: the 47 from 2026-10-26, #407's soc2020_major_group=3 and #408's twelve. The register merged cleanly and its static check passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#414 writes and counts each package's rows as it reads them, so the bundle no longer holds every row at once. The 16 GB CI runner could not fit this branch's bundle without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bundle job now fits on the runner (3,578,392 KiB peak with #414's streaming build) and failed only on the pinned counts. The new pins come from that run's uploaded reports: 326,495 facts, 12 packages and source tables, 275 geography keys and 46 period keys. Each change is attributed per package from the packages' own facts reports, and the attributions sum to the bundle's changes. Semantic duplicate keys (2,324) and warnings are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review nit on #410: Chapter 4 of the Market Value Survey 2024 prints "£2m or more" in AT4.1 and AT4.2 but "£2m plus" as the column header in AT4.3 and AT4.4, and the package files both under gbp_2m_or_more. The comment above the proposed labels now names every printed wording. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review on #410, round 2, item 3: Chronicle keeps the codes each table prints, and the consumer maps one vintage to the other. Microcosm's local-authority crosswalk already lists E08000038 and E08000039 as aliases of E08000016 and E08000019 (PolicyEngine/microcosm#931, item (e)), so the doc section now says so instead of pointing at a Chronicle register change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UK land value inputs for chronicle#408: MHCLG land values, VOA's 2026 stock, EPC D3, council tax by authority, the Market Value Survey, land by sector and HMRC stamp tables
#410 merged into bundle-stream-consumer-rows after #414 had merged that branch into main, so chronicle#408's packages never reached main. This branch starts from that merge (41c5b37) and brings in main's #413 (chronicle#412's area inputs). The one conflict, .gitignore, keeps both PRs' raw-only entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22
marked this pull request as ready for review
November 5, 2026 16:32
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This puts #410's code on main. #410 merged into
bundle-stream-consumer-rowson 5 November at 16:07 UTC, 86 minutes after #414 had merged that branch into main, so chronicle#408's packages never reached main. #410 has the full description of what they add.What the branch holds
bundle-stream-consumer-rows. Its parents are UK land value inputs for chronicle#408: MHCLG land values, VOA's 2026 stock, EPC D3, council tax by authority, the Market Value Survey, land by sector and HMRC stamp tables #410's head (d2737e6) and Write and count bundle rows per package instead of holding them all #414's head (b71d03d), which main already has..gitignore, where both PRs added raw-only entries at the end. Both blocks are kept.docs/pe-uk-source-checklist.mdmerged without conflict.Tests
Run locally on 9db960f:
ruff check chronicle policyengine_chronicle db scripts testsis clean.scripts/build_dimension_value_register.py --static-checkfinds no problems..gitignoreor walk every manifest.Not run locally: the bundle job's two test steps,
tests/test_chronicle_uk_national_2025.pyandtests/test_chronicle_bundle.py. That job took 3h16m on #410's CI, and it runs again on this PR.Refs #408. Item 6 waits on #409.
🤖 Generated with Claude Code