[anth-volk]

Fixes #977

Part of #968.

What changes

With this change, the contract and the gateway accept a UK local-area breakdown result (kind: local_areas) with or without two new optional fields. No executor sends them yet: #975 makes the executor fill them, and it merges only after this PR is merged and deployed.

Field	Type	Meaning
constituency_withheld_codes	list[str] | None, default None	The ONS codes, sorted, of the constituencies the run estimated but disclosure control did not publish.
local_authority_withheld_codes	list[str] | None, default None	The same for local authorities.

• An area the run never estimated, such as one with no households in the dataset, is in neither list.
• [] means that no area of the level was withheld.
• None means that the result does not report them. Results without the fields still validate, and their JSON leaves both keys out rather than writing null.

Why an unreported list is left out of the JSON

In simulation-deploy.yml, a push to main runs the beta job and then the prod job. Each one calls simulation-deploy.reusable.yml, where:

1. deploy_executor ("Deploy and unit test Modal executor") redeploys the versioned executor app for the pinned .py version, policyengine-simulation-py6-10-0, under the same name, then runs the executor's unit tests.
2. deploy_gateway ("Deploy and unit test Modal gateway") needs deploy_executor. It then redeploys the stable gateway app, policyengine-simulation-gateway.
3. update_routing needs both, and integration runs after it.

Both Modal images mount the contract from the checkout (add_local_python_source(..., "policyengine_simulation_contract", ...)). So a merge of this PR redeploys the 6.10.0 executor with the new contract while the old gateway is still serving, and the old gateway's contract forbids unknown keys.

With the fields as plain = None defaults, the executor's breakdown output (LocalAreaBreakdownBuilder.serialize, which calls model_dump(mode="json")) wrote both keys as null. The executor's existing test_a_local_area_breakdown_publishes_both_levels_from_one_release failed on the first commit alone for that reason, and the old gateway would have reported such a breakdown job as failed.

Both fields now use field-level exclude_if, as #919 does for x and y. A None list is left out of every dump, while [] is kept. Until #975 deploys, the executor's breakdown output carries the same keys as before, so the old gateway accepts it. Once this PR's gateway is deployed, it accepts the fields too.

Commits

1. 762f2c6f Add optional withheld-area code lists to the local-area breakdown result. This is Fill the withheld-area code lists in UK local-area breakdown results #975's first commit: the two fields, the gateway's golden OpenAPI, the gateway README's table of breakdown fields, and an entrypoint OpenAPI test. Seen failing first: the contract tests (extra_forbidden on the new keys) and the entrypoint test (KeyError: 'constituency_withheld_codes').
2. c9b3a89a Leave unreported withheld-code lists out of breakdown result JSON. Adds exclude_if to both fields. The contract now requires pydantic 2.12 or later, the first release with exclude_if. Every lock already resolves 2.13, and the five locks that include the contract record only the new requirement. Seen failing first: two new contract tests, which found None for both keys in the dump, and the executor's breakdown test above.
3. f78fb6b3 Merge origin/main. Brings in Record when each policyengine.py route was published and latest changed #961 and Pass is_tax through to detailed_budget #972, which also change the contract and the golden OpenAPI. The golden still equals the generated document.
4. b4d1f32a Describe the withheld-code fields the contract now accepts in the changelog. Documentation only.

Order with #975

#975 keeps the executor commits and must not merge until this PR is merged and deployed. After this PR merges, main gets merged into #975, so its diff then shows only the executor commits.

Overlap with #919

#919 makes the same pydantic>=2.12 change and adds its own _is_none helper to macro_output.py. Whichever PR merges second resolves the text conflict and keeps one helper.

Checks run

• Focused tests:
  • the contract's test_local_area_breakdown.py, test_gateway_models.py and test_macro_output_uk_geography.py;
  • the entrypoint's test_openapi.py;
  • the gateway's test_openapi_golden.py, test_local_area_breakdown.py and test_endpoints.py;
  • the executor's test_simulation_output_uk_local_areas.py, test_simulation_output_geographic.py, test_simulation_output_impacts.py, test_simulation_api_contracts.py and test_runtime_evaluated_annotations.py;
  • the Stage 12 persistence library's tests, since its lock changed;
  • apis-integ's test_calculate.py without the beta_only tests, against a client regenerated with ./scripts/generate-clients.sh.
• uv lock --check with uv 0.13.6 passes in each package whose lock changed.
• make check passes: ruff format and lint, and Pyright in every package, with no errors.
• python scripts/run_quality_guards.py with GITHUB_BASE_REF=main passes every guard.
• Every check above ran without Fill the withheld-area code lists in UK local-area breakdown results #975's executor commits.

Known check failure

The PR description check requires the first line to be Fixes #ISSUE_NUMBER. This description starts with Part of #968 on purpose, because #975 is the PR that fixes #968, so that check fails here.

The work was done red-green: each behaviour change's tests were written first and seen failing for the expected reason, then the code was written, and both were committed together.

🤖 Generated with Claude Code