[anth-volk]

Fixes #4133

Stacked on #4130.

Summary

The release run (the run for the "Update PolicyEngine API" commit) stops repeating the checks its merge run passed on the same code, and runs the release test suite on six runners. The two only help together: with either one alone, the other remains the longer branch before the staging deploy.

• check-release-commit (new job): .github/scripts/check_release_commit.sh gathers what .github/release_commit.py reads, and that module decides whether this run may skip the checks. It writes merge_run_checks_passed (true or false) and merge_run_url as job outputs. The job runs on every push; on a merge commit its steps do nothing, so the checks that need it never need a skipped job.
• The five checks (lint, quality-guards, alembic-v1-check, alembic-v2-check, v2-integration-check) need check-release-commit and are skipped only when merge_run_checks_passed is true. !cancelled() runs them when the check job fails.
• publish-git-tag also needs check-release-commit, runs whatever the checks ended, and its new step python .github/release_commit.py require-checks (with JOB_RESULTS: ${{ toJSON(needs) }}) fails unless either the checks were skipped because check-release-commit said so and none of them failed, or lint, both Alembic checks and the integration tests passed in this run. These are the checks it needed before; the quality guards still block no deploy job, as before.
• Every release job after the checks starts its if: with !cancelled() and checks the result of each job it needs, and the jobs in migrate-v1-cloud-sql.yml and seed-v2-database.yml have if: ${{ !cancelled() }}. GitHub skips a job whose needs include a skipped job, even through the jobs between them, unless its if: calls a status function (actions/runner#2205). Without this, skipping the checks would skip every deploy job after them, and the run would still finish green.
• release-tests is a matrix of six shards (fail-fast: false). run_release_tests.sh requires RELEASE_TEST_SHARD_INDEX, RELEASE_TEST_SHARD_COUNT and RELEASE_TEST_SHARD_REPORT_DIRECTORY and loads the pytest plugin tests/release_test_shard_plugin.py (through PYTEST_ADDOPTS). Each shard uploads what it collected and ran as the artifact release-test-shard-<n>.
• check-release-test-shards (new job) downloads every shard's report and runs python -m scripts.release_test_shards check-coverage --shard-count 6. deploy-cloud-run-staging needs it.
• docs/engineering/skills/testing.md, ci-workflows.md and alembic-migrations.md describe the shards, the check, and the !cancelled() rule for release jobs.

No new dependency, and no inline ruff suppressions are added.

The release commit check

release_commit.py check lets the release run skip the five checks only when both hold:

1. The release commit changes nothing but release files. From git diff --raw --no-renames against its single parent:
   • pyproject.toml and uv.lock are edited regular files (status M, mode 100644), and each differs from the parent's copy in exactly one line of the file: the version = "..." line in the [project] table, and the version line of the [[package]] entry named after the project (policyengine-api). Any other change, such as a uv lock that re-resolved a dependency, fails the check.
   • CHANGELOG.md is an edited regular file.
   • Files directly in the fragment directory ([tool.towncrier] directory, read through .github/changelog_fragments.py) are deleted, never added or edited; .gitkeep and anything in a subfolder count as other files.
   • Any other path fails the check, as does a release commit with more than one parent.
2. The merge commit's Push run passed every check. From GET /repos/{repo}/actions/workflows/push.yml/runs?head_sha=<parent>&event=push and each run's jobs: every job named Lint, Quality guards, or <check name> / ... for the three reusable checks, ended completed and success.

When either fails, or the script cannot read git or the API, it prints the reasons as notices and writes false, and the checks run as before. Run against this repository's history, the change check accepts each of the 15 most recent release commits (b5cf71dc8 to b93b0b3b6). Run read-only against the real API, the whole check returns true for 5.14.3 (merge run #2614) and false for release #2590, whose merge run failed the quality guards.

How the release tests are split

• Every shard collects the whole suite (the make test paths, about 3,980 tests), so all of them see the same tests.
• tests/release_test_shard_plugin.py (pytest_collection_modifyitems, trylast) keeps only the tests in the files scripts/release_test_shards.py assigns to its shard, and reports the rest as deselected. A file's tests stay together, so module- and class-scoped fixtures are set up once.
• Files are assigned heaviest first, each to the shard with the least weight so far; ties go to the lower shard number, and equal weights are taken in path order. A file's weight is its measured seconds in MEASURED_FILE_SECONDS (the 20 files over 10 seconds in a local run with --durations=0), or else its test count times 0.04 seconds. The assignment depends only on the collected files and their test counts, so it is the same on every runner, and each test runs in exactly one shard.
• check-release-test-shards fails unless all six shards reported, every shard collected the same node IDs, and each collected test ran in exactly one shard. It lists any test that ran in no shard, in two shards, or ran without being collected.

Locally, the six shards of the real suite ran side by side: all 3,984 tests passed, each in exactly one shard (the check said so), in 98 to 175 seconds of test time per shard. The heaviest shards hold test_country_spm.py alone and test_import_side_effects.py with one small file.

Job dependencies on a release run

Before (with #4125, #4127 and #4130; minutes from the start of the run):

lint, quality-guards, alembic-v1/v2, v2-integration (10.2) --+
ensure-staging-model (0.8) ----------------------------------+-> publish-git-tag -+-> migrate-v1-staging (2.0) --+
                                                                                  +-> seed-v2-staging (5.9) -----+
release-tests, one runner (about 16 to 19) ------------------------------------------------------------------+-> deploy-cloud-run-staging, from about 17 to 19

After:

check-release-commit (about 0.5) --+-> lint, quality-guards, alembic-v1/v2, v2-integration (skipped when the merge run passed them)
                                   +--------------------------------+
ensure-staging-model (0.8) ----------------------------------------+-> publish-git-tag (require-checks) -+-> migrate-v1-staging (2.0) --+
                                                                                                         +-> seed-v2-staging (5.9) -----+
release-tests, 6 shards (about 5 to 6) -> check-release-test-shards (about 0.4) ---------------------------------------------------------+-> deploy-cloud-run-staging, from about 7

Everything after deploy-cloud-run-staging keeps its order. Expected saving: about 10 to 12 minutes per release beyond the three pull requests below (estimate). The merge run's checks start about 10 to 15 seconds later, after check-release-commit finishes doing nothing.

When the check finds a reason to run the checks, the release run behaves as before except for the shards: the tag and the staging database jobs wait for the checks (about 10 minutes), and the staging deploy starts at about 16.4.

Environment variables

Job	Variable	Provider	How it reaches the step
check-release-commit	GH_TOKEN	the run's GITHUB_TOKEN (GitHub)	step env: from github.token; job permissions: contents: read, actions: read
check-release-commit	GITHUB_REPOSITORY, GITHUB_OUTPUT	the runner's default variables	every step
check-release-commit	PYTHON	not set; the script runs python3 from actions/setup-python
publish-git-tag	JOB_RESULTS	the needs context: each needed job's result, and check-release-commit's outputs	step env: from toJSON(needs)
release-tests	RELEASE_TEST_SHARD_INDEX	the matrix	step env: from matrix.shard_index
release-tests	RELEASE_TEST_SHARD_COUNT	literal "6"	step env:
release-tests	RELEASE_TEST_SHARD_REPORT_DIRECTORY	runner.temp (runner context)	step env:
release-tests	PYTEST_ADDOPTS	literal, set by run_release_tests.sh	env before make test
release-tests	everything the tests read before	as in #4125: staging environment variables and secrets, repository and organization secrets, literals	unchanged job and step env:, environment: staging
release-tests, check-release-test-shards	artifact upload and download	the runner's own Actions runtime token, which actions/upload-artifact and actions/download-artifact use for the same run	no workflow setting; check-release-test-shards has permissions: contents: read
v2-integration-check	CODECOV_TOKEN	repository and organization secret	caller secrets:, unchanged

Secret and variable names were confirmed with read-only gh api calls that list names only. No new secret or variable is needed. Every other job keeps its environment:, env:, secrets: and permissions:; only their if: changes.

Google Cloud sign-in: no job that signs in to Google Cloud moves or changes environment. Neither new job signs in.

Unverified until the first release after merge

push.yml runs only on pushes to master, so this pull request's CI runs none of these jobs.

• That a release run whose checks are skipped still runs every deploy job. This depends on GitHub applying !cancelled() and the explicit needs.<job>.result conditions on every release job, and on the jobs inside migrate-v1-cloud-sql.yml and seed-v2-database.yml. If any of them were skipped, the run would finish without deploying. Check the first release's job list.
• That the run's GITHUB_TOKEN with actions: read can list the parent's Push runs and their jobs. The same calls worked read-only with a personal token.
• That check-release-commit succeeds on merge runs with every step skipped, so the checks start.
• actions/upload-artifact@v4 and actions/download-artifact@v4, used here for the first time in this repository.
• That the shard plugin loads under make test (coverage run -m pytest) on the runner; it does locally.
• That every shard collects the same tests on GitHub's runners; check-release-test-shards checks this on each release.
• The shard times and the time saved. The weights were measured on a laptop.
• If check-release-commit itself fails, the checks run, and publish-git-tag requires them. Whether GitHub then runs the jobs inside the three reusable check workflows is unverified; if it skips them, publish-git-tag fails and the release stops before the tag.

Release commits whose checks are skipped no longer upload integration coverage to Codecov.

Testing

• Red: f682f2a (tests only). pytest --continue-on-collection-errors tests/unit/test_release_commit.py tests/unit/test_release_test_shards.py tests/unit/test_release_run_checks.py tests/unit/test_deploy_pipeline_scripts.py tests/unit/test_cloud_run_push_workflow.py: 11 failed, 26 passed, 2 errors (the two new modules do not exist yet).
• Green: dde97a9. The same command: 88 passed.
• The workflow and script structure tests, with the fragment and version bump tests (14 modules): 260 passed.
• The six shards of the real make test suite, run side by side locally with a Redis each: 3,984 passed, and check-coverage passed.
• ruff format --check . and ruff check . with the ruff version uv.lock pins (0.15.12): clean.
• python scripts/run_quality_guards.py: every guard passed. python -m scripts.rules guard --base-ref origin/master: passed. The rules tool's tests: OK. python -m scripts.rules render: no diff.
• uv lock --check: passed. shellcheck on check_release_commit.sh and run_release_tests.sh: clean. push.yml, migrate-v1-cloud-sql.yml and seed-v2-database.yml parse as YAML and pass actionlint.

🤖 Generated with Claude Code