Skip to content

Report when the deployed policyengine.py version went live - #4084

Merged
PavelMakarchuk merged 4 commits into
masterfrom
report-live-since
Oct 28, 2026
Merged

PavelMakarchuk merged 4 commits into
masterfrom
report-live-since

Conversation

@PavelMakarchuk

@PavelMakarchuk PavelMakarchuk commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4083

Summary

  • New script .github/scripts/resolve_policyengine_live_since.sh: reads the .py version from uv.lock, and the DEPLOYED_POLICYENGINE_VERSION and POLICYENGINE_VERSION_LIVE_SINCE of the revision serving 100% of traffic (steps.previous.outputs.revision). Same version → keeps that date; a new version or a first deploy → the deploy time; a revision without a recorded date (deployed before this change) or a failed gcloud lookup → empty, i.e. unknown, rather than a later, wrong date. A failed lookup never fails the deploy.
  • deploy_cloud_run_candidate.sh adds both values to the candidate's environment; push.yml passes PREVIOUS_CLOUD_RUN_REVISION to the staging and production deploy steps (single-line run: unchanged).
  • GET /deployment (next to /health) returns policyengine_version (installed), policyengine_version_live_since and revision (K_REVISION). The date stays null after merge until the next .py version is deployed, because the version now live started before dates were recorded.

Why not a deploy timestamp: the API deploys several times a day while .py stays the same, so "last deployed" is nearly always today. The app-v2 Deployments developer tool (PolicyEngine/policyengine-app-v2#1450) will read /deployment for its "Live since" column; the household API already reports the equivalent.

Note: the date is the candidate deploy time; traffic moves to the candidate after its smoke checks, minutes later.

Testing

  • tests/unit/test_policyengine_live_since.py (10 tests): same version keeps the date; a new version and a first deploy get the deploy time; a revision without a date (with or without a recorded version) and a failed gcloud give an empty date; a lock without policyengine fails; the dry-run candidate deploy sets both variables; /deployment returns the fields and a null date without the variable.
  • tests/unit/test_cloud_run_candidate_deploy.py, test_cloud_run_push_workflow.py, test_asgi_factory.py: 78 passed.
  • ruff format, ruff check on changed files; shellcheck -x on both scripts; python scripts/run_quality_guards.py: all passed.

🤖 Generated with Claude Code

The candidate deploy now reads the .py version from uv.lock and the
version and date recorded on the revision serving 100% of traffic. A
deploy that keeps the version keeps the date; a new version, a revision
without a date or a failed lookup gets the deploy time. Both values go to
Cloud Run as DEPLOYED_POLICYENGINE_VERSION and
POLICYENGINE_VERSION_LIVE_SINCE, and GET /deployment reports them with the
Cloud Run revision.

Fixes #4083

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A serving revision with the same .py version but no recorded date was
deployed before dates were recorded, so its version may have been live for
weeks; a failed lookup tells nothing either. Both now leave the date empty
(GET /deployment reports null) instead of the deploy time. Only a new .py
version or a first deploy records the deploy time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PavelMakarchuk added a commit to PolicyEngine/policyengine-app-v2 that referenced this pull request Oct 27, 2026
The API's GET /deployment (PolicyEngine/policyengine-api#4084) reports
policyengine_version_live_since, and the gateway's /versions
(PolicyEngine/policyengine-sim-api#961) reports latest_since. The page
reads both when present; an API without the route, or a gateway without
the field, shows no date as before. A missing gateway worker keeps the
API's own date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.84%. Comparing base (3fde3da) to head (7cd3e9e).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4084       +/-   ##
===========================================
+ Coverage   50.82%   87.84%   +37.02%     
===========================================
  Files         254      270       +16     
  Lines       14635    16070     +1435     
  Branches     2339     2627      +288     
===========================================
+ Hits         7438    14117     +6679     
+ Misses       6570     1284     -5286     
- Partials      627      669       +42     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@PavelMakarchuk
PavelMakarchuk marked this pull request as ready for review October 27, 2026 20:11

@anth-volk anth-volk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code-quality notes from a review of this PR; four inline comments below.

Environment variables, traced from where each value comes from to where it is used:

Variable Provided by How it reaches the code
PREVIOUS_CLOUD_RUN_REVISION revision output of the "Capture current Cloud Run … state" step (gcloud run services describe) Step-level env: on the staging and production candidate deploy steps
CLOUD_RUN_PROJECT, CLOUD_RUN_REGION Defaults in cloud_run_set_defaults The new script sources cloud_run_env.sh itself
DEPLOYED_POLICYENGINE_VERSION, POLICYENGINE_VERSION_LIVE_SINCE Output of resolve_policyengine_live_since.sh Added to --set-env-vars of gcloud run deploy; read back from .spec.containers[0].env, which is correct for this single-container service
K_REVISION Set by Cloud Run at runtime Not configured in the repository
GCLOUD_BIN, POLICYENGINE_LOCK_FILE, POLICYENGINE_LIVE_SINCE_NOW, CLOUD_RUN_DRY_RUN Test overrides Not set in the workflows, so the defaults apply

One thing I couldn't confirm from the repository: whether the deploy service account (GCP_DEPLOY_SERVICE_ACCOUNT) has run.revisions.get. If it doesn't, gcloud run revisions describe fails and the date is silently empty on every deploy instead of failing the deploy.

Comment thread .github/workflows/push.yml
Comment thread .github/scripts/resolve_policyengine_live_since.sh Outdated
Comment thread policyengine_api/fastapi_routes/health.py
Comment thread policyengine_api/fastapi_routes/health.py Outdated
PavelMakarchuk and others added 2 commits October 27, 2026 18:06
- resolve_policyengine_live_since.sh reads the .py version with the new
  `locked_packages.py bundle-version` command (the relock's tomllib reader),
  which fails unless policyengine is locked at exactly one version.
- /deployment is in the health route group (migration registry and
  ROUTE_GROUP_BY_PATH), so its request logs no longer show `unknown`;
  migration contract docs regenerated.
- /deployment reports the date only when DEPLOYED_POLICYENGINE_VERSION
  matches the installed version, so a date is never paired with another
  version.
- Tests: both candidate deploy steps receive PREVIOUS_CLOUD_RUN_REVISION;
  bundle-version reads one version and refuses two; the route drops a date
  for another version and logs under health.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk

Copy link
Copy Markdown
Collaborator Author

@anth-volk thanks — all four inline points are addressed in 7cd3e9e (replies inline).

On run.revisions.get: the deploy account already needs it. The same staging and production deploy jobs run gcloud run revisions describe in resolve_cloud_run_candidate_state.sh and set_cloud_run_revision.sh, so the new script needs no new permission. If the call failed anyway, the date is left empty (unknown) and the deploy continues.

Also merged current master into the branch. Ready for another look.

@anth-volk anth-volk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, all four points are addressed in 7cd3e9e, and you're right about run.revisions.get: the same deploy jobs already run gcloud run revisions describe in resolve_cloud_run_candidate_state.sh and set_cloud_run_revision.sh.

One minor, optional point: a lock without policyengine now fails with the relock's message ("…so the packages its extras pin can't be held back"), which reads oddly during a deploy, and the script's own "policyengine is not locked" check can no longer run, because set -e exits as soon as locked_packages.py fails.

@PavelMakarchuk
PavelMakarchuk merged commit c280b85 into master Oct 28, 2026
15 checks passed
PavelMakarchuk added a commit to PolicyEngine/policyengine-app-v2 that referenced this pull request Oct 28, 2026
…ce serves (#1450)

* Add a Deployments developer tool: which .py and data build each service serves

/{country}/dev-tools/deployments shows, for the US and the UK, which
policyengine.py version, country model and data build each PolicyEngine
service declares in its lock file, and what the running service serves.

- Declared: each service's lock on its default branch pins .py; that .py
  tag's bundle manifest certifies one data build per country. The latest .py
  on PyPI is the reference.
- Served: api.policyengine.org's /v2/tax-benefit-models/by-country/{country}
  reports the served .py (the full /metadata is ~75 MB for the US), and the
  simulation gateway's /versions reports the latest route's .py, model and
  data build. A service whose served versions differ from its lock is marked
  "Not deployed".

Services: the API, app v3 (through the API), the simulation API, the
household API (no microdata, so no data build) and API v2 alpha. Moved here
from PolicyEngine/calibration-diagnostics#271: this is PolicyEngine
infrastructure, not Microcosm calibration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show each service as a card with declared and served side by side

The seven-column table needed about 1,100 px, so on narrower windows the
data-build column was cut off and every cell wrapped. Each service is now
a card: the service and its links on top, then a Declared and a Served
column with the same labeled rows, stacked on narrow screens. The
latest-bundle summary uses the same labeled rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a drift overview table above the per-service cards

The cards show detail but not drift across services at a glance. An
overview table now lists every service in one row: deployment status,
status against the latest .py, and the .py, model and data-build values.
Where a running service serves something other than its lock, the cell
turns red and shows "lock X / serving Y". Build IDs are shortened to their
date and hash ("2026-10-19 · 6cba5d71"); the full ID is in the tooltip and
in the card. Each service name links to its card under "Details".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep only the overview table on the Deployments page

Drop the per-service detail cards and the note under the table. The
repository, lock and runtime-report links move under each service name,
and each status label shows its explanation as a tooltip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Tidy the latest-bundle summary and shorten every Microcosm build ID

- The latest-bundle box shows .py, model and data build as labeled values
  in one row; the "every reporting service is deployed" label is gone (the
  red "N not deployed" label still appears when a service drifts).
- Overview build IDs drop the repeated producer, country and year prefix:
  "spm · 2026-09-15", "national · 2026-10-19 · 6cba5d71". They wrap only
  between parts, and the table fits a 1024 px window with a drifted row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Name the latest data build the same way as the table

The latest-bundle box showed the full build ID while the table showed the
short form, so the two could not be compared by eye. The box now uses the
same short form ("national · 2026-10-25 · a6299504", full ID in the
tooltip), the same value size as the table, and a larger heading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Read the API's data build from its gateway worker and flag a missing one

The API sends each US and UK economy simulation to the simulation
gateway's worker for its own .py. The page now looks up that worker in
the gateway's /versions instead of inferring the data build from the .py
bundle:

- the worker exists and reports the certified build: the API's data build
  is reported, not inferred;
- no worker for the API's .py: "No worker" (economy simulations fail);
- the worker reports another build than its .py certifies: "Not deployed",
  with both builds named.

App v3 inherits the API's report, so it gets the same check. When the
gateway report cannot be read, the API falls back to the .py bundle as
before. The header counts every service that needs attention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Read the household API's served versions; drop the retired API v2 alpha

- Household API: its /versions lists a `current` channel (what users get)
  and a `frontier` channel (the next rollout). The page compares `current`
  with the lock and names `frontier` in the explanation. Today it shows
  real drift: current serves policyengine-us 2.42.4 while the lock declares
  2.53.1, which runs only on frontier.
- API v2 alpha: the repository is archived and v2.api.policyengine.org no
  longer resolves, so the service is removed from the list instead of
  showing a lock file for something that does not run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Say "live differs from main" instead of "not deployed"

A service whose running version differs from its lock is still deployed
and serving; only the version differs. The column is now "Live vs main"
with "Same as main" / "Differs from main", drift cells read "main X /
live Y", and explanations say "the lock on main declares".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show newest releases in the header; name household channels; calmer drift

- The header is now "Latest versions" with two aligned rows: what the
  latest .py certifies (.py, country model, Core, data build) and the
  newest release of each (PyPI for .py, the country model and Core;
  Microcosm's public releases API for its current data build).
- The household API's model cell uses its own channel names, "current" and
  "frontier", when either differs from main.
- Drift cells keep the table's font and size, label each line in grey and
  colour only the differing value; no red cell background. The
  "N services need attention" label is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Drop the status columns and align channel lines

The overview keeps Service, .py, country model and data build. Where the
running service differs from main, a cell lists the values as labeled
lines ("main"/"live", or the household API's "current"/"frontier"); labels
and values now share the table's monospace font and baseline, without red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Link table data builds to Microcosm; drop the lock and report links

Each Microcosm data build in the overview links to its release page on the
Microcosm calibration dashboard, like the header already does. The
service cell keeps its repository link and drops the "lock" and "report"
links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a "Live since" column from the services' own reports

The household API reports `deployed_at` for each channel in /versions; the
table now shows when its `current` channel (what users get) went live.
Services that report no deploy time show a dash until they do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Read live-since dates from the API and the simulation gateway

The API's GET /deployment (PolicyEngine/policyengine-api#4084) reports
policyengine_version_live_since, and the gateway's /versions
(PolicyEngine/policyengine-sim-api#961) reports latest_since. The page
reads both when present; an API without the route, or a gateway without
the field, shows no date as before. A missing gateway worker keeps the
API's own date.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report when the deployed policyengine.py version went live

2 participants