Skip to content

Give the YAML test job 45 minutes, not 30 - #2697

Merged
juaristi22 merged 1 commit into
mainfrom
ci-yaml-test-timeout
Oct 27, 2026
Merged

juaristi22 merged 1 commit into
mainfrom
ci-yaml-test-timeout

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #2698

What and why

The Test / YAML job's limit goes from 30 minutes to 45.

On 27 October, 8 of the 16 YAML jobs that finished in pull request and merge-queue runs hit the 30-minute limit (two more were cancelled by newer pushes). Each had already logged its N passed summary with no failures. The other 8 took 22 to 28 minutes.

A cancelled YAML job fails Test / Result. In the merge queue, Result runs only once the Python shards finish, about an hour in, and the queue then drops the pull request; adding it back runs the whole suite again. That took #2695 out of the queue today (run 40541059060).

45 minutes leaves room for a slow runner and still stops a hung job. It adds no time to a run, since Result waits about an hour for the Python shards anyway.

This doesn't explain why the job slowed down: on 10 October it took about 4 minutes (run 38085920460). Most of the test step's time now falls outside pytest's own reported duration. In three passing jobs (runs 40496057048, 40516287528, and attempt 2 of 40525399987), pytest reported about 8 minutes, but its 5107 passed summary appeared in the log 16 to 17 minutes into the step, and the step ended about 8 minutes after that.

Test plan

  • Policy changes: not applicable, CI only.
  • make format: not applicable, no Python files changed.
  • Changelog fragment changelog.d/ci-yaml-test-timeout.changed.md (.github/check_changelog_fragments.py passes).
  • policyengine_uk/tests/code_health/test_ci_runner_variables.py and test_ci_test_shards.py pass locally (126 passed).
  • This pull request's own run passes Test / YAML and Test / Result under the new limit (.github/** counts as code, so the full suite runs).

Reproducer: n/a: CI workflow and changelog fragment only; nothing under policyengine_uk/, pyproject.toml or uv.lock changes.

axiom: n/a: CI only, no policy change

🤖 Generated with Claude Code

On 27 October 8 of 16 pull request and merge-queue YAML jobs hit the
30-minute limit, each after its tests had passed; the other 8 took 22
to 28 minutes. In merge-queue run 40541059060 the cancelled job failed
Result once the shards had finished, which took #2695 out of the
queue. Result waits about an hour for the Python shards, so the longer
limit adds no time to a run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 marked this pull request as ready for review October 27, 2026 20:32
@vahid-ahmadi

Copy link
Copy Markdown
Collaborator

Review: #2697 at f464059

The change is right: one timeout-minutes line on the YAML job, and the changelog fragment passes. Because Release test calls the same reusable workflow, the new limit covers both callers. No musts.

Should: 45 minutes is already nearly used up by this pull request's own run

Test / YAML on this pull request (run 40559394037) took 41m36s, which leaves 3.4 minutes of headroom. Here is where the time went in the "Run the YAML tests" step (18:23:06 to 19:03:49):

Point Time into step
test session starts 0m12s
5101 passed ... in 620.42s (0:10:20) 27m02s
Step ends 40m43s

So the gap after the summary was 13.7 minutes here, longer than the 8 minutes in the description, and the description's "passing jobs take 22 to 28 minutes" is already out of date. The Release test / YAML job for the same commit took 23m17s, so the spread between runners is wide. On the next slow runner, 45 will cancel a job that has passed, which is the failure this pull request is meant to stop.

Your own argument supports a higher limit: Result waits about an hour for the Python shards, so a limit of 60 adds no wall-clock time to a normal run and still stops a job that hangs. I suggest 60, and updating the comment and changelog wording to give the range we have now seen (23 to 42 minutes). If you'd rather keep 45, please note in the description that this run used 41.6 of it.

Note

I'll approve once the limit question is settled either way.

🤖 Generated with Claude Code

@juaristi22
juaristi22 added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit fad5f60 Oct 27, 2026
38 of 39 checks passed
@juaristi22
juaristi22 deleted the ci-yaml-test-timeout branch October 27, 2026 21:39
MaxGhenis added a commit that referenced this pull request Oct 27, 2026
Brings in #2697 (YAML job limit 45 minutes) so CI reruns on the new limit.
No conflicts; the PR's own diff is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit to PolicyEngine/microcosm that referenced this pull request Oct 27, 2026
2.162.14 is the first release on PyPI that carries
PolicyEngine/policyengine-uk#2695: 2.162.13's release run stopped at its
YAML test job's 30-minute limit, so it never published, and
PolicyEngine/policyengine-uk#2697 raised the limit. Its package code outside
the tests is the head commit the previous pin installed (b71ce793), so the
interim git source goes and the lock takes the registry release.

This pair's local-area cut stays the build at db50ae8, which records
policyengine-uk 2.162.12; its national is built at this commit on 2.162.14.
The uprating and bonus-sacrifice pins keep every value, concept coverage
finds no input added or removed, and the coverage manifest, the H2 spine
fixture, the version mentions and the US worker identity's lock digest move
with the lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The YAML test job hits its 30-minute limit after its tests pass

2 participants