[jammyfu]

Why the daily-update keeps failing (root cause)

In the last 3 weeks the repo has had 9 daily-update runs (workflow first ran 2026-10-30): 2 failed, 6 succeeded, 1 manual rerun was still running when this PR was opened. Both failures are the same design flaw: the Verify step ran the whole unit-test suite against the data the job had just fetched, so anything that moves with GitHub failed the run, and a failed run discards the whole day's update (nothing is committed).

Run	Time (UTC)	Failed check	Trigger
41122642510	2026-10-30 20:22	test_hooks.test_every_ranked_new_repo_has_a_hook_or_is_listed_as_pending — 22 newly ranked repos had no hook yet (CI has no model key)	schedule
41235403615	2026-10-31 19:41	test_repo.test_stats_cover_every_project_and_are_consistent — affaan-m/ECC history_total vs stars differed by 3 (limit was 2)	schedule

Previous fixes (#14 hooks, #15 slack 2 -> 5) each raised one limit; the next live value would have failed again. No rate-limit or push-conflict failures were seen yet, but neither was handled (only 5xx was retried; git push had no retry).

Changes

• Daily run validates data, not code: python3 tools/verify.py --daily = compile + new tools/data_checks.py + README freshness. Unit tests still run in ci.yml on every PR/push.
• tools/data_checks.py: errors fail (project list mismatch, missing growth, negative gain, inconsistent per_day, no snapshot for today, history_total vs stars off by more than max(50, 1%), more than 25% of repos fetched as missing = outage/rate limit). Warnings (::warning::) for small history/star drift (> 5) and a few missing repos; the run stays green.
• api_get: 5 attempts; backoff for 5xx and network errors; 429 and rate-limited 403 honour Retry-After / X-RateLimit-Reset (capped at 120 s); plain 403/404/422 stay final.
• Workflow push step: up to 5 attempts, git pull --rebase origin main between them (main moves when a PR merges mid-run). Concurrency group daily-update was already there (kept, tested).
• tools/local_daily.sh uses --daily too. test_repo no longer asserts live drift (delegates to data_checks, errors only).

Files

tools/data_checks.py (new), tools/verify.py, tools/update_stars.py, tools/risers_core.py, tools/local_daily.sh, .github/workflows/daily-update.yml, tests/test_daily_robustness.py (new, 19 tests), tests/test_repo.py, tests/test_core.py, docs/DAILY_UPDATE.md, governance docs (CURRENT_PLAN.md PLAN-004, TODO_BACKLOG.md, WORKLOG/DECISIONS/CHANGELOG).

Verification

python3 tools/verify.py            # 222 tests OK, README current
python3 tools/verify.py --daily    # data check passed: 70 projects, 0 warnings
GITHUB_EVENT_NAME=schedule python3 tools/verify.py --daily
python3 tools/next_plan.py --check

Not merged. After merge, trigger daily-update once (workflow_dispatch) and confirm it commits.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T09:10:40.156352Z	0357b76	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.