fix(ci): daily-update validates data instead of unit-testing live data; API + push retries

.github/workflows/daily-update.yml

@@ -81,7 +81,9 @@ jobs:
         run: python3 tools/update_stars.py --slot "$SLOT"
       - name: Verify
         if: steps.gate.outputs.run == 'true'
-        run: python3 tools/verify.py
+        # Data checks only (tools/data_checks.py): hard errors fail, drift against the GitHub API warns.
+        # The unit tests run in ci.yml on every pull request, not against live data here.
+        run: python3 tools/verify.py --daily
       - name: Commit and push if anything changed
         if: steps.gate.outputs.run == 'true'
         run: |
@@ -93,4 +95,14 @@ jobs:
           git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
           git add data rankings latest README*.md docs/HOW_IT_WORKS*.md docs/DETAILS*.md
           git commit -m "chore(data): daily star update $(date -u +%F)"
-          git push origin HEAD:main
+          # main can move while the job runs (a merged PR): rebase onto it and retry instead of failing the day.
+          for attempt in 1 2 3 4 5; do
+            if git push origin HEAD:main; then
+              exit 0
+            fi
+            echo "push rejected (attempt $attempt); rebasing onto origin/main"
+            sleep $((attempt * 5))
+            git pull --rebase origin main || { git rebase --abort || true; echo "rebase conflict" >&2; exit 1; }
+          done
+          echo "could not push after 5 attempts" >&2
+          exit 1

CURRENT_PLAN.md

@@ -2,40 +2,39 @@
 
 ## Plan ID
 
-PLAN-003
+PLAN-004
 
 ## Goal
 
-Polish the generated README first screen and stop baking an update date into the badge URL; the date must come from `data/stats.json` and refresh on every daily-update run.
+Stop the recurring daily-update failures at the root: the scheduled run must validate the data it generated (hard errors fail, GitHub API timing drift warns) instead of running the unit-test suite against live data, and must survive rate limits and a moving `main`.
 
 ## In Scope
 
-- `tools/render_readme.py` and `tools/i18n.py` (README / HOW_IT_WORKS / DETAILS copy)
-- Regenerated `README*.md` (and the other generated pages the renderer writes)
-- Tests that the update badge is dynamic and the stamp text is filled from `stats.json`
-- Concise first-screen copy: what the list is, the Actions schedule, how numbers are computed, how to use or contribute, links
+- `tools/data_checks.py`, `tools/verify.py --daily`, `.github/workflows/daily-update.yml`, `tools/local_daily.sh`
+- `tools/update_stars.py` API retry/backoff, `tools/risers_core.py` hard limit for history drift
+- Tests in `tests/test_daily_robustness.py`; docs in `docs/DAILY_UPDATE.md` and governance files
 
 ## Out of Scope
 
-- Ranking logic, categories, the ⚠ suspected-crack rules, `data/stats.json` numbers
-- Secrets, the daily-update cron times, GEO files (`docs/FAQ.md`, `docs/FACTS.md`, `CITATION.cff`, social preview)
-- Merging to `main`
+- Ranking logic, README copy, cron times, secrets, merging to `main`
 
 ## Tasks
 
-- [x] Replace the static `updated-YYYY--MM--DD` shields badge with a dynamic badge that reads `today_utc` from `data/stats.json`
-- [x] Keep the stamp line filled from `stats.generated_at` on every `render_readme` / daily-update run
-- [x] Add two first-screen FAQ items (how numbers are computed; how to use or contribute) linking existing `docs/FAQ.md` anchors
-- [x] Add a test that templates contain no hard-coded update date and that the rendered stamp follows `generated_at`
+- [x] Group the failed runs by cause (41122642510, 41235403615)
+- [x] Add `tools/data_checks.py` with error and warning severities and `verify.py --daily`
+- [x] Wire the workflow and `local_daily.sh` to `--daily`; unit tests stay in `ci.yml`
+- [x] Retry 429 / rate-limited 403 / 5xx / network errors; retry the push after rebasing onto `main`
+- [x] Regression tests for all of the above
 - [ ] Owner reviews the PR; merge only with approval
 
 ## Verification
 
 - `python3 tools/verify.py`
+- `python3 tools/verify.py --daily`
 - `python3 tools/next_plan.py --check`
 
 ## Next Candidates
 
-- After merge, confirm the next `daily-update` run commits data
+- After merge, confirm the next scheduled `daily-update` run commits data
 - Thin Developer Tools / Design / Data new-repo boards: see `TODO_BACKLOG.md`
 - Acceleration column (this week vs the week before)

TODO_BACKLOG.md

@@ -2,7 +2,7 @@
 
 ## Now
 
-- [ ] After the history-slack PR is merged, confirm the next `daily-update` run commits data (the 2026-11-01 pattern of Verify blocking a 3-star `history_total` gap must not recur)
+- [ ] After the daily-update reliability PR (PLAN-004) is merged, confirm the next scheduled `daily-update` run commits data (Verify must only fail on real data errors)
 - [ ] Confirm the first unattended cron runs after the merge: `cn-0000` (Beijing) and `pt-0000` (Los Angeles) folders appear under `rankings/<date>/` (`gh run list --workflow daily-update.yml`)
 - [ ] Developer Tools new-repo board: 15 of 20 entries were flagged ⚠ in the last run because star-farmed repos use `cli` / `developer-tools`; decide whether to add a quality signal (owner call)
 - [ ] Design and Data new-repo boards are thin (0-2 entries): widen with specific topics only (no generic ones; `tests/test_spam.py` guards that)

docs/DAILY_UPDATE.md

@@ -85,8 +85,17 @@ would skip. The Beijing slot and the next day's runs still refresh the data, so
 
 ## Option A: GitHub Actions (preferred when it runs)
 
-`.github/workflows/daily-update.yml` regenerates the data, runs `tools/verify.py`, and commits to `main` if anything changed.
+`.github/workflows/daily-update.yml` regenerates the data, runs `tools/verify.py --daily`, and commits to `main` if anything changed.
 It uses only first-party `actions/*` and the built-in `GITHUB_TOKEN`.
+
+Why `--daily` (root cause of the 2026-10-30 and 2026-10-31 failures): the run used to execute the whole unit-test suite
+against the data it had just fetched, so anything that moves with GitHub (a new repo without a hook yet, a 3-star gap between
+`/stargazers/history` and `/repos`) failed the run and the day's update was lost. Now the daily run only validates the data
+(`tools/data_checks.py`): **errors** fail (project list mismatch, negative or inconsistent growth, no snapshot for today,
+history vs stars off by more than `max(50, 1 %)`, more than 25 % of repos fetched as missing) and **warnings** (history vs stars
+off by more than `HISTORY_TOTAL_STAR_SLACK`, a few missing repos) show as `::warning::` annotations. The unit tests still run in
+`ci.yml` on every pull request. API calls retry 5xx, network errors, `429` and rate-limited `403` with backoff (honouring
+`Retry-After` / `X-RateLimit-Reset`, capped at 120 s); the push step rebases onto `main` and retries up to 5 times.
 `.github/workflows/ci.yml` runs `tools/verify.py` on every pull request and push to `main` (no network).
 
 Scheduled workflows run from the default branch only.
@@ -115,7 +124,7 @@ gh api repos/jammyfu/github-risers/check-runs/<job_id>/annotations --jq '.[].mes
 
 ```bash
 GITHUB_TOKEN=$(gh auth token) python3 tools/update_stars.py   # fetch + render only
-tools/local_daily.sh                  # update, verify, commit, push to main (any time)
+tools/local_daily.sh                  # update, verify (--daily), commit, push to main (any time)
 tools/local_daily.sh --no-push        # update, verify, commit locally
 tools/local_daily.sh --slot beijing   # only proceeds if it is 00:xx in Asia/Shanghai
 tools/local_daily.sh --slot la        # only proceeds if it is 00:xx in America/Los_Angeles (DST-aware)

docs/project-governance/CHANGELOG.md

@@ -1,5 +1,10 @@
 # CHANGELOG.md
 
+## 2026-11-01 (daily-update reliability)
+
+- The daily run no longer runs the unit tests against live data. `python3 tools/verify.py --daily` validates the generated data (`tools/data_checks.py`): real errors fail, GitHub API timing drift only warns. CI still runs every unit test on pull requests.
+- API retries for rate limits (`429`, rate-limited `403`), 5xx and network errors with capped backoff; the push step rebases onto `main` and retries.
+
 ## 2026-11-01 (README date and first screen)
 
 - The README `updated` badge is a dynamic shields.io JSON badge that reads `today_utc` from `data/stats.json`. The stamp line is filled from `generated_at` on every daily-update render. No update date is typed into `tools/i18n.py`.

docs/project-governance/DECISIONS.md

@@ -68,6 +68,14 @@
 - The daily-update Verify step (`tests/test_repo.py`) still requires a full history sum to stay close to `stargazers_count`. The allowed absolute delta is now 5 (`tools/risers_core.py::HISTORY_TOTAL_STAR_SLACK`).
 - A 3-star gap is accepted; a gap larger than 5 still fails. The check is not removed.
 
+## 2026-11-01 (daily-update root cause)
+
+### The daily run validates data; it does not run the unit tests against live data
+
+- Failures: run 41122642510 (2026-10-30 20:22Z, hook test: new repos without a hook) and run 41235403615 (2026-10-31 19:41Z, `history_total` vs `stars` off by 3). Each fix raised one limit; the next live value would have failed again, and a failed run discards the whole day's update.
+- Decision: `tools/verify.py --daily` = compile + `tools/data_checks.py` (hard errors fail, API-timing drift warns) + README freshness. Unit tests stay in CI (`ci.yml`) on every PR. Real data errors still fail.
+- Plus: `api_get` retries 429 / rate-limited 403 / 5xx / network errors with capped backoff; the push step rebases and retries.
+
 ## 2026-11-01 (README update date)
 
 ### Do not bake the update date into the badge URL

docs/project-governance/WORKLOG.md

@@ -59,6 +59,12 @@
 - Gaps found: stale `CURRENT_PLAN.md`/`MASTER_PLAN.md`/`TODO_BACKLOG.md`/governance files; dirty-worktree and record-order rules only in chat; no check of the plan shape; no regression test that `verify.py` and `local_daily.sh` avoid PATH/`shell=True`.
 - Added `tools/next_plan.py --check`, `tests/test_harness.py`, and wrote the rules into `AGENTS.md` and `docs/`.
 
+## 2026-11-01 — daily-update root cause (PLAN-004)
+
+- Failed runs: 41122642510 (hook completeness test on live data) and 41235403615 (`history_total` vs `stars` gap of 3). Common cause: Verify ran the full unit-test suite on freshly fetched data.
+- Added `tools/data_checks.py`, `verify.py --daily`, rate-limit/5xx retry in `api_get`, push retry with rebase; tests in `tests/test_daily_robustness.py`.
+- Verification: `python3 tools/verify.py` green; `python3 tools/verify.py --daily` green.
+
 ## 2026-11-01 — README date and first-screen polish (PLAN-003)
 
 - Replaced the static `updated-YYYY--MM--DD` shields badge with a dynamic JSON badge that reads `$.today_utc` from `data/stats.json` on `main`.

tests/test_core.py

@@ -56,9 +56,10 @@ def test_history_total_slack_accepts_a_three_star_gap(self):
         self.assertTrue(core.history_total_within_slack(100, 95))
         self.assertTrue(core.history_total_within_slack(100, 100))
         self.assertFalse(core.history_total_within_slack(100, 106))
-        src = Path(__file__).resolve().parents[1].joinpath("tests/test_repo.py").read_text("utf-8")
-        self.assertIn("history_total_within_slack", src)
-        self.assertNotRegex(src, r"assertLessEqual\(abs\(p\[.history_total.\] - p\[.stars.\]\), 2")
+        src = Path(__file__).resolve().parents[1].joinpath("tools/data_checks.py").read_text("utf-8")
+        self.assertIn("history_total_within_slack", src)  # the daily data check warns on this drift
+        repo_tests = Path(__file__).resolve().parents[1].joinpath("tests/test_repo.py").read_text("utf-8")
+        self.assertNotIn("history_total_within_slack", repo_tests)  # drift must not fail a unit test
 
 
 class RankTests(unittest.TestCase):