Skip to content

Add companion-check workflow: docs PRs auto-merge once their companion code PR lands - #629

Open
kriszyp wants to merge 6 commits into
mainfrom
kris/companion-check
Open

Add companion-check workflow: docs PRs auto-merge once their companion code PR lands#629
kriszyp wants to merge 6 commits into
mainfrom
kris/companion-check

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 15, 2026

Copy link
Copy Markdown
Member

Adds a companion-check commit status driven by Depends-on: markers in PR bodies, so a docs PR documenting an unmerged code change can be approved and armed for auto-merge, then land automatically once its companion PR (e.g. harper#2147) merges. PRs without a marker get an immediate success, so once companion-check becomes a required check it never blocks ordinary docs PRs.

For the human reviewer

  1. No-marker means silent success. Chosen so the required check never gates ordinary docs PRs; the alternative (explicit Depends-on: none opt-out) reaches every author's habits. To cover the typo risk this creates, everything else fails closed: empty markers, unparseable refs, and any non-ref residue on a marker line all post failure. Reversible in-script.
  2. Poll, not push. A 15-minute cron sweep polls companion state instead of the companion repos dispatching events here. Costs up to 15 minutes of merge latency and a modest API budget (~2 calls per open PR per sweep, dep lookups memoized and capped at 10 per PR); avoids wiring tokens/workflows into harper and harper-pro. Reversible later without changing the marker contract.
  3. Per-repo copy, not a reusable workflow. This lands here and (separately) in harper-pro as harper-pro#704. If a third repo wants it, promoting to HarperFast/.github is the moment to deduplicate; the committed test harness makes drift detectable.
  4. Inline github-script, tested by extraction. The ~150-line script stays in the YAML (no build step, no action packaging); scripts/companion-check.test.mjs extracts it by its 12-space indent and runs 25+ scenario assertions, wired into the Validate matrix as Workflow Tests. The extraction is a stateful parse of the script: | block (until dedent) and asserts loudly if it comes back empty.
  5. pull_request_target with a secret in scope, on a public repo. Mitigations: the workflow never checks out PR code, the body is only ever data inside github-script, and the optional COMPANION_CHECK_TOKEN is used exclusively for same-org refs on non-fork PRs — a fork PR referencing a private repo blocks with an explanatory failure rather than probing with the secret. A maintenance rule at the top of the file states the invariant.
  6. Definitive 404s fail red. A companion that doesn't resolve (typo, private repo without the token, token lacking access) posts failure "not found or inaccessible" rather than an eternal pending blaming a token. Both states block; the failure is more honest about needing human attention.
  7. Accepted residuals: a millisecond TOCTOU between the sweep's freshness re-read and its status write (commit statuses have no CAS; the surviving stale direction is pending, which blocks); markers inside fenced code blocks are parsed as live (quote the word, not the marker form, when writing about it); statuses (not check runs) were chosen for required-check simplicity, so there is no re-run button — the 15-minute sweep or a workflow_dispatch is the retry path.

Not in this PR (needs repo admin, after merge): enable the companion-check required status check in branch protection, run one workflow_dispatch backfill sweep, and optionally add COMPANION_CHECK_TOKEN (fine-grained PAT or app token with PR read on harper-pro) for docs PRs that depend on private-repo PRs. Repo auto-merge (allow_auto_merge) is already enabled.

Verification

npm run test:workflows (added to the Validate matrix) passes: 25+ assertions over the extracted production script covering marker parsing (all three documented forms, shorthand, dedupe, cap, empty/partial/traversal markers failing closed), state reduction (merged/open/closed/missing/unreadable precedence), sweep semantics (no-repost dedup, no-marker backfill, dropped-webhook healing, race-guard skip on changed PRs), error isolation (HTTP 500 degrades to pending; a thrown refresh posts pending rather than preserving stale success), and secret guards (withheld for foreign orgs and fork PRs). npm run format:check clean. The workflow itself cannot run end-to-end from this branch (schedule fires on the default branch only, and the status context becomes selectable in branch protection only after a first main run) — post-merge verification is a manual workflow_dispatch sweep plus arming #623 as the live trial.

Review coverage

Authored by Claude Fable 5. Cross-model review across 4 rounds (full @ f2c4478, deltas @ 2fd988e / e6093a1, final @ 71750b6 = HEAD): codex graded leg ✓ (gpt-5.6-sol @ HEAD), gemini via agy (default model) ✓ every round, Harper-domain adjudication ✓ (claude-opus-5, last @ e6093a1; pruned on the final narrow delta), cursor-grok ✗ (output-format failure), cursor-composer ✗ (pruned round 1, format failure @ e6093a1). Verdict trail CHANGES → CHANGES → CHANGES → COMMENTS; the remaining findings are the accepted residuals in the ledger above.

Human-Review-Need: 4 @ 71750b6

kriszyp and others added 5 commits August 14, 2026 19:20
…repos

Adds a workflow posting a companion-check commit status driven by
Depends-on: markers in PR bodies, so a docs PR can be approved and
armed for auto-merge, then merge automatically once the feature PR
it documents lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fail closed on unparseable markers, support repo#N shorthand, bound and
dedupe refs, isolate per-dep/per-PR errors, restrict the cross-repo
token to same-org refs, guard sweep/event races, and skip no-marker PRs
in the cron sweep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-2 review fixes: any non-ref residue on a Depends-on line fails the
check; empty markers fail closed; token-confirmed 404s fail as not-found
instead of pending on a token hint; the sweep reconciles every open PR
(backfill + dropped-webhook healing); dep lookups are memoized per run;
path-traversal segments rejected; API base from context.apiUrl. Adds a
self-contained node test harness covering the embedded script.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A missing or unreadable-without-secret companion now blocks with an
honest red failure instead of pending on a token hint that misdiagnoses
typos; transport/auth errors keep the pending Cannot-read form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a test harness (scripts/companion-check.test.mjs) and registers it under package.json to validate the embedded GitHub script in the companion-check workflow. The reviewer pointed out that the current YAML script extraction logic is fragile and suggested replacing it with a stateful parser to robustly extract only the script block.

Comment thread scripts/companion-check.test.mjs Outdated
@github-actions
github-actions Bot temporarily deployed to pr-629 August 15, 2026 02:00 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-629

This preview will update automatically when you push new commits.

…on (review feedback)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-629

This preview will update automatically when you push new commits.

@kriszyp
kriszyp marked this pull request as ready for review August 15, 2026 04:03
@kriszyp
kriszyp requested a review from a team as a code owner August 15, 2026 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant