Skip to content

fix(issue-pr-loop): finish open PRs before starting new issues - #431

Merged
thomhurst merged 2 commits into
mainfrom
loop-wip-limit
Nov 2, 2026
Merged

thomhurst merged 2 commits into
mainfrom
loop-wip-limit

Conversation

@thomhurst

@thomhurst thomhurst commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Workers skipped every PR whose checks were still running and claimed new issues instead, so open PRs piled up and the chance of conflicts grew.

Changes to scripts/Start-IssuePrLoop.ps1:

  • New -MaxOpenPrs (default 8, 0 = no limit). While that many of the loop's open PRs (drafts included) lack needs-human/on-hold, no new issue is queued; stranded branches, stale in-progress claims and finished parents still are. The held count appears in the survey stats ("held by the open-PR limit").
  • A PR with an unresolved review thread, or a changes-requested review newer than both the head commit and the loop's last reply, is no longer skipped for pending checks. It ranks with failed-CI PRs, is not Light, and its prompt says to handle the feedback without waiting for checks. Conversation comments do not count, since a review with no findings is also a comment.
  • Get-PrActivity fetches head commit time and thread resolution in one paginated GraphQL query (gh pr list --json commits exceeds GitHub's node limit).

The matching SKILL.md priority-list change went to main directly (4d52aec).

Ported from thomhurst/TUnit#7423. This repo has no Test-IssuePrLoop.ps1; the script parses cleanly, and a throwaway harness that loads the survey functions passed the same scenarios as TUnit's tests (feedback wakes a pending PR, a loop reply or an older review does not, parked PRs don't count toward the limit, drafts do, a zero limit holds nothing).

Summary by CodeRabbit

  • New Features
    • Added a configurable limit on open pull requests, defaulting to 8; set it to 0 to disable the limit. Draft pull requests count toward the limit, while parked pull requests do not.
  • Improvements
    • When the limit is reached, new issue candidates are skipped while existing work remains eligible.
    • Pull requests with unresolved review threads or newer changes-requested feedback can be selected while checks are running.
    • Issues with unpublished local commits and issues at the open-pull-request limit receive higher priority.

Workers skipped every PR whose checks were still running and claimed new
issues instead, so open PRs piled up and conflicts grew more likely.

- Add -MaxOpenPrs (default 8, 0 = no limit): while that many of the
  loop's open PRs lack needs-human/on-hold, no new issue is queued.
  Stranded branches, stale claims and finished parents still are. The
  held count shows in the survey stats.
- A PR with an unresolved review thread, or a changes-requested review
  newer than the last push and the loop's last reply, is worked even
  while its checks run, ranks with failed-CI PRs, and is not Light.
- Get-PrActivity reads head commit time and thread resolution with one
  paginated GraphQL query (gh pr list --json commits exceeds the node
  limit).

Ported from thomhurst/TUnit#7423.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thomhurst
thomhurst deployed to Pull Requests November 2, 2026 23:30 — with GitHub Actions Active
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T23:32:32.712049Z 6eb4ada PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

The script adds a configurable limit for new issue candidates based on open, unparked PRs. It also retrieves review activity and updates PR selection so that qualifying feedback can make a PR eligible while checks are pending.

Changes: Issue and PR survey controls

Layer / File(s) Summary
Configure and apply the open-PR limit
scripts/Start-IssuePrLoop.ps1
Adds -MaxOpenPrs, with a default of 8 and a valid range of 0 to 1000. A value of 0 disables the limit. When the positive limit is reached, the survey skips new issue candidates and increments the issue-wip statistic.
Select PRs with actionable review feedback
scripts/Start-IssuePrLoop.ps1
Retrieves PR head commit times and unresolved review-thread counts. Unresolved threads qualify as feedback. A changes-requested review qualifies when it is newer than both the head commit and the loop’s latest review or marker comment. Pending-check PRs remain eligible when conflicting or carrying qualifying feedback; the selection ranks clean PRs first, followed by conflicting PRs, then failed or feedback-bearing PRs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: Low · up to 77648

The new open-PR limit is a soft cap and can be exceeded when several loop copies run in parallel. A reopened review thread can also stay ignored on a deferred PR until the deferral expires. PRs with more than 100 review threads may be missed as having feedback. None of these breaks the loop, but the author should be aware of them or follow up.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit checks the PR queue at dawn
Counts open branches, then hops along
New review notes bring waiting work in view
Pending checks do not hide feedback anew
At the limit, new issue picks pause
The loop reports each skipped cause
And the rabbit nibbles clover by the log

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thomhurst
thomhurst deployed to Pull Requests November 2, 2026 23:31 — with GitHub Actions Active

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6eb4ada871

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +271 to +272
# checks still run. Conversation comments do not count: a finished review with no
# findings is also a comment, and would send units to PRs with nothing to do.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Include actionable conversation reviews in feedback detection

When a reviewer posts findings as a PR conversation comment while checks are pending, this function returns false unless there is also an unresolved thread or a qualifying CHANGES_REQUESTED review. Get-Candidates consequently skips the PR and can start another issue despite the outstanding findings. AGENTS.md explicitly identifies conversation comments as a review surface used by Claude and requires checking both surfaces every cycle. Distinguish actionable conversation reviews from empty acknowledgments rather than excluding that surface entirely.

AGENTS.md reference: AGENTS.md:L63-L63

Useful? React with 👍 / 👎.

function Get-PrActivity {
$query = 'query($owner: String!, $name: String!, $cursor: String) { repository(owner: $owner, name: $name) { ' +
'pullRequests(states: OPEN, first: 100, after: $cursor) { pageInfo { hasNextPage endCursor } ' +
'nodes { number commits(last: 1) { nodes { commit { committedDate } } } reviewThreads(first: 100) { nodes { isResolved } } } } } }'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Paginate review threads before counting unresolved feedback

For a PR with more than 100 review threads, this query reads only the first 100 threads; the surrounding pagination advances the PR connection, not each PR's thread connection. If those initial threads are resolved and a later thread contains new feedback without a CHANGES_REQUESTED review, UnresolvedThreads becomes zero and the pending-check filter skips actionable work. Request thread pageInfo and fetch the remaining pages before deciding that the PR has no unresolved feedback.

Useful? React with 👍 / 👎.

@thomhurst
thomhurst merged commit 99a27cf into main Nov 2, 2026
10 of 11 checks passed
@greptile-apps

greptile-apps Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR changes which work the issue-PR loop sends to agents.

The PR appears safe to merge, with non-blocking gaps that can delay work on review feedback.

Findings

  1. P2 Reopened feedback still waits ▶
  2. P2 Later threads are missed ▶

Summary

The PR adds MaxOpenPrs to hold new issues while existing PRs accumulate. It also lets workers handle review feedback before checks finish.

  • The loop sends PRs with fresh review feedback to an agent before checks finish.
  • The loop holds new issues when too many PRs are open.

Reviews (1) · Last reviewed commit: "docs(issue-pr-loop): rewrap the survey s..." · Reviewed by Greptile

else { '' }
# A green, clean PR usually needs only the merge gate or a disposition reply.
New-Candidate 'pr' ([int]$pr.number) $rank (Get-PrFingerprint $pr) ($rank -eq 0)
New-Candidate 'pr' ([int]$pr.number) $rank (Get-PrFingerprint $pr) ($rank -eq 0) $hint

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Reopened feedback still waits

Reopening a resolved thread can leave a waiting PR skipped. Test-HasNewFeedback detects the thread, but Get-PrFingerprint excludes thread state. Select-Item therefore keeps the existing wait until checks finish or it expires.

Include thread state in both survey and post-unit fingerprints so reopened feedback wakes the PR without repeatedly waking unchanged feedback.

function Get-PrActivity {
$query = 'query($owner: String!, $name: String!, $cursor: String) { repository(owner: $owner, name: $name) { ' +
'pullRequests(states: OPEN, first: 100, after: $cursor) { pageInfo { hasNextPage endCursor } ' +
'nodes { number commits(last: 1) { nodes { commit { committedDate } } } reviewThreads(first: 100) { nodes { isResolved } } } } } }'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Later threads are missed

Get-PrActivity reads only the first 100 review threads, including resolved threads. If those are resolved and a later thread remains open, UnresolvedThreads becomes zero. A pending PR without a newer changes-requested review is then skipped until checks finish.

Fetch all thread pages before treating the PR as having no unresolved feedback.

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Review

The change looks sound. I read the diff and checked that the helpers it relies on exist in the PR head ($loopMarker, New-Candidate with its $Hint parameter, and $prFields including reviews, comments and author). I did not run the script.

What I checked

  • The -MaxOpenPrs limit counts drafts but not parked PRs. Stranded branches, stale claims and finished parents still run.
  • Feedback detection treats an unresolved thread or a newer CHANGES_REQUESTED review as feedback. It ignores conversation comments, and the loop's own replies and marker comments move the baseline forward.
  • Get-PrActivity is paginated and asks only for the head commit.

Minor points, not blocking

  1. reviewThreads(first: 100) is not paginated. A PR with more than 100 threads would count only the first 100. That is unlikely, but a bot-heavy PR could reach it. Counting only the unresolved threads among them is safe, though threads past the first 100 are missed.
  2. When Get-PrActivity fails, the whole survey throws. Falling back to an empty activity map would degrade to the old behavior (pending PRs skipped) instead of stopping the loop. Whether a loud failure is preferable is your call.
  3. The PR description says there is no Test-IssuePrLoop.ps1 here. The ported TUnit scenarios (feedback wakes a pending PR, a loop reply does not, parked PRs don't count, drafts do, a zero limit holds nothing) would be worth committing as a small Pester or script test. Otherwise this logic is only verified by a throwaway harness.

LGTM overall.

This branch was successfully deployed

1 active deployment
Pull Requests — 7764889d Deployed Nov 2, 2026 by thomhurst via modularpipeline #1675
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant