docs(stovepipe): record stage RFC, aligned with the shipped implementation - #394
Open
mnoah1 wants to merge 4 commits into
Open
docs(stovepipe): record stage RFC, aligned with the shipped implementation#394mnoah1 wants to merge 4 commits into
mnoah1 wants to merge 4 commits into
Conversation
mnoah1
marked this pull request as ready for review
July 16, 2026 19:33
mnoah1
force-pushed
the
mnoah1/record-step
branch
2 times, most recently
from
August 4, 2026 20:22
4dc7628 to
7b28df2
Compare
mnoah1
commented
Aug 4, 2026
| | `Project` | Empty for the whole repository; stable project id for Phase 2 | | ||
| | `Degree` | Health degree in the closed interval `[0, 1]`; `0` is green and `1` is fully broken | | ||
| | `RequestID` | Request that established the fact | | ||
| | `CreatedAt` | Millisecond timestamp at which the fact was first recorded | |
Contributor
Author
There was a problem hiding this comment.
Thoughts on timestamp? We don't track timestamps elsewhere in the system but for these more long-term persisted facts, do we want to maintain some kind of timestamped history?
roychying
reviewed
Aug 5, 2026
|
|
||
| A fact is written only when the request reaches a `succeeded` or `failed` verdict. A `cancelled` build is acked with no fact recorded. Callers gating deployments treat an absent fact as not green. | ||
|
|
||
| The fail-closed path also produces no fact, as the DLQ reconciler forces `failed` and never publishes to `record`. |
Contributor
There was a problem hiding this comment.
I don't think never publishes to record holds when the fail-close originates in build_dlq. if build ack keeps failing, once retry_count hits MaxAttempts, the subscriber itself moves it to the DLQ, leaving one or more live poll chains behind in buildsignal step, and publishRecord fires anyway, record then writes a degree-1 fact regardless of the build result.
mnoah1
force-pushed
the
mnoah1/record-step
branch
from
August 13, 2026 15:58
7b28df2 to
2f229dd
Compare
mnoah1
changed the title
…mework The record stage landed in #555 while this doc still read as a proposal, so the RFC described a design rather than the code. Re-frames it as-built and corrects the contract details that drifted: ValidationFact carries no Queue field (the queue is the store binding and the leading PK column), Get takes (uri, project), the Record payload carries queue_name beside the request id, degrees are the named DegreeGreen/DegreeBroken constants, LastGreenRequestID is shipped rather than proposed, and the bookmark's CAS conflict is absorbed by the retry loop rather than returned. Adds the two algorithm steps the doc omitted (deserialize, per-queue storage resolution) and the ordering rationale for advancing the bookmark only after the fact is durable. Replaces the inline Hooks design with integration against hook-framework.md, which supersedes it: a HookEvent published to a per-domain hook topic and dispatched to a pluggable extension, not a Notify() call out of the stage. Records where the publish belongs, the event shape, and the decision that `type` names the scope (validation.repository.recorded) rather than the outcome, since scope is the axis consumers cannot route on later. Documents that record_dlq has no consumer even though dead-lettering is enabled on the subscription, so rejected messages accumulate unread.
…uarantee Rebasing brought ref promotion and the two commit-timestamp metrics into the record stage, none of which the RFC described: promotion is gated on holding the last-green bookmark so an older green commit cannot drag the ref backwards, and the equal-id case reports holding it so a crash before promoting is recoverable. Also corrects an overstated claim. The fail-closed path was documented as never reaching record, but buildsignal publishes unconditionally once a build goes terminal, outside the guard that makes its state write idempotent. A request forced to failed by buildsignal_dlq reconciliation can therefore be carried into a permanent broken fact by a surviving poll chain that observed success. Records the trace, notes that build_dlq is orphaned alongside record_dlq, and opens the question of whether observed success should outrank a forced failure.
mnoah1
force-pushed
the
mnoah1/record-step
branch
from
August 13, 2026 16:18
2f229dd to
74f0a48
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updating record step doc to align with current implementation (now mostly implemented through phase 1), and forward looking plans for Hooks and future phases.