feat(orchestrator): build extension profiles from configuration - #545
Merged
Conversation
This was referenced Aug 7, 2026
behinddwalls
marked this pull request as ready for review
August 7, 2026 18:24
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 7, 2026 18:36
0c95872 to
2390a96
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 7, 2026 21:21
2390a96 to
1a26c63
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 7, 2026 22:19
1a26c63 to
a2b03cd
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
2 times, most recently
from
August 11, 2026 21:06
5a99ea1 to
71c6078
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 11, 2026 22:39
71c6078 to
33143de
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 11, 2026 22:52
33143de to
edac343
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 12, 2026 01:57
edac343 to
d72a5b4
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
2 times, most recently
from
August 12, 2026 19:04
b1d2195 to
f8e4063
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 12, 2026 20:24
f8e4063 to
2c7736e
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 12, 2026 20:47
2c7736e to
f904b2f
Compare
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
2 times, most recently
from
August 12, 2026 21:37
420022e to
f904b2f
Compare
## Summary ### Why? Which implementation of each extension a queue resolved to was hardcoded: the build runner was the fake one for every queue regardless of what was available, and the change provider was a global all-or-nothing environment gate. A deployment could not run one queue against a real provider next to one running entirely on fakes, which is what a stack serving both a test environment and a live repository needs. ### What? `PROFILES_CONFIG_PATH` names a YAML file selecting the change provider, build runner, and conflict analyzer per queue. Each extension is independently optional, so a queue that differs only in its analyzer says only that. `kind` is an open string rather than a closed schema, so supporting a new provider is a new value and an implementation behind it, not a change to the file's shape. The file holds no secret: each integration names the environment variable carrying its credential. The `pathoverlap` analyzer takes a `by` granularity — `file`, or `directory` to coarsen a queue's conflicts to a shared parent. It defaults to `file`, the narrower of the two: a default that widened what conflicts would serialize a queue more than its configuration asked for. With no config file the built-in example topology applies, reproducing the previous behavior exactly — including the per-queue analyzers the E2E suite depends on, and a routing change provider that still falls back to the fake when no token is set. That is what keeps the existing suite meaningful as a regression gate. Extensions are reused across queues configured alike. This is load-bearing for the build runner: the build and buildsignal controllers look it up separately and the fake holds a build's outcome in memory, so two instances would lose the result between triggering a build and polling it. ## Test Plan ✅ `bazel test //service/submitqueue/orchestrator/server:go_default_test` — pins the built-in topology against what the E2E suite expects, covers per-extension inheritance, provider defaults, every validation rejection, that queues configured alike share one build runner, that each queue's analyzer behaves as configured, and that a missing token fails at startup rather than mid-merge. ✅ Added with the `by` granularity: it defaults to `file`, `directory` is accepted, and an unrecognized value is rejected at load rather than at the first queue that resolves it.
behinddwalls
force-pushed
the
sq/orchestrator-profiles
branch
from
August 12, 2026 21:41
f904b2f to
782e786
Compare
mnoah1
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
Which implementation of each extension a queue resolved to was hardcoded: the build runner was the fake one for every queue regardless of what was available, and the change provider was a global all-or-nothing environment gate. A deployment could not run one queue against a real provider next to one running entirely on fakes, which is what a stack serving both a test environment and a live repository needs.
What?
PROFILES_CONFIG_PATHnames a YAML file selecting the change provider, build runner, and conflict analyzer per queue. Each extension is independently optional, so a queue that differs only in its analyzer says only that.kindis an open string rather than a closed schema, so supporting a new provider is a new value and an implementation behind it, not a change to the file's shape.The file holds no secret: each integration names the environment variable carrying its credential.
The
pathoverlapanalyzer takes abygranularity —file, ordirectoryto coarsen a queue's conflicts to a shared parent. It defaults tofile, the narrower of the two: a default that widened what conflicts would serialize a queue more than its configuration asked for.With no config file the built-in example topology applies, reproducing the previous behavior exactly — including the per-queue analyzers the E2E suite depends on, and a routing change provider that still falls back to the fake when no token is set. That is what keeps the existing suite meaningful as a regression gate.
Extensions are reused across queues configured alike. This is load-bearing for the build runner: the build and buildsignal controllers look it up separately and the fake holds a build's outcome in memory, so two instances would lose the result between triggering a build and polling it.
Test Plan
✅
bazel test //service/submitqueue/orchestrator/server:go_default_test— pins the built-in topology against what the E2E suite expects, covers per-extension inheritance, provider defaults, every validation rejection, that queues configured alike share one build runner, that each queue's analyzer behaves as configured, and that a missing token fails at startup rather than mid-merge.✅ Added with the
bygranularity: it defaults tofile,directoryis accepted, and an unrecognized value is rejected at load rather than at the first queue that resolves it.