From 0c5638f0c238dd6d48a7d9ca47d964c74180f665 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 12:15:00 +0000 Subject: [PATCH 01/33] feat(watch): never mirror the kube-root-ca.crt ConfigMap The cluster's root CA publisher writes kube-root-ca.crt into every namespace. It holds that one cluster's CA, changes on every rebuild, and the publisher overwrites any other value, so a copy in Git can be neither edited nor restored. Mirroring it only produced a committer-authored reconcile commit per namespace on every cluster rebuild. Both gatherings now leave it out: the snapshot fold (desiredFromObject) and the live route, which counts each dropped event as watch_events_total{outcome="cluster_generated"}. A snapshot that left it out says so in one info line. A copy already in Git is treated like any document the cluster does not hold: prune mode Always sweeps it, OnEvent leaves it for a human to delete. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/TODO.md | 5 +- docs/UPGRADING.md | 9 ++ docs/configuration.md | 7 ++ docs/interpreting-metrics.md | 2 +- internal/telemetry/exporter.go | 2 + internal/watch/cluster_generated.go | 29 ++++++ internal/watch/cluster_generated_test.go | 89 +++++++++++++++++++ internal/watch/desired_state_change_filter.go | 5 ++ internal/watch/scope_resolve.go | 3 + internal/watch/target_watch.go | 12 +++ internal/watch/watch_event_metrics.go | 1 + test/e2e/commit_request_e2e_test.go | 7 +- test/e2e/inplace_edit_e2e_test.go | 14 +-- 13 files changed, 168 insertions(+), 17 deletions(-) create mode 100644 internal/watch/cluster_generated.go create mode 100644 internal/watch/cluster_generated_test.go diff --git a/docs/TODO.md b/docs/TODO.md index 4519174b..80b41eff 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -116,8 +116,9 @@ Their argument for each is in [gitops-api-open-asks.md](future/gitops-api-open-a `Invalidate` re-issues one (`internal/git/render_fidelity_gate.go`), and close the retention revision-mismatch question in [watch-plane-status-convergence-failures.md](design/watch-plane-status-convergence-failures.md) §3. -- [ ] Filter more cluster-generated noise, such as `kube-root-ca.crt`; decide which objects and - whether a user can opt back in (`resource_policy.go` is a type-level denylist only). +- [ ] Filter more cluster-generated noise beyond `kube-root-ca.crt` (always left out, no opt-in: + `internal/watch/cluster_generated.go`), such as OpenShift's `openshift-service-ca.crt`, once + measured on such a cluster. - [ ] A manifest whose CRD is missing refuses the whole folder (`IssueUnresolvedKRM`, `manifestanalyzer/acceptance.go`): keep that, or defer only that document. The scan already records `unresolved-mapping` instead of failing. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 176c54ea..c90d3c01 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,6 +7,15 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. +## `kube-root-ca.crt` is not mirrored + +The `kube-root-ca.crt` ConfigMap the cluster writes into every namespace is never mirrored, whatever +a `WatchRule` selects: it holds that one cluster's CA, and the cluster overwrites any other value. +A copy already in Git is left like any document the cluster does not hold. Under prune mode +`Always` the next resync removes it; under `OnEvent` (the default) delete it from Git yourself. If an +applier applied that folder, it prunes the ConfigMap when the file goes, and the cluster recreates +it at once. + ## Upgrading from 0.53.0: recreate the connection objects in one pass **Breaking, with no in-place conversion.** The objects that connect a folder to a cluster, a diff --git a/docs/configuration.md b/docs/configuration.md index 821f1592..b0ed9a6f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -2708,6 +2708,13 @@ The important fields are: - `spec.rules[].sourceNamespace`: the source-cluster namespace that item watches; omitted means the rule's own namespace +**The `kube-root-ca.crt` ConfigMap is never mirrored**, whatever a rule selects. The cluster writes +it into every namespace, it holds that one cluster's CA, and the cluster overwrites any other value, +so a copy in Git can be neither edited nor restored. A copy already in Git is treated like any +document the cluster does not hold: prune mode `Always` removes it on the next resync, and under +`OnEvent` you delete it yourself. Each snapshot that leaves it out logs `left out cluster-generated +objects on purpose`, and `watch_events_total{outcome="cluster_generated"}` counts its live events. + ### Watching a different source namespace Set `spec.rules[].sourceNamespace` to mirror a namespace other than the one the `WatchRule` lives in diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index 7a03c665..34d5ac43 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -190,7 +190,7 @@ boundary, the commit, the push. Background: | Metric | Type | Labels | Notes | | --- | --- | --- | --- | -| `watch_events_total` | counter | `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource`, `outcome` | The ingest census: every delivered watch event, exactly once. `outcome` is `routed` / `unchanged` / `not_object` / `bookmark` / `shutdown` / `stream_error` / `route_failed`. | +| `watch_events_total` | counter | `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource`, `outcome` | The ingest census: every delivered watch event, exactly once. `outcome` is `routed` / `unchanged` / `cluster_generated` (`kube-root-ca.crt`, never mirrored) / `not_object` / `bookmark` / `shutdown` / `stream_error` / `route_failed`. | | `watch_event_handling_seconds` | histogram | `source_cluster`, `group`, `version`, `resource` | How long a stream was **busy** on one event, attribution wait included. Occupancy, not queue delay. | | `watch_sessions_ended_total` | counter | `source_cluster`, `group`, `version`, `resource`, `reason` | `closed` (the API server's routine watch timeout: the most common ending on a healthy cluster) / `expired` (cursor out of history, forcing a rebuild) / `error` / `stopped`. | | `watch_replay_duration_seconds` | histogram | `source_cluster`, `group`, `version`, `resource` | Time to `initial-events-end`: what a `410` storm charges. | diff --git a/internal/telemetry/exporter.go b/internal/telemetry/exporter.go index df5230ea..10cfc1ab 100644 --- a/internal/telemetry/exporter.go +++ b/internal/telemetry/exporter.go @@ -149,6 +149,8 @@ var ( // unchanged — a live UPDATE whose sanitized content equals what Git already // holds: the /status-churn case, expected in volume. // bookmark — a watch bookmark, carrying cursor progress and no object. + // cluster_generated — an object the cluster itself writes and Git never holds + // (kube-root-ca.crt), dropped on purpose. // shutdown — the stream was cancelled while the event waited for its author. // Not loss: a restart replays and resyncs the object. // diff --git a/internal/watch/cluster_generated.go b/internal/watch/cluster_generated.go new file mode 100644 index 00000000..42cd174e --- /dev/null +++ b/internal/watch/cluster_generated.go @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" +) + +// kubeRootCAConfigMap is the ConfigMap kube-controller-manager's root CA publisher writes into every +// namespace. Its content is the cluster's CA bundle: it means nothing outside that one cluster, it +// changes whenever the cluster is rebuilt, and the publisher overwrites any other value at once, so +// a copy in Git can neither be edited nor restored. +const kubeRootCAConfigMap = "kube-root-ca.crt" + +// clusterGenerated reports whether an object is one the cluster itself writes and Git never holds, +// whatever a WatchRule selects. Unlike resource_policy.go, which excludes whole types, this +// excludes single objects of a type that is mirrored. +// +// Both gatherings ask it: the snapshot fold (desiredFromObject) leaves the object out of the +// desired set, and the live route drops its events. An object already in Git is therefore treated +// like any document the cluster does not hold: prune mode "Always" sweeps it on the next resync, +// and "OnEvent" leaves it for a human to delete. +func clusterGenerated(gvr schema.GroupVersionResource, u *unstructured.Unstructured) bool { + if u == nil { + return false + } + return gvr.Group == "" && gvr.Resource == "configmaps" && u.GetName() == kubeRootCAConfigMap +} diff --git a/internal/watch/cluster_generated_test.go b/internal/watch/cluster_generated_test.go new file mode 100644 index 00000000..a87f6118 --- /dev/null +++ b/internal/watch/cluster_generated_test.go @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "context" + "maps" + "slices" + "testing" + + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + k8stypes "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/watch" + + "github.com/ConfigButler/gitops-reverser/internal/reconcile" + "github.com/ConfigButler/gitops-reverser/internal/telemetry" + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +func rootCAConfigMapObject(rv string) *unstructured.Unstructured { + obj := configMapObject(rv) + obj.SetName(kubeRootCAConfigMap) + return obj +} + +func TestClusterGenerated_OnlyTheRootCAConfigMap(t *testing.T) { + secrets := schema.GroupVersionResource{Version: "v1", Resource: "secrets"} + named := func(name string) *unstructured.Unstructured { + obj := configMapObject("1") + obj.SetName(name) + return obj + } + + assert.True(t, clusterGenerated(configmapsGVR, named(kubeRootCAConfigMap))) + assert.False(t, clusterGenerated(configmapsGVR, named("demo")), "any other ConfigMap is mirrored") + assert.False(t, clusterGenerated(secrets, named(kubeRootCAConfigMap)), + "the name alone is not enough: only the ConfigMap is the publisher's") + assert.False(t, clusterGenerated(configmapsGVR, nil)) +} + +// The snapshot leaves it out, so a resync never writes it and, under prune mode "Always", sweeps a +// copy Git already holds: the same as for any object the cluster does not hold. +func TestSnapshotFromList_LeavesOutTheRootCAConfigMap(t *testing.T) { + demo, rootCA := configMapObject("6"), rootCAConfigMapObject("7") + demo.SetUID("demo-uid") + rootCA.SetUID("root-ca-uid") + list := &unstructured.UnstructuredList{Items: []unstructured.Unstructured{*demo, *rootCA}} + + snapshot := snapshotFromList(configmapsGVR, list) + + require.Len(t, snapshot.desired, 1) + assert.Equal(t, "demo", snapshot.desired[0].Resource.Name) + assert.Equal(t, []k8stypes.UID{"demo-uid"}, slices.Collect(maps.Keys(snapshot.baselines)), + "no baseline for an object Git never holds") + assert.Equal(t, 1, snapshot.generated, "counted, so the snapshot can say it left it out") +} + +// Every live event for it is dropped, a DELETED one too: the publisher recreates it at once, and a +// delete mirrored on its own would only make Git's answer depend on which event came last. +func TestRouteLiveTargetWatchEvent_DropsTheRootCAConfigMap(t *testing.T) { + reader, err := telemetry.InitTestExporter() + require.NoError(t, err) + + gitDest := types.NewResourceReference("my-target", "my-ns") + enqueuer := &recordingEnqueuer{} + stream := reconcile.NewGitTargetEventStream(gitDest.Name, gitDest.Namespace, enqueuer, logr.Discard()) + router := &EventRouter{ + Log: logr.Discard(), + gitTargetStreams: map[string]*reconcile.GitTargetEventStream{gitDest.Key(): stream}, + } + manager := &Manager{EventRouter: router} + key := targetWatchKey{GVR: configmapsGVR, Namespace: "apps"} + + for i, eventType := range []watch.EventType{watch.Added, watch.Modified, watch.Deleted} { + rv, err := manager.routeLiveTargetWatchEvent(context.Background(), logr.Discard(), gitDest, + testStream(key), watch.Event{Type: eventType, Object: rootCAConfigMapObject(string(rune('1' + i)))}) + require.NoError(t, err) + assert.Equal(t, string(rune('1'+i)), rv, "the cursor still moves past it") + } + + assert.Empty(t, enqueuer.events, "nothing reaches the writer") + generated, ok := telemetry.CollectInt64Sum(reader, watchEventsMetric, ingestMatch(watchOutcomeGenerated)) + require.True(t, ok) + assert.Equal(t, int64(3), generated, "each drop is in the ingest census") +} diff --git a/internal/watch/desired_state_change_filter.go b/internal/watch/desired_state_change_filter.go index 68f3b190..4073058a 100644 --- a/internal/watch/desired_state_change_filter.go +++ b/internal/watch/desired_state_change_filter.go @@ -125,11 +125,16 @@ type replaySnapshot struct { // owners is the applier that owns each object some applier owns, read from the raw object // before sanitize strips it (git.AppliedByOf). owners map[types.ResourceIdentifier]git.AppliedBy + // generated counts the cluster-generated objects left out (clusterGenerated), for the log. + generated int } // add folds one observed object into the snapshot. An object desiredFromObject excludes (a // Terminating one) gets no baseline either: Git does not hold it. func (r *replaySnapshot) add(gvr schema.GroupVersionResource, u *unstructured.Unstructured) { + if clusterGenerated(gvr, u) { + r.generated++ + } desired, ok := desiredFromObject(gvr, u) if !ok { return diff --git a/internal/watch/scope_resolve.go b/internal/watch/scope_resolve.go index 09949a67..0635721e 100644 --- a/internal/watch/scope_resolve.go +++ b/internal/watch/scope_resolve.go @@ -94,6 +94,9 @@ func desiredFromObject( if u.GetDeletionTimestamp() != nil { return manifestanalyzer.DesiredResource{}, false } + if clusterGenerated(gvr, u) { + return manifestanalyzer.DesiredResource{}, false + } id := types.NewResourceIdentifier(gvr.Group, gvr.Version, gvr.Resource, u.GetNamespace(), u.GetName()) return manifestanalyzer.DesiredResource{Resource: id, Object: sanitize.Sanitize(u)}, true } diff --git a/internal/watch/target_watch.go b/internal/watch/target_watch.go index dc6364bc..c7368452 100644 --- a/internal/watch/target_watch.go +++ b/internal/watch/target_watch.go @@ -1075,6 +1075,11 @@ func (m *Manager) enqueueReplayResync( // its content is the baseline for those events. A refused or never-completed snapshot never // gets here, and the stream keeps the baselines of what the worker did accept. stream.changes.adopt(snapshot.baselines) + if snapshot.generated > 0 { + log.Info("target snapshot left out cluster-generated objects on purpose; Git never holds them", + "gitDest", gitDest.String(), "gvr", stream.key.GVR.String(), "namespace", stream.key.Namespace, + "name", kubeRootCAConfigMap, "count", snapshot.generated) + } log.V(1).Info("target replay resync enqueued", "gitDest", gitDest.String(), "gvr", stream.key.GVR.String(), "resourceVersion", resourceVersion, "count", len(snapshot.desired)) @@ -1191,6 +1196,13 @@ func (m *Manager) routeLiveTargetWatchEvent( "gvr", stream.key.GVR.String(), "type", string(ev.Type)) return rv, nil } + if clusterGenerated(stream.key.GVR, u) { + recordWatchEvent(ctx, gitDest, stream.key.GVR, watchOutcomeGenerated) + log.V(1).Info("target watch skipped a cluster-generated object on purpose", + "gitDest", gitDest.String(), "gvr", stream.key.GVR.String(), + "namespace", u.GetNamespace(), "name", u.GetName(), "type", string(ev.Type)) + return rv, nil + } op := operationForLiveTargetWatchEvent(ev.Type, u) event := targetWatchGitEvent(stream.key.GVR, u, op) // Stamp the producing collection and stream incarnation before the event leaves the stream. diff --git a/internal/watch/watch_event_metrics.go b/internal/watch/watch_event_metrics.go index 69be3655..14ec0edb 100644 --- a/internal/watch/watch_event_metrics.go +++ b/internal/watch/watch_event_metrics.go @@ -30,6 +30,7 @@ const ( watchOutcomeUnchanged = "unchanged" watchOutcomeNotObject = "not_object" watchOutcomeBookmark = "bookmark" + watchOutcomeGenerated = "cluster_generated" watchOutcomeRouteFailed = "route_failed" watchOutcomeShutdown = "shutdown" watchOutcomeStreamError = "stream_error" diff --git a/test/e2e/commit_request_e2e_test.go b/test/e2e/commit_request_e2e_test.go index 1f6a7bbf..8b8635db 100644 --- a/test/e2e/commit_request_e2e_test.go +++ b/test/e2e/commit_request_e2e_test.go @@ -603,10 +603,9 @@ func setupWindowedDeploymentPipeline(name, basePath, commitWindow string) window By(fmt.Sprintf("creating GitTarget with spec.commit.window=%s", commitWindow)) createValidatedGitTargetWithCommitWindow(p.gitTarget, p.ns, p.gitProvider, basePath, commitWindow) - // Watch Deployments, not ConfigMaps: a fresh namespace contains NO Deployments, whereas every - // namespace is pre-populated with a kube-root-ca.crt ConfigMap that a configmaps WatchRule - // would match — its initial reconcile would establish main before the spec's own edit, hiding - // the pure "branch not even created until something is finalized" behaviour these suites test. + // Watch Deployments: a fresh namespace contains NO Deployments, so the initial reconcile + // commits nothing and cannot establish main before the spec's own edit, which would hide the + // pure "branch not even created until something is finalized" behaviour these suites test. applyDeploymentWatchRule(p.ns, p.watchRule, p.gitTarget) verifyResourceStatus("watchrule", p.watchRule, p.ns, "True", "Succeeded", "") diff --git a/test/e2e/inplace_edit_e2e_test.go b/test/e2e/inplace_edit_e2e_test.go index 6edf5dbb..50c28aa9 100644 --- a/test/e2e/inplace_edit_e2e_test.go +++ b/test/e2e/inplace_edit_e2e_test.go @@ -272,16 +272,10 @@ var _ = Describe( // holds every line the human wrote, in order, and that anything added to it is a `resources:` // entry. // -// It replaces a byte-equality assertion, which is no longer the right question. This namespace -// holds a ConfigMap nobody in the test created — the cluster's own `kube-root-ca.crt` — and the -// WatchRule selects every ConfigMap, so the operator has a watched resource with no document in -// Git. Placement gives it a file beside this folder's one kustomization and registers it, which -// is the documented kustomize-root behaviour (and what the new-file-placement spec asserts -// directly). Before sibling inference was deleted, that resource was appended to the existing -// bundle instead, and the bundle was already listed — so the build file happened to stay -// byte-identical. Byte equality was therefore pinning a side effect of inference, not the -// property this spec is about: a hand-authored build file must not be reordered, reformatted, or -// have anything taken out of it. +// It replaces a byte-equality assertion, which pinned a side effect rather than the property this +// spec is about: a hand-authored build file must not be reordered, reformatted, or have anything +// taken out of it. A watched object with no document in Git is given a file beside the folder's +// kustomization and registered there, which is the documented kustomize-root behaviour. func expectKustomizationOnlyGainedResourceEntries(g Gomega, original, committed string) { GinkgoHelper() From a77b82c5dcd3a56d38ce3bc023a34ebf15130f63 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 12:27:12 +0000 Subject: [PATCH 02/33] feat(applier): converge the applier on the worker's own pushes at spec.convergeInterval A push of GitOps Reverser's own asks no apply: the flush already put its edits on the cluster. But the applier's own state stayed behind the branch until somebody else pushed: a Kustomization's status.lastAppliedRevision and its GitRepository's artifact, and whatever reads them (Flux commit-status notifications, a step that publishes what was applied). FluxClaim and ArgoCDClaim gain spec.convergeInterval. Once an applier has stayed behind the tip that long, a converge (CatchUp) is pending for it. The clock starts on the first pass that finds it behind and again on every send, so a burst of saves costs one apply and an applier at the tip costs nothing. The interval is read from the claim on every pass that finds the applier behind, so a change applies from the next wake. FluxClaim defaults to 1m; ArgoCDClaim to 0s, since a sync runs hooks. An interval converge yields to an edit: its fetch writes nothing to the cluster, but its apply would put a tip that lacks an edit mirrored after the send, flipping that object until a Repair. Its decision (ApplyRequest.Current, asked just before the resume) stops holding once a live edit is mirrored after the send, and it waits a fresh interval. Every other reason applies regardless, as S1 requires. The e2e FluxClaims pin 0s, so the specs that count applies keep counting only the ones an edit or a push asks for; a new Manager Flux Applier spec sets 10s and waits for lastAppliedRevision and the artifact to reach the tip of a mirrored API edit. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/applierclaim_types.go | 14 ++ api/v1alpha3/argocdclaim_types.go | 28 +++ api/v1alpha3/fluxclaim_types.go | 30 +++ api/v1alpha3/zz_generated.deepcopy.go | 14 +- .../bases/configbutler.ai_argocdclaims.yaml | 18 ++ .../crd/bases/configbutler.ai_fluxclaims.yaml | 18 ++ docs/UPGRADING.md | 8 + docs/bi-directional-flux.md | 3 +- docs/configuration.md | 2 + docs/design/apply-trigger.md | 18 +- hack/crdfields/fields.yaml | 12 ++ internal/git/applier_claim.go | 125 ++++++++++- internal/git/apply_converge_test.go | 194 ++++++++++++++++++ internal/git/branch_worker.go | 5 + test/e2e/applier_claim.go | 13 ++ test/e2e/flux_applier_e2e_test.go | 34 +++ 16 files changed, 528 insertions(+), 8 deletions(-) create mode 100644 internal/git/apply_converge_test.go diff --git a/api/v1alpha3/applierclaim_types.go b/api/v1alpha3/applierclaim_types.go index 5b678f17..04768bdb 100644 --- a/api/v1alpha3/applierclaim_types.go +++ b/api/v1alpha3/applierclaim_types.go @@ -3,6 +3,8 @@ package v1alpha3 import ( + "time" + meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" @@ -190,6 +192,10 @@ type ApplierClaim interface { // false when the applier has none it shares this way or the applier claim has not read it. Appliers // sharing one have one operation at a time. SharedSource() (source types.NamespacedName, known bool) + // ConvergeInterval is spec.convergeInterval, or the kind's default when it is omitted: how long + // the applier may stay behind the branch after a push of GitOps Reverser's own. Zero never + // converges for that reason. + ConvergeInterval() time.Duration // Operation is status.operation, and SetOperation replaces it and status.repairPending. Operation() *ApplyOperation RepairPending() bool @@ -214,6 +220,14 @@ func NewApplierClaim(kind string) ApplierClaim { } } +// durationOr is d, or fallback when it is omitted. +func durationOr(d *metav1.Duration, fallback time.Duration) time.Duration { + if d == nil { + return fallback + } + return d.Duration +} + // defaultedNamespace is namespace, or fallback when it is empty. func defaultedNamespace(namespace, fallback string) string { if namespace == "" { diff --git a/api/v1alpha3/argocdclaim_types.go b/api/v1alpha3/argocdclaim_types.go index f0ad8ea1..4f7bee26 100644 --- a/api/v1alpha3/argocdclaim_types.go +++ b/api/v1alpha3/argocdclaim_types.go @@ -4,6 +4,7 @@ package v1alpha3 import ( "strings" + "time" meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -54,6 +55,25 @@ type ArgoCDClaimSpec struct { // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="spec.applicationRef is immutable; delete and recreate the ArgoCDClaim to hold another Application" // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.applicationRef.name must not be empty" ApplicationRef meta.NamespacedObjectReference `json:"applicationRef"` + + // Why off by default, where a FluxClaim's is on: a sync runs the Application's hooks, which + // cost more than a no-op apply, and Argo CD's own refresh already compares the live state + // against the tip without a sync. Only the last sync's revision lags. + + // ConvergeInterval is how long the Application may stay behind the branch + // after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a + // push changes nothing on the cluster, which already holds the edit, but + // leaves the Application's last sync at the commit before it. Once the + // interval has passed, the tip is synced, hooks included: a burst of saves + // costs one sync, and an Application at the tip costs none. "0s" never + // syncs for this reason. Otherwise at least "10s" and at most "24h". + // Defaults to "0s". + // +optional + // +kubebuilder:validation:Type=string + // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" + // +kubebuilder:default="0s" + // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.convergeInterval must be 0s, or a Go duration from 10s to 24h" + ConvergeInterval *metav1.Duration `json:"convergeInterval,omitempty"` } // ArgoCDClaimStatus is what the applier claim knows about its Application, and @@ -195,6 +215,9 @@ func init() { SchemeBuilder.Register(&ArgoCDClaim{}, &ArgoCDClaimList{}) } +// DefaultArgoCDConvergeInterval is an ArgoCDClaim's spec.convergeInterval when it is omitted: off. +const DefaultArgoCDConvergeInterval = 0 + var _ ApplierClaim = (*ArgoCDClaim)(nil) // ClaimKind is ArgoCDClaim. @@ -235,6 +258,11 @@ func (t *ArgoCDClaim) SharedSource() (types.NamespacedName, bool) { return types.NamespacedName{}, false } +// ConvergeInterval is spec.convergeInterval, or DefaultArgoCDConvergeInterval when it is omitted. +func (t *ArgoCDClaim) ConvergeInterval() time.Duration { + return durationOr(t.Spec.ConvergeInterval, DefaultArgoCDConvergeInterval) +} + // Operation is status.operation. func (t *ArgoCDClaim) Operation() *ApplyOperation { return t.Status.Operation } diff --git a/api/v1alpha3/fluxclaim_types.go b/api/v1alpha3/fluxclaim_types.go index b4cb8e1c..10f847dc 100644 --- a/api/v1alpha3/fluxclaim_types.go +++ b/api/v1alpha3/fluxclaim_types.go @@ -3,6 +3,8 @@ package v1alpha3 import ( + "time" + meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" @@ -45,6 +47,26 @@ type FluxClaimSpec struct { // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="spec.kustomizationRef is immutable; delete and recreate the FluxClaim to hold another Kustomization" // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.kustomizationRef.name must not be empty" KustomizationRef meta.NamespacedObjectReference `json:"kustomizationRef"` + + // Why a timer: a push of GitOps Reverser's own needs no apply for the cluster (the flush + // already put the edit there), so none is asked for at once; but what reads the applier's own + // state (its applied revision, its source artifact) would stay at the last foreign commit until + // someone else pushed. One apply per interval keeps that level and batches a burst of saves. + + // ConvergeInterval is how long the Kustomization may stay behind the + // branch after a push of GitOps Reverser's own, as a Go duration ("1m", + // "90s"). Such a push changes nothing on the cluster, which already holds + // the edit, but leaves status.lastAppliedRevision and the GitRepository's + // artifact at the commit before it. Once the interval has passed, the tip is + // applied: a burst of saves costs one apply, and a Kustomization at the tip + // costs none. "0s" never applies for this reason. Otherwise at least "10s" + // and at most "24h". Defaults to "1m". + // +optional + // +kubebuilder:validation:Type=string + // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" + // +kubebuilder:default="1m" + // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.convergeInterval must be 0s, or a Go duration from 10s to 24h" + ConvergeInterval *metav1.Duration `json:"convergeInterval,omitempty"` } // FluxClaimStatus is what the applier claim knows about its Kustomization, and @@ -198,6 +220,9 @@ func init() { SchemeBuilder.Register(&FluxClaim{}, &FluxClaimList{}) } +// DefaultFluxConvergeInterval is a FluxClaim's spec.convergeInterval when it is omitted. +const DefaultFluxConvergeInterval = time.Minute + var _ ApplierClaim = (*FluxClaim)(nil) // ClaimKind is FluxClaim. @@ -236,6 +261,11 @@ func (t *FluxClaim) SharedSource() (types.NamespacedName, bool) { return types.NamespacedName{Namespace: source.Namespace, Name: source.Name}, true } +// ConvergeInterval is spec.convergeInterval, or DefaultFluxConvergeInterval when it is omitted. +func (t *FluxClaim) ConvergeInterval() time.Duration { + return durationOr(t.Spec.ConvergeInterval, DefaultFluxConvergeInterval) +} + // Operation is status.operation. func (t *FluxClaim) Operation() *ApplyOperation { return t.Status.Operation } diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 62900075..65ac7381 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -88,7 +88,7 @@ func (in *ArgoCDClaim) DeepCopyInto(out *ArgoCDClaim) { *out = *in out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - out.Spec = in.Spec + in.Spec.DeepCopyInto(&out.Spec) in.Status.DeepCopyInto(&out.Status) } @@ -162,6 +162,11 @@ func (in *ArgoCDClaimSpec) DeepCopyInto(out *ArgoCDClaimSpec) { *out = *in out.KubeProviderRef = in.KubeProviderRef out.ApplicationRef = in.ApplicationRef + if in.ConvergeInterval != nil { + in, out := &in.ConvergeInterval, &out.ConvergeInterval + *out = new(v1.Duration) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ArgoCDClaimSpec. @@ -629,7 +634,7 @@ func (in *FluxClaim) DeepCopyInto(out *FluxClaim) { *out = *in out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - out.Spec = in.Spec + in.Spec.DeepCopyInto(&out.Spec) in.Status.DeepCopyInto(&out.Status) } @@ -709,6 +714,11 @@ func (in *FluxClaimSpec) DeepCopyInto(out *FluxClaimSpec) { *out = *in out.KubeProviderRef = in.KubeProviderRef out.KustomizationRef = in.KustomizationRef + if in.ConvergeInterval != nil { + in, out := &in.ConvergeInterval, &out.ConvergeInterval + *out = new(v1.Duration) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new FluxClaimSpec. diff --git a/config/crd/bases/configbutler.ai_argocdclaims.yaml b/config/crd/bases/configbutler.ai_argocdclaims.yaml index 0a5162e8..f505a258 100644 --- a/config/crd/bases/configbutler.ai_argocdclaims.yaml +++ b/config/crd/bases/configbutler.ai_argocdclaims.yaml @@ -124,6 +124,24 @@ spec: rule: self == oldSelf - message: spec.applicationRef.name must not be empty rule: self.name != '' + convergeInterval: + default: 0s + description: |- + ConvergeInterval is how long the Application may stay behind the branch + after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a + push changes nothing on the cluster, which already holds the edit, but + leaves the Application's last sync at the commit before it. Once the + interval has passed, the tip is synced, hooks included: a burst of saves + costs one sync, and an Application at the tip costs none. "0s" never + syncs for this reason. Otherwise at least "10s" and at most "24h". + Defaults to "0s". + pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ + type: string + x-kubernetes-validations: + - message: spec.convergeInterval must be 0s, or a Go duration from + 10s to 24h + rule: duration(self) == duration('0s') || (duration(self) >= duration('10s') + && duration(self) <= duration('24h')) kubeProviderRef: description: |- KubeProviderRef names the KubeProvider of the cluster that runs the diff --git a/config/crd/bases/configbutler.ai_fluxclaims.yaml b/config/crd/bases/configbutler.ai_fluxclaims.yaml index d05eb531..78abcfa2 100644 --- a/config/crd/bases/configbutler.ai_fluxclaims.yaml +++ b/config/crd/bases/configbutler.ai_fluxclaims.yaml @@ -90,6 +90,24 @@ spec: spec: description: spec defines the desired state of FluxClaim. properties: + convergeInterval: + default: 1m + description: |- + ConvergeInterval is how long the Kustomization may stay behind the + branch after a push of GitOps Reverser's own, as a Go duration ("1m", + "90s"). Such a push changes nothing on the cluster, which already holds + the edit, but leaves status.lastAppliedRevision and the GitRepository's + artifact at the commit before it. Once the interval has passed, the tip is + applied: a burst of saves costs one apply, and a Kustomization at the tip + costs none. "0s" never applies for this reason. Otherwise at least "10s" + and at most "24h". Defaults to "1m". + pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ + type: string + x-kubernetes-validations: + - message: spec.convergeInterval must be 0s, or a Go duration from + 10s to 24h + rule: duration(self) == duration('0s') || (duration(self) >= duration('10s') + && duration(self) <= duration('24h')) kubeProviderRef: description: |- KubeProviderRef names the KubeProvider of the cluster that runs the diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index c90d3c01..858955dd 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,6 +7,14 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. +## A `FluxClaim` applies the tip a minute after GitOps Reverser's own pushes + +A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a +`Kustomization`'s `status.lastAppliedRevision` and its `GitRepository`'s artifact stayed at the last +commit somebody else pushed. `spec.convergeInterval` bounds that lag: once the applier has stayed +behind the tip that long, the tip is applied, one apply for a whole burst of saves. A `FluxClaim` +defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` defaults to `0s`. + ## `kube-root-ca.crt` is not mirrored The `kube-root-ca.crt` ConfigMap the cluster writes into every namespace is never mirrored, whatever diff --git a/docs/bi-directional-flux.md b/docs/bi-directional-flux.md index e4dc209f..f0cb93dc 100644 --- a/docs/bi-directional-flux.md +++ b/docs/bi-directional-flux.md @@ -582,11 +582,12 @@ applied the tip is asked nothing. | Every operation checks the target again (incarnation, branch, admission, consent) and the source binding before it resumes, records itself first, and never asks for something weaker; a stop mid-apply suspends the `Kustomization` | [`dispatcher_test.go`](../internal/applier/dispatcher_test.go) | | A failed attempt is decided again; each `Kustomization` keeps its own backoff; a shared one is asked once; appliers sharing a source apply one at a time; nothing is sent while no flush can be made; an undo of an edit in the apply window is mirrored, and the applier's own write is not | [`apply_attempt_test.go`](../internal/git/apply_attempt_test.go), [`apply_groups_test.go`](../internal/git/apply_groups_test.go), [`applier_claim_test.go`](../internal/git/applier_claim_test.go), [`apply_window_test.go`](../internal/git/apply_window_test.go) | | A newly named `Kustomization` converges once; the startup converge asks nothing of one at the tip; an API create is adopted even at the tip; nothing is asked before the initial snapshots are published, or for an empty tip | [`apply_catchup_test.go`](../internal/git/apply_catchup_test.go), [`applier_claim_test.go`](../internal/git/applier_claim_test.go) | +| A burst of the worker's own pushes converges once after `spec.convergeInterval`; `0s` never does; a changed interval applies from the next wake; an interval converge yields to an edit before its resume and waits a fresh interval, where a `ForeignCommit` does not | [`apply_converge_test.go`](../internal/git/apply_converge_test.go) | | An object another applier owns is neither mirrored nor swept | [`apply_ownership_test.go`](../internal/git/apply_ownership_test.go) | | An applier claim nothing names is `Ready` and holds nothing; named from a `GitTarget` on another branch than its source, it is `SourceMismatch` and adopts nothing | [Manager Flux Applier](../test/e2e/flux_applier_e2e_test.go) | | A `ForeignCommit` for a `SourceMismatch` applier never stamps the other branch's source and records `ApplyRequestFailed`; once the source is fixed the applier claim adopts the `Kustomization` and Flux reaches the tip; dropping the reference hands it back, and naming the applier claim again adopts it anew | Manager Flux Applier | | A Gitea push webhook to the operator's receiver gets a Git-side push applied; with no receiver and no API edit, the refresh finds one | Manager Flux Applier | -| A failed apply is suspended again and retried; a `flux resume` is suspended again with a `ClaimedResourceReconciled` Event; deleting the applier claim hands the `Kustomization` back | Manager Flux Applier | +| A failed apply is suspended again and retried; a `flux resume` is suspended again with a `ClaimedResourceReconciled` Event; a mirrored API edit's tip is applied as a `CatchUp` at `spec.convergeInterval`, so `lastAppliedRevision` and the artifact reach it; deleting the applier claim hands the `Kustomization` back | Manager Flux Applier | | A manager restarted during an apply suspends the `Kustomization` as it stops; the new one completes that same operation and leaves it suspended | [Flux Restart Recovery](../test/e2e/flux_restart_recovery_e2e_test.go) | | Two API edits seconds apart end at the later value with no extra commit | [Bi-directional (Flux)](../test/e2e/flux_bi_directional_e2e_test.go) | | A Git-side push with an API edit on top lands both, and Flux applies the replayed commit | Bi-directional (Flux) | diff --git a/docs/configuration.md b/docs/configuration.md index b0ed9a6f..20e05ad9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -159,6 +159,7 @@ The references are immutable: to hold another `Kustomization`, delete the applie |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Kustomization`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `kustomizationRef` | **required** | The `Kustomization` that applies the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | +| `convergeInterval` | `1m` | How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by an interval. `0s` never applies for this reason; otherwise `10s` to `24h` | ### `ArgoCDClaim` (namespaced): the Argo CD `Application` GitOps Reverser asks to sync @@ -168,6 +169,7 @@ The references are immutable: to hold another `Application`, delete the applier |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Application`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | +| `convergeInterval` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | ### `CommitRequest` (namespaced): save now diff --git a/docs/design/apply-trigger.md b/docs/design/apply-trigger.md index 6d6635d5..6118ea04 100644 --- a/docs/design/apply-trigger.md +++ b/docs/design/apply-trigger.md @@ -57,7 +57,7 @@ overwrites it is withheld and repaired. Four reasons make one due: | `ForeignCommit` | The branch gained a commit GitOps Reverser did not push. The flush replays the pending writes onto it and pushes them first, so the applied commit carries the API side's answer | Every applier on the branch | | `Refusal` | An edit to a document the folder holds was refused at the write boundary, and the target opted in with `onRefusal: Reapply` and is not suspended. At most one request a minute per target; a refusal inside that minute is carried by a trailing one | The refusing target's applier | | `Repair` | The apply window withheld a late apply over a newer API edit, so the cluster runs an older value than Git | The applier that wrote it | -| `CatchUp` | Ownership, not safety ([below](#catchup-adopt-and-converge)). An **adopt**: the API created an object in the folder, or a resync added one to Git, and an apply stamps it as the applier's, so a later Git-side delete prunes it. A **converge**: once when the worker starts, and once when a `GitTarget` first names the trigger, unless the applier already applied the tip | The folder's applier for an adopt; for a converge, every applier on the branch at start, and the named one on first naming. Flux and Argo CD alike | +| `CatchUp` | Ownership, not safety ([below](#catchup-adopt-and-converge)). An **adopt**: the API created an object in the folder, or a resync added one to Git, and an apply stamps it as the applier's, so a later Git-side delete prunes it. A **converge**: once when the worker starts, once when a `GitTarget` first names the trigger, and once the applier has stayed behind the tip for the claim's `spec.convergeInterval`, unless the applier already applied the tip | The folder's applier for an adopt; for a converge, every applier on the branch at start, and the named one on first naming. Flux and Argo CD alike | No applier is asked for anything before every initial snapshot of the `GitTarget`s it covers is published, and a push of GitOps Reverser's own leaves nothing pending: the cluster already runs the @@ -259,6 +259,22 @@ deletes its file the applier leaves it on the cluster, and a later resync writes uid. The worker keeps, per applier, the commit the last succeeded operation applied, from the operation's report and, after a restart, from the trigger's `status.operation`. A converge of that commit asks nothing. +- **Converge on an interval.** A push of the worker's own asks nothing at once, but it leaves the + applier's own state behind the branch: a `Kustomization`'s `status.lastAppliedRevision` and its + `GitRepository`'s artifact, an `Application`'s last sync, and whatever reads them (Flux's + commit-status notifications, a step that publishes what was applied). So once an applier has + stayed behind the tip for its claim's `spec.convergeInterval`, a converge is pending for it. The + clock starts on the first pass that finds it behind and again on every send, so a burst of saves + inside one interval costs one apply, and an applier at the tip costs nothing. A `FluxClaim` + defaults to `1m`; an `ArgoCDClaim` to `0s`, never, since a sync runs the `Application`'s hooks + and Argo CD's refresh already compares the live state against the tip. +- **An interval converge yields to an edit.** Its fetch writes nothing to the cluster; its apply + puts the fetched tip there, and an edit mirrored after the send is not in that tip, so the apply + would revert it and a `Repair` would put it back. Such a converge's decision stops holding once a + live edit is mirrored after its send (`ApplyStale`, asked just before the resume), and it waits a + fresh interval rather than going again after the next flush. While a burst of edits lasts, it + lands in the first quiet moment. Every other reason applies regardless, as the flush guarantees + ([above](#flush-then-apply)). - **No apply before every initial snapshot is published.** An applier is asked for nothing, for any reason, until every collection of every `GitTarget` it covers has a settled snapshot (applied, refused or failed; tracked per collection on the watch side) and the worker has published every diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 57d8422b..4f697b33 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -258,6 +258,13 @@ kinds: doc: >- The `Kustomization` that applies the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) + - path: convergeInterval + doc: >- + How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. + Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the + `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is + applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by + an interval. `0s` never applies for this reason; otherwise `10s` to `24h` - kind: ArgoCDClaim title: "`ArgoCDClaim` (namespaced): the Argo CD `Application` GitOps Reverser asks to sync" @@ -271,6 +278,11 @@ kinds: doc: >- The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) + - path: convergeInterval + doc: >- + As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a + push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and + Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` - kind: CommitRequest title: "`CommitRequest` (namespaced): save now" diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 7f4aac63..1b1267db 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -14,6 +14,7 @@ import ( "k8s.io/apimachinery/pkg/api/equality" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" k8stypes "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" "github.com/ConfigButler/gitops-reverser/api/v1alpha3" itypes "github.com/ConfigButler/gitops-reverser/internal/types" @@ -242,6 +243,14 @@ type applierState struct { // named is the GitTarget incarnations that named the applier so far: one naming it for the first // time leaves a converge pending. named map[k8stypes.UID]bool + // convergeInterval is how long the applier may stay behind the tip before a converge is pending + // for it, zero for never, as last read from its applier claim, and behindSince is when this worker first found it behind, zero while + // it is not, or since the last attempt was sent (scheduleConverges). + convergeInterval time.Duration + behindSince time.Time + // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, + // so its attempt yields to an edit (convergeHolds). + lag bool // appliedRevision is the revision the last operation that succeeded applied, as the applier // reported it, recovered on restart from the applier claim's status.operation. A converge of the commit // it names asks nothing. @@ -279,6 +288,12 @@ type applierState struct { followedSuspend string } +// onlyLag reports whether scheduleConverges' converge is all that is pending for the applier, given its +// unsent refusals: an attempt carrying nothing else yields to an edit (convergeHolds). +func (s *applierState) onlyLag(refusals map[refusalKey]unsentRefusal) bool { + return s.lag && !s.foreign && !s.adopt && !s.repair && len(refusals) == 0 +} + // repairPending reports whether a Repair is pending: not yet carried, or carried by the attempt out. func (s *applierState) repairPending() bool { return s.repair || (s.inflight != nil && s.inflight.repair) @@ -293,6 +308,8 @@ type inflightAttempt struct { converge bool repair bool refusals map[refusalKey]unsentRefusal + // lag is whether it carries only scheduleConverges' converge. + lag bool // phase is the last report of its operation, empty before the first. phase ApplyOutcome // previous is the operation recorded before this attempt's, with the attempt it belonged to, @@ -363,10 +380,12 @@ func (l *branchWorkerEventLoop) serviceAppliers() { now := time.Now() l.settleApplyResults(now) l.followRecordedOperations(now) + converging := l.scheduleConverges(now) work := l.takeReasons() // A listing that failed is retried on the applier timer (listRetryAt), which every arming below // keeps. closing := earliest(earliest(l.reportApplyWindows(now), l.snapshotHoldDeadline()), l.listRetryAt) + closing = earliest(closing, converging) if !work { l.endFlush() @@ -400,6 +419,65 @@ func (l *branchWorkerEventLoop) serviceAppliers() { l.armApplierTimer(now, wait) } +// scheduleConverges leaves a converge pending for each applier that has stayed behind the tip for its +// convergeInterval, and returns when the next one still waiting comes due, zero when none waits. +// +// A push of the worker's own leaves nothing pending (applyInbox.observe): the flush put its edits on +// the cluster already. But the applier's own state, its applied revision and its source, stays at the +// commit before, and other things read it. The interval bounds that lag. The clock starts on the first +// pass that finds the applier behind, and starts again when an attempt is sent, so a burst of pushes +// inside one interval costs one apply, and an applier at the tip costs nothing. The converge then +// waits like any other: for the barrier, the backoff and the flush (dueAppliers), and asks nothing +// of an applier that applied the tip meanwhile. +func (l *branchWorkerEventLoop) scheduleConverges(now time.Time) time.Time { + tip := l.w.inbox.lastTip() + var next time.Time + for _, state := range l.appliers { + if tip == "" || revisionNamesCommit(state.appliedRevision, tip) { + state.behindSince = time.Time{} + continue + } + state.convergeInterval = l.w.convergeIntervalOf(state) + if state.convergeInterval <= 0 { + state.behindSince = time.Time{} + continue + } + if state.behindSince.IsZero() { + state.behindSince = now + } + if state.converge || l.attemptOut(state) { + continue + } + due := state.behindSince.Add(state.convergeInterval) + if now.Before(due) { + next = earliest(next, due) + continue + } + state.converge, state.lag = true, true + l.w.Log.V(1).Info("The applier stayed behind the tip for its converge interval; a converge is pending", + "branch", l.w.Branch, "applier", state.ref.String(), "applied", state.appliedRevision, "tip", tip, + "interval", state.convergeInterval.String()) + } + return next +} + +// convergeIntervalOf reads the applier's spec.convergeInterval from its applier claim, on every pass +// that finds it behind, so a change applies from the next wake rather than from the next listing, +// which a push of the worker's own does not make. The client is cached, so this costs no round trip. +// An applier claim that cannot be read keeps the interval last read. +func (w *BranchWorker) convergeIntervalOf(state *applierState) time.Duration { + ref := state.ref.ApplierClaim + applierClaim := v1alpha3.NewApplierClaim(ref.Kind) + if applierClaim == nil { + return 0 + } + key := client.ObjectKey{Namespace: ref.Namespace, Name: ref.Name} + if err := w.Client.Get(w.ctx, key, applierClaim); err != nil { + return state.convergeInterval + } + return applierClaim.ConvergeInterval() +} + // takeReasons moves the reasons the inbox holds onto the appliers listed now, and reports whether // any applier has work. A listing that fails leaves them in the inbox, and sets listRetryAt, when // the applier timer looks again; only a listing that succeeds clears it (followRecordedOperations @@ -422,6 +500,7 @@ func (l *branchWorkerEventLoop) takeReasons() bool { for _, state := range l.appliers { state.foreign = state.foreign || foreign state.converge = state.converge || converge + state.lag = state.lag && !converge } return true } @@ -445,7 +524,7 @@ func (l *branchWorkerEventLoop) dueAppliers(now time.Time) ([]*applierState, tim } refusals := l.unsentRefusalsFor(state) if l.alreadyConverged(state, refusals, tip) { - state.converge = false + state.converge, state.lag = false, false l.w.Log.V(1).Info("The applier already applied the tip; the converge asks nothing", "branch", l.w.Branch, "applier", state.ref.String(), "commit", tip) } @@ -651,7 +730,7 @@ func (l *branchWorkerEventLoop) listApplier(ref ApplierRef, named []branchTarget state.targets = named for _, target := range named { if !state.named[target.uid] { - state.named[target.uid], state.converge = true, true + state.named[target.uid], state.converge, state.lag = true, true, false } } } @@ -730,8 +809,10 @@ func (l *branchWorkerEventLoop) sendAttempt(state *applierState, tip string, ref id: l.nextApplyID, foreign: state.foreign, adopt: state.adopt, converge: state.converge, repair: state.repair, refusals: refusals, abandon: abandon, previous: state.operation, previousID: state.operationID, + lag: state.onlyLag(refusals), } - state.foreign, state.adopt, state.converge, state.repair = false, false, false, false + state.foreign, state.adopt, state.converge, state.repair, state.lag = false, false, false, false, false + state.behindSince = time.Time{} state.inflight = attempt state.lastTurn = attempt.id l.reportApplyWindow(state) @@ -767,7 +848,7 @@ func (l *branchWorkerEventLoop) sendAttempt(state *applierState, tip string, ref Record: record, Targets: targets, PriorFailures: state.failures, - Current: func() bool { return w.decisionHolds(tip) }, + Current: w.currentFor(tip, attempt.lag), Report: w.reportApplyResult, Abandoned: abandoned.Done(), }) @@ -808,6 +889,14 @@ func (l *branchWorkerEventLoop) settleApplyResults(now time.Time) { } continue case ApplyStale: + if attempt.lag { + // It yielded to an edit, or to a newer tip: it only catches the applier's state + // up, so it waits a fresh interval rather than going again after the next flush. + state.behindSince = now + l.w.Log.V(1).Info("A converge yielded to an edit before its resume; waiting a fresh interval", + "branch", l.w.Branch, "applier", state.ref.String()) + continue + } // The decision went stale before the write: nothing failed, and the next wake // decides again, at the tip as it is then. l.restoreAttempt(state, attempt) @@ -1005,6 +1094,13 @@ func (l *branchWorkerEventLoop) publishUnresolved() { func (l *branchWorkerEventLoop) restoreAttempt(state *applierState, attempt *inflightAttempt) { state.foreign = state.foreign || attempt.foreign state.adopt = state.adopt || attempt.adopt + // The converge given back is only scheduleConverges' when that is all either side holds. + switch { + case attempt.converge && !attempt.lag: + state.lag = false + case attempt.lag && !state.converge: + state.lag = true + } state.converge = state.converge || attempt.converge state.repair = state.repair || attempt.repair for key, refusal := range attempt.refusals { @@ -1048,6 +1144,27 @@ func (w *BranchWorker) decisionHolds(tip string) bool { return w.inbox.tipIs(tip) } +// currentFor is ApplyRequest.Current for an attempt decided at tip: decisionHolds, and for an attempt +// that only catches the applier's state up (lag), convergeHolds as well. +func (w *BranchWorker) currentFor(tip string, lag bool) func() bool { + if !lag { + return func() bool { return w.decisionHolds(tip) } + } + edits := w.liveEdits.Load() + return func() bool { return w.convergeHolds(tip, edits) } +} + +// convergeHolds is the decision of a converge that only catches the applier's state up +// (scheduleConverges): the branch is still at tip, and no live edit was mirrored since the send, the +// count of which was edits then. The fetch before the resume writes nothing to the cluster; the apply +// after it would put tip there, and an edit made meanwhile is not in tip, so the apply would revert it +// and a Repair would put it back: a flip on the cluster, for an apply nothing needed. Such a converge +// yields instead, and waits a fresh interval. An apply with any other reason does not: S1 keeps +// continuous editing from starving it, and the apply window repairs what it overwrites. +func (w *BranchWorker) convergeHolds(tip string, edits uint64) bool { + return w.decisionHolds(tip) && w.liveEdits.Load() == edits +} + // reportApplyResult is ApplyRequest.Report: it records the report and wakes the loop to settle it. // It is called from the requester's goroutine. func (w *BranchWorker) reportApplyResult(res ApplyResult) { diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go new file mode 100644 index 00000000..a0084c01 --- /dev/null +++ b/internal/git/apply_converge_test.go @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" +) + +// spec.convergeInterval: a push of the worker's own asks nothing at once, but an applier left behind +// the tip for the interval is asked a CatchUp of it, so its own state (applied revision, source +// artifact) catches up with the branch. + +// convergedAt starts loop's applier, and has it apply the first tip, "aaa", at startup. +func convergedAt(t *testing.T, loop *branchWorkerEventLoop, requester *recordingRequester) { + t.Helper() + const commit = "aaa" + loop.w.inbox.observe(RemoteObservation{Commit: commit, By: ObservedByFetch}) + loop.serviceAppliers() + startup := requester.taken() + require.Len(t, startup, 1, "the startup converge") + startup[0].Report(ApplyResult{ID: startup[0].ID, Outcome: ApplySucceeded, AppliedRevision: commit}) + loop.serviceAppliers() + require.Empty(t, requester.taken()) +} + +// intervalPassed moves the clock of every applier's converge back by more than its interval. +func intervalPassed(t *testing.T, loop *branchWorkerEventLoop) { + t.Helper() + for _, state := range loop.appliers { + require.False(t, state.behindSince.IsZero(), "the clock runs only for an applier behind the tip") + state.behindSince = state.behindSince.Add(-state.convergeInterval - time.Second) + } +} + +func convergeEvery(interval time.Duration) func(*configv1alpha3.FluxClaim) { + return func(c *configv1alpha3.FluxClaim) { c.Spec.ConvergeInterval = &metav1.Duration{Duration: interval} } +} + +// TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval: each push asks nothing at once and the +// timer is armed for the interval; once it has passed, one CatchUp of the newest tip goes, and an +// applier at the tip is not clocked any more. +func TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + state := loop.appliers[applierRef("alpha")] + + for _, commit := range []string{"bbb", "ccc", "ddd"} { + loop.w.inbox.observe(RemoteObservation{Commit: commit, By: ObservedByPush}) + loop.serviceAppliers() + require.Empty(t, requester.taken(), "the worker's own push asks nothing at once") + } + require.NotNil(t, loop.applierTimer, "the timer wakes the loop when the interval has passed") + assert.Equal(t, configv1alpha3.DefaultFluxConvergeInterval, state.convergeInterval, "a FluxClaim's default") + + intervalPassed(t, loop) + loop.serviceAppliers() + requests := requester.taken() + require.Len(t, requests, 1, "one apply for the whole burst") + assert.Equal(t, configv1alpha3.ApplyReasonCatchUp, requests[0].Reason) + assert.Equal(t, "ddd", requests[0].Commit) + assert.True(t, state.behindSince.IsZero(), "a send starts the clock again") + + requests[0].Report(ApplyResult{ID: requests[0].ID, Outcome: ApplySucceeded, AppliedRevision: "main@sha1:ddd"}) + loop.serviceAppliers() + assert.Empty(t, requester.taken()) + assert.True(t, state.behindSince.IsZero(), "an applier at the tip is not clocked") +} + +// TestConverge_APushDuringAnAttemptWaitsAFreshInterval: the attempt applied the tip it was sent for, +// and the push that landed meanwhile waits an interval from the send, not from before it. +func TestConverge_APushDuringAnAttemptWaitsAFreshInterval(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + loop.serviceAppliers() + intervalPassed(t, loop) + loop.serviceAppliers() + sent := requester.taken() + require.Len(t, sent, 1) + + loop.w.inbox.observe(RemoteObservation{Commit: "ccc", By: ObservedByPush}) + loop.serviceAppliers() + sent[0].Report(ApplyResult{ID: sent[0].ID, Outcome: ApplySucceeded, AppliedRevision: "bbb"}) + loop.serviceAppliers() + assert.Empty(t, requester.taken(), "behind again, but the interval starts at the send") + + intervalPassed(t, loop) + loop.serviceAppliers() + requests := requester.taken() + require.Len(t, requests, 1) + assert.Equal(t, "ccc", requests[0].Commit) +} + +// TestConverge_ZeroNeverConverges: "0s" keeps a push of the worker's own asking nothing, ever; an +// ArgoCDClaim defaults to it, because a sync runs the Application's hooks. +func TestConverge_ZeroNeverConverges(t *testing.T) { + for name, objects := range map[string][]client.Object{ + "flux 0s": {applierTestTarget("alpha", true), fluxClaim("alpha", convergeEvery(0))}, + "argo cd": {applierTestTarget("alpha", true, argoApplier)}, + } { + t.Run(name, func(t *testing.T) { + loop, requester := manualLoop(t, objects...) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + loop.serviceAppliers() + + for _, state := range loop.appliers { + assert.Zero(t, state.convergeInterval) + assert.True(t, state.behindSince.IsZero(), "no clock runs") + } + assert.Empty(t, requester.taken()) + }) + } +} + +// TestConverge_YieldsToAnEditBeforeItsResume: an interval converge's decision stops holding once a +// live edit is mirrored after its send, since the tip it fetched lacks that edit and applying it would +// flip the object on the cluster. Stale, it waits a fresh interval, then goes for the newer tip. +func TestConverge_YieldsToAnEditBeforeItsResume(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + loop.serviceAppliers() + intervalPassed(t, loop) + loop.serviceAppliers() + sent := requester.taken() + require.Len(t, sent, 1) + require.True(t, sent[0].Current(), "nothing happened since the send") + + loop.handleLiveEvents(&WriteRequest{Events: []Event{windowEvent("alpha", "2", "kubectl-edit")}}) + assert.False(t, sent[0].Current(), "an edit arrived during the fetch") + + report(sent[0], ApplyStale) + loop.openWindow = nil + loop.serviceAppliers() + assert.Empty(t, requester.taken(), "not again at once: a fresh interval") + state := loop.appliers[applierRef("alpha")] + assert.False(t, state.converge) + + loop.w.inbox.observe(RemoteObservation{Commit: "ccc", By: ObservedByPush}) + intervalPassed(t, loop) + loop.serviceAppliers() + again := requester.taken() + require.Len(t, again, 1) + assert.Equal(t, "ccc", again[0].Commit) +} + +// TestConverge_AnyOtherReasonDoesNotYield: a ForeignCommit is not a catch-up, so an edit does not make +// it stale; the apply window repairs what it overwrites (S1). +func TestConverge_AnyOtherReasonDoesNotYield(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByFetch}) + loop.serviceAppliers() + sent := requester.taken() + require.Len(t, sent, 1) + require.Equal(t, configv1alpha3.ApplyReasonForeignCommit, sent[0].Reason) + + loop.handleLiveEvents(&WriteRequest{Events: []Event{windowEvent("alpha", "2", "kubectl-edit")}}) + assert.True(t, sent[0].Current()) +} + +// TestConverge_AChangedIntervalAppliesFromTheNextWake: the interval is read from the applier claim on +// every pass that finds the applier behind, so a change needs no listing, which a push of the +// worker's own does not make. +func TestConverge_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true), fluxClaim("alpha", convergeEvery(0))) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + loop.serviceAppliers() + state := loop.appliers[applierRef("alpha")] + require.True(t, state.behindSince.IsZero(), "off") + + claim := &configv1alpha3.FluxClaim{} + require.NoError(t, loop.w.Client.Get(t.Context(), client.ObjectKey{Namespace: "team-a", Name: "alpha"}, claim)) + convergeEvery(30 * time.Second)(claim) + require.NoError(t, loop.w.Client.Update(t.Context(), claim)) + + loop.serviceAppliers() + assert.Equal(t, 30*time.Second, state.convergeInterval) + intervalPassed(t, loop) + loop.serviceAppliers() + requests := requester.taken() + require.Len(t, requests, 1) + assert.Equal(t, "bbb", requests[0].Commit) +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index e8f0ea65..96e81cf1 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -163,6 +163,10 @@ type BranchWorker struct { applyEpoch atomic.Uint64 appliersAhead atomic.Bool hasAppliers atomic.Bool + // liveEdits counts the live events the loop has mirrored, for the requester's goroutine to read: a + // converge that only catches the applier's state up yields to an edit that arrives before its + // resume (convergeHolds). + liveEdits atomic.Uint64 // initialSnapshots reports whether every collection a GitTarget watches has a settled // snapshot, and resyncRecheck asks the watch side for fresh snapshots of a GitTarget. Set by the // WorkerManager before Start; nil in the CLI and in tests (apply_epoch.go). @@ -1275,6 +1279,7 @@ func (l *branchWorkerEventLoop) handleLiveEvents(request *WriteRequest) { if !l.mirrors(event) { continue } + l.w.liveEdits.Add(1) l.appendLiveEvent(event) } } diff --git a/test/e2e/applier_claim.go b/test/e2e/applier_claim.go index 42c22ebb..0e925c2b 100644 --- a/test/e2e/applier_claim.go +++ b/test/e2e/applier_claim.go @@ -20,6 +20,10 @@ const ( // createFluxClaim creates the FluxClaim name in namespace, holding the Kustomization // flux-system/ in the operator's own cluster (the install's "local" KubeProvider). +// +// Its spec.convergeInterval is "0s", not the "1m" default: the specs count the applies an edit or a +// push asks for, and a converge a minute after the worker's own push would land inside their holds. +// The spec that tests the converge sets one (setFluxClaimConvergeInterval). func createFluxClaim(namespace, name, kustomization string) { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 @@ -32,11 +36,20 @@ spec: kustomizationRef: namespace: flux-system name: %s + convergeInterval: 0s `, name, namespace, localKubeProviderRef(), kustomization) _, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") Expect(err).NotTo(HaveOccurred(), "failed to create FluxClaim %s/%s", namespace, name) } +// setFluxClaimConvergeInterval sets the FluxClaim's spec.convergeInterval. +func setFluxClaimConvergeInterval(namespace, name, interval string) { + GinkgoHelper() + _, err := kubectlRunInNamespace(namespace, "patch", fluxClaimResource, name, "--type=merge", "-p", + fmt.Sprintf(`{"spec":{"convergeInterval":%q}}`, interval)) + Expect(err).NotTo(HaveOccurred(), "failed to set the convergeInterval of FluxClaim %s/%s", namespace, name) +} + // createArgoCDClaim creates the ArgoCDClaim name in namespace, holding the Application // / in the operator's own cluster. func createArgoCDClaim(namespace, name, appNamespace, application string) { diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index 7534ca92..bdd4a161 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -295,6 +295,32 @@ data: verifyResourceCondition(fluxClaimResource, fluxName, testNs, "InControl", "True", "Suspended", "") }) + // A push of GitOps Reverser's own asks no apply at once, but the Kustomization's own state follows + // the branch at the applier claim's spec.convergeInterval: its lastAppliedRevision and its + // GitRepository's artifact reach the tip a mirrored API edit made, with no push by anyone else. + It("brings the applied revision to its own pushes' tip at the convergeInterval", func() { + setFluxClaimConvergeInterval(testNs, fluxName, "10s") + DeferCleanup(setFluxClaimConvergeInterval, testNs, fluxName, "0s") + + By("mirroring an API edit: the branch moves by GitOps Reverser's own push") + applyConfigMap(testNs, "applier-converge-edit") + Eventually(func(g Gomega) { + g.Expect(runGitIn(repo.CheckoutDir, "fetch", "origin", "main")).To(Succeed()) + g.Expect(gitShowFiles(repo.CheckoutDir)).To(ContainSubstring("applier-converge-edit")) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + + By("a CatchUp applies that tip within the interval, and the Kustomization is held again") + Eventually(func(g Gomega) { + head := remoteHeadSHA(g, repo.CheckoutDir) + g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(head)) + g.Expect(fluxArtifactRevision(g, fluxName)).To(ContainSubstring(head)) + done := readClaimOperation(g, testNs, fluxClaimResource, fluxName) + g.Expect(done.Reason).To(Equal("CatchUp")) + g.Expect(done.Phase).To(Equal("Succeeded")) + g.Expect(kustomizationSuspended(g, fluxName)).To(BeTrue()) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + }) + // Deleting the applier claim hands the Kustomization back: its finalizer waits until nothing is // running and releases spec.suspend, so Flux applies on its own again. It("hands the Kustomization back to Flux when the applier claim is deleted", func() { @@ -418,6 +444,14 @@ func fluxLastAppliedRevision(g Gomega, name string) string { return strings.TrimSpace(out) } +// fluxArtifactRevision is the revision of the GitRepository name's artifact. +func fluxArtifactRevision(g Gomega, name string) string { + out, err := kubectlRunInNamespace("flux-system", "get", "gitrepository", name, + "-o", "jsonpath={.status.artifact.revision}") + g.Expect(err).NotTo(HaveOccurred()) + return strings.TrimSpace(out) +} + // gitTargetEventMessages is every message of the Events with reason recorded on a GitTarget. func gitTargetEventMessages(g Gomega, testNs, gitTarget, reason string) string { out, err := kubectlRunInNamespace(testNs, "get", "events", "--field-selector", From 91020f48272bccc958cf5c2b995dc32851065cff Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 13:38:25 +0000 Subject: [PATCH 03/33] feat(commitrequest): spec.catchUp asks the applier to apply a save at once; convergeInterval is catchUpAfter The claim field is renamed catchUpAfter: it is how long the applier may stay behind the branch before it catches up, and the apply shows as reason CatchUp. "Reconcile" would suggest Flux's periodic drift correction, which the claim turns off. A CommitRequest gains spec.catchUp. "AsDefinedOnClaim" (the default) leaves the apply to the claim's catchUpAfter; "ImmediatelyAfterPush" leaves a converge pending for the GitTarget's applier as soon as the save's commit is pushed. It was asked for, so it does not yield to an edit. A save that pushes no commit, or whose GitTarget names no applier claim, asks nothing. The field is immutable like the rest of the spec. The Manager Flux Applier converge spec now times an UPDATE: a create is adopted at once, so it never proved the interval. The failed-apply spec takes its ConfigMap out of Git before its namespace goes, which left every later apply of the folder failing. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/applierclaim_types.go | 4 +- api/v1alpha3/argocdclaim_types.go | 16 ++-- api/v1alpha3/commitrequest_types.go | 26 ++++++- api/v1alpha3/fluxclaim_types.go | 16 ++-- api/v1alpha3/zz_generated.deepcopy.go | 8 +- .../bases/configbutler.ai_argocdclaims.yaml | 8 +- .../bases/configbutler.ai_commitrequests.yaml | 19 ++++- .../crd/bases/configbutler.ai_fluxclaims.yaml | 8 +- docs/UPGRADING.md | 4 +- docs/bi-directional-flux.md | 4 +- docs/configuration.md | 5 +- docs/design/apply-trigger.md | 8 +- hack/crdfields/fields.yaml | 8 +- .../controller/commitrequest_controller.go | 1 + internal/git/applier_claim.go | 56 ++++++++++---- internal/git/apply_converge_test.go | 76 +++++++++++++------ internal/git/branch_worker.go | 3 + internal/git/commit_request_attach.go | 5 ++ internal/git/commit_request_attach_loop.go | 4 + test/e2e/applier_claim.go | 14 ++-- test/e2e/flux_applier_e2e_test.go | 61 ++++++++++++--- 21 files changed, 258 insertions(+), 96 deletions(-) diff --git a/api/v1alpha3/applierclaim_types.go b/api/v1alpha3/applierclaim_types.go index 04768bdb..cf997c09 100644 --- a/api/v1alpha3/applierclaim_types.go +++ b/api/v1alpha3/applierclaim_types.go @@ -192,10 +192,10 @@ type ApplierClaim interface { // false when the applier has none it shares this way or the applier claim has not read it. Appliers // sharing one have one operation at a time. SharedSource() (source types.NamespacedName, known bool) - // ConvergeInterval is spec.convergeInterval, or the kind's default when it is omitted: how long + // CatchUpAfter is spec.catchUpAfter, or the kind's default when it is omitted: how long // the applier may stay behind the branch after a push of GitOps Reverser's own. Zero never // converges for that reason. - ConvergeInterval() time.Duration + CatchUpAfter() time.Duration // Operation is status.operation, and SetOperation replaces it and status.repairPending. Operation() *ApplyOperation RepairPending() bool diff --git a/api/v1alpha3/argocdclaim_types.go b/api/v1alpha3/argocdclaim_types.go index 4f7bee26..f237d90d 100644 --- a/api/v1alpha3/argocdclaim_types.go +++ b/api/v1alpha3/argocdclaim_types.go @@ -60,7 +60,7 @@ type ArgoCDClaimSpec struct { // cost more than a no-op apply, and Argo CD's own refresh already compares the live state // against the tip without a sync. Only the last sync's revision lags. - // ConvergeInterval is how long the Application may stay behind the branch + // CatchUpAfter is how long the Application may stay behind the branch // after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a // push changes nothing on the cluster, which already holds the edit, but // leaves the Application's last sync at the commit before it. Once the @@ -72,8 +72,8 @@ type ArgoCDClaimSpec struct { // +kubebuilder:validation:Type=string // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" // +kubebuilder:default="0s" - // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.convergeInterval must be 0s, or a Go duration from 10s to 24h" - ConvergeInterval *metav1.Duration `json:"convergeInterval,omitempty"` + // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.catchUpAfter must be 0s, or a Go duration from 10s to 24h" + CatchUpAfter *metav1.Duration `json:"catchUpAfter,omitempty"` } // ArgoCDClaimStatus is what the applier claim knows about its Application, and @@ -215,8 +215,8 @@ func init() { SchemeBuilder.Register(&ArgoCDClaim{}, &ArgoCDClaimList{}) } -// DefaultArgoCDConvergeInterval is an ArgoCDClaim's spec.convergeInterval when it is omitted: off. -const DefaultArgoCDConvergeInterval = 0 +// DefaultArgoCDCatchUpAfter is an ArgoCDClaim's spec.catchUpAfter when it is omitted: off. +const DefaultArgoCDCatchUpAfter = 0 var _ ApplierClaim = (*ArgoCDClaim)(nil) @@ -258,9 +258,9 @@ func (t *ArgoCDClaim) SharedSource() (types.NamespacedName, bool) { return types.NamespacedName{}, false } -// ConvergeInterval is spec.convergeInterval, or DefaultArgoCDConvergeInterval when it is omitted. -func (t *ArgoCDClaim) ConvergeInterval() time.Duration { - return durationOr(t.Spec.ConvergeInterval, DefaultArgoCDConvergeInterval) +// CatchUpAfter is spec.catchUpAfter, or DefaultArgoCDCatchUpAfter when it is omitted. +func (t *ArgoCDClaim) CatchUpAfter() time.Duration { + return durationOr(t.Spec.CatchUpAfter, DefaultArgoCDCatchUpAfter) } // Operation is status.operation. diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index a67819c9..b7b0edcb 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -22,7 +22,7 @@ import ( // including one that only touches metadata — was rejected as a spec change. // CommitRequestWindow compares its durations as durations, which compares what // the fields mean instead of how they were spelled. -// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit)",message="CommitRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="CommitRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.whenNothingToCommit) || self.whenNothingToCommit != 'CommitEmpty' || has(self.message)",message="whenNothingToCommit: CommitEmpty requires spec.message, because a message is all an empty commit records" type CommitRequestSpec struct { // GitTargetRef names the GitTarget, in this namespace, whose commit window @@ -71,8 +71,32 @@ type CommitRequestSpec struct { // +kubebuilder:validation:Enum=Resolve;CommitEmpty // +kubebuilder:default=Resolve WhenNothingToCommit NothingToCommitAction `json:"whenNothingToCommit,omitempty"` + + // CatchUp decides when the GitTarget's applier is asked to apply the commit + // this request pushes. "AsDefinedOnClaim" leaves it to the applier claim's + // spec.catchUpAfter. "ImmediatelyAfterPush" asks as soon as the commit is + // pushed, which costs one apply per request. The cluster already holds the + // saved objects; the apply brings the applier's own state, such as a + // Kustomization's status.lastAppliedRevision, to the commit. A request that + // pushes no commit, or whose GitTarget names no applier claim, asks for + // nothing. Defaults to "AsDefinedOnClaim". + // +optional + // +kubebuilder:validation:Enum=AsDefinedOnClaim;ImmediatelyAfterPush + // +kubebuilder:default=AsDefinedOnClaim + CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } +// CatchUpPolicy is when a CommitRequest's commit is applied by the GitTarget's +// applier. +type CatchUpPolicy string + +const ( + // CatchUpAsDefinedOnClaim leaves the apply to the applier claim's spec.catchUpAfter. + CatchUpAsDefinedOnClaim CatchUpPolicy = "AsDefinedOnClaim" + // CatchUpImmediatelyAfterPush asks the applier to apply the tip as soon as the commit is pushed. + CatchUpImmediatelyAfterPush CatchUpPolicy = "ImmediatelyAfterPush" +) + // NothingToCommitAction is what a CommitRequest does when it ends with nothing // to commit. type NothingToCommitAction string diff --git a/api/v1alpha3/fluxclaim_types.go b/api/v1alpha3/fluxclaim_types.go index 10f847dc..9e0cb43a 100644 --- a/api/v1alpha3/fluxclaim_types.go +++ b/api/v1alpha3/fluxclaim_types.go @@ -53,7 +53,7 @@ type FluxClaimSpec struct { // state (its applied revision, its source artifact) would stay at the last foreign commit until // someone else pushed. One apply per interval keeps that level and batches a burst of saves. - // ConvergeInterval is how long the Kustomization may stay behind the + // CatchUpAfter is how long the Kustomization may stay behind the // branch after a push of GitOps Reverser's own, as a Go duration ("1m", // "90s"). Such a push changes nothing on the cluster, which already holds // the edit, but leaves status.lastAppliedRevision and the GitRepository's @@ -65,8 +65,8 @@ type FluxClaimSpec struct { // +kubebuilder:validation:Type=string // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" // +kubebuilder:default="1m" - // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.convergeInterval must be 0s, or a Go duration from 10s to 24h" - ConvergeInterval *metav1.Duration `json:"convergeInterval,omitempty"` + // +kubebuilder:validation:XValidation:rule="duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h'))",message="spec.catchUpAfter must be 0s, or a Go duration from 10s to 24h" + CatchUpAfter *metav1.Duration `json:"catchUpAfter,omitempty"` } // FluxClaimStatus is what the applier claim knows about its Kustomization, and @@ -220,8 +220,8 @@ func init() { SchemeBuilder.Register(&FluxClaim{}, &FluxClaimList{}) } -// DefaultFluxConvergeInterval is a FluxClaim's spec.convergeInterval when it is omitted. -const DefaultFluxConvergeInterval = time.Minute +// DefaultFluxCatchUpAfter is a FluxClaim's spec.catchUpAfter when it is omitted. +const DefaultFluxCatchUpAfter = time.Minute var _ ApplierClaim = (*FluxClaim)(nil) @@ -261,9 +261,9 @@ func (t *FluxClaim) SharedSource() (types.NamespacedName, bool) { return types.NamespacedName{Namespace: source.Namespace, Name: source.Name}, true } -// ConvergeInterval is spec.convergeInterval, or DefaultFluxConvergeInterval when it is omitted. -func (t *FluxClaim) ConvergeInterval() time.Duration { - return durationOr(t.Spec.ConvergeInterval, DefaultFluxConvergeInterval) +// CatchUpAfter is spec.catchUpAfter, or DefaultFluxCatchUpAfter when it is omitted. +func (t *FluxClaim) CatchUpAfter() time.Duration { + return durationOr(t.Spec.CatchUpAfter, DefaultFluxCatchUpAfter) } // Operation is status.operation. diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 65ac7381..032972e0 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -162,8 +162,8 @@ func (in *ArgoCDClaimSpec) DeepCopyInto(out *ArgoCDClaimSpec) { *out = *in out.KubeProviderRef = in.KubeProviderRef out.ApplicationRef = in.ApplicationRef - if in.ConvergeInterval != nil { - in, out := &in.ConvergeInterval, &out.ConvergeInterval + if in.CatchUpAfter != nil { + in, out := &in.CatchUpAfter, &out.CatchUpAfter *out = new(v1.Duration) **out = **in } @@ -714,8 +714,8 @@ func (in *FluxClaimSpec) DeepCopyInto(out *FluxClaimSpec) { *out = *in out.KubeProviderRef = in.KubeProviderRef out.KustomizationRef = in.KustomizationRef - if in.ConvergeInterval != nil { - in, out := &in.ConvergeInterval, &out.ConvergeInterval + if in.CatchUpAfter != nil { + in, out := &in.CatchUpAfter, &out.CatchUpAfter *out = new(v1.Duration) **out = **in } diff --git a/config/crd/bases/configbutler.ai_argocdclaims.yaml b/config/crd/bases/configbutler.ai_argocdclaims.yaml index f505a258..e6769fb0 100644 --- a/config/crd/bases/configbutler.ai_argocdclaims.yaml +++ b/config/crd/bases/configbutler.ai_argocdclaims.yaml @@ -124,10 +124,10 @@ spec: rule: self == oldSelf - message: spec.applicationRef.name must not be empty rule: self.name != '' - convergeInterval: + catchUpAfter: default: 0s description: |- - ConvergeInterval is how long the Application may stay behind the branch + CatchUpAfter is how long the Application may stay behind the branch after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a push changes nothing on the cluster, which already holds the edit, but leaves the Application's last sync at the commit before it. Once the @@ -138,8 +138,8 @@ spec: pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ type: string x-kubernetes-validations: - - message: spec.convergeInterval must be 0s, or a Go duration from - 10s to 24h + - message: spec.catchUpAfter must be 0s, or a Go duration from 10s + to 24h rule: duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h')) kubeProviderRef: diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 90b56c58..9eb05ffd 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -77,6 +77,21 @@ spec: spec defines the desired state of CommitRequest. It is immutable after creation. properties: + catchUp: + default: AsDefinedOnClaim + description: |- + CatchUp decides when the GitTarget's applier is asked to apply the commit + this request pushes. "AsDefinedOnClaim" leaves it to the applier claim's + spec.catchUpAfter. "ImmediatelyAfterPush" asks as soon as the commit is + pushed, which costs one apply per request. The cluster already holds the + saved objects; the apply brings the applier's own state, such as a + Kustomization's status.lastAppliedRevision, to the commit. A request that + pushes no commit, or whose GitTarget names no applier claim, asks for + nothing. Defaults to "AsDefinedOnClaim". + enum: + - AsDefinedOnClaim + - ImmediatelyAfterPush + type: string gitTargetRef: description: |- GitTargetRef names the GitTarget, in this namespace, whose commit window @@ -203,7 +218,9 @@ spec: == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) - || self.whenNothingToCommit == oldSelf.whenNothingToCommit) + || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) + == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == + oldSelf.catchUp) - message: 'whenNothingToCommit: CommitEmpty requires spec.message, because a message is all an empty commit records' rule: '!has(self.whenNothingToCommit) || self.whenNothingToCommit != diff --git a/config/crd/bases/configbutler.ai_fluxclaims.yaml b/config/crd/bases/configbutler.ai_fluxclaims.yaml index 78abcfa2..2393a155 100644 --- a/config/crd/bases/configbutler.ai_fluxclaims.yaml +++ b/config/crd/bases/configbutler.ai_fluxclaims.yaml @@ -90,10 +90,10 @@ spec: spec: description: spec defines the desired state of FluxClaim. properties: - convergeInterval: + catchUpAfter: default: 1m description: |- - ConvergeInterval is how long the Kustomization may stay behind the + CatchUpAfter is how long the Kustomization may stay behind the branch after a push of GitOps Reverser's own, as a Go duration ("1m", "90s"). Such a push changes nothing on the cluster, which already holds the edit, but leaves status.lastAppliedRevision and the GitRepository's @@ -104,8 +104,8 @@ spec: pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ type: string x-kubernetes-validations: - - message: spec.convergeInterval must be 0s, or a Go duration from - 10s to 24h + - message: spec.catchUpAfter must be 0s, or a Go duration from 10s + to 24h rule: duration(self) == duration('0s') || (duration(self) >= duration('10s') && duration(self) <= duration('24h')) kubeProviderRef: diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 858955dd..954db55d 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -11,9 +11,11 @@ We are pre-1.0, so breaking changes bump the **minor** version (release-please i A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a `Kustomization`'s `status.lastAppliedRevision` and its `GitRepository`'s artifact stayed at the last -commit somebody else pushed. `spec.convergeInterval` bounds that lag: once the applier has stayed +commit somebody else pushed. `spec.catchUpAfter` bounds that lag: once the applier has stayed behind the tip that long, the tip is applied, one apply for a whole burst of saves. A `FluxClaim` defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` defaults to `0s`. +A `CommitRequest` that sets `catchUp: ImmediatelyAfterPush` has its commit applied as soon as it is +pushed, whatever the claim's interval. ## `kube-root-ca.crt` is not mirrored diff --git a/docs/bi-directional-flux.md b/docs/bi-directional-flux.md index f0cb93dc..13efc35a 100644 --- a/docs/bi-directional-flux.md +++ b/docs/bi-directional-flux.md @@ -582,12 +582,12 @@ applied the tip is asked nothing. | Every operation checks the target again (incarnation, branch, admission, consent) and the source binding before it resumes, records itself first, and never asks for something weaker; a stop mid-apply suspends the `Kustomization` | [`dispatcher_test.go`](../internal/applier/dispatcher_test.go) | | A failed attempt is decided again; each `Kustomization` keeps its own backoff; a shared one is asked once; appliers sharing a source apply one at a time; nothing is sent while no flush can be made; an undo of an edit in the apply window is mirrored, and the applier's own write is not | [`apply_attempt_test.go`](../internal/git/apply_attempt_test.go), [`apply_groups_test.go`](../internal/git/apply_groups_test.go), [`applier_claim_test.go`](../internal/git/applier_claim_test.go), [`apply_window_test.go`](../internal/git/apply_window_test.go) | | A newly named `Kustomization` converges once; the startup converge asks nothing of one at the tip; an API create is adopted even at the tip; nothing is asked before the initial snapshots are published, or for an empty tip | [`apply_catchup_test.go`](../internal/git/apply_catchup_test.go), [`applier_claim_test.go`](../internal/git/applier_claim_test.go) | -| A burst of the worker's own pushes converges once after `spec.convergeInterval`; `0s` never does; a changed interval applies from the next wake; an interval converge yields to an edit before its resume and waits a fresh interval, where a `ForeignCommit` does not | [`apply_converge_test.go`](../internal/git/apply_converge_test.go) | +| A burst of the worker's own pushes converges once after `spec.catchUpAfter`; `0s` never does; a changed interval applies from the next wake; an interval converge yields to an edit before its resume and waits a fresh interval, where a `ForeignCommit` does not | [`apply_converge_test.go`](../internal/git/apply_converge_test.go) | | An object another applier owns is neither mirrored nor swept | [`apply_ownership_test.go`](../internal/git/apply_ownership_test.go) | | An applier claim nothing names is `Ready` and holds nothing; named from a `GitTarget` on another branch than its source, it is `SourceMismatch` and adopts nothing | [Manager Flux Applier](../test/e2e/flux_applier_e2e_test.go) | | A `ForeignCommit` for a `SourceMismatch` applier never stamps the other branch's source and records `ApplyRequestFailed`; once the source is fixed the applier claim adopts the `Kustomization` and Flux reaches the tip; dropping the reference hands it back, and naming the applier claim again adopts it anew | Manager Flux Applier | | A Gitea push webhook to the operator's receiver gets a Git-side push applied; with no receiver and no API edit, the refresh finds one | Manager Flux Applier | -| A failed apply is suspended again and retried; a `flux resume` is suspended again with a `ClaimedResourceReconciled` Event; a mirrored API edit's tip is applied as a `CatchUp` at `spec.convergeInterval`, so `lastAppliedRevision` and the artifact reach it; deleting the applier claim hands the `Kustomization` back | Manager Flux Applier | +| A failed apply is suspended again and retried; a `flux resume` is suspended again with a `ClaimedResourceReconciled` Event; a mirrored API edit's tip is applied as a `CatchUp` at `spec.catchUpAfter`, so `lastAppliedRevision` and the artifact reach it; deleting the applier claim hands the `Kustomization` back | Manager Flux Applier | | A manager restarted during an apply suspends the `Kustomization` as it stops; the new one completes that same operation and leaves it suspended | [Flux Restart Recovery](../test/e2e/flux_restart_recovery_e2e_test.go) | | Two API edits seconds apart end at the later value with no extra commit | [Bi-directional (Flux)](../test/e2e/flux_bi_directional_e2e_test.go) | | A Git-side push with an API edit on top lands both, and Flux applies the replayed commit | Bi-directional (Flux) | diff --git a/docs/configuration.md b/docs/configuration.md index 20e05ad9..f03fc789 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -159,7 +159,7 @@ The references are immutable: to hold another `Kustomization`, delete the applie |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Kustomization`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `kustomizationRef` | **required** | The `Kustomization` that applies the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | -| `convergeInterval` | `1m` | How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by an interval. `0s` never applies for this reason; otherwise `10s` to `24h` | +| `catchUpAfter` | `1m` | How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by an interval. `0s` never applies for this reason; otherwise `10s` to `24h` | ### `ArgoCDClaim` (namespaced): the Argo CD `Application` GitOps Reverser asks to sync @@ -169,7 +169,7 @@ The references are immutable: to hold another `Application`, delete the applier |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Application`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | -| `convergeInterval` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | +| `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | ### `CommitRequest` (namespaced): save now @@ -182,6 +182,7 @@ The references are immutable: to hold another `Application`, delete the applier | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | +| `catchUp` | `AsDefinedOnClaim` | `ImmediatelyAfterPush` asks the `GitTarget`'s applier to apply the commit as soon as it is pushed, one apply per request, instead of after the applier claim's `catchUpAfter` | diff --git a/docs/design/apply-trigger.md b/docs/design/apply-trigger.md index 6118ea04..d5a86a05 100644 --- a/docs/design/apply-trigger.md +++ b/docs/design/apply-trigger.md @@ -57,7 +57,7 @@ overwrites it is withheld and repaired. Four reasons make one due: | `ForeignCommit` | The branch gained a commit GitOps Reverser did not push. The flush replays the pending writes onto it and pushes them first, so the applied commit carries the API side's answer | Every applier on the branch | | `Refusal` | An edit to a document the folder holds was refused at the write boundary, and the target opted in with `onRefusal: Reapply` and is not suspended. At most one request a minute per target; a refusal inside that minute is carried by a trailing one | The refusing target's applier | | `Repair` | The apply window withheld a late apply over a newer API edit, so the cluster runs an older value than Git | The applier that wrote it | -| `CatchUp` | Ownership, not safety ([below](#catchup-adopt-and-converge)). An **adopt**: the API created an object in the folder, or a resync added one to Git, and an apply stamps it as the applier's, so a later Git-side delete prunes it. A **converge**: once when the worker starts, once when a `GitTarget` first names the trigger, and once the applier has stayed behind the tip for the claim's `spec.convergeInterval`, unless the applier already applied the tip | The folder's applier for an adopt; for a converge, every applier on the branch at start, and the named one on first naming. Flux and Argo CD alike | +| `CatchUp` | Ownership, not safety ([below](#catchup-adopt-and-converge)). An **adopt**: the API created an object in the folder, or a resync added one to Git, and an apply stamps it as the applier's, so a later Git-side delete prunes it. A **converge**: once when the worker starts, once when a `GitTarget` first names the trigger, and once the applier has stayed behind the tip for the claim's `spec.catchUpAfter`, unless the applier already applied the tip | The folder's applier for an adopt; for a converge, every applier on the branch at start, and the named one on first naming. Flux and Argo CD alike | No applier is asked for anything before every initial snapshot of the `GitTarget`s it covers is published, and a push of GitOps Reverser's own leaves nothing pending: the cluster already runs the @@ -263,7 +263,7 @@ deletes its file the applier leaves it on the cluster, and a later resync writes applier's own state behind the branch: a `Kustomization`'s `status.lastAppliedRevision` and its `GitRepository`'s artifact, an `Application`'s last sync, and whatever reads them (Flux's commit-status notifications, a step that publishes what was applied). So once an applier has - stayed behind the tip for its claim's `spec.convergeInterval`, a converge is pending for it. The + stayed behind the tip for its claim's `spec.catchUpAfter`, a converge is pending for it. The clock starts on the first pass that finds it behind and again on every send, so a burst of saves inside one interval costs one apply, and an applier at the tip costs nothing. A `FluxClaim` defaults to `1m`; an `ArgoCDClaim` to `0s`, never, since a sync runs the `Application`'s hooks @@ -275,6 +275,10 @@ deletes its file the applier leaves it on the cluster, and a later resync writes fresh interval rather than going again after the next flush. While a burst of edits lasts, it lands in the first quiet moment. Every other reason applies regardless, as the flush guarantees ([above](#flush-then-apply)). +- **A save can ask to catch up at once.** A `CommitRequest` with `catchUp: ImmediatelyAfterPush` + leaves a converge pending for its `GitTarget`'s applier as soon as its commit is pushed, without + waiting for `catchUpAfter`. It was asked for, so it does not yield to an edit. A save that pushed + no commit asks nothing. - **No apply before every initial snapshot is published.** An applier is asked for nothing, for any reason, until every collection of every `GitTarget` it covers has a settled snapshot (applied, refused or failed; tracked per collection on the watch side) and the worker has published every diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 4f697b33..0a53d099 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -258,7 +258,7 @@ kinds: doc: >- The `Kustomization` that applies the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) - - path: convergeInterval + - path: catchUpAfter doc: >- How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the @@ -278,7 +278,7 @@ kinds: doc: >- The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) - - path: convergeInterval + - path: catchUpAfter doc: >- As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and @@ -305,3 +305,7 @@ kinds: doc: Close the attached window this long after the attach - path: whenNothingToCommit doc: "`CommitEmpty` records the message in an empty commit when nothing changed" + - path: catchUp + doc: >- + `ImmediatelyAfterPush` asks the `GitTarget`'s applier to apply the commit as soon as it is pushed, + one apply per request, instead of after the applier claim's `catchUpAfter` diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index a99ef732..67ab23a4 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -255,6 +255,7 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques IdleTimeout: window.idleTimeout, MaxDuration: window.maxDuration, CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpImmediatelyAfterPush, } if time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout() { // Past the bound the controller only withdraws: an attach sent now would sit behind the diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 1b1267db..8fef4a81 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -243,11 +243,11 @@ type applierState struct { // named is the GitTarget incarnations that named the applier so far: one naming it for the first // time leaves a converge pending. named map[k8stypes.UID]bool - // convergeInterval is how long the applier may stay behind the tip before a converge is pending + // catchUpAfter is how long the applier may stay behind the tip before a converge is pending // for it, zero for never, as last read from its applier claim, and behindSince is when this worker first found it behind, zero while // it is not, or since the last attempt was sent (scheduleConverges). - convergeInterval time.Duration - behindSince time.Time + catchUpAfter time.Duration + behindSince time.Time // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, // so its attempt yields to an edit (convergeHolds). lag bool @@ -420,7 +420,7 @@ func (l *branchWorkerEventLoop) serviceAppliers() { } // scheduleConverges leaves a converge pending for each applier that has stayed behind the tip for its -// convergeInterval, and returns when the next one still waiting comes due, zero when none waits. +// catchUpAfter, and returns when the next one still waiting comes due, zero when none waits. // // A push of the worker's own leaves nothing pending (applyInbox.observe): the flush put its edits on // the cluster already. But the applier's own state, its applied revision and its source, stays at the @@ -437,8 +437,8 @@ func (l *branchWorkerEventLoop) scheduleConverges(now time.Time) time.Time { state.behindSince = time.Time{} continue } - state.convergeInterval = l.w.convergeIntervalOf(state) - if state.convergeInterval <= 0 { + state.catchUpAfter = l.w.catchUpAfterOf(state) + if state.catchUpAfter <= 0 { state.behindSince = time.Time{} continue } @@ -448,24 +448,24 @@ func (l *branchWorkerEventLoop) scheduleConverges(now time.Time) time.Time { if state.converge || l.attemptOut(state) { continue } - due := state.behindSince.Add(state.convergeInterval) + due := state.behindSince.Add(state.catchUpAfter) if now.Before(due) { next = earliest(next, due) continue } state.converge, state.lag = true, true - l.w.Log.V(1).Info("The applier stayed behind the tip for its converge interval; a converge is pending", + l.w.Log.V(1).Info("The applier stayed behind the tip for its catchUpAfter; a converge is pending", "branch", l.w.Branch, "applier", state.ref.String(), "applied", state.appliedRevision, "tip", tip, - "interval", state.convergeInterval.String()) + "interval", state.catchUpAfter.String()) } return next } -// convergeIntervalOf reads the applier's spec.convergeInterval from its applier claim, on every pass +// catchUpAfterOf reads the applier's spec.catchUpAfter from its applier claim, on every pass // that finds it behind, so a change applies from the next wake rather than from the next listing, // which a push of the worker's own does not make. The client is cached, so this costs no round trip. // An applier claim that cannot be read keeps the interval last read. -func (w *BranchWorker) convergeIntervalOf(state *applierState) time.Duration { +func (w *BranchWorker) catchUpAfterOf(state *applierState) time.Duration { ref := state.ref.ApplierClaim applierClaim := v1alpha3.NewApplierClaim(ref.Kind) if applierClaim == nil { @@ -473,9 +473,9 @@ func (w *BranchWorker) convergeIntervalOf(state *applierState) time.Duration { } key := client.ObjectKey{Namespace: ref.Namespace, Name: ref.Name} if err := w.Client.Get(w.ctx, key, applierClaim); err != nil { - return state.convergeInterval + return state.catchUpAfter } - return applierClaim.ConvergeInterval() + return applierClaim.CatchUpAfter() } // takeReasons moves the reasons the inbox holds onto the appliers listed now, and reports whether @@ -485,7 +485,7 @@ func (w *BranchWorker) convergeIntervalOf(state *applierState) time.Duration { func (l *branchWorkerEventLoop) takeReasons() bool { w := l.w _, foreign, converge := w.inbox.take() - if !foreign && !converge && len(l.unsentRefusals) == 0 && !l.appliersHaveWork() { + if !foreign && !converge && len(l.unsentRefusals) == 0 && len(l.savedCatchUps) == 0 && !l.appliersHaveWork() { return false } targets, err := w.appliersOnBranch(w.ctx) @@ -502,9 +502,37 @@ func (l *branchWorkerEventLoop) takeReasons() bool { state.converge = state.converge || converge state.lag = state.lag && !converge } + l.takeSavedCatchUps() return true } +// catchUpAfterSave leaves a converge pending for the applier of a CommitRequest's GitTarget, whose +// commit was just pushed and which asked for it (catchUp: ImmediatelyAfterPush). It is taken onto the +// applier when the appliers are next listed, at the end of this wake. +func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest) { + if l.savedCatchUps == nil { + l.savedCatchUps = map[itypes.ResourceReference]bool{} + } + l.savedCatchUps[itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace)] = true +} + +// takeSavedCatchUps leaves a converge pending for every applier a pushed CommitRequest asked to catch +// up. It was asked for, so it is no lag converge: it does not yield to an edit (convergeHolds). A +// GitTarget that names no applier asks nothing. +func (l *branchWorkerEventLoop) takeSavedCatchUps() { + for _, state := range l.appliers { + for _, target := range state.targets { + if !l.savedCatchUps[target.ref] { + continue + } + state.converge, state.lag = true, false + l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", + "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", target.ref.String()) + } + } + l.savedCatchUps = nil +} + // dueAppliers is the appliers an attempt is due for now, in turn order, and the earliest time one // still waiting comes due. The appliers are taken in turn order, so in a group, which has one attempt // out at a time, the attempt goes to the applier whose turn is longest ago. diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index a0084c01..9456a25a 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -14,7 +14,7 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// spec.convergeInterval: a push of the worker's own asks nothing at once, but an applier left behind +// spec.catchUpAfter: a push of the worker's own asks nothing at once, but an applier left behind // the tip for the interval is asked a CatchUp of it, so its own state (applied revision, source // artifact) catches up with the branch. @@ -36,18 +36,18 @@ func intervalPassed(t *testing.T, loop *branchWorkerEventLoop) { t.Helper() for _, state := range loop.appliers { require.False(t, state.behindSince.IsZero(), "the clock runs only for an applier behind the tip") - state.behindSince = state.behindSince.Add(-state.convergeInterval - time.Second) + state.behindSince = state.behindSince.Add(-state.catchUpAfter - time.Second) } } -func convergeEvery(interval time.Duration) func(*configv1alpha3.FluxClaim) { - return func(c *configv1alpha3.FluxClaim) { c.Spec.ConvergeInterval = &metav1.Duration{Duration: interval} } +func catchUpEvery(interval time.Duration) func(*configv1alpha3.FluxClaim) { + return func(c *configv1alpha3.FluxClaim) { c.Spec.CatchUpAfter = &metav1.Duration{Duration: interval} } } -// TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval: each push asks nothing at once and the +// TestCatchUpAfter_ABurstOfOwnPushesConvergesOnceAfterTheInterval: each push asks nothing at once and the // timer is armed for the interval; once it has passed, one CatchUp of the newest tip goes, and an // applier at the tip is not clocked any more. -func TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval(t *testing.T) { +func TestCatchUpAfter_ABurstOfOwnPushesConvergesOnceAfterTheInterval(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) state := loop.appliers[applierRef("alpha")] @@ -58,7 +58,7 @@ func TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval(t *testing.T) { require.Empty(t, requester.taken(), "the worker's own push asks nothing at once") } require.NotNil(t, loop.applierTimer, "the timer wakes the loop when the interval has passed") - assert.Equal(t, configv1alpha3.DefaultFluxConvergeInterval, state.convergeInterval, "a FluxClaim's default") + assert.Equal(t, configv1alpha3.DefaultFluxCatchUpAfter, state.catchUpAfter, "a FluxClaim's default") intervalPassed(t, loop) loop.serviceAppliers() @@ -74,9 +74,9 @@ func TestConverge_ABurstOfOwnPushesConvergesOnceAfterTheInterval(t *testing.T) { assert.True(t, state.behindSince.IsZero(), "an applier at the tip is not clocked") } -// TestConverge_APushDuringAnAttemptWaitsAFreshInterval: the attempt applied the tip it was sent for, +// TestCatchUpAfter_APushDuringAnAttemptWaitsAFreshInterval: the attempt applied the tip it was sent for, // and the push that landed meanwhile waits an interval from the send, not from before it. -func TestConverge_APushDuringAnAttemptWaitsAFreshInterval(t *testing.T) { +func TestCatchUpAfter_APushDuringAnAttemptWaitsAFreshInterval(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) @@ -99,11 +99,11 @@ func TestConverge_APushDuringAnAttemptWaitsAFreshInterval(t *testing.T) { assert.Equal(t, "ccc", requests[0].Commit) } -// TestConverge_ZeroNeverConverges: "0s" keeps a push of the worker's own asking nothing, ever; an +// TestCatchUpAfter_ZeroNeverConverges: "0s" keeps a push of the worker's own asking nothing, ever; an // ArgoCDClaim defaults to it, because a sync runs the Application's hooks. -func TestConverge_ZeroNeverConverges(t *testing.T) { +func TestCatchUpAfter_ZeroNeverConverges(t *testing.T) { for name, objects := range map[string][]client.Object{ - "flux 0s": {applierTestTarget("alpha", true), fluxClaim("alpha", convergeEvery(0))}, + "flux 0s": {applierTestTarget("alpha", true), fluxClaim("alpha", catchUpEvery(0))}, "argo cd": {applierTestTarget("alpha", true, argoApplier)}, } { t.Run(name, func(t *testing.T) { @@ -113,7 +113,7 @@ func TestConverge_ZeroNeverConverges(t *testing.T) { loop.serviceAppliers() for _, state := range loop.appliers { - assert.Zero(t, state.convergeInterval) + assert.Zero(t, state.catchUpAfter) assert.True(t, state.behindSince.IsZero(), "no clock runs") } assert.Empty(t, requester.taken()) @@ -121,10 +121,10 @@ func TestConverge_ZeroNeverConverges(t *testing.T) { } } -// TestConverge_YieldsToAnEditBeforeItsResume: an interval converge's decision stops holding once a +// TestCatchUpAfter_YieldsToAnEditBeforeItsResume: an interval converge's decision stops holding once a // live edit is mirrored after its send, since the tip it fetched lacks that edit and applying it would // flip the object on the cluster. Stale, it waits a fresh interval, then goes for the newer tip. -func TestConverge_YieldsToAnEditBeforeItsResume(t *testing.T) { +func TestCatchUpAfter_YieldsToAnEditBeforeItsResume(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) @@ -153,9 +153,9 @@ func TestConverge_YieldsToAnEditBeforeItsResume(t *testing.T) { assert.Equal(t, "ccc", again[0].Commit) } -// TestConverge_AnyOtherReasonDoesNotYield: a ForeignCommit is not a catch-up, so an edit does not make +// TestCatchUpAfter_AnyOtherReasonDoesNotYield: a ForeignCommit is not a catch-up, so an edit does not make // it stale; the apply window repairs what it overwrites (S1). -func TestConverge_AnyOtherReasonDoesNotYield(t *testing.T) { +func TestCatchUpAfter_AnyOtherReasonDoesNotYield(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByFetch}) @@ -168,11 +168,11 @@ func TestConverge_AnyOtherReasonDoesNotYield(t *testing.T) { assert.True(t, sent[0].Current()) } -// TestConverge_AChangedIntervalAppliesFromTheNextWake: the interval is read from the applier claim on +// TestCatchUpAfter_AChangedIntervalAppliesFromTheNextWake: the interval is read from the applier claim on // every pass that finds the applier behind, so a change needs no listing, which a push of the // worker's own does not make. -func TestConverge_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { - loop, requester := manualLoop(t, applierTestTarget("alpha", true), fluxClaim("alpha", convergeEvery(0))) +func TestCatchUpAfter_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true), fluxClaim("alpha", catchUpEvery(0))) convergedAt(t, loop, requester) loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) loop.serviceAppliers() @@ -181,14 +181,46 @@ func TestConverge_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { claim := &configv1alpha3.FluxClaim{} require.NoError(t, loop.w.Client.Get(t.Context(), client.ObjectKey{Namespace: "team-a", Name: "alpha"}, claim)) - convergeEvery(30 * time.Second)(claim) + catchUpEvery(30 * time.Second)(claim) require.NoError(t, loop.w.Client.Update(t.Context(), claim)) loop.serviceAppliers() - assert.Equal(t, 30*time.Second, state.convergeInterval) + assert.Equal(t, 30*time.Second, state.catchUpAfter) intervalPassed(t, loop) loop.serviceAppliers() requests := requester.taken() require.Len(t, requests, 1) assert.Equal(t, "bbb", requests[0].Commit) } + +// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a CommitRequest with catchUp: ImmediatelyAfterPush +// leaves a converge pending for its GitTarget's applier once its commit is pushed, without waiting +// for catchUpAfter; one that pushed no commit asks nothing. It does not yield to an edit: it was +// asked for. +func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + save := func(name string, commit string) { + id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingCommitRequest{ + id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, + } + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + } + + save("nothing-pushed", "") + loop.serviceAppliers() + require.Empty(t, requester.taken(), "no commit, nothing to catch up with") + + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + save("pushed", "bbb") + loop.serviceAppliers() + requests := requester.taken() + require.Len(t, requests, 1, "at once, not after catchUpAfter") + assert.Equal(t, configv1alpha3.ApplyReasonCatchUp, requests[0].Reason) + assert.Equal(t, "bbb", requests[0].Commit) + + loop.handleLiveEvents(&WriteRequest{Events: []Event{windowEvent("alpha", "2", "kubectl-edit")}}) + assert.True(t, requests[0].Current(), "asked for, so it does not yield to an edit") +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 96e81cf1..974ab912 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1044,6 +1044,9 @@ type branchWorkerEventLoop struct { // window opens (then it attaches to it) or its attach deadline passes. // Loop-goroutine only. pendingCRs map[commitRequestID]*pendingCommitRequest + // savedCatchUps is the GitTargets whose pushed CommitRequest asked their applier to catch up at + // once (catchUp: ImmediatelyAfterPush), until the appliers are next listed (takeReasons). + savedCatchUps map[itypes.ResourceReference]bool // crSeq numbers registrations, so competing requests attach first come, first served. crSeq uint64 // refusalTimer fires at the earliest deadline in refusalPending. See refusal_reapply.go. diff --git a/internal/git/commit_request_attach.go b/internal/git/commit_request_attach.go index 2a0ff5d8..3965e8db 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -146,6 +146,9 @@ type AttachCommitRequest struct { // CommitEmpty records Message in an empty commit when the request ends with nothing to // commit: whenNothingToCommit: CommitEmpty. CommitEmpty bool + // CatchUpNow asks the GitTarget's applier to apply the tip as soon as the request's commit is + // pushed: catchUp: ImmediatelyAfterPush. + CatchUpNow bool } // commitRequestID is the worker-local key for a CommitRequest: its namespaced @@ -184,6 +187,8 @@ type pendingCommitRequest struct { // commitEmpty records the message in an empty commit when the request ends with nothing to // commit. Never for a WindowMismatch: that would claim more than happened. commitEmpty bool + // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. + catchUpNow bool // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/commit_request_attach_loop.go index ee312315..a29a2274 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -213,6 +213,7 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque idleTimeout: req.IdleTimeout, maxDuration: req.MaxDuration, commitEmpty: req.CommitEmpty, + catchUpNow: req.CatchUpNow, } l.pendingCRs[id] = pcr l.w.setCommitRequestPhase(id, PhaseWaitingForWindow) @@ -472,6 +473,9 @@ func (l *branchWorkerEventLoop) resolveCommitRequest(id commitRequestID, result result.Branch = l.w.Branch } l.w.recordCommitRequestOutcome(id, result) + if pcr := l.pendingCRs[id]; pcr != nil && pcr.catchUpNow && result.Commit != "" { + l.catchUpAfterSave(pcr) + } delete(l.pendingCRs, id) l.w.setCommitRequestPhase(id, "") l.w.Log.Info("CommitRequest resolved", diff --git a/test/e2e/applier_claim.go b/test/e2e/applier_claim.go index 0e925c2b..f2d0cc2d 100644 --- a/test/e2e/applier_claim.go +++ b/test/e2e/applier_claim.go @@ -21,9 +21,9 @@ const ( // createFluxClaim creates the FluxClaim name in namespace, holding the Kustomization // flux-system/ in the operator's own cluster (the install's "local" KubeProvider). // -// Its spec.convergeInterval is "0s", not the "1m" default: the specs count the applies an edit or a +// Its spec.catchUpAfter is "0s", not the "1m" default: the specs count the applies an edit or a // push asks for, and a converge a minute after the worker's own push would land inside their holds. -// The spec that tests the converge sets one (setFluxClaimConvergeInterval). +// The spec that tests the converge sets one (setFluxClaimCatchUpAfter). func createFluxClaim(namespace, name, kustomization string) { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 @@ -36,18 +36,18 @@ spec: kustomizationRef: namespace: flux-system name: %s - convergeInterval: 0s + catchUpAfter: 0s `, name, namespace, localKubeProviderRef(), kustomization) _, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") Expect(err).NotTo(HaveOccurred(), "failed to create FluxClaim %s/%s", namespace, name) } -// setFluxClaimConvergeInterval sets the FluxClaim's spec.convergeInterval. -func setFluxClaimConvergeInterval(namespace, name, interval string) { +// setFluxClaimCatchUpAfter sets the FluxClaim's spec.catchUpAfter. +func setFluxClaimCatchUpAfter(namespace, name, interval string) { GinkgoHelper() _, err := kubectlRunInNamespace(namespace, "patch", fluxClaimResource, name, "--type=merge", "-p", - fmt.Sprintf(`{"spec":{"convergeInterval":%q}}`, interval)) - Expect(err).NotTo(HaveOccurred(), "failed to set the convergeInterval of FluxClaim %s/%s", namespace, name) + fmt.Sprintf(`{"spec":{"catchUpAfter":%q}}`, interval)) + Expect(err).NotTo(HaveOccurred(), "failed to set the catchUpAfter of FluxClaim %s/%s", namespace, name) } // createArgoCDClaim creates the ArgoCDClaim name in namespace, holding the Application diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index bdd4a161..44f82610 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -293,28 +293,65 @@ data: g.Expect(kustomizationSuspended(g, fluxName)).To(BeTrue(), "suspended again") }, 2*time.Minute, 2*time.Second).Should(Succeed()) verifyResourceCondition(fluxClaimResource, fluxName, testNs, "InControl", "True", "Suspended", "") + + // The namespace goes when the spec ends. A ConfigMap left in Git for it would fail every later + // apply of the folder, so it leaves Git first, and the folder applies without it. + By("taking the ConfigMap out of Git again, before its namespace goes") + removeFileFromOutside(repo, fluxApplierFolder+"/later.yaml", "e2e: another writer removes the ConfigMap") + Eventually(func(g Gomega) { + head := remoteHeadSHA(g, repo.CheckoutDir) + g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(head)) + g.Expect(readClaimOperation(g, testNs, fluxClaimResource, fluxName).Phase).To(Equal("Succeeded")) + }, 3*time.Minute, 2*time.Second).Should(Succeed()) }) // A push of GitOps Reverser's own asks no apply at once, but the Kustomization's own state follows - // the branch at the applier claim's spec.convergeInterval: its lastAppliedRevision and its + // the branch at the applier claim's spec.catchUpAfter: its lastAppliedRevision and its // GitRepository's artifact reach the tip a mirrored API edit made, with no push by anyone else. - It("brings the applied revision to its own pushes' tip at the convergeInterval", func() { - setFluxClaimConvergeInterval(testNs, fluxName, "10s") - DeferCleanup(setFluxClaimConvergeInterval, testNs, fluxName, "0s") - - By("mirroring an API edit: the branch moves by GitOps Reverser's own push") - applyConfigMap(testNs, "applier-converge-edit") + // + // The timed edit is an UPDATE of an object the Kustomization already applied. A create would not + // do: it is adopted at once (a CatchUp that stamps Flux's owner labels), with no interval at all. + It("brings the applied revision to its own pushes' tip at the catchUpAfter", func() { + const name = "applier-converge-edit" + + By("creating a ConfigMap, and waiting until its adopt has applied the tip") + applyConfigMap(testNs, name) + var adoptedHead string + var adopted claimOperation Eventually(func(g Gomega) { g.Expect(runGitIn(repo.CheckoutDir, "fetch", "origin", "main")).To(Succeed()) - g.Expect(gitShowFiles(repo.CheckoutDir)).To(ContainSubstring("applier-converge-edit")) + g.Expect(gitShowFiles(repo.CheckoutDir)).To(ContainSubstring(name)) + adoptedHead = remoteHeadSHA(g, repo.CheckoutDir) + g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(adoptedHead)) + adopted = readClaimOperation(g, testNs, fluxClaimResource, fluxName) + g.Expect(adopted.Phase).To(Equal("Succeeded")) + }, 3*time.Minute, 2*time.Second).Should(Succeed()) + + // No reset afterwards: the next spec deletes the applier claim. + setFluxClaimCatchUpAfter(testNs, fluxName, "30s") + + By("editing it through the API: the branch moves by GitOps Reverser's own push") + _, err := kubectlRunInNamespace(testNs, "patch", "configmap", name, "--type=merge", "-p", + `{"data":{"test-key":"converged"}}`) + Expect(err).NotTo(HaveOccurred()) + var editedHead string + Eventually(func(g Gomega) { + editedHead = remoteHeadSHA(g, repo.CheckoutDir) + g.Expect(editedHead).NotTo(Equal(adoptedHead), "the edit is pushed") }, 2*time.Minute, 2*time.Second).Should(Succeed()) - By("a CatchUp applies that tip within the interval, and the Kustomization is held again") + By("no apply at once: the cluster already holds the edit") + Consistently(func(g Gomega) { + g.Expect(readClaimOperation(g, testNs, fluxClaimResource, fluxName).Token).To(Equal(adopted.Token)) + g.Expect(fluxLastAppliedRevision(g, fluxName)).NotTo(ContainSubstring(editedHead)) + }, 10*time.Second, 2*time.Second).Should(Succeed()) + + By("once the interval has passed, a CatchUp applies the tip, and the Kustomization is held again") Eventually(func(g Gomega) { - head := remoteHeadSHA(g, repo.CheckoutDir) - g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(head)) - g.Expect(fluxArtifactRevision(g, fluxName)).To(ContainSubstring(head)) + g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(editedHead)) + g.Expect(fluxArtifactRevision(g, fluxName)).To(ContainSubstring(editedHead)) done := readClaimOperation(g, testNs, fluxClaimResource, fluxName) + g.Expect(done.Token).NotTo(Equal(adopted.Token)) g.Expect(done.Reason).To(Equal("CatchUp")) g.Expect(done.Phase).To(Equal("Succeeded")) g.Expect(kustomizationSuspended(g, fluxName)).To(BeTrue()) From 4c8a6f8171c5a8ecc11ec6e6d311615edd4f7881 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 14:36:10 +0000 Subject: [PATCH 04/33] feat(commitrequest): report the apply a save asked for in an Applied condition A CommitRequest with catchUp: ImmediatelyAfterPush now reports how the apply it asked for went, beside Ready and never changing it: Ready stays "the save is pushed". - Unknown, Applying or Retrying: the apply runs, or did not complete and is asked again with the applier's backoff; the message says why. - True, Applied: status.appliedRevision names what the applier applied. - False, NothingPushed, NoApplierClaim or ApplierGone: nothing to apply, or no applier to ask. - Unknown, Lost: GitOps Reverser restarted before it learnt. The branch worker keeps, per applier, the saves waiting for an apply. The attempt sent next carries them, so the one that succeeds settles them (any apply sent after a save's push holds its commit); one that does not gives them back. An applier already at the tip settles them without an apply. The controller follows a Ready request that asked to catch up until Applied is settled, then expires it as any finished request. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/commitrequest_types.go | 20 ++ .../bases/configbutler.ai_commitrequests.yaml | 23 +++ docs/UPGRADING.md | 3 +- docs/configuration.md | 22 +++ docs/design/apply-trigger.md | 5 +- internal/controller/commitrequest_apply.go | 104 +++++++++++ .../controller/commitrequest_apply_test.go | 105 +++++++++++ .../controller/commitrequest_controller.go | 6 + .../commitrequest_controller_unit_test.go | 13 ++ internal/controller/constants.go | 4 + internal/git/applier_claim.go | 62 +++---- internal/git/apply_converge_test.go | 75 ++++++++ internal/git/branch_worker.go | 10 +- internal/git/commit_request_attach_loop.go | 5 +- internal/git/save_apply.go | 171 ++++++++++++++++++ internal/watch/event_router.go | 18 ++ test/e2e/flux_applier_e2e_test.go | 45 +++++ 17 files changed, 649 insertions(+), 42 deletions(-) create mode 100644 internal/controller/commitrequest_apply.go create mode 100644 internal/controller/commitrequest_apply_test.go create mode 100644 internal/git/save_apply.go diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index b7b0edcb..d9fb391b 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -204,6 +204,19 @@ type CommitRequestStatus struct { // once the commit is on the remote. AuthorAttributed says whether the // request's creator, recorded at admission, names the commit author; False is // not a failure and does not affect Ready. + // + // Applied is set only with spec.catchUp "ImmediatelyAfterPush", once Ready is + // True. It says whether the GitTarget's applier applied a commit holding this + // request's: + // + // - Unknown, reason Applying or Retrying: the apply runs, or failed and is + // asked again; the message says why. + // - True, reason Applied: status.appliedRevision names what was applied. + // - False, reason NothingPushed, NoApplierClaim or ApplierGone: there was + // nothing to apply, or no applier to ask. + // - Unknown, reason Lost: GitOps Reverser restarted before it knew. + // + // It never affects Ready: the save is done once it is pushed. // +optional // +listType=map // +listMapKey=type @@ -218,6 +231,12 @@ type CommitRequestStatus struct { // Commit is the commit hash. Set once the commit is pushed (Pushed=True). // +optional Commit string `json:"commit,omitempty"` + + // AppliedRevision is the revision the GitTarget's applier reported applying + // for this request, as in a Kustomization's status.lastAppliedRevision. Set + // once Applied is True. + // +optional + AppliedRevision string `json:"appliedRevision,omitempty"` } // +kubebuilder:object:root=true @@ -228,6 +247,7 @@ type CommitRequestStatus struct { // +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.commit` // +kubebuilder:printcolumn:name="AuthorAttributed",type=string,JSONPath=`.status.conditions[?(@.type=="AuthorAttributed")].status`,priority=1 // +kubebuilder:printcolumn:name="Pushed",type=string,JSONPath=`.status.conditions[?(@.type=="Pushed")].status`,priority=1 +// +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Branch",type=string,JSONPath=`.status.branch`,priority=1 // +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 9eb05ffd..52f94b3a 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -35,6 +35,10 @@ spec: name: Pushed priority: 1 type: string + - jsonPath: .status.conditions[?(@.type=="Applied")].status + name: Applied + priority: 1 + type: string - jsonPath: .status.branch name: Branch priority: 1 @@ -228,6 +232,12 @@ spec: status: description: status defines the observed state of CommitRequest. properties: + appliedRevision: + description: |- + AppliedRevision is the revision the GitTarget's applier reported applying + for this request, as in a Kustomization's status.lastAppliedRevision. Set + once Applied is True. + type: string branch: description: Branch is the branch the commit is made on. Set once the request ends. @@ -244,6 +254,19 @@ spec: once the commit is on the remote. AuthorAttributed says whether the request's creator, recorded at admission, names the commit author; False is not a failure and does not affect Ready. + + Applied is set only with spec.catchUp "ImmediatelyAfterPush", once Ready is + True. It says whether the GitTarget's applier applied a commit holding this + request's: + + - Unknown, reason Applying or Retrying: the apply runs, or failed and is + asked again; the message says why. + - True, reason Applied: status.appliedRevision names what was applied. + - False, reason NothingPushed, NoApplierClaim or ApplierGone: there was + nothing to apply, or no applier to ask. + - Unknown, reason Lost: GitOps Reverser restarted before it knew. + + It never affects Ready: the save is done once it is pushed. items: description: Condition contains details for one aspect of the current state of this API Resource. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 954db55d..2ced05b2 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -15,7 +15,8 @@ commit somebody else pushed. `spec.catchUpAfter` bounds that lag: once the appli behind the tip that long, the tip is applied, one apply for a whole burst of saves. A `FluxClaim` defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` defaults to `0s`. A `CommitRequest` that sets `catchUp: ImmediatelyAfterPush` has its commit applied as soon as it is -pushed, whatever the claim's interval. +pushed, whatever the claim's interval, and reports the apply in its `Applied` condition and +`status.appliedRevision`. ## `kube-root-ca.crt` is not mirrored diff --git a/docs/configuration.md b/docs/configuration.md index f03fc789..679030d8 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -3121,6 +3121,28 @@ request produced a pushed commit: watch window, whose Git author remains either the configured committer or the explicit unresolved author according to the watch attribution outcome. - **Pushed**: `True` once the commit, empty or not, is in the remote repository. +- **Applied**: only with [`catchUp: ImmediatelyAfterPush`](#applying-a-save-at-once), once `Ready=True`. + It never changes `Ready`. + +### Applying a save at once + +A save is in Git once it is pushed, and the cluster already runs it, so by default nothing applies +it: the applier claim's `catchUpAfter` brings the applier's own state (a `Kustomization`'s +`status.lastAppliedRevision`, its `GitRepository`'s artifact) level later. A request that needs that +state at once, to publish what was applied, sets `spec.catchUp: ImmediatelyAfterPush`. Its +`GitTarget`'s applier is then asked to apply the tip as soon as the commit is pushed, one apply per +such request, and the request reports the apply in its `Applied` condition: + +| `Applied` | Reason | Means | +| --- | --- | --- | +| `Unknown` | `Applying` | the apply is asked for or runs | +| `Unknown` | `Retrying` | it did not complete and is asked again; the message says why | +| `True` | `Applied` | applied; `status.appliedRevision` names the revision, as the applier reported it | +| `False` | `NothingPushed`, `NoApplierClaim`, `ApplierGone` | nothing to apply, or no applier to ask | +| `Unknown` | `Lost` | GitOps Reverser restarted before it learnt how the apply went | + +Wait for `Applied=True` for "saved and applied"; `Ready=True` stays "saved". A finished request is +deleted only after its `Applied` condition is settled. ### Finished requests are deleted after 48 hours diff --git a/docs/design/apply-trigger.md b/docs/design/apply-trigger.md index d5a86a05..8eeb82ba 100644 --- a/docs/design/apply-trigger.md +++ b/docs/design/apply-trigger.md @@ -278,7 +278,10 @@ deletes its file the applier leaves it on the cluster, and a later resync writes - **A save can ask to catch up at once.** A `CommitRequest` with `catchUp: ImmediatelyAfterPush` leaves a converge pending for its `GitTarget`'s applier as soon as its commit is pushed, without waiting for `catchUpAfter`. It was asked for, so it does not yield to an edit. A save that pushed - no commit asks nothing. + no commit asks nothing. The save waits on its applier until an attempt that carries it succeeds + (any apply sent after its push holds its commit), and the request's `Applied` condition reports + it: retried with the applier's backoff, settled applied, or settled not applied when there is no + applier, nothing was pushed, or the applier stops being named (`save_apply.go`). - **No apply before every initial snapshot is published.** An applier is asked for nothing, for any reason, until every collection of every `GitTarget` it covers has a settled snapshot (applied, refused or failed; tracked per collection on the watch side) and the worker has published every diff --git a/internal/controller/commitrequest_apply.go b/internal/controller/commitrequest_apply.go new file mode 100644 index 00000000..2c8b0617 --- /dev/null +++ b/internal/controller/commitrequest_apply.go @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "time" + + "k8s.io/apimachinery/pkg/api/equality" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + logf "sigs.k8s.io/controller-runtime/pkg/log" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +// A CommitRequest with catchUp: ImmediatelyAfterPush keeps being reconciled once it is Ready, to +// report in its Applied condition how the apply it asked for went (git.SaveApply). Ready is the +// save's alone and never changes here. + +// crReasonApplyLost is the Applied reason for an apply the worker knows nothing of: GitOps Reverser +// restarted before it learnt how it went, or the record aged out. +const crReasonApplyLost = "Lost" + +// applyLostAfter is how long, from Ready, a request may find its worker knowing nothing of its apply +// before Applied says it is lost. The worker records the apply as the request resolves, so only a +// restart, or a record past its TTL, leaves it unknown. +const applyLostAfter = time.Minute + +// followsApply reports whether a finished request still reports its apply: it asked for one, its +// save ended Ready, and Applied is not settled yet. +func followsApply(cr *configbutleraiv1alpha3.CommitRequest) bool { + return cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpImmediatelyAfterPush && + conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) && !applyReported(cr) +} + +// applyReported reports whether Applied is settled: True, False, or Unknown because it was lost. +func applyReported(cr *configbutleraiv1alpha3.CommitRequest) bool { + c := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeApplied) + return c != nil && (c.Status != metav1.ConditionUnknown || c.Reason == crReasonApplyLost) +} + +// followApply reads where the request's apply stands, writes it to Applied, and comes back until it +// is settled; then the request expires as any finished one does. +func (r *CommitRequestReconciler) followApply( + ctx context.Context, + cr *configbutleraiv1alpha3.CommitRequest, +) (ctrl.Result, error) { + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") + attach := git.AttachCommitRequest{ + Namespace: cr.Namespace, Name: cr.Name, UID: string(cr.UID), + GitTargetName: cr.Spec.GitTargetRef.Name, GitTargetNamespace: cr.Namespace, + } + apply, known, err := r.Finalizer.LookupCommitRequestApply(ctx, attach) + if err != nil { + log.V(1).Info("Cannot read the CommitRequest's apply yet; will retry", + "name", client.ObjectKeyFromObject(cr), "err", err.Error()) + return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + } + if !known { + apply = unknownApply(cr) + } + before := cr.Status.DeepCopy() + setAppliedCondition(cr, apply) + if !equality.Semantic.DeepEqual(*before, cr.Status) { + if err := r.Status().Update(ctx, cr); err != nil { + // A conflict reads the request again; anything else retries the same way. + return ctrl.Result{}, client.IgnoreNotFound(err) + } + log.Info("CommitRequest apply reported", "name", client.ObjectKeyFromObject(cr), + "reason", apply.Reason, "appliedRevision", cr.Status.AppliedRevision, "message", apply.Message) + } + if applyReported(cr) { + return r.expireFinished(ctx, cr) + } + return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil +} + +// unknownApply is the apply of a request its worker knows nothing of: still being taken on, shortly +// after Ready, and lost after applyLostAfter. +func unknownApply(cr *configbutleraiv1alpha3.CommitRequest) git.SaveApply { + if ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady); ready != nil && + time.Since(ready.LastTransitionTime.Time) < applyLostAfter { + return git.SaveApply{Reason: git.SaveApplyApplying, Message: "waiting for the branch worker to take the apply"} + } + return git.SaveApply{Reason: crReasonApplyLost, + Message: "GitOps Reverser does not know how the apply went: it restarted before it learnt, or kept no record"} +} + +// setAppliedCondition writes apply to Applied and status.appliedRevision. +func setAppliedCondition(cr *configbutleraiv1alpha3.CommitRequest, apply git.SaveApply) { + status := metav1.ConditionUnknown + switch { + case apply.Applied: + status = metav1.ConditionTrue + cr.Status.AppliedRevision = apply.AppliedRevision + case apply.Done: + status = metav1.ConditionFalse + } + setCommitRequestCondition(cr, ConditionTypeApplied, status, apply.Reason, apply.Message) +} diff --git a/internal/controller/commitrequest_apply_test.go b/internal/controller/commitrequest_apply_test.go new file mode 100644 index 00000000..7fa0e2f3 --- /dev/null +++ b/internal/controller/commitrequest_apply_test.go @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +func catchingUp(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { + cr.Spec.CatchUp = configv1alpha3.CatchUpImmediatelyAfterPush + return cr +} + +// TestCommitRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with +// catchUp: ImmediatelyAfterPush is reconciled on, and Applied follows the worker's report until it +// is settled. Ready stays the save's. +func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) + f := committedFinalizer() + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + + reconcileCommitRequest(t, r, "save-1") + assert.Zero(t, f.applyLookups, "the save resolves first") + + f.apply, f.applyKnown = git.SaveApply{Reason: git.SaveApplyRetrying, Message: "the apply did not complete"}, true + res := reconcileCommitRequest(t, r, "save-1") + assert.Positive(t, res.RequeueAfter, "followed until it is settled") + got := fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyRetrying) + requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) + + f.apply = git.SaveApply{ + Done: true, + Applied: true, + Reason: git.SaveApplyApplied, + AppliedRevision: "main@sha1:c0ffee", + } + reconcileCommitRequest(t, r, "save-1") + got = fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, metav1.ConditionTrue, git.SaveApplyApplied) + assert.Equal(t, "main@sha1:c0ffee", got.Status.AppliedRevision) + + lookups := f.applyLookups + reconcileCommitRequest(t, r, "save-1") + assert.Equal(t, lookups, f.applyLookups, "settled: not looked up again") +} + +// TestCommitRequestReconcile_NothingToApplyIsFalse: a settled apply that applied nothing is +// Applied=False, and does not touch Ready. +func TestCommitRequestReconcile_NothingToApplyIsFalse(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) + f := committedFinalizer() + f.apply, f.applyKnown = git.SaveApply{Done: true, Reason: git.SaveApplyNoApplier}, true + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + + reconcileCommitRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, metav1.ConditionFalse, git.SaveApplyNoApplier) + requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) +} + +// TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply +// shortly after Ready is still taking it on; long after, it restarted, and Applied says it is lost. +func TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) + f := committedFinalizer() + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileCommitRequest(t, r, "save-1") + + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyApplying) + + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + ready.LastTransitionTime = metav1.NewTime(time.Now().Add(-2 * applyLostAfter)) + require.NoError(t, c.Status().Update(context.Background(), &got)) + reconcileCommitRequest(t, r, "save-1") + got = fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, crReasonApplyLost) + assert.False(t, followsApply(&got), "lost is settled") +} + +// TestCommitRequestReconcile_NoCatchUpReportsNoApply: the default asks no apply, and has no Applied. +func TestCommitRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save-1")) + f := committedFinalizer() + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + + reconcileCommitRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") + assert.Nil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeApplied)) + assert.Zero(t, f.applyLookups) +} diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index 67ab23a4..aa561b6d 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -37,6 +37,9 @@ type CommitRequestFinalizer interface { // answer: resolved with git.ErrCommitRequestWithdrawn when it is cancelled, the real outcome // when the worker got there first, or resolved=false with the worker's phase while it decides. WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + // LookupCommitRequestApply reports where the apply a request with catchUp: + // ImmediatelyAfterPush asked for stands, and false while the worker knows nothing of it. + LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) } // CommandAuthorLookup resolves the author of a CommitRequest from the submitter @@ -202,6 +205,9 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques } if done { if commitRequest != nil { + if followsApply(commitRequest) { + return r.followApply(ctx, commitRequest) + } return r.expireFinished(ctx, commitRequest) } return ctrl.Result{}, nil diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index 5d432bc9..82e2348f 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -45,6 +45,19 @@ type fakeFinalizer struct { withdrawErr error withdrawPending bool withdrawCalls []git.AttachCommitRequest + + // The apply a request with catchUp: ImmediatelyAfterPush asked for, and the lookups made. + apply git.SaveApply + applyKnown bool + applyErr error + applyLookups int +} + +func (f *fakeFinalizer) LookupCommitRequestApply( + _ context.Context, _ git.AttachCommitRequest, +) (git.SaveApply, bool, error) { + f.applyLookups++ + return f.apply, f.applyKnown, f.applyErr } func (f *fakeFinalizer) WithdrawCommitRequest( diff --git a/internal/controller/constants.go b/internal/controller/constants.go index 92d636cf..05763c67 100644 --- a/internal/controller/constants.go +++ b/internal/controller/constants.go @@ -98,6 +98,10 @@ const ( // ConditionTypePushed indicates whether a CommitRequest's commit reached the // remote repository. ConditionTypePushed = "Pushed" + // ConditionTypeApplied reports, for a CommitRequest with catchUp: ImmediatelyAfterPush, whether + // its GitTarget's applier applied a commit that holds the request's: Unknown while it applies or + // retries, True once applied, False when there was nothing to apply. It never affects Ready. + ConditionTypeApplied = "Applied" // KubeProviderConditionValidated reports whether a KubeProvider's credential is safe and // resolvable: an inCluster provider is Validated while the installation lists it in diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 8fef4a81..156a1d22 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -251,6 +251,9 @@ type applierState struct { // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, // so its attempt yields to an edit (convergeHolds). lag bool + // waitingSaves is the CommitRequests waiting for this applier's next apply that succeeds + // (save_apply.go); the attempt sent next carries them. + waitingSaves map[commitRequestID]bool // appliedRevision is the revision the last operation that succeeded applied, as the applier // reported it, recovered on restart from the applier claim's status.operation. A converge of the commit // it names asks nothing. @@ -288,6 +291,17 @@ type applierState struct { followedSuspend string } +// restoreLag sets lag for the converge an attempt gives back, before it is merged: it is only +// scheduleConverges' when that is all either side holds. +func (s *applierState) restoreLag(attempt *inflightAttempt) { + switch { + case attempt.converge && !attempt.lag: + s.lag = false + case attempt.lag && !s.converge: + s.lag = true + } +} + // onlyLag reports whether scheduleConverges' converge is all that is pending for the applier, given its // unsent refusals: an attempt carrying nothing else yields to an edit (convergeHolds). func (s *applierState) onlyLag(refusals map[refusalKey]unsentRefusal) bool { @@ -310,6 +324,8 @@ type inflightAttempt struct { refusals map[refusalKey]unsentRefusal // lag is whether it carries only scheduleConverges' converge. lag bool + // saves is the CommitRequests whose apply it carries (save_apply.go). + saves map[commitRequestID]bool // phase is the last report of its operation, empty before the first. phase ApplyOutcome // previous is the operation recorded before this attempt's, with the attempt it belonged to, @@ -506,33 +522,6 @@ func (l *branchWorkerEventLoop) takeReasons() bool { return true } -// catchUpAfterSave leaves a converge pending for the applier of a CommitRequest's GitTarget, whose -// commit was just pushed and which asked for it (catchUp: ImmediatelyAfterPush). It is taken onto the -// applier when the appliers are next listed, at the end of this wake. -func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest) { - if l.savedCatchUps == nil { - l.savedCatchUps = map[itypes.ResourceReference]bool{} - } - l.savedCatchUps[itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace)] = true -} - -// takeSavedCatchUps leaves a converge pending for every applier a pushed CommitRequest asked to catch -// up. It was asked for, so it is no lag converge: it does not yield to an edit (convergeHolds). A -// GitTarget that names no applier asks nothing. -func (l *branchWorkerEventLoop) takeSavedCatchUps() { - for _, state := range l.appliers { - for _, target := range state.targets { - if !l.savedCatchUps[target.ref] { - continue - } - state.converge, state.lag = true, false - l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", - "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", target.ref.String()) - } - } - l.savedCatchUps = nil -} - // dueAppliers is the appliers an attempt is due for now, in turn order, and the earliest time one // still waiting comes due. The appliers are taken in turn order, so in a group, which has one attempt // out at a time, the attempt goes to the applier whose turn is longest ago. @@ -553,6 +542,8 @@ func (l *branchWorkerEventLoop) dueAppliers(now time.Time) ([]*applierState, tim refusals := l.unsentRefusalsFor(state) if l.alreadyConverged(state, refusals, tip) { state.converge, state.lag = false, false + l.settleSaves(state.waitingSaves, state.appliedRevision) + state.waitingSaves = nil l.w.Log.V(1).Info("The applier already applied the tip; the converge asks nothing", "branch", l.w.Branch, "applier", state.ref.String(), "commit", tip) } @@ -727,6 +718,7 @@ func (l *branchWorkerEventLoop) syncAppliers(targets []branchTarget) { l.forgetFolders(listed) for ref, state := range l.appliers { if _, ok := named[ref]; !ok { + l.dropSaves(state) if state.inflight != nil { state.inflight.abandon() } @@ -837,9 +829,10 @@ func (l *branchWorkerEventLoop) sendAttempt(state *applierState, tip string, ref id: l.nextApplyID, foreign: state.foreign, adopt: state.adopt, converge: state.converge, repair: state.repair, refusals: refusals, abandon: abandon, previous: state.operation, previousID: state.operationID, - lag: state.onlyLag(refusals), + lag: state.onlyLag(refusals), saves: state.waitingSaves, } state.foreign, state.adopt, state.converge, state.repair, state.lag = false, false, false, false, false + state.waitingSaves = nil state.behindSince = time.Time{} state.inflight = attempt state.lastTurn = attempt.id @@ -915,6 +908,7 @@ func (l *branchWorkerEventLoop) settleApplyResults(now time.Time) { if state.appliedRevision == "" && state.operation != nil { state.appliedRevision = state.operation.Commit } + l.settleSaves(attempt.saves, state.appliedRevision) continue case ApplyStale: if attempt.lag { @@ -935,6 +929,7 @@ func (l *branchWorkerEventLoop) settleApplyResults(now time.Time) { state.failures++ state.retryAt = now.Add(applierRetryDelay(state.failures, res.Outcome == ApplyRejected)) l.restoreAttempt(state, attempt) + l.retrySaves(attempt.saves, errString(res.Err)) l.lookIfSourceMoved(state, res) message := "An apply attempt did not deliver; deciding it again after its backoff" if res.Outcome == ApplyFailed { @@ -1122,15 +1117,12 @@ func (l *branchWorkerEventLoop) publishUnresolved() { func (l *branchWorkerEventLoop) restoreAttempt(state *applierState, attempt *inflightAttempt) { state.foreign = state.foreign || attempt.foreign state.adopt = state.adopt || attempt.adopt - // The converge given back is only scheduleConverges' when that is all either side holds. - switch { - case attempt.converge && !attempt.lag: - state.lag = false - case attempt.lag && !state.converge: - state.lag = true - } + state.restoreLag(attempt) state.converge = state.converge || attempt.converge state.repair = state.repair || attempt.repair + for id := range attempt.saves { + state.waitSave(id) + } for key, refusal := range attempt.refusals { if !l.w.refusalStands(key) { continue diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index 9456a25a..0e580158 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -224,3 +224,78 @@ func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { loop.handleLiveEvents(&WriteRequest{Events: []Event{windowEvent("alpha", "2", "kubectl-edit")}}) assert.True(t, requests[0].Current(), "asked for, so it does not yield to an edit") } + +// TestSaveApply_FollowsTheAttemptThatCarriesIt: a save that asked to catch up waits on its applier, +// is carried by the attempt sent next, is retried when that attempt does not succeed, and is +// settled applied by the one that does. One that pushed nothing, or whose GitTarget names no +// applier, is settled at once as not applied. +func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true), applierTestTarget("loner", false)) + convergedAt(t, loop, requester) + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + save := func(name, target, commit string) commitRequestID { + id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingCommitRequest{ + id: id, gitTargetName: target, gitTargetNamespace: "team-a", catchUpNow: true, + } + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + return id + } + lookup := func(id commitRequestID) SaveApply { + apply, known := loop.w.LookupSaveApply(id.Namespace, id.Name, id.UID) + require.True(t, known, "the worker knows %s", id.Name) + return apply + } + + nothing := save("nothing", "alpha", "") + assert.Equal(t, SaveApply{Done: true, Reason: SaveApplyNothingPushed, Message: lookup(nothing).Message}, + lookup(nothing)) + + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + pushed := save("pushed", "alpha", "bbb") + alone := save("alone", "loner", "bbb") + assert.Equal(t, SaveApplyApplying, lookup(pushed).Reason) + loop.serviceAppliers() + assert.True(t, lookup(alone).Done) + assert.Equal(t, SaveApplyNoApplier, lookup(alone).Reason) + + first := requester.taken() + require.Len(t, first, 1) + report(first[0], ApplyFailed) + loop.serviceAppliers() + assert.Equal(t, SaveApplyRetrying, lookup(pushed).Reason) + assert.False(t, lookup(pushed).Done) + + dueNow(loop) + loop.serviceAppliers() + again := requester.taken() + require.Len(t, again, 1, "the retry carries the save") + again[0].Report(ApplyResult{ID: again[0].ID, Outcome: ApplySucceeded, AppliedRevision: "main@sha1:bbb"}) + loop.serviceAppliers() + settled := lookup(pushed) + assert.True(t, settled.Done) + assert.True(t, settled.Applied) + assert.Equal(t, "main@sha1:bbb", settled.AppliedRevision) +} + +// TestSaveApply_AnApplierAlreadyAtTheTipSettlesIt: when the applier applied the tip before the save's +// converge went, nothing is asked, and the save is applied all the same. +func TestSaveApply_AnApplierAlreadyAtTheTipSettlesIt(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + convergedAt(t, loop, requester) + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "late", UID: "uid-late"} + loop.pendingCRs[id] = &pendingCommitRequest{ + id: id, + gitTargetName: "alpha", + gitTargetNamespace: "team-a", + catchUpNow: true, + } + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "aaa"}) + + loop.serviceAppliers() + assert.Empty(t, requester.taken()) + apply, known := loop.w.LookupSaveApply(id.Namespace, id.Name, id.UID) + require.True(t, known) + assert.True(t, apply.Applied) +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 974ab912..2fd207c8 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -370,6 +370,9 @@ type BranchWorker struct { // crPhases holds where each unresolved CommitRequest stands, for the controller to report // while it waits. Guarded by crOutcomesMu; the event loop is the only writer. crPhases map[commitRequestID]CommitRequestPhase + // crApplies holds where the apply each CommitRequest asked for stands (save_apply.go). Guarded by + // crOutcomesMu; the event loop is the only writer. + crApplies map[commitRequestID]saveApplyEntry // crQueuedAttaches counts, per request, the attaches on the FIFO that the loop has not handled // yet. An outcome is not collected while any are queued: the outcome is what turns a late attach // into a no-op, so collecting it first would let that attach register a withdrawn request @@ -1044,9 +1047,10 @@ type branchWorkerEventLoop struct { // window opens (then it attaches to it) or its attach deadline passes. // Loop-goroutine only. pendingCRs map[commitRequestID]*pendingCommitRequest - // savedCatchUps is the GitTargets whose pushed CommitRequest asked their applier to catch up at - // once (catchUp: ImmediatelyAfterPush), until the appliers are next listed (takeReasons). - savedCatchUps map[itypes.ResourceReference]bool + // savedCatchUps is the CommitRequests, by GitTarget, whose pushed commit asked their applier to + // catch up at once (catchUp: ImmediatelyAfterPush), until the appliers are next listed + // (takeReasons, save_apply.go). + savedCatchUps map[itypes.ResourceReference][]commitRequestID // crSeq numbers registrations, so competing requests attach first come, first served. crSeq uint64 // refusalTimer fires at the earliest deadline in refusalPending. See refusal_reapply.go. diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/commit_request_attach_loop.go index a29a2274..b088bb53 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -60,6 +60,7 @@ func (w *BranchWorker) recordCommitRequestOutcomeLocked(id commitRequestID, resu w.crOwners.release(k, w) } } + w.collectSaveApplies(now) } // countQueuedAttach records an attach entering the FIFO. The caller holds pendingResyncsMu, so the @@ -473,8 +474,8 @@ func (l *branchWorkerEventLoop) resolveCommitRequest(id commitRequestID, result result.Branch = l.w.Branch } l.w.recordCommitRequestOutcome(id, result) - if pcr := l.pendingCRs[id]; pcr != nil && pcr.catchUpNow && result.Commit != "" { - l.catchUpAfterSave(pcr) + if pcr := l.pendingCRs[id]; pcr != nil && pcr.catchUpNow { + l.catchUpAfterSave(pcr, result.Commit) } delete(l.pendingCRs, id) l.w.setCommitRequestPhase(id, "") diff --git a/internal/git/save_apply.go b/internal/git/save_apply.go new file mode 100644 index 00000000..48d3d3c1 --- /dev/null +++ b/internal/git/save_apply.go @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "time" + + itypes "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// A CommitRequest with catchUp: ImmediatelyAfterPush asks its GitTarget's applier to apply the tip +// as soon as its commit is pushed, and the controller reports how that went in its Applied +// condition. The save's own outcome is decided by the push alone; this is reported beside it and +// never changes it. +// +// The worker keeps, per applier, the saves waiting for an apply. The attempt that goes out carries +// them, so the one that succeeds settles them: any apply sent after a save's push applies a tip +// that holds the save's commit. An attempt that does not succeed gives them back, and the next +// carries them again. A save whose GitTarget names no applier, or that pushed no commit, or whose +// applier stops being named, is settled as not applied. + +// Why a save's apply was settled, or why it still waits: the Applied condition's reasons. +const ( + SaveApplyApplying = "Applying" + SaveApplyRetrying = "Retrying" + SaveApplyApplied = "Applied" + SaveApplyNothingPushed = "NothingPushed" + SaveApplyNoApplier = "NoApplierClaim" + SaveApplyApplierGone = "ApplierGone" +) + +// SaveApply is how the apply a CommitRequest asked for stands. +type SaveApply struct { + // Done is whether it is settled: it changes no more. + Done bool + // Applied is whether the applier applied a commit that holds the save's. + Applied bool + // Reason is one of the SaveApply* reasons, and Message says more. + Reason string + Message string + // AppliedRevision is the revision the applier reported applying, once Applied. + AppliedRevision string +} + +type saveApplyEntry struct { + apply SaveApply + updatedAt time.Time +} + +// recordSaveApply stores where a save's apply stands, for the controller to read. The event loop is +// the only writer; the entries are collected with the CommitRequest outcomes. +func (w *BranchWorker) recordSaveApply(id commitRequestID, apply SaveApply) { + w.crOutcomesMu.Lock() + defer w.crOutcomesMu.Unlock() + if w.crApplies == nil { + w.crApplies = map[commitRequestID]saveApplyEntry{} + } + w.crApplies[id] = saveApplyEntry{apply: apply, updatedAt: time.Now()} +} + +// LookupSaveApply reports where the apply a CommitRequest asked for stands, and false while the +// worker knows nothing of it. +func (w *BranchWorker) LookupSaveApply(namespace, name, uid string) (SaveApply, bool) { + w.crOutcomesMu.Lock() + defer w.crOutcomesMu.Unlock() + entry, ok := w.crApplies[commitRequestID{Namespace: namespace, Name: name, UID: uid}] + return entry.apply, ok +} + +// collectSaveApplies drops the entries older than the outcome TTL. The caller holds crOutcomesMu. +func (w *BranchWorker) collectSaveApplies(now time.Time) { + for id, entry := range w.crApplies { + if now.Sub(entry.updatedAt) > commitRequestOutcomeTTL { + delete(w.crApplies, id) + } + } +} + +// catchUpAfterSave is called as a CommitRequest that asked to catch up resolves. With a pushed +// commit, its GitTarget's applier gets a converge when the appliers are next listed, at the end of +// this wake (takeSavedCatchUps); with none, there is nothing to apply. +func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, commit string) { + if commit == "" { + l.w.recordSaveApply(pcr.id, SaveApply{Done: true, Reason: SaveApplyNothingPushed, + Message: "the request pushed no commit, so there is nothing to apply"}) + return + } + if l.savedCatchUps == nil { + l.savedCatchUps = map[itypes.ResourceReference][]commitRequestID{} + } + ref := itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace) + l.savedCatchUps[ref] = append(l.savedCatchUps[ref], pcr.id) + l.w.recordSaveApply(pcr.id, SaveApply{Reason: SaveApplyApplying, + Message: "commit " + commit + " is pushed; its applier is asked to apply it"}) +} + +// takeSavedCatchUps leaves a converge pending for every applier a pushed CommitRequest asked to catch +// up, and records the request as waiting on it. It was asked for, so it is no lag converge: it does +// not yield to an edit (convergeHolds). A request whose GitTarget names no applier is settled. +func (l *branchWorkerEventLoop) takeSavedCatchUps() { + for ref, ids := range l.savedCatchUps { + state := l.applierNaming(ref) + if state == nil { + for _, id := range ids { + l.w.recordSaveApply(id, SaveApply{Done: true, Reason: SaveApplyNoApplier, + Message: "GitTarget " + ref.String() + " names no applier claim that may be asked"}) + } + continue + } + state.converge, state.lag = true, false + for _, id := range ids { + state.waitSave(id) + } + l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", + "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", ref.String(), "requests", len(ids)) + } + l.savedCatchUps = nil +} + +// applierNaming is the applier a GitTarget names, as last listed, or nil. +func (l *branchWorkerEventLoop) applierNaming(ref itypes.ResourceReference) *applierState { + for _, state := range l.appliers { + for _, target := range state.targets { + if target.ref == ref { + return state + } + } + } + return nil +} + +// waitSave records a CommitRequest as waiting for the applier's next apply that succeeds. +func (s *applierState) waitSave(id commitRequestID) { + if s.waitingSaves == nil { + s.waitingSaves = map[commitRequestID]bool{} + } + s.waitingSaves[id] = true +} + +// settleSaves settles every save in saves as applied at revision. +func (l *branchWorkerEventLoop) settleSaves(saves map[commitRequestID]bool, revision string) { + for id := range saves { + l.w.recordSaveApply(id, SaveApply{Done: true, Applied: true, Reason: SaveApplyApplied, + AppliedRevision: revision, Message: "the applier applied " + revision}) + } +} + +// retrySaves records the saves an attempt that did not succeed gave back as waiting again, with why. +func (l *branchWorkerEventLoop) retrySaves(saves map[commitRequestID]bool, why string) { + for id := range saves { + l.w.recordSaveApply(id, SaveApply{Reason: SaveApplyRetrying, + Message: "the apply did not complete, and is asked again: " + why}) + } +} + +// dropSaves settles the saves of an applier no GitTarget names any more: nobody asks it for anything. +func (l *branchWorkerEventLoop) dropSaves(state *applierState) { + saves := state.waitingSaves + if state.inflight != nil { + for id := range state.inflight.saves { + if saves == nil { + saves = map[commitRequestID]bool{} + } + saves[id] = true + } + } + for id := range saves { + l.w.recordSaveApply(id, SaveApply{Done: true, Reason: SaveApplyApplierGone, + Message: "applier " + state.ref.String() + " is no longer named on this branch"}) + } +} diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index dd26e83e..277b39cb 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -161,6 +161,24 @@ func (r *EventRouter) WithdrawCommitRequest( return git.FinalizeResult{Branch: branch, Phase: phase}, false, nil } +// LookupCommitRequestApply reports where the apply a CommitRequest asked for stands, from the +// branch worker of its GitTarget, and false while that worker knows nothing of it or none runs. +func (r *EventRouter) LookupCommitRequestApply( + ctx context.Context, + attach git.AttachCommitRequest, +) (git.SaveApply, bool, error) { + worker, _, err := r.commitRequestWorker(ctx, attach) + if errors.Is(err, errGitTargetGone) { + return git.SaveApply{Done: true, Reason: git.SaveApplyApplierGone, + Message: "the GitTarget is gone, so its applier is asked for nothing"}, true, nil + } + if err != nil || worker == nil { + return git.SaveApply{}, false, err + } + apply, known := worker.LookupSaveApply(attach.Namespace, attach.Name, attach.UID) + return apply, known, nil +} + // heldBecause is why a request waiting for its push has not reached the remote: the worker's // publication report while it cannot publish, empty otherwise. func heldBecause(worker *git.BranchWorker, phase git.CommitRequestPhase) string { diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index 44f82610..926bb977 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -358,6 +358,51 @@ data: }, 2*time.Minute, 2*time.Second).Should(Succeed()) }) + // A save that asks to catch up is applied as soon as it is pushed, with catchUpAfter off, and its + // CommitRequest reports the apply: Applied=True with the revision Flux applied. + It("applies a save that asks to catch up, and reports it on the CommitRequest", func() { + const configMap, request = "applier-converge-edit", "applier-save-catch-up" + setFluxClaimCatchUpAfter(testNs, fluxName, "0s") + + By("saving an API edit with catchUp: ImmediatelyAfterPush") + _, err := kubectlRunWithStdin(testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: CommitRequest +metadata: + name: %s + namespace: %s +spec: + gitTargetRef: + name: %s + message: "e2e: save and apply" + catchUp: ImmediatelyAfterPush + window: + attachTimeout: "4s" + maxDuration: "4s" +`, request, testNs, destName), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred()) + _, err = kubectlRunInNamespace(testNs, "patch", "configmap", configMap, "--type=merge", "-p", + `{"data":{"test-key":"saved"}}`) + Expect(err).NotTo(HaveOccurred()) + + By("the save is pushed, and Ready says only that") + var commit string + Eventually(func(g Gomega) { + g.Expect(commitRequestCondition(g, testNs, request, "Ready")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, request, "{.status.conditions[?(@.type==\"Ready\")].reason}")). + To(Equal(commitRequestReasonCommitted)) + commit = commitRequestField(g, testNs, request, "{.status.commit}") + g.Expect(commit).NotTo(BeEmpty()) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + + By("the applier applies it at once, and Applied reports the revision") + Eventually(func(g Gomega) { + g.Expect(commitRequestCondition(g, testNs, request, "Applied")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, request, "{.status.appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(commit)) + g.Expect(readClaimOperation(g, testNs, fluxClaimResource, fluxName).Reason).To(Equal("CatchUp")) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + }) + // Deleting the applier claim hands the Kustomization back: its finalizer waits until nothing is // running and releases spec.suspend, so Flux applies on its own again. It("hands the Kustomization back to Flux when the applier claim is deleted", func() { From 59a78482ed72d54d7a07a66a0eb09043cf8f341d Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 14:55:45 +0000 Subject: [PATCH 05/33] fix(kustomize): keep a release's values file, and find generator entries by identity Four fixes to writing a live ConfigMap back into its configMapGenerator: - An input a HelmRelease or Argo CD Application also names as a values file counts as shared: a delete of the generated ConfigMap keeps it, and a write into it is refused, since the kustomize render cannot show what the edit does to the release. - A generator entry is found in the kustomization's bytes as they stand, by its name and namespace, not by the index it had before the batch. Deleting two generated ConfigMaps of one kustomization in one flush no longer aborts the second. - The render check keys a generated object by its namespace too, so two entries of one name in different namespaces are two objects: deleting or writing one is checked against that one. - A live object the generator already renders plans no write, so it needs no input it may write: a generator in a read-only base, or one two render roots share, no longer fails a resync that changes nothing. Co-Authored-By: Claude Opus 5.5 (1M context) --- internal/git/generator_write.go | 2 + internal/git/generator_write_test.go | 35 +++++ internal/git/manifestedit/kustomization.go | 86 ++++++----- .../git/manifestedit/kustomization_test.go | 50 +++++-- internal/manifestanalyzer/generator_write.go | 86 +++++++++-- internal/manifestanalyzer/generators.go | 19 ++- internal/manifestanalyzer/generators_test.go | 133 +++++++++++++++++- internal/manifestanalyzer/override_chain.go | 10 +- internal/manifestanalyzer/render_verify.go | 14 +- internal/manifestanalyzer/store.go | 4 +- 10 files changed, 368 insertions(+), 71 deletions(-) diff --git a/internal/git/generator_write.go b/internal/git/generator_write.go index 9d243e90..bfbd3fd0 100644 --- a/internal/git/generator_write.go +++ b/internal/git/generator_write.go @@ -69,6 +69,7 @@ func (wb *writeBatch) writeGenerated( SourcePath: gen.GeneratedIn, Kind: desired.GetKind(), Name: desired.GetName(), + Namespace: gen.Rendered.GetNamespace(), Desired: desired, Governed: true, }) @@ -139,6 +140,7 @@ func (wb *writeBatch) removeGenerated( SourcePath: gen.GeneratedIn, Kind: obj.GetKind(), Name: obj.GetName(), + Namespace: gen.Rendered.GetNamespace(), Removed: true, }) wb.putToKustomize = true diff --git a/internal/git/generator_write_test.go b/internal/git/generator_write_test.go index efb44648..68e5a3e0 100644 --- a/internal/git/generator_write_test.go +++ b/internal/git/generator_write_test.go @@ -251,3 +251,38 @@ func TestResync_SweepsAGeneratedConfigMapTheClusterNoLongerHas(t *testing.T) { assertFileBytes(t, envPath, generatorFlagsEnv, "onEvent never sweeps") }) } + +// Two generated ConfigMaps deleted in one batch: the second entry moved up when the first went, and +// its removal still finds it, so both entries and their env files go. +func TestPlanFlush_TwoDeletedGeneratedConfigMapsInOneBatch(t *testing.T) { + worktree := newWorktreeForTest(t) + root := worktree.Filesystem().Root() + seedFile(t, root, "kustomization.yaml", `configMapGenerator: + - name: coffee-flags + envs: + - flags.env + - name: coffee-menu + envs: + - menu.env +generatorOptions: + disableNameSuffixHash: true +`) + seedFile(t, root, "flags.env", "ENABLE_VOUCHERS=true\n") + seedFile(t, root, "menu.env", "ESPRESSO=275\n") + commitCorpusWorktree(t, worktree, "seed two generators") + deleted := func(name string) Event { + return Event{ + Identifier: types.NewResourceIdentifier("", "v1", "configmaps", "coffee", name), + Operation: "DELETE", + } + } + + changed, err := flushEventsForTest(t, newContentWriter(types.SensitiveResourcePolicy{}), worktree, corpusMapper(), + deleted("coffee-flags"), deleted("coffee-menu")) + require.NoError(t, err) + assert.True(t, changed) + assert.NoFileExists(t, filepath.Join(root, "flags.env")) + assert.NoFileExists(t, filepath.Join(root, "menu.env")) + assertFileBytes(t, filepath.Join(root, "kustomization.yaml"), + "generatorOptions:\n disableNameSuffixHash: true\n", "both entries are gone") +} diff --git a/internal/git/manifestedit/kustomization.go b/internal/git/manifestedit/kustomization.go index d4b0b7e6..621efd14 100644 --- a/internal/git/manifestedit/kustomization.go +++ b/internal/git/manifestedit/kustomization.go @@ -482,33 +482,32 @@ type LiteralEdit struct { Remove bool } -// EditConfigMapGeneratorLiterals applies literal edits to configMapGenerator[entryIndex], which -// must be named entryName, preserving comments and key order. An item is edited in place, so its -// position in the list and every other item keep their bytes. All-or-nothing like its siblings: -// a drifted entry skips the whole call with a diagnostic. +// GeneratorEntryID names a configMapGenerator entry by its own name and namespace fields, as +// kustomize names what it generates: two entries may share a name in different namespaces. +type GeneratorEntryID struct { + Name, Namespace string +} + +// EditConfigMapGeneratorLiterals applies literal edits to the configMapGenerator entry id names, +// preserving comments and key order. An item is edited in place, so its position in the list and +// every other item keep their bytes. All-or-nothing like its siblings: an entry the file does not +// hold exactly once skips the whole call with a diagnostic. func EditConfigMapGeneratorLiterals( path string, content []byte, - entryIndex int, - entryName string, + id GeneratorEntryID, edits []LiteralEdit, ) (EditResult, []Diagnostic) { - return editGeneratorEntry(path, content, entryIndex, entryName, func(_, entry *yaml.Node) error { - return applyLiteralEdits(entry, edits) + return editGeneratorEntry(path, content, id, func(_, seq *yaml.Node, index int) error { + return applyLiteralEdits(seq.Content[index], edits) }) } -// RemoveConfigMapGeneratorEntry drops configMapGenerator[entryIndex], which must be named -// entryName. An emptied configMapGenerator: drops the key, as an emptied patches: does. -func RemoveConfigMapGeneratorEntry( - path string, - content []byte, - entryIndex int, - entryName string, -) (EditResult, []Diagnostic) { - return editGeneratorEntry(path, content, entryIndex, entryName, func(root, _ *yaml.Node) error { - seq := nodeMapGet(root, configMapGeneratorKey) - seq.Content = append(seq.Content[:entryIndex], seq.Content[entryIndex+1:]...) +// RemoveConfigMapGeneratorEntry drops the configMapGenerator entry id names. An emptied +// configMapGenerator: drops the key, as an emptied patches: does. +func RemoveConfigMapGeneratorEntry(path string, content []byte, id GeneratorEntryID) (EditResult, []Diagnostic) { + return editGeneratorEntry(path, content, id, func(root, seq *yaml.Node, index int) error { + seq.Content = append(seq.Content[:index], seq.Content[index+1:]...) if len(seq.Content) == 0 { nodeMapDelete(root, configMapGeneratorKey) } @@ -518,14 +517,14 @@ func RemoveConfigMapGeneratorEntry( const configMapGeneratorKey = "configMapGenerator" -// editGeneratorEntry locates configMapGenerator[entryIndex], verifies its name, runs edit on it, -// and re-encodes the kustomization. +// editGeneratorEntry finds the one configMapGenerator entry id names in content as it stands, runs +// edit on it, and re-encodes the kustomization. The entry is found by identity, not by position, so +// an earlier edit of the same file, such as a removed entry before it, cannot point it elsewhere. func editGeneratorEntry( path string, content []byte, - entryIndex int, - entryName string, - edit func(root, entry *yaml.Node) error, + id GeneratorEntryID, + edit func(root, seq *yaml.Node, index int) error, ) (EditResult, []Diagnostic) { skip := func(format string, args ...interface{}) (EditResult, []Diagnostic) { return EditResult{Content: content, Mode: EditSkipped}, @@ -537,17 +536,24 @@ func editGeneratorEntry( } target := docs[idx].body seq := nodeMapGet(root, configMapGeneratorKey) - if seq == nil || seq.Kind != yaml.SequenceNode || entryIndex < 0 || entryIndex >= len(seq.Content) { - return skip("kustomization %s has no configMapGenerator entry %d", path, entryIndex) + if seq == nil || seq.Kind != yaml.SequenceNode { + return skip("kustomization %s has no configMapGenerator", path) } - entry := seq.Content[entryIndex] - if entry.Kind != yaml.MappingNode { - return skip("kustomization %s: configMapGenerator entry %d is not a mapping", path, entryIndex) + index := -1 + for i, entry := range seq.Content { + if entry.Kind != yaml.MappingNode || scalarField(entry, "name") != id.Name || + scalarField(entry, "namespace") != id.Namespace { + continue + } + if index >= 0 { + return skip("kustomization %s holds more than one configMapGenerator entry %s", path, id) + } + index = i } - if name := nodeMapGet(entry, "name"); name == nil || name.Value != entryName { - return skip("kustomization %s: configMapGenerator entry %d is not named %q", path, entryIndex, entryName) + if index < 0 { + return skip("kustomization %s holds no configMapGenerator entry %s", path, id) } - if err := edit(root, entry); err != nil { + if err := edit(root, seq, index); err != nil { return skip("kustomization %s: %v", path, err) } encoded, err := encodeNode(root) @@ -562,6 +568,22 @@ func editGeneratorEntry( return EditResult{Content: []byte(joinDocuments(docs)), Mode: EditPatched}, nil } +// String names the entry in a diagnostic, with its namespace when it has one. +func (id GeneratorEntryID) String() string { + if id.Namespace == "" { + return fmt.Sprintf("named %q", id.Name) + } + return fmt.Sprintf("named %q in namespace %q", id.Name, id.Namespace) +} + +// scalarField is the value of a mapping's scalar field, "" when it has none. +func scalarField(mapping *yaml.Node, key string) string { + if v := nodeMapGet(mapping, key); v != nil && v.Kind == yaml.ScalarNode { + return v.Value + } + return "" +} + // applyLiteralEdits edits an entry's literals: items, creating the list for a first literal. func applyLiteralEdits(entry *yaml.Node, edits []LiteralEdit) error { seq := nodeMapGet(entry, "literals") diff --git a/internal/git/manifestedit/kustomization_test.go b/internal/git/manifestedit/kustomization_test.go index 1fa7ebae..24633e6d 100644 --- a/internal/git/manifestedit/kustomization_test.go +++ b/internal/git/manifestedit/kustomization_test.go @@ -365,24 +365,54 @@ func TestAppendKustomizationResource_RefusalsLeaveContentUntouched(t *testing.T) } } -// A configMapGenerator edit pins the entry by index AND name, so a drifted file is skipped rather -// than mis-edited, exactly as an images: edit is. -func TestConfigMapGeneratorEdits_SkipADriftedEntry(t *testing.T) { +// A configMapGenerator edit finds its entry by name and namespace in the bytes as they stand, so an +// entry the file does not hold, or holds twice, is skipped rather than mis-edited. +func TestConfigMapGeneratorEdits_SkipAnEntryNotHeldOnce(t *testing.T) { content := []byte("configMapGenerator:\n - name: menu\n literals:\n - A=1\n") - res, _ := EditConfigMapGeneratorLiterals("kustomization.yaml", content, 0, "flags", + res, _ := EditConfigMapGeneratorLiterals("kustomization.yaml", content, GeneratorEntryID{Name: "flags"}, []LiteralEdit{{Key: "A", Value: "2"}}) require.Equal(t, EditSkipped, res.Mode) - res, _ = RemoveConfigMapGeneratorEntry("kustomization.yaml", content, 1, "menu") + res, _ = RemoveConfigMapGeneratorEntry("kustomization.yaml", content, + GeneratorEntryID{Name: "menu", Namespace: "eu"}) + require.Equal(t, EditSkipped, res.Mode, "the entry has no namespace") + twice := []byte("configMapGenerator:\n - name: menu\n - name: menu\n") + res, _ = RemoveConfigMapGeneratorEntry("kustomization.yaml", twice, GeneratorEntryID{Name: "menu"}) require.Equal(t, EditSkipped, res.Mode) - res, _ = EditConfigMapGeneratorLiterals("kustomization.yaml", content, 0, "menu", + res, _ = EditConfigMapGeneratorLiterals("kustomization.yaml", content, GeneratorEntryID{Name: "menu"}, []LiteralEdit{{Key: "A", Remove: true}, {Key: "Z", Remove: true}}) require.Equal(t, EditPatched, res.Mode) require.Equal(t, "configMapGenerator:\n - name: menu\n", string(res.Content), "an emptied literals: goes, and removing an absent key is a no-op") } +// TestConfigMapGeneratorEdits_FollowAnEarlierEditOfTheFile: removing the first of two entries moves +// the second up, and the edits that follow still find it; entries of one name in two namespaces are +// told apart. +func TestConfigMapGeneratorEdits_FollowAnEarlierEditOfTheFile(t *testing.T) { + content := []byte(`configMapGenerator: + - name: menu + namespace: eu + literals: [A=1] + - name: menu + namespace: us + literals: [A=1] + - name: flags + literals: [B=1] +`) + res, diags := RemoveConfigMapGeneratorEntry("kustomization.yaml", content, + GeneratorEntryID{Name: "menu", Namespace: "eu"}) + require.Equal(t, EditPatched, res.Mode, "%v", diags) + res, diags = EditConfigMapGeneratorLiterals("kustomization.yaml", res.Content, GeneratorEntryID{Name: "flags"}, + []LiteralEdit{{Key: "B", Value: "2"}}) + require.Equal(t, EditPatched, res.Mode, "%v", diags) + res, diags = RemoveConfigMapGeneratorEntry("kustomization.yaml", res.Content, + GeneratorEntryID{Name: "menu", Namespace: "us"}) + require.Equal(t, EditPatched, res.Mode, "%v", diags) + require.Equal(t, "configMapGenerator:\n - name: flags\n literals: [B=2]\n", string(res.Content)) +} + // Comments in a literals: list are the author's, and an edit keeps them: an edited item keeps its // head and line comments, a removed item takes only its own, and a comment below the list stays // below it whichever item ends up last. @@ -442,8 +472,12 @@ func TestEditConfigMapGeneratorLiterals_KeepsComments(t *testing.T) { } for name, tc := range cases { t.Run(name, func(t *testing.T) { - res, diags := EditConfigMapGeneratorLiterals("kustomization.yaml", []byte(in), 0, "branding", - []LiteralEdit{tc.edit}) + res, diags := EditConfigMapGeneratorLiterals( + "kustomization.yaml", + []byte(in), + GeneratorEntryID{Name: "branding"}, + []LiteralEdit{tc.edit}, + ) require.Equal(t, EditPatched, res.Mode, "%v", diags) require.Equal(t, tc.want, string(res.Content)) }) diff --git a/internal/manifestanalyzer/generator_write.go b/internal/manifestanalyzer/generator_write.go index 0d51e1a9..0e58e488 100644 --- a/internal/manifestanalyzer/generator_write.go +++ b/internal/manifestanalyzer/generator_write.go @@ -53,8 +53,13 @@ type GeneratedObject struct { // Roots are the render roots that render this entry, in any namespace, sorted. Roots []string // SharedInputs are the entry's input files something else also reads: another generator - // entry, a resources: entry or a patch. A delete of the object keeps them. + // entry, a resources: entry, a patch or a release's values files. A delete of the object + // keeps them. SharedInputs map[string]struct{} + // ValuesInputs are the entry's input files a release names as a values file: an Argo CD + // Application's helm.valueFiles or a Flux HelmRelease's valuesFiles. A write to one is + // refused, since the render shows nothing of what it does to the release. + ValuesInputs map[string]struct{} // Resource is the API-side identity of the rendered object, as a document's ResourceIdentity // is: set only when the registry resolves its GVK to a followable resource. The resync sweep // drops only a generated object it can name this way. @@ -109,8 +114,13 @@ type generatedRender struct { } // generatedObjects groups the generated renders of a scan into one GeneratedObject per entry and -// namespace, each naming every root that renders the entry. -func generatedObjects(renders []generatedRender, kusts map[string]*kustomizationDoc) []*GeneratedObject { +// namespace, each naming every root that renders the entry. valuesFiles are the files a release +// names as a values file (helmValueFileRefs). +func generatedObjects( + renders []generatedRender, + kusts map[string]*kustomizationDoc, + valuesFiles map[string]struct{}, +) []*GeneratedObject { type entryKey struct{ in, kind, name string } type objectKey struct { entry entryKey @@ -118,7 +128,7 @@ func generatedObjects(renders []generatedRender, kusts map[string]*kustomization } roots := map[entryKey]map[string]struct{}{} byObject := map[objectKey]*GeneratedObject{} - inputUses := inputUseCounts(kusts) + inputUses := inputUseCounts(kusts, valuesFiles) var order []objectKey for _, r := range renders { ek := entryKey{in: r.in, kind: r.object.GetKind(), name: r.object.GetName()} @@ -136,6 +146,7 @@ func generatedObjects(renders []generatedRender, kusts map[string]*kustomization Entry: entry, Rendered: r.object, SharedInputs: sharedInputsOf(entry, inputUses), + ValuesInputs: valuesInputsOf(entry, valuesFiles), } order = append(order, ok) } @@ -149,8 +160,8 @@ func generatedObjects(renders []generatedRender, kusts map[string]*kustomization } // inputUseCounts counts, per file, how many generator entries, resources: entries and patches of -// the scan read it. -func inputUseCounts(kusts map[string]*kustomizationDoc) map[string]int { +// the scan read it, and one more for a file a release names as a values file. +func inputUseCounts(kusts map[string]*kustomizationDoc, valuesFiles map[string]struct{}) map[string]int { uses := map[string]int{} for _, doc := range kusts { for _, entry := range doc.configMapGenerators { @@ -169,6 +180,9 @@ func inputUseCounts(kusts map[string]*kustomizationDoc) map[string]int { for p := range reachedResourceFiles(kusts) { uses[p]++ } + for p := range valuesFiles { + uses[p]++ + } return uses } @@ -186,6 +200,20 @@ func sharedInputsOf(entry *ConfigMapGeneratorEntry, uses map[string]int) map[str return out } +// valuesInputsOf is the entry's inputs a release names as a values file. +func valuesInputsOf(entry *ConfigMapGeneratorEntry, valuesFiles map[string]struct{}) map[string]struct{} { + if entry == nil { + return nil + } + out := map[string]struct{}{} + for _, p := range entry.inputPaths() { + if _, ok := valuesFiles[p]; ok { + out[p] = struct{}{} + } + } + return out +} + // generatorEntryFor finds the entry that rendered obj: the one entry of its name, or, when two // entries share a name, the one whose own namespace is the object's. func generatorEntryFor(doc *kustomizationDoc, obj *unstructured.Unstructured) *ConfigMapGeneratorEntry { @@ -250,12 +278,19 @@ var errGeneratedWrite = errors.New("no destination for the change") // the inputs of the generator that renders gen. writeScope is the write jail (empty: everything // is writable), and read returns an input file's current bytes. An error is a refusal and names // the reason; nothing has been written. +// +// A desired object the generator already renders plans nothing, and so needs no input it may +// write: a generator in a read-only base, or one two render roots share, is matched as it stands. +// The oracle still proves the render equals the live object. func PlanGeneratedWrite( gen *GeneratedObject, desired *unstructured.Unstructured, writeScope string, read func(path string) ([]byte, bool), ) ([]GeneratedWrite, error) { + if rendersUnchanged(gen.Rendered, desired) { + return nil, nil + } entry, err := writableEntry(gen, writeScope) if err != nil { return nil, err @@ -275,7 +310,9 @@ func PlanGeneratedWrite( errGeneratedWrite) } - plan := generatorPlan{entry: entry, writeScope: writeScope, read: read, files: map[string][]byte{}} + plan := generatorPlan{ + entry: entry, writeScope: writeScope, valuesInputs: gen.ValuesInputs, read: read, files: map[string][]byte{}, + } liveData, _, _ := unstructured.NestedStringMap(desired.Object, "data") for _, key := range sortedKeysOfSet(unionStringKeys(liveData, current.data)) { if err := plan.key(key, liveData, current); err != nil { @@ -288,6 +325,22 @@ func PlanGeneratedWrite( return plan.writes(read), nil } +// rendersUnchanged reports whether rendered, the generator's render as Git stands, already is +// desired, setting the namespace aside as changedOutsideData does: the oracle compares it. +func rendersUnchanged(rendered, desired *unstructured.Unstructured) bool { + if len(changedOutsideData(desired, rendered)) > 0 { + return false + } + for _, field := range []string{"data", "binaryData"} { + want, _, _ := unstructured.NestedStringMap(desired.Object, field) + have, _, _ := unstructured.NestedStringMap(rendered.Object, field) + if !equalStringMaps(want, have) { + return false + } + } + return true +} + // writableEntry is the entry behind gen, or the refusal that says why its inputs are not this // target's to write: no single entry, an entry two render roots share, or one in a read-only base. func writableEntry(gen *GeneratedObject, writeScope string) (*ConfigMapGeneratorEntry, error) { @@ -324,8 +377,7 @@ func PlanGeneratedDelete( if !ok { return nil, fmt.Errorf("%w: cannot read %s", errGeneratedWrite, entry.KustomizationPath) } - res, diags := manifestedit.RemoveConfigMapGeneratorEntry( - entry.KustomizationPath, kustomization, entry.Index, entry.Name) + res, diags := manifestedit.RemoveConfigMapGeneratorEntry(entry.KustomizationPath, kustomization, entry.ID()) if res.Mode != manifestedit.EditPatched { return nil, fmt.Errorf("%w: the entry cannot be removed from %s%s", errGeneratedWrite, entry.KustomizationPath, diagnosticSuffix(diags)) @@ -366,8 +418,10 @@ func diagnosticSuffix(diags []manifestedit.Diagnostic) string { type generatorPlan struct { entry *ConfigMapGeneratorEntry writeScope string - read func(string) ([]byte, bool) - files map[string][]byte + // valuesInputs are the inputs a release names as a values file, which no write may touch. + valuesInputs map[string]struct{} + read func(string) ([]byte, bool) + files map[string][]byte // literals are the literals: edits, applied to the kustomization in one call at the end. literals []manifestedit.LiteralEdit } @@ -429,8 +483,7 @@ func (p *generatorPlan) applyLiterals() error { return nil } path := p.entry.KustomizationPath - res, diags := manifestedit.EditConfigMapGeneratorLiterals( - path, p.content(path), p.entry.Index, p.entry.Name, p.literals) + res, diags := manifestedit.EditConfigMapGeneratorLiterals(path, p.content(path), p.entry.ID(), p.literals) switch res.Mode { case manifestedit.EditPatched: p.files[path] = res.Content @@ -469,11 +522,16 @@ func (p *generatorPlan) setEnv(path, key, value string) error { return nil } -// writable refuses an input file outside the write jail. +// writable refuses an input file outside the write jail, and one a release reads as a values file: +// the oracle renders the kustomization, and cannot see what the edit does to the release. func (p *generatorPlan) writable(path string) error { if p.writeScope != "" && !pathWithin(path, p.writeScope) { return fmt.Errorf("%w: %s is outside spec.path", errGeneratedWrite, path) } + if _, values := p.valuesInputs[path]; values { + return fmt.Errorf("%w: %s is also a release's values file, which the render cannot check", + errGeneratedWrite, path) + } return nil } diff --git a/internal/manifestanalyzer/generators.go b/internal/manifestanalyzer/generators.go index 625d397b..f338a137 100644 --- a/internal/manifestanalyzer/generators.go +++ b/internal/manifestanalyzer/generators.go @@ -35,8 +35,6 @@ const featureGeneratorBehavior = "configMapGenerator-behavior" type ConfigMapGeneratorEntry struct { // KustomizationPath is the kustomization file declaring the entry (slash). KustomizationPath string - // Index is the entry's position in configMapGenerator:. - Index int // Name and Namespace are the entry's own fields. Namespace is usually empty: a namespace: // transformer supplies it. Name, Namespace string @@ -55,6 +53,12 @@ type GeneratorFileSource struct { Path string } +// ID names the entry in its kustomization: by its name and namespace, which an edit of the file +// leaves as they are, where its position may move. +func (e ConfigMapGeneratorEntry) ID() manifestedit.GeneratorEntryID { + return manifestedit.GeneratorEntryID{Name: e.Name, Namespace: e.Namespace} +} + // inputPaths lists every file the entry reads. func (e ConfigMapGeneratorEntry) inputPaths() []string { out := make([]string, 0, len(e.Files)+len(e.Envs)) @@ -75,7 +79,7 @@ func configMapGenerators(k kustypes.Kustomization, kustomizationPath string) ([] globalHashOff := k.GeneratorOptions != nil && k.GeneratorOptions.DisableNameSuffixHash features := map[string]struct{}{} var out []ConfigMapGeneratorEntry - for i, g := range k.ConfigMapGenerator { + for _, g := range k.ConfigMapGenerator { switch g.Behavior { case "", "create": // kustomize ORs the two (types.MergeGlobalOptionsIntoLocal): either one disables it. @@ -87,7 +91,7 @@ func configMapGenerators(k kustypes.Kustomization, kustomizationPath string) ([] // it would not make the entry adoptable. features[featureGeneratorBehavior] = struct{}{} } - entry, ok := generatorEntry(g, kustomizationPath, dir, i) + entry, ok := generatorEntry(g, kustomizationPath, dir) if ok { out = append(out, entry) } @@ -97,14 +101,9 @@ func configMapGenerators(k kustypes.Kustomization, kustomizationPath string) ([] // generatorEntry resolves one entry's sources. ok is false when a files: source does not parse // the way kustomize parses it. -func generatorEntry( - g kustypes.ConfigMapArgs, - kustomizationPath, dir string, - index int, -) (ConfigMapGeneratorEntry, bool) { +func generatorEntry(g kustypes.ConfigMapArgs, kustomizationPath, dir string) (ConfigMapGeneratorEntry, bool) { entry := ConfigMapGeneratorEntry{ KustomizationPath: kustomizationPath, - Index: index, Name: g.Name, Namespace: g.Namespace, Literals: append([]string(nil), g.LiteralSources...), diff --git a/internal/manifestanalyzer/generators_test.go b/internal/manifestanalyzer/generators_test.go index 0aca5ef8..20153574 100644 --- a/internal/manifestanalyzer/generators_test.go +++ b/internal/manifestanalyzer/generators_test.go @@ -406,7 +406,8 @@ func TestLiteralValue_AgreesWithKustomize(t *testing.T) { } kustomization := []byte("configMapGenerator:\n - name: probe\n literals: [SEED=x]\n" + "generatorOptions:\n disableNameSuffixHash: true\n") - res, diags := manifestedit.EditConfigMapGeneratorLiterals("kustomization.yaml", kustomization, 0, "probe", edits) + res, diags := manifestedit.EditConfigMapGeneratorLiterals("kustomization.yaml", kustomization, + manifestedit.GeneratorEntryID{Name: "probe"}, edits) require.Equal(t, manifestedit.EditPatched, res.Mode, "%v", diags) rendered, err := renderRoot([]manifestedit.FileContent{{Path: "kustomization.yaml", Content: res.Content}}, ".") @@ -444,3 +445,133 @@ func TestBuildPlan_SweepsAGeneratedObject(t *testing.T) { structureOnly := buildStoreFS(context.Background(), fsys, nil, WriterAllowlist()) require.Empty(t, drops(structureOnly, allInScope), "a scan with no registry names nothing to drop") } + +// A live object the generator already renders plans no write, so it needs no input it may write: +// a generator two render roots share, or one in a read-only base, is matched as it stands. +func TestPlanGeneratedWrite_AnUnchangedObjectNeedsNoWritableEntry(t *testing.T) { + sharedBase := fstest.MapFS{ + "base/kustomization.yaml": {Data: []byte("configMapGenerator:\n - name: menu\n envs: [menu.env]\n" + + "generatorOptions:\n disableNameSuffixHash: true\n")}, + "base/menu.env": {Data: []byte("A=1\n")}, + "overlays/a/kustomization.yaml": {Data: []byte("resources: [../../base]\n")}, + "overlays/b/kustomization.yaml": {Data: []byte("resources: [../../base]\n")}, + } + cases := map[string]struct { + fsys fstest.MapFS + live *unstructured.Unstructured + writeScope string + }{ + "shared by two render roots": { + fsys: sharedBase, + live: liveConfigMap("", "menu", map[string]interface{}{"A": "1"}), + }, + "in a read-only base": { + fsys: coffeeFS(), + live: liveConfigMap("coffee", "coffee-flags", map[string]interface{}{ + "ENABLE_VOUCHERS": "true", "SHOW_BANNER": "true", + }), + writeScope: "overlays/prod", + }, + } + for name, tc := range cases { + t.Run(name, func(t *testing.T) { + generated := generatedFor(t, tc.fsys, tc.live) + require.NotEmpty(t, generated) + writes, err := PlanGeneratedWrite(generated[0], tc.live, tc.writeScope, func(string) ([]byte, bool) { + t.Fatal("an unchanged object reads no input") + return nil, false + }) + require.NoError(t, err) + require.Empty(t, writes) + }) + } +} + +// An input a HelmRelease also names as a values file is the release's too, and the render cannot +// show what an edit does to the release: a delete keeps it, and a write to it is refused. +func TestGeneratedInputs_AReleasesValuesFileIsKeptAndNeverWritten(t *testing.T) { + fsys := fstest.MapFS{ + "kustomization.yaml": {Data: []byte("resources: [release.yaml]\n" + + "configMapGenerator:\n - {name: chart-values, files: [values.yaml], literals: [MODE=a]}\n" + + "generatorOptions:\n disableNameSuffixHash: true\n")}, + "release.yaml": {Data: []byte("apiVersion: helm.toolkit.fluxcd.io/v2\nkind: HelmRelease\n" + + "metadata: {name: shop}\nspec:\n chart:\n spec:\n chart: shop\n valuesFiles: [values.yaml]\n")}, + "values.yaml": {Data: []byte("replicas: 2\n")}, + } + read := func(p string) ([]byte, bool) { + f, ok := fsys[p] + if !ok { + return nil, false + } + return f.Data, true + } + live := liveConfigMap("", "chart-values", map[string]interface{}{"values.yaml": "replicas: 2\n", "MODE": "a"}) + generated := generatedFor(t, fsys, live) + require.Len(t, generated, 1) + + writes, err := PlanGeneratedDelete(generated[0], "", read) + require.NoError(t, err) + require.Equal(t, []string{"kustomization.yaml"}, pathsOf(toFileContents(writes)), "values.yaml stays") + + _, err = PlanGeneratedWrite( + generated[0], + liveConfigMap( + "", + "chart-values", + map[string]interface{}{"values.yaml": "replicas: 3\n", "MODE": "a"}, + ), + "", + read, + ) + require.ErrorContains(t, err, "values.yaml is also a release's values file") + + writes, err = PlanGeneratedWrite( + generated[0], + liveConfigMap( + "", + "chart-values", + map[string]interface{}{"values.yaml": "replicas: 2\n", "MODE": "b"}, + ), + "", + read, + ) + require.NoError(t, err, "a key of another input is written as before") + require.Equal(t, []string{"kustomization.yaml"}, pathsOf(toFileContents(writes))) +} + +func toFileContents(writes []GeneratedWrite) []manifestedit.FileContent { + out := make([]manifestedit.FileContent, 0, len(writes)) + for _, w := range writes { + out = append(out, manifestedit.FileContent{Path: w.Path, Content: w.Content}) + } + return out +} + +// Two generated objects of one name, from one kustomization, in two namespaces are two objects to +// the oracle: removing one is not refused because the other still renders, and a write to one is +// checked against that one. +func TestVerifyBatchRenders_GeneratedObjectsInTwoNamespaces(t *testing.T) { + kustomization := func(entries string) manifestedit.FileContent { + return manifestedit.FileContent{Path: "kustomization.yaml", Content: []byte("configMapGenerator:\n" + entries + + "generatorOptions:\n disableNameSuffixHash: true\n")} + } + eu := " - {name: menu, namespace: eu, literals: [A=1]}\n" + us := " - {name: menu, namespace: us, literals: [A=1]}\n" + before := []manifestedit.FileContent{kustomization(eu + us)} + + removed := WriteIntent{SourcePath: "kustomization.yaml", Kind: "ConfigMap", Name: "menu", Namespace: "eu", + Removed: true} + require.NoError(t, VerifyBatchRenders(before, []manifestedit.FileContent{kustomization(us)}, + []WriteIntent{removed}, VerifyOptions{})) + + written := liveConfigMap("us", "menu", map[string]interface{}{"A": "2"}) + intent := WriteIntent{SourcePath: "kustomization.yaml", Kind: "ConfigMap", Name: "menu", Namespace: "us", + Desired: written, Governed: true} + after := []manifestedit.FileContent{kustomization(eu + " - {name: menu, namespace: us, literals: [A=2]}\n")} + require.NoError(t, VerifyBatchRenders(before, after, []WriteIntent{intent}, VerifyOptions{})) + + intent.Namespace = "eu" + written.SetNamespace("eu") + require.Error(t, VerifyBatchRenders(before, after, []WriteIntent{intent}, VerifyOptions{}), + "eu did not move, and us moved without an intent") +} diff --git a/internal/manifestanalyzer/override_chain.go b/internal/manifestanalyzer/override_chain.go index bf9ec341..becafe21 100644 --- a/internal/manifestanalyzer/override_chain.go +++ b/internal/manifestanalyzer/override_chain.go @@ -29,12 +29,18 @@ import ( const reasonRenderFailed manifestedit.DiagReason = "kustomize-build-failed" // chainKey identifies a rendered document: the source file kustomize says produced -// it, plus its kind and name. Namespace is deliberately absent — a namespace -// transformer rewrites it, and the source document may not carry one at all. +// it, plus its kind and name. Namespace is deliberately absent for a document — a +// namespace transformer rewrites it, and the source document may not carry one at all. +// +// A generated object is the exception. Its origin is the kustomization whose generator +// made it, and one kustomization can generate two objects of one name in different +// namespaces (entries with their own namespace:), so namespace, as rendered, tells +// them apart. It is empty for everything else. type chainKey struct { originPath string kind string name string + namespace string } // renderRoots returns the kustomization directories no other kustomization in the diff --git a/internal/manifestanalyzer/render_verify.go b/internal/manifestanalyzer/render_verify.go index 9ed74612..f3c65358 100644 --- a/internal/manifestanalyzer/render_verify.go +++ b/internal/manifestanalyzer/render_verify.go @@ -49,6 +49,10 @@ type WriteIntent struct { SourcePath string // Kind and Name identify the document within that file. Kind, Name string + // Namespace is set only for a generated object, whose SourcePath is the kustomization that + // generated it: the namespace the render gives it, which tells apart two objects of one name + // that kustomization generates (see chainKey). + Namespace string // Desired is the live object the document must render to. It is the whole point of // the check, and it is nil only when Removed or Unchecked says there is nothing to // check against. @@ -72,7 +76,7 @@ type WriteIntent struct { } func (w WriteIntent) key() chainKey { - return chainKey{originPath: filepathToSlash(w.SourcePath), kind: w.Kind, name: w.Name} + return chainKey{originPath: filepathToSlash(w.SourcePath), kind: w.Kind, name: w.Name, namespace: w.Namespace} } // RenderRefusedError is the oracle's verdict: the bytes the flush was about to commit do @@ -314,11 +318,15 @@ func sameObject(a, b *unstructured.Unstructured) bool { // renderedByKey indexes a render by the document each object came from. The key carries the // ORIGIN FILE, not just kind/name, so two same-named objects rendered from different source // files stay distinct. A generated object is keyed by the kustomization that generated it, which -// is also the SourcePath its write intent names. +// is also the SourcePath its write intent names, and by its namespace. func renderedByKey(objects []renderedObject) map[chainKey]renderedObject { out := make(map[chainKey]renderedObject, len(objects)) for _, o := range objects { - out[chainKey{originPath: o.sourceKey(), kind: o.Object.GetKind(), name: o.Object.GetName()}] = o + key := chainKey{originPath: o.sourceKey(), kind: o.Object.GetKind(), name: o.Object.GetName()} + if o.OriginPath == "" { + key.namespace = o.Object.GetNamespace() + } + out[key] = o } return out } diff --git a/internal/manifestanalyzer/store.go b/internal/manifestanalyzer/store.go index ecd813d4..1a4947b7 100644 --- a/internal/manifestanalyzer/store.go +++ b/internal/manifestanalyzer/store.go @@ -462,7 +462,9 @@ func buildStore( renderInputs: reachedResourceFiles(kusts), renderedDocuments: renderedDocumentsOf(ovAssignments), GeneratorInputs: pathSet(scan.InputFiles), - generated: generatedObjects(generated, kusts), + // A generator input is out of the YAML files, so the values files it may be are looked + // for among the render's. + generated: generatedObjects(generated, kusts, helmValueFileRefs(scan.RenderFiles())), } store.resolveGeneratedResources(ctx, lookup, settings.declaredNamespace) From c838fceb129e823ba1902b7ab299e29b3f77a54d Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 14:55:45 +0000 Subject: [PATCH 06/33] fix(commitrequest): settle Applied by the commit the applier applied; catchUp is AfterPush - spec.catchUp "ImmediatelyAfterPush" is "AfterPush". - A save waiting for its apply keeps the commit it pushed, and is settled Applied only when the applied revision holds that commit. One whose commit a force-push dropped from the branch is False, CommitDropped; one whose applied revision the checkout does not hold yet keeps waiting while the remote is looked at. - A save whose GitTarget names no applier claim, or stopped naming the one asked, is Applied=True (NoApplierClaim, ApplierGone): nothing applies the branch, and the cluster already holds what was saved. - catchUpAfter "0s" drops a converge only its interval left pending, waiting behind the barrier or a backoff; one a save asked for stays. - The descriptions say when an apply is asked for rather than promise how long the applier may lag, name the latest tip rather than the request's commit, explain Ready, Pushed and Applied together, and list the timed and save-asked catch-ups among the claims' triggers. Co-Authored-By: Claude Opus 5.5 (1M context) --- .coverage-baseline | 2 +- api/v1alpha3/argocdclaim_types.go | 22 ++- api/v1alpha3/commitrequest_types.go | 67 ++++--- api/v1alpha3/fluxclaim_types.go | 23 ++- .../bases/configbutler.ai_argocdclaims.yaml | 22 ++- .../bases/configbutler.ai_commitrequests.yaml | 61 ++++--- .../crd/bases/configbutler.ai_fluxclaims.yaml | 23 ++- docs/UPGRADING.md | 12 +- docs/configuration.md | 31 ++-- docs/design/apply-trigger.md | 17 +- hack/crdfields/fields.yaml | 24 +-- internal/controller/commitrequest_apply.go | 4 +- .../controller/commitrequest_apply_test.go | 42 +++-- .../controller/commitrequest_controller.go | 4 +- .../commitrequest_controller_unit_test.go | 2 +- internal/controller/constants.go | 2 +- internal/git/applier_claim.go | 25 ++- internal/git/apply_converge_test.go | 136 ++++++++++++++- internal/git/branch_worker.go | 4 +- internal/git/commit_request_attach.go | 2 +- internal/git/save_apply.go | 165 ++++++++++++++---- test/e2e/flux_applier_e2e_test.go | 4 +- 22 files changed, 490 insertions(+), 204 deletions(-) diff --git a/.coverage-baseline b/.coverage-baseline index f32b65c8..b0184ecd 100644 --- a/.coverage-baseline +++ b/.coverage-baseline @@ -1 +1 @@ -80.3 +80.6 diff --git a/api/v1alpha3/argocdclaim_types.go b/api/v1alpha3/argocdclaim_types.go index f237d90d..eeedf011 100644 --- a/api/v1alpha3/argocdclaim_types.go +++ b/api/v1alpha3/argocdclaim_types.go @@ -60,14 +60,16 @@ type ArgoCDClaimSpec struct { // cost more than a no-op apply, and Argo CD's own refresh already compares the live state // against the tip without a sync. Only the last sync's revision lags. - // CatchUpAfter is how long the Application may stay behind the branch - // after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a - // push changes nothing on the cluster, which already holds the edit, but - // leaves the Application's last sync at the commit before it. Once the - // interval has passed, the tip is synced, hooks included: a burst of saves - // costs one sync, and an Application at the tip costs none. "0s" never - // syncs for this reason. Otherwise at least "10s" and at most "24h". - // Defaults to "0s". + // CatchUpAfter is how long the Application stays behind the branch after + // a push of GitOps Reverser's own before a sync of the tip is asked for, + // as a Go duration ("5m"). Such a push changes nothing on the cluster, + // which already holds the edit, but leaves the Application's last sync at + // the commit before it. The interval starts when the Application is first + // seen behind, so a burst of saves costs one sync, hooks included, and one + // at the tip costs none. The sync can still wait for startup, a retry + // backoff or edits being published. "0s" never asks for this reason, and + // drops a sync it asked for that has not started. Otherwise at least "10s" + // and at most "24h". Defaults to "0s". // +optional // +kubebuilder:validation:Type=string // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" @@ -182,7 +184,9 @@ type ArgoCDClaimSource struct { // - after a refused edit, on a GitTarget with spec.onRefusal "Reapply"; // - after a sync reverted an edit made through the API; // - after objects were created through the API in its folders; -// - once at startup, and when a GitTarget first names this ArgoCDClaim. +// - once at startup, and when a GitTarget first names this ArgoCDClaim; +// - once spec.catchUpAfter passed since a push of GitOps Reverser's own; +// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type ArgoCDClaim struct { diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index d9fb391b..9f086d55 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -72,16 +72,16 @@ type CommitRequestSpec struct { // +kubebuilder:default=Resolve WhenNothingToCommit NothingToCommitAction `json:"whenNothingToCommit,omitempty"` - // CatchUp decides when the GitTarget's applier is asked to apply the commit - // this request pushes. "AsDefinedOnClaim" leaves it to the applier claim's - // spec.catchUpAfter. "ImmediatelyAfterPush" asks as soon as the commit is - // pushed, which costs one apply per request. The cluster already holds the - // saved objects; the apply brings the applier's own state, such as a - // Kustomization's status.lastAppliedRevision, to the commit. A request that - // pushes no commit, or whose GitTarget names no applier claim, asks for - // nothing. Defaults to "AsDefinedOnClaim". + // CatchUp decides when the GitTarget's applier is asked to apply the branch + // after this request pushes a commit. "AsDefinedOnClaim" leaves it to the + // applier claim's spec.catchUpAfter. "AfterPush" asks right after the push. + // The apply takes the latest branch tip, which may hold later commits, and + // saves pushed close together can share one apply; the Applied condition + // reports how it went. The cluster already holds the saved objects: + // the apply brings the applier's own state, such as a Kustomization's + // status.lastAppliedRevision, to the tip. Defaults to "AsDefinedOnClaim". // +optional - // +kubebuilder:validation:Enum=AsDefinedOnClaim;ImmediatelyAfterPush + // +kubebuilder:validation:Enum=AsDefinedOnClaim;AfterPush // +kubebuilder:default=AsDefinedOnClaim CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } @@ -93,8 +93,8 @@ type CatchUpPolicy string const ( // CatchUpAsDefinedOnClaim leaves the apply to the applier claim's spec.catchUpAfter. CatchUpAsDefinedOnClaim CatchUpPolicy = "AsDefinedOnClaim" - // CatchUpImmediatelyAfterPush asks the applier to apply the tip as soon as the commit is pushed. - CatchUpImmediatelyAfterPush CatchUpPolicy = "ImmediatelyAfterPush" + // CatchUpAfterPush asks the applier to apply the tip right after the commit is pushed. + CatchUpAfterPush CatchUpPolicy = "AfterPush" ) // NothingToCommitAction is what a CommitRequest does when it ends with nothing @@ -193,30 +193,38 @@ type CommitRequestWindow struct { // not a failure and does not affect Ready; the final Git author remains // the matching watch window's author. // - Pushed (domain): True once the commit is in the remote repository. +// - Applied (domain): with spec.catchUp "AfterPush", whether the applier +// caught up with the commit. It never affects Ready. type CommitRequestStatus struct { // ObservedGeneration is the latest generation observed by the controller. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty"` - // Conditions are the latest observations of this CommitRequest's state. Ready, - // Reconciling and Stalled follow kstatus: Ready is True once the request ended - // without error, with a pushed commit or with nothing to save. Pushed is True - // once the commit is on the remote. AuthorAttributed says whether the - // request's creator, recorded at admission, names the commit author; False is - // not a failure and does not affect Ready. + // Conditions are the latest observations of this CommitRequest's state. + // Reconciling and Stalled follow kstatus. AuthorAttributed says whether the + // request's creator, recorded at admission, names the commit author; False + // is not a failure and does not affect Ready. // - // Applied is set only with spec.catchUp "ImmediatelyAfterPush", once Ready is - // True. It says whether the GitTarget's applier applied a commit holding this - // request's: + // Three conditions say how far the save got: // - // - Unknown, reason Applying or Retrying: the apply runs, or failed and is - // asked again; the message says why. - // - True, reason Applied: status.appliedRevision names what was applied. - // - False, reason NothingPushed, NoApplierClaim or ApplierGone: there was - // nothing to apply, or no applier to ask. - // - Unknown, reason Lost: GitOps Reverser restarted before it knew. + // - Ready (kstatus): True once the request ended without error, with a + // pushed commit or with nothing to save. It never waits for an apply. + // - Pushed: True once the commit is on the remote. + // - Applied: set only with spec.catchUp "AfterPush", once Ready is True. + // + // Applied says whether the cluster's applier caught up with the commit: // - // It never affects Ready: the save is done once it is pushed. + // - Unknown, reason Applying or Retrying: the apply runs, or did not + // complete and is asked again; the message says why. + // - True, reason Applied: the applier applied a revision that holds the + // commit, named in status.appliedRevision. + // - True, reason NoApplierClaim or ApplierGone: the GitTarget names no + // applier claim, or stopped naming the one asked, and the cluster + // already holds the saved objects. + // - False, reason NothingPushed: there is no commit to apply. + // - False, reason CommitDropped: the branch no longer holds the commit; + // its history was rewritten after the push. + // - Unknown, reason Lost: GitOps Reverser restarted before it knew. // +optional // +listType=map // +listMapKey=type @@ -233,8 +241,9 @@ type CommitRequestStatus struct { Commit string `json:"commit,omitempty"` // AppliedRevision is the revision the GitTarget's applier reported applying - // for this request, as in a Kustomization's status.lastAppliedRevision. Set - // once Applied is True. + // for this request, as in a Kustomization's status.lastAppliedRevision. It + // holds the request's commit, and may be a later one. Set once Applied is + // True with reason Applied. // +optional AppliedRevision string `json:"appliedRevision,omitempty"` } diff --git a/api/v1alpha3/fluxclaim_types.go b/api/v1alpha3/fluxclaim_types.go index 9e0cb43a..e50554f2 100644 --- a/api/v1alpha3/fluxclaim_types.go +++ b/api/v1alpha3/fluxclaim_types.go @@ -53,14 +53,17 @@ type FluxClaimSpec struct { // state (its applied revision, its source artifact) would stay at the last foreign commit until // someone else pushed. One apply per interval keeps that level and batches a burst of saves. - // CatchUpAfter is how long the Kustomization may stay behind the - // branch after a push of GitOps Reverser's own, as a Go duration ("1m", - // "90s"). Such a push changes nothing on the cluster, which already holds - // the edit, but leaves status.lastAppliedRevision and the GitRepository's - // artifact at the commit before it. Once the interval has passed, the tip is - // applied: a burst of saves costs one apply, and a Kustomization at the tip - // costs none. "0s" never applies for this reason. Otherwise at least "10s" - // and at most "24h". Defaults to "1m". + // CatchUpAfter is how long the Kustomization stays behind the branch after + // a push of GitOps Reverser's own before an apply of the tip is asked for, + // as a Go duration ("1m", "90s"). Such a push changes nothing on the + // cluster, which already holds the edit, but leaves + // status.lastAppliedRevision and the GitRepository's artifact at the commit + // before it. The interval starts when the Kustomization is first seen + // behind, so a burst of saves costs one apply, and one at the tip costs + // none. The apply can still wait for startup, a retry backoff or edits + // being published. "0s" never asks for this reason, and drops an apply it + // asked for that has not started. Otherwise at least "10s" and at most + // "24h". Defaults to "1m". // +optional // +kubebuilder:validation:Type=string // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" @@ -187,7 +190,9 @@ type FluxClaimSource struct { // - after a refused edit, on a GitTarget with spec.onRefusal "Reapply"; // - after an apply reverted an edit made through the API; // - after objects were created through the API in its folders; -// - once at startup, and when a GitTarget first names this FluxClaim. +// - once at startup, and when a GitTarget first names this FluxClaim; +// - once spec.catchUpAfter passed since a push of GitOps Reverser's own; +// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type FluxClaim struct { diff --git a/config/crd/bases/configbutler.ai_argocdclaims.yaml b/config/crd/bases/configbutler.ai_argocdclaims.yaml index e6769fb0..55d99bb1 100644 --- a/config/crd/bases/configbutler.ai_argocdclaims.yaml +++ b/config/crd/bases/configbutler.ai_argocdclaims.yaml @@ -66,7 +66,9 @@ spec: - after a refused edit, on a GitTarget with spec.onRefusal "Reapply"; - after a sync reverted an edit made through the API; - after objects were created through the API in its folders; - - once at startup, and when a GitTarget first names this ArgoCDClaim. + - once at startup, and when a GitTarget first names this ArgoCDClaim; + - once spec.catchUpAfter passed since a push of GitOps Reverser's own; + - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: @@ -127,14 +129,16 @@ spec: catchUpAfter: default: 0s description: |- - CatchUpAfter is how long the Application may stay behind the branch - after a push of GitOps Reverser's own, as a Go duration ("5m"). Such a - push changes nothing on the cluster, which already holds the edit, but - leaves the Application's last sync at the commit before it. Once the - interval has passed, the tip is synced, hooks included: a burst of saves - costs one sync, and an Application at the tip costs none. "0s" never - syncs for this reason. Otherwise at least "10s" and at most "24h". - Defaults to "0s". + CatchUpAfter is how long the Application stays behind the branch after + a push of GitOps Reverser's own before a sync of the tip is asked for, + as a Go duration ("5m"). Such a push changes nothing on the cluster, + which already holds the edit, but leaves the Application's last sync at + the commit before it. The interval starts when the Application is first + seen behind, so a burst of saves costs one sync, hooks included, and one + at the tip costs none. The sync can still wait for startup, a retry + backoff or edits being published. "0s" never asks for this reason, and + drops a sync it asked for that has not started. Otherwise at least "10s" + and at most "24h". Defaults to "0s". pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ type: string x-kubernetes-validations: diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 52f94b3a..e373fccb 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -84,17 +84,17 @@ spec: catchUp: default: AsDefinedOnClaim description: |- - CatchUp decides when the GitTarget's applier is asked to apply the commit - this request pushes. "AsDefinedOnClaim" leaves it to the applier claim's - spec.catchUpAfter. "ImmediatelyAfterPush" asks as soon as the commit is - pushed, which costs one apply per request. The cluster already holds the - saved objects; the apply brings the applier's own state, such as a - Kustomization's status.lastAppliedRevision, to the commit. A request that - pushes no commit, or whose GitTarget names no applier claim, asks for - nothing. Defaults to "AsDefinedOnClaim". + CatchUp decides when the GitTarget's applier is asked to apply the branch + after this request pushes a commit. "AsDefinedOnClaim" leaves it to the + applier claim's spec.catchUpAfter. "AfterPush" asks right after the push. + The apply takes the latest branch tip, which may hold later commits, and + saves pushed close together can share one apply; the Applied condition + reports how it went. The cluster already holds the saved objects: + the apply brings the applier's own state, such as a Kustomization's + status.lastAppliedRevision, to the tip. Defaults to "AsDefinedOnClaim". enum: - AsDefinedOnClaim - - ImmediatelyAfterPush + - AfterPush type: string gitTargetRef: description: |- @@ -235,8 +235,9 @@ spec: appliedRevision: description: |- AppliedRevision is the revision the GitTarget's applier reported applying - for this request, as in a Kustomization's status.lastAppliedRevision. Set - once Applied is True. + for this request, as in a Kustomization's status.lastAppliedRevision. It + holds the request's commit, and may be a later one. Set once Applied is + True with reason Applied. type: string branch: description: Branch is the branch the commit is made on. Set once @@ -248,25 +249,31 @@ spec: type: string conditions: description: |- - Conditions are the latest observations of this CommitRequest's state. Ready, - Reconciling and Stalled follow kstatus: Ready is True once the request ended - without error, with a pushed commit or with nothing to save. Pushed is True - once the commit is on the remote. AuthorAttributed says whether the - request's creator, recorded at admission, names the commit author; False is - not a failure and does not affect Ready. + Conditions are the latest observations of this CommitRequest's state. + Reconciling and Stalled follow kstatus. AuthorAttributed says whether the + request's creator, recorded at admission, names the commit author; False + is not a failure and does not affect Ready. - Applied is set only with spec.catchUp "ImmediatelyAfterPush", once Ready is - True. It says whether the GitTarget's applier applied a commit holding this - request's: + Three conditions say how far the save got: - - Unknown, reason Applying or Retrying: the apply runs, or failed and is - asked again; the message says why. - - True, reason Applied: status.appliedRevision names what was applied. - - False, reason NothingPushed, NoApplierClaim or ApplierGone: there was - nothing to apply, or no applier to ask. - - Unknown, reason Lost: GitOps Reverser restarted before it knew. + - Ready (kstatus): True once the request ended without error, with a + pushed commit or with nothing to save. It never waits for an apply. + - Pushed: True once the commit is on the remote. + - Applied: set only with spec.catchUp "AfterPush", once Ready is True. + + Applied says whether the cluster's applier caught up with the commit: - It never affects Ready: the save is done once it is pushed. + - Unknown, reason Applying or Retrying: the apply runs, or did not + complete and is asked again; the message says why. + - True, reason Applied: the applier applied a revision that holds the + commit, named in status.appliedRevision. + - True, reason NoApplierClaim or ApplierGone: the GitTarget names no + applier claim, or stopped naming the one asked, and the cluster + already holds the saved objects. + - False, reason NothingPushed: there is no commit to apply. + - False, reason CommitDropped: the branch no longer holds the commit; + its history was rewritten after the push. + - Unknown, reason Lost: GitOps Reverser restarted before it knew. items: description: Condition contains details for one aspect of the current state of this API Resource. diff --git a/config/crd/bases/configbutler.ai_fluxclaims.yaml b/config/crd/bases/configbutler.ai_fluxclaims.yaml index 2393a155..385b5edb 100644 --- a/config/crd/bases/configbutler.ai_fluxclaims.yaml +++ b/config/crd/bases/configbutler.ai_fluxclaims.yaml @@ -66,7 +66,9 @@ spec: - after a refused edit, on a GitTarget with spec.onRefusal "Reapply"; - after an apply reverted an edit made through the API; - after objects were created through the API in its folders; - - once at startup, and when a GitTarget first names this FluxClaim. + - once at startup, and when a GitTarget first names this FluxClaim; + - once spec.catchUpAfter passed since a push of GitOps Reverser's own; + - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: @@ -93,14 +95,17 @@ spec: catchUpAfter: default: 1m description: |- - CatchUpAfter is how long the Kustomization may stay behind the - branch after a push of GitOps Reverser's own, as a Go duration ("1m", - "90s"). Such a push changes nothing on the cluster, which already holds - the edit, but leaves status.lastAppliedRevision and the GitRepository's - artifact at the commit before it. Once the interval has passed, the tip is - applied: a burst of saves costs one apply, and a Kustomization at the tip - costs none. "0s" never applies for this reason. Otherwise at least "10s" - and at most "24h". Defaults to "1m". + CatchUpAfter is how long the Kustomization stays behind the branch after + a push of GitOps Reverser's own before an apply of the tip is asked for, + as a Go duration ("1m", "90s"). Such a push changes nothing on the + cluster, which already holds the edit, but leaves + status.lastAppliedRevision and the GitRepository's artifact at the commit + before it. The interval starts when the Kustomization is first seen + behind, so a burst of saves costs one apply, and one at the tip costs + none. The apply can still wait for startup, a retry backoff or edits + being published. "0s" never asks for this reason, and drops an apply it + asked for that has not started. Otherwise at least "10s" and at most + "24h". Defaults to "1m". pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ type: string x-kubernetes-validations: diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 2ced05b2..75003d1e 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -11,12 +11,12 @@ We are pre-1.0, so breaking changes bump the **minor** version (release-please i A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a `Kustomization`'s `status.lastAppliedRevision` and its `GitRepository`'s artifact stayed at the last -commit somebody else pushed. `spec.catchUpAfter` bounds that lag: once the applier has stayed -behind the tip that long, the tip is applied, one apply for a whole burst of saves. A `FluxClaim` -defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` defaults to `0s`. -A `CommitRequest` that sets `catchUp: ImmediatelyAfterPush` has its commit applied as soon as it is -pushed, whatever the claim's interval, and reports the apply in its `Applied` condition and -`status.appliedRevision`. +commit somebody else pushed. `spec.catchUpAfter` sets when that lag is caught up: once the applier +has stayed behind the tip that long, an apply of the tip is asked for, one for a whole burst of +saves. A `FluxClaim` defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` +defaults to `0s`. A `CommitRequest` that sets `catchUp: AfterPush` has the tip applied right after +its commit is pushed, whatever the claim's interval, and reports the outcome in its `Applied` +condition and `status.appliedRevision`. ## `kube-root-ca.crt` is not mirrored diff --git a/docs/configuration.md b/docs/configuration.md index 679030d8..6fafc0df 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -159,7 +159,7 @@ The references are immutable: to hold another `Kustomization`, delete the applie |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Kustomization`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `kustomizationRef` | **required** | The `Kustomization` that applies the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | -| `catchUpAfter` | `1m` | How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by an interval. `0s` never applies for this reason; otherwise `10s` to `24h` | +| `catchUpAfter` | `1m` | How long the `Kustomization` stays behind the branch after a push of GitOps Reverser's own before an apply of the tip is asked for. Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it. The interval starts when the `Kustomization` is first seen behind, so a burst of saves costs one apply; the apply can still wait for startup, a retry backoff or edits being published, and an edit arriving before the resume defers it by an interval. `0s` never asks for this reason, and drops an apply it asked for that has not started; otherwise `10s` to `24h` | ### `ArgoCDClaim` (namespaced): the Argo CD `Application` GitOps Reverser asks to sync @@ -169,7 +169,7 @@ The references are immutable: to hold another `Application`, delete the applier |---|---|---| | `kubeProviderRef` | **required** | The `KubeProvider` whose cluster holds the `Application`: the applier claim's own namespace, or `namespace` names another. Its `allowedApplierClaims` must admit this namespace | | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | -| `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | +| `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync stays behind the branch after a push of GitOps Reverser's own before a sync of the tip is asked for. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | ### `CommitRequest` (namespaced): save now @@ -182,7 +182,7 @@ The references are immutable: to hold another `Application`, delete the applier | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | -| `catchUp` | `AsDefinedOnClaim` | `ImmediatelyAfterPush` asks the `GitTarget`'s applier to apply the commit as soon as it is pushed, one apply per request, instead of after the applier claim's `catchUpAfter` | +| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it | @@ -1895,10 +1895,11 @@ for a `Kustomization` and an `Application`. | `ForeignCommit` | The branch gained a commit GitOps Reverser did not push. The pending writes are replayed onto it and pushed first, so the request names that pushed commit | Every applier declared on the branch, once per applier | | `Refusal` | An eligible refused edit, with `spec.onRefusal: Reapply` | This target's applier | | `Repair` | An apply reverted an API edit made after the flush; Git kept the edit | That applier | -| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, and once when a `GitTarget` first names the applier claim | To adopt, that folder's applier; to converge, every applier declared on the branch at start, and the newly named one | +| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, once when a `GitTarget` first names the applier claim, once the applier claim's `catchUpAfter` has passed since a push of GitOps Reverser's own, and after a `CommitRequest` with [`catchUp: AfterPush`](#applying-a-save-at-once) pushed a commit | To adopt, that folder's applier; to converge, every applier declared on the branch at start, the newly named one, the one left behind, or the requesting `GitTarget`'s | -An API edit asks for no apply, and neither does a push of GitOps Reverser's own: the cluster already -runs what they hold. A create is different, because an applier prunes only what it applied: an +An API edit asks for no apply, and neither does a push of GitOps Reverser's own, at once: the cluster +already runs what they hold. Only the applier's own state is left behind, which `catchUpAfter` +catches up later. A create is different, because an applier prunes only what it applied: an object the API created (an `ADDED` event whose newest field managers are not an applier's) carries neither Flux's owner labels nor Argo CD's tracking annotation, so a later Git-side delete of its file would leave it running. An adopt is therefore applied even at the commit the applier last @@ -3121,24 +3122,28 @@ request produced a pushed commit: watch window, whose Git author remains either the configured committer or the explicit unresolved author according to the watch attribution outcome. - **Pushed**: `True` once the commit, empty or not, is in the remote repository. -- **Applied**: only with [`catchUp: ImmediatelyAfterPush`](#applying-a-save-at-once), once `Ready=True`. - It never changes `Ready`. +- **Applied**: only with [`catchUp: AfterPush`](#applying-a-save-at-once), once `Ready=True`: + whether the applier caught up with the commit. It never changes `Ready`, so `Ready=True` alone + does not mean the commit was applied. ### Applying a save at once A save is in Git once it is pushed, and the cluster already runs it, so by default nothing applies it: the applier claim's `catchUpAfter` brings the applier's own state (a `Kustomization`'s `status.lastAppliedRevision`, its `GitRepository`'s artifact) level later. A request that needs that -state at once, to publish what was applied, sets `spec.catchUp: ImmediatelyAfterPush`. Its -`GitTarget`'s applier is then asked to apply the tip as soon as the commit is pushed, one apply per -such request, and the request reports the apply in its `Applied` condition: +state at once, to publish what was applied, sets `spec.catchUp: AfterPush`. Its +`GitTarget`'s applier is then asked to apply the branch tip right after the commit is pushed. The +tip may hold later commits, saves pushed close together can share one apply, and an applier +already at the tip needs none. The request reports the outcome in its `Applied` condition: | `Applied` | Reason | Means | | --- | --- | --- | | `Unknown` | `Applying` | the apply is asked for or runs | | `Unknown` | `Retrying` | it did not complete and is asked again; the message says why | -| `True` | `Applied` | applied; `status.appliedRevision` names the revision, as the applier reported it | -| `False` | `NothingPushed`, `NoApplierClaim`, `ApplierGone` | nothing to apply, or no applier to ask | +| `True` | `Applied` | the applier applied a revision that holds the commit; `status.appliedRevision` names it, as the applier reported it | +| `True` | `NoApplierClaim`, `ApplierGone` | the `GitTarget` names no applier claim, or stopped naming the one asked: nothing applies the branch, and the cluster already holds what was saved | +| `False` | `NothingPushed` | the request pushed no commit, so there is nothing to apply | +| `False` | `CommitDropped` | the branch no longer holds the commit: its history was rewritten after the push | | `Unknown` | `Lost` | GitOps Reverser restarted before it learnt how the apply went | Wait for `Applied=True` for "saved and applied"; `Ready=True` stays "saved". A finished request is diff --git a/docs/design/apply-trigger.md b/docs/design/apply-trigger.md index 8eeb82ba..d5d4b438 100644 --- a/docs/design/apply-trigger.md +++ b/docs/design/apply-trigger.md @@ -267,7 +267,9 @@ deletes its file the applier leaves it on the cluster, and a later resync writes clock starts on the first pass that finds it behind and again on every send, so a burst of saves inside one interval costs one apply, and an applier at the tip costs nothing. A `FluxClaim` defaults to `1m`; an `ArgoCDClaim` to `0s`, never, since a sync runs the `Application`'s hooks - and Argo CD's refresh already compares the live state against the tip. + and Argo CD's refresh already compares the live state against the tip. Setting `0s` also drops + an interval converge still pending, behind the barrier or a backoff; a converge something else + asked for stays. - **An interval converge yields to an edit.** Its fetch writes nothing to the cluster; its apply puts the fetched tip there, and an edit mirrored after the send is not in that tip, so the apply would revert it and a `Repair` would put it back. Such a converge's decision stops holding once a @@ -275,13 +277,16 @@ deletes its file the applier leaves it on the cluster, and a later resync writes fresh interval rather than going again after the next flush. While a burst of edits lasts, it lands in the first quiet moment. Every other reason applies regardless, as the flush guarantees ([above](#flush-then-apply)). -- **A save can ask to catch up at once.** A `CommitRequest` with `catchUp: ImmediatelyAfterPush` +- **A save can ask to catch up at once.** A `CommitRequest` with `catchUp: AfterPush` leaves a converge pending for its `GitTarget`'s applier as soon as its commit is pushed, without waiting for `catchUpAfter`. It was asked for, so it does not yield to an edit. A save that pushed - no commit asks nothing. The save waits on its applier until an attempt that carries it succeeds - (any apply sent after its push holds its commit), and the request's `Applied` condition reports - it: retried with the applier's backoff, settled applied, or settled not applied when there is no - applier, nothing was pushed, or the applier stops being named (`save_apply.go`). + no commit asks nothing. The save waits on its applier, with its commit, until an attempt that + carries it succeeds, and is settled applied only when the applied revision holds that commit: a + force-push that dropped it settles it not applied, and a revision the checkout does not hold yet + keeps it waiting while the remote is looked at. The request's `Applied` condition reports it: + retried with the applier's backoff, settled applied, settled applied when no applier is named + (the cluster already holds the save), or settled not applied when nothing was pushed + (`save_apply.go`). - **No apply before every initial snapshot is published.** An applier is asked for nothing, for any reason, until every collection of every `GitTarget` it covers has a settled snapshot (applied, refused or failed; tracked per collection on the watch side) and the worker has published every diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 0a53d099..5405d3b8 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -260,11 +260,13 @@ kinds: [who applies this folder](#who-applies-this-folder-specapplierclaimref) - path: catchUpAfter doc: >- - How long the `Kustomization` may stay behind the branch after a push of GitOps Reverser's own. - Such a push needs no apply for the cluster, but leaves `status.lastAppliedRevision` and the - `GitRepository`'s artifact at the commit before it; once the interval has passed, the tip is - applied. A burst of saves costs one apply, and an edit arriving before the resume defers it by - an interval. `0s` never applies for this reason; otherwise `10s` to `24h` + How long the `Kustomization` stays behind the branch after a push of GitOps Reverser's own + before an apply of the tip is asked for. Such a push needs no apply for the cluster, but leaves + `status.lastAppliedRevision` and the `GitRepository`'s artifact at the commit before it. The + interval starts when the `Kustomization` is first seen behind, so a burst of saves costs one + apply; the apply can still wait for startup, a retry backoff or edits being published, and an + edit arriving before the resume defers it by an interval. `0s` never asks for this reason, and + drops an apply it asked for that has not started; otherwise `10s` to `24h` - kind: ArgoCDClaim title: "`ArgoCDClaim` (namespaced): the Argo CD `Application` GitOps Reverser asks to sync" @@ -280,9 +282,10 @@ kinds: naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) - path: catchUpAfter doc: >- - As a `FluxClaim`'s: how long the `Application`'s last sync may stay behind the branch after a - push of GitOps Reverser's own. Off by default, since a sync runs the `Application`'s hooks and - Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` + As a `FluxClaim`'s: how long the `Application`'s last sync stays behind the branch after a push + of GitOps Reverser's own before a sync of the tip is asked for. Off by default, since a sync runs + the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. + Otherwise `10s` to `24h` - kind: CommitRequest title: "`CommitRequest` (namespaced): save now" @@ -307,5 +310,6 @@ kinds: doc: "`CommitEmpty` records the message in an empty commit when nothing changed" - path: catchUp doc: >- - `ImmediatelyAfterPush` asks the `GitTarget`'s applier to apply the commit as soon as it is pushed, - one apply per request, instead of after the applier claim's `catchUpAfter` + `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is + pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, + and saves pushed close together can share one apply; the `Applied` condition reports it diff --git a/internal/controller/commitrequest_apply.go b/internal/controller/commitrequest_apply.go index 2c8b0617..58d4c442 100644 --- a/internal/controller/commitrequest_apply.go +++ b/internal/controller/commitrequest_apply.go @@ -17,7 +17,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -// A CommitRequest with catchUp: ImmediatelyAfterPush keeps being reconciled once it is Ready, to +// A CommitRequest with catchUp: AfterPush keeps being reconciled once it is Ready, to // report in its Applied condition how the apply it asked for went (git.SaveApply). Ready is the // save's alone and never changes here. @@ -33,7 +33,7 @@ const applyLostAfter = time.Minute // followsApply reports whether a finished request still reports its apply: it asked for one, its // save ended Ready, and Applied is not settled yet. func followsApply(cr *configbutleraiv1alpha3.CommitRequest) bool { - return cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpImmediatelyAfterPush && + return cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush && conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) && !applyReported(cr) } diff --git a/internal/controller/commitrequest_apply_test.go b/internal/controller/commitrequest_apply_test.go index 7fa0e2f3..470d4ec5 100644 --- a/internal/controller/commitrequest_apply_test.go +++ b/internal/controller/commitrequest_apply_test.go @@ -17,12 +17,12 @@ import ( ) func catchingUp(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { - cr.Spec.CatchUp = configv1alpha3.CatchUpImmediatelyAfterPush + cr.Spec.CatchUp = configv1alpha3.CatchUpAfterPush return cr } // TestCommitRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with -// catchUp: ImmediatelyAfterPush is reconciled on, and Applied follows the worker's report until it +// catchUp: AfterPush is reconciled on, and Applied follows the worker's report until it // is settled. Ready stays the save's. func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) @@ -55,19 +55,31 @@ func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { assert.Equal(t, lookups, f.applyLookups, "settled: not looked up again") } -// TestCommitRequestReconcile_NothingToApplyIsFalse: a settled apply that applied nothing is -// Applied=False, and does not touch Ready. -func TestCommitRequestReconcile_NothingToApplyIsFalse(t *testing.T) { - c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) - f := committedFinalizer() - f.apply, f.applyKnown = git.SaveApply{Done: true, Reason: git.SaveApplyNoApplier}, true - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - - reconcileCommitRequest(t, r, "save-1") - reconcileCommitRequest(t, r, "save-1") - got := fetchCommitRequest(t, c, "save-1") - requireCondition(t, got, ConditionTypeApplied, metav1.ConditionFalse, git.SaveApplyNoApplier) - requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) +// TestCommitRequestReconcile_ASettledApplyWithoutAnApplier: a request that pushed nothing is +// Applied=False; one whose GitTarget names no applier is Applied=True, since the cluster already holds +// what it saved and nothing applies the branch. Neither touches Ready. +func TestCommitRequestReconcile_ASettledApplyWithoutAnApplier(t *testing.T) { + for _, tc := range []struct { + apply git.SaveApply + want metav1.ConditionStatus + }{ + {git.SaveApply{Done: true, Reason: git.SaveApplyNothingPushed}, metav1.ConditionFalse}, + {git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyNoApplier}, metav1.ConditionTrue}, + } { + t.Run(tc.apply.Reason, func(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) + f := committedFinalizer() + f.apply, f.applyKnown = tc.apply, true + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + + reconcileCommitRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") + requireCondition(t, got, ConditionTypeApplied, tc.want, tc.apply.Reason) + requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) + assert.Empty(t, got.Status.AppliedRevision) + }) + } } // TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index aa561b6d..33d58e52 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -38,7 +38,7 @@ type CommitRequestFinalizer interface { // when the worker got there first, or resolved=false with the worker's phase while it decides. WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) // LookupCommitRequestApply reports where the apply a request with catchUp: - // ImmediatelyAfterPush asked for stands, and false while the worker knows nothing of it. + // AfterPush asked for stands, and false while the worker knows nothing of it. LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) } @@ -261,7 +261,7 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques IdleTimeout: window.idleTimeout, MaxDuration: window.maxDuration, CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpImmediatelyAfterPush, + CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, } if time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout() { // Past the bound the controller only withdraws: an attach sent now would sit behind the diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index 82e2348f..6b7ac9e0 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -46,7 +46,7 @@ type fakeFinalizer struct { withdrawPending bool withdrawCalls []git.AttachCommitRequest - // The apply a request with catchUp: ImmediatelyAfterPush asked for, and the lookups made. + // The apply a request with catchUp: AfterPush asked for, and the lookups made. apply git.SaveApply applyKnown bool applyErr error diff --git a/internal/controller/constants.go b/internal/controller/constants.go index 05763c67..aaebcc87 100644 --- a/internal/controller/constants.go +++ b/internal/controller/constants.go @@ -98,7 +98,7 @@ const ( // ConditionTypePushed indicates whether a CommitRequest's commit reached the // remote repository. ConditionTypePushed = "Pushed" - // ConditionTypeApplied reports, for a CommitRequest with catchUp: ImmediatelyAfterPush, whether + // ConditionTypeApplied reports, for a CommitRequest with catchUp: AfterPush, whether // its GitTarget's applier applied a commit that holds the request's: Unknown while it applies or // retries, True once applied, False when there was nothing to apply. It never affects Ready. ConditionTypeApplied = "Applied" diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 156a1d22..678904da 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -251,9 +251,9 @@ type applierState struct { // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, // so its attempt yields to an edit (convergeHolds). lag bool - // waitingSaves is the CommitRequests waiting for this applier's next apply that succeeds - // (save_apply.go); the attempt sent next carries them. - waitingSaves map[commitRequestID]bool + // waitingSaves is the CommitRequests waiting for this applier's next apply that succeeds, each + // with the commit it pushed (save_apply.go); the attempt sent next carries them. + waitingSaves map[commitRequestID]string // appliedRevision is the revision the last operation that succeeded applied, as the applier // reported it, recovered on restart from the applier claim's status.operation. A converge of the commit // it names asks nothing. @@ -324,8 +324,9 @@ type inflightAttempt struct { refusals map[refusalKey]unsentRefusal // lag is whether it carries only scheduleConverges' converge. lag bool - // saves is the CommitRequests whose apply it carries (save_apply.go). - saves map[commitRequestID]bool + // saves is the CommitRequests whose apply it carries, each with the commit it pushed + // (save_apply.go). + saves map[commitRequestID]string // phase is the last report of its operation, empty before the first. phase ApplyOutcome // previous is the operation recorded before this attempt's, with the attempt it belonged to, @@ -455,7 +456,12 @@ func (l *branchWorkerEventLoop) scheduleConverges(now time.Time) time.Time { } state.catchUpAfter = l.w.catchUpAfterOf(state) if state.catchUpAfter <= 0 { + // Off: a converge only this schedule left pending, waiting on the barrier or a backoff, + // goes with it. Anything else that asks for the tip keeps it. state.behindSince = time.Time{} + if state.lag { + state.converge, state.lag = false, false + } continue } if state.behindSince.IsZero() { @@ -542,8 +548,9 @@ func (l *branchWorkerEventLoop) dueAppliers(now time.Time) ([]*applierState, tim refusals := l.unsentRefusalsFor(state) if l.alreadyConverged(state, refusals, tip) { state.converge, state.lag = false, false - l.settleSaves(state.waitingSaves, state.appliedRevision) + saves := state.waitingSaves state.waitingSaves = nil + l.settleSaves(state, saves, state.appliedRevision) l.w.Log.V(1).Info("The applier already applied the tip; the converge asks nothing", "branch", l.w.Branch, "applier", state.ref.String(), "commit", tip) } @@ -908,7 +915,7 @@ func (l *branchWorkerEventLoop) settleApplyResults(now time.Time) { if state.appliedRevision == "" && state.operation != nil { state.appliedRevision = state.operation.Commit } - l.settleSaves(attempt.saves, state.appliedRevision) + l.settleSaves(state, attempt.saves, state.appliedRevision) continue case ApplyStale: if attempt.lag { @@ -1120,8 +1127,8 @@ func (l *branchWorkerEventLoop) restoreAttempt(state *applierState, attempt *inf state.restoreLag(attempt) state.converge = state.converge || attempt.converge state.repair = state.repair || attempt.repair - for id := range attempt.saves { - state.waitSave(id) + for id, commit := range attempt.saves { + state.waitSave(id, commit) } for key, refusal := range attempt.refusals { if !l.w.refusalStands(key) { diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index 0e580158..8c33b6e6 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -3,12 +3,17 @@ package git import ( + "os" + "path/filepath" + "strings" "testing" "time" + gogit "github.com/go-git/go-git/v6" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + k8stypes "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" @@ -193,7 +198,7 @@ func TestCatchUpAfter_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { assert.Equal(t, "bbb", requests[0].Commit) } -// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a CommitRequest with catchUp: ImmediatelyAfterPush +// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a CommitRequest with catchUp: AfterPush // leaves a converge pending for its GitTarget's applier once its commit is pushed, without waiting // for catchUpAfter; one that pushed no commit asks nothing. It does not yield to an edit: it was // asked for. @@ -227,8 +232,8 @@ func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { // TestSaveApply_FollowsTheAttemptThatCarriesIt: a save that asked to catch up waits on its applier, // is carried by the attempt sent next, is retried when that attempt does not succeed, and is -// settled applied by the one that does. One that pushed nothing, or whose GitTarget names no -// applier, is settled at once as not applied. +// settled applied by the one that does. One that pushed nothing is settled at once as not applied, +// and one whose GitTarget names no applier as applied: nothing applies the branch. func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true), applierTestTarget("loner", false)) convergedAt(t, loop, requester) @@ -257,6 +262,7 @@ func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { assert.Equal(t, SaveApplyApplying, lookup(pushed).Reason) loop.serviceAppliers() assert.True(t, lookup(alone).Done) + assert.True(t, lookup(alone).Applied, "no applier applies the branch; the cluster holds the save") assert.Equal(t, SaveApplyNoApplier, lookup(alone).Reason) first := requester.taken() @@ -299,3 +305,127 @@ func TestSaveApply_AnApplierAlreadyAtTheTipSettlesIt(t *testing.T) { require.True(t, known) assert.True(t, apply.Applied) } + +// TestCatchUpAfter_TurningItOffCancelsAPendingConverge: a converge only catchUpAfter left pending, +// waiting out the backoff of an attempt that failed, goes when catchUpAfter becomes "0s". A save that +// asks to catch up still goes: catchUpAfter does not decide it. +func TestCatchUpAfter_TurningItOffCancelsAPendingConverge(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true), fluxClaim("alpha", catchUpEvery(30*time.Second))) + convergedAt(t, loop, requester) + loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) + loop.serviceAppliers() + intervalPassed(t, loop) + loop.serviceAppliers() + sent := requester.taken() + require.Len(t, sent, 1) + report(sent[0], ApplyFailed) + loop.serviceAppliers() + state := loop.appliers[applierRef("alpha")] + require.True(t, state.converge, "given back, waiting out its backoff") + + claim := &configv1alpha3.FluxClaim{} + require.NoError(t, loop.w.Client.Get(t.Context(), client.ObjectKey{Namespace: "team-a", Name: "alpha"}, claim)) + catchUpEvery(0)(claim) + require.NoError(t, loop.w.Client.Update(t.Context(), claim)) + dueNow(loop) + loop.serviceAppliers() + assert.Empty(t, requester.taken(), "off: the timed converge is gone") + assert.False(t, state.converge) + + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "asks", UID: "uid-asks"} + loop.pendingCRs[id] = &pendingCommitRequest{ + id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, + } + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "bbb"}) + loop.serviceAppliers() + asked := requester.taken() + require.Len(t, asked, 1, "a save that asks is no timed converge") + assert.Equal(t, "bbb", asked[0].Commit) +} + +// saveHistory makes the worker's checkout hold base, saved on it, later on saved, and rewritten on base +// beside saved: the history a force-push that drops saved leaves. It returns them in that order. +func saveHistory(t *testing.T, w *BranchWorker) (string, string, string, string) { + t.Helper() + w.GitBranchUID = k8stypes.UID("uid-" + strings.ToLower(t.Name())) + dir := w.repoPath() + t.Cleanup(func() { _ = os.RemoveAll(dir) }) + repo, err := gogit.PlainInit(dir, false) + require.NoError(t, err) + require.NoError(t, PinExplicitSigningPolicy(repo)) + worktree, err := repo.Worktree() + require.NoError(t, err) + // In the folder of applierTestTarget("alpha"), which an apply waits for. + require.NoError(t, os.MkdirAll(filepath.Join(dir, "apps", "alpha"), 0o750)) + file := filepath.Join("apps", "alpha", "a.yaml") + baseHash := commitFileChange(t, worktree, dir, file, "base") + savedHash := commitFileChange(t, worktree, dir, file, "saved") + laterHash := commitFileChange(t, worktree, dir, file, "later") + require.NoError(t, worktree.Checkout(&gogit.CheckoutOptions{Hash: baseHash, Force: true})) + rewrittenHash := commitFileChange(t, worktree, dir, file, "rewritten") + return baseHash.String(), savedHash.String(), laterHash.String(), rewrittenHash.String() +} + +// TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit: an apply that succeeds settles a save as +// applied only when the applied revision holds the save's commit. One whose commit a force-push +// dropped from the branch is settled as not applied; one the checkout cannot tell about waits. +func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { + cases := map[string]struct { + tip func(base, saved, later, rewritten string) (string, ObservationSource) + applied func(tip string) string + want SaveApply + }{ + "a later commit holds it": { + tip: func(_, _, later, _ string) (string, ObservationSource) { return later, ObservedByPush }, + applied: func(tip string) string { return "main@sha1:" + tip }, + want: SaveApply{Done: true, Applied: true, Reason: SaveApplyApplied}, + }, + "a force-push dropped it": { + tip: func(_, _, _, rewritten string) (string, ObservationSource) { return rewritten, ObservedByFetch }, + applied: func(tip string) string { return "main@sha1:" + tip }, + want: SaveApply{Done: true, Reason: SaveApplyCommitDropped}, + }, + "the checkout lacks the applied commit": { + tip: func(_, _, later, _ string) (string, ObservationSource) { return later, ObservedByPush }, + applied: func(string) string { return "main@sha1:" + strings.Repeat("f", 40) }, + want: SaveApply{Reason: SaveApplyRetrying}, + }, + } + for name, tc := range cases { + t.Run(name, func(t *testing.T) { + loop, requester := manualLoop(t, applierTestTarget("alpha", true)) + base, saved, later, rewritten := saveHistory(t, loop.w) + loop.w.inbox.observe(RemoteObservation{Commit: base, By: ObservedByFetch}) + loop.serviceAppliers() + startup := requester.taken() + require.Len(t, startup, 1) + startup[0].Report(ApplyResult{ID: startup[0].ID, Outcome: ApplySucceeded, AppliedRevision: base}) + loop.serviceAppliers() + + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "save", UID: "uid-save"} + loop.pendingCRs[id] = &pendingCommitRequest{ + id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, + } + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: saved}) + tip, by := tc.tip(base, saved, later, rewritten) + loop.w.inbox.observe(RemoteObservation{Commit: tip, By: by}) + loop.serviceAppliers() + sent := requester.taken() + require.Len(t, sent, 1) + require.Equal(t, tip, sent[0].Commit) + sent[0].Report(ApplyResult{ID: sent[0].ID, Outcome: ApplySucceeded, AppliedRevision: tc.applied(tip)}) + loop.serviceAppliers() + + got, known := loop.w.LookupSaveApply(id.Namespace, id.Name, id.UID) + require.True(t, known) + assert.Equal(t, tc.want.Done, got.Done, got.Message) + assert.Equal(t, tc.want.Applied, got.Applied, got.Message) + assert.Equal(t, tc.want.Reason, got.Reason, got.Message) + if !got.Done { + assert.Contains(t, loop.appliers[applierRef("alpha")].waitingSaves, id, "it waits on") + } + }) + } +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 2fd207c8..41acca1a 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1048,9 +1048,9 @@ type branchWorkerEventLoop struct { // Loop-goroutine only. pendingCRs map[commitRequestID]*pendingCommitRequest // savedCatchUps is the CommitRequests, by GitTarget, whose pushed commit asked their applier to - // catch up at once (catchUp: ImmediatelyAfterPush), until the appliers are next listed + // catch up at once (catchUp: AfterPush), with that commit, until the appliers are next listed // (takeReasons, save_apply.go). - savedCatchUps map[itypes.ResourceReference][]commitRequestID + savedCatchUps map[itypes.ResourceReference]map[commitRequestID]string // crSeq numbers registrations, so competing requests attach first come, first served. crSeq uint64 // refusalTimer fires at the earliest deadline in refusalPending. See refusal_reapply.go. diff --git a/internal/git/commit_request_attach.go b/internal/git/commit_request_attach.go index 3965e8db..c247d746 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -147,7 +147,7 @@ type AttachCommitRequest struct { // commit: whenNothingToCommit: CommitEmpty. CommitEmpty bool // CatchUpNow asks the GitTarget's applier to apply the tip as soon as the request's commit is - // pushed: catchUp: ImmediatelyAfterPush. + // pushed: catchUp: AfterPush. CatchUpNow bool } diff --git a/internal/git/save_apply.go b/internal/git/save_apply.go index 48d3d3c1..bcc9e22b 100644 --- a/internal/git/save_apply.go +++ b/internal/git/save_apply.go @@ -3,21 +3,26 @@ package git import ( + "maps" + "strings" "time" + gogit "github.com/go-git/go-git/v6" + "github.com/go-git/go-git/v6/plumbing" + itypes "github.com/ConfigButler/gitops-reverser/internal/types" ) -// A CommitRequest with catchUp: ImmediatelyAfterPush asks its GitTarget's applier to apply the tip -// as soon as its commit is pushed, and the controller reports how that went in its Applied -// condition. The save's own outcome is decided by the push alone; this is reported beside it and -// never changes it. +// A CommitRequest with catchUp: AfterPush asks its GitTarget's applier to apply the tip as soon as its +// commit is pushed, and the controller reports how that went in its Applied condition. The save's own +// outcome is decided by the push alone; this is reported beside it and never changes it. // -// The worker keeps, per applier, the saves waiting for an apply. The attempt that goes out carries -// them, so the one that succeeds settles them: any apply sent after a save's push applies a tip -// that holds the save's commit. An attempt that does not succeed gives them back, and the next -// carries them again. A save whose GitTarget names no applier, or that pushed no commit, or whose -// applier stops being named, is settled as not applied. +// The worker keeps, per applier, the saves waiting for an apply, each with the commit it pushed. The +// attempt that goes out carries them, and the one that succeeds settles each save whose commit the +// applied revision holds (settleSaves). An attempt that does not succeed gives them back, and the next +// carries them again. A save that pushed no commit is settled as not applied. One whose GitTarget names +// no applier, or whose applier stops being named, is settled as applied: nothing applies the branch, +// and the cluster, where the saved objects came from, already holds them. // Why a save's apply was settled, or why it still waits: the Applied condition's reasons. const ( @@ -27,13 +32,15 @@ const ( SaveApplyNothingPushed = "NothingPushed" SaveApplyNoApplier = "NoApplierClaim" SaveApplyApplierGone = "ApplierGone" + SaveApplyCommitDropped = "CommitDropped" ) // SaveApply is how the apply a CommitRequest asked for stands. type SaveApply struct { // Done is whether it is settled: it changes no more. Done bool - // Applied is whether the applier applied a commit that holds the save's. + // Applied is whether the applier applied a commit that holds the save's, or no applier applies + // the branch. Applied bool // Reason is one of the SaveApply* reasons, and Message says more. Reason string @@ -86,10 +93,13 @@ func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, comm return } if l.savedCatchUps == nil { - l.savedCatchUps = map[itypes.ResourceReference][]commitRequestID{} + l.savedCatchUps = map[itypes.ResourceReference]map[commitRequestID]string{} } ref := itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace) - l.savedCatchUps[ref] = append(l.savedCatchUps[ref], pcr.id) + if l.savedCatchUps[ref] == nil { + l.savedCatchUps[ref] = map[commitRequestID]string{} + } + l.savedCatchUps[ref][pcr.id] = commit l.w.recordSaveApply(pcr.id, SaveApply{Reason: SaveApplyApplying, Message: "commit " + commit + " is pushed; its applier is asked to apply it"}) } @@ -98,21 +108,22 @@ func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, comm // up, and records the request as waiting on it. It was asked for, so it is no lag converge: it does // not yield to an edit (convergeHolds). A request whose GitTarget names no applier is settled. func (l *branchWorkerEventLoop) takeSavedCatchUps() { - for ref, ids := range l.savedCatchUps { + for ref, saves := range l.savedCatchUps { state := l.applierNaming(ref) if state == nil { - for _, id := range ids { - l.w.recordSaveApply(id, SaveApply{Done: true, Reason: SaveApplyNoApplier, - Message: "GitTarget " + ref.String() + " names no applier claim that may be asked"}) + for id := range saves { + l.w.recordSaveApply(id, SaveApply{Done: true, Applied: true, Reason: SaveApplyNoApplier, + Message: "GitTarget " + ref.String() + " names no applier claim, so nothing applies the " + + "branch; the cluster already holds the saved objects"}) } continue } state.converge, state.lag = true, false - for _, id := range ids { - state.waitSave(id) + for id, commit := range saves { + state.waitSave(id, commit) } l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", - "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", ref.String(), "requests", len(ids)) + "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", ref.String(), "requests", len(saves)) } l.savedCatchUps = nil } @@ -129,43 +140,121 @@ func (l *branchWorkerEventLoop) applierNaming(ref itypes.ResourceReference) *app return nil } -// waitSave records a CommitRequest as waiting for the applier's next apply that succeeds. -func (s *applierState) waitSave(id commitRequestID) { +// waitSave records a CommitRequest, which pushed commit, as waiting for the applier's next apply that +// succeeds. +func (s *applierState) waitSave(id commitRequestID, commit string) { if s.waitingSaves == nil { - s.waitingSaves = map[commitRequestID]bool{} + s.waitingSaves = map[commitRequestID]string{} } - s.waitingSaves[id] = true + s.waitingSaves[id] = commit } -// settleSaves settles every save in saves as applied at revision. -func (l *branchWorkerEventLoop) settleSaves(saves map[commitRequestID]bool, revision string) { - for id := range saves { - l.w.recordSaveApply(id, SaveApply{Done: true, Applied: true, Reason: SaveApplyApplied, - AppliedRevision: revision, Message: "the applier applied " + revision}) +// settleSaves settles the saves an apply of revision answered, by what revision holds: +// +// - a save whose commit revision names or descends from is applied; +// - one whose commit the branch no longer holds either is not, and never will be: the branch's +// history was rewritten after the push; +// - one whose commit the branch holds and revision does not waits for the converge left pending; +// - one the checkout cannot tell about, because it lacks a commit, waits as well, and the remote is +// looked at: revision is a commit pushed elsewhere, and its fetch brings the apply that settles it. +func (l *branchWorkerEventLoop) settleSaves(state *applierState, saves map[commitRequestID]string, revision string) { + if len(saves) == 0 { + return + } + repo, err := gogit.PlainOpen(l.w.repoPath()) + if err != nil { + repo = nil + } + applied, tip := revisionCommit(revision), l.w.inbox.lastTip() + look := false + for id, commit := range saves { + held, known := commitHolds(repo, applied, commit) + if held { + l.w.recordSaveApply(id, SaveApply{Done: true, Applied: true, Reason: SaveApplyApplied, + AppliedRevision: revision, Message: "the applier applied " + revision}) + continue + } + onBranch, branchKnown := commitHolds(repo, tip, commit) + switch { + case known && branchKnown && !onBranch: + l.w.recordSaveApply(id, SaveApply{Done: true, Reason: SaveApplyCommitDropped, Message: "the applier " + + "applied " + revision + ", and the branch no longer holds commit " + commit + ": its history was " + + "rewritten after the push"}) + continue + case known: + state.converge, state.lag = true, false + l.w.recordSaveApply(id, SaveApply{ + Reason: SaveApplyRetrying, + Message: "the applier applied " + revision + ", which does not hold commit " + commit + "; asking again", + }) + default: + look = look || applied != "" + l.w.recordSaveApply(id, SaveApply{Reason: SaveApplyRetrying, Message: "the applier applied " + revision + + ", and whether it holds commit " + commit + " is not known here yet; looking at the remote"}) + } + state.waitSave(id, commit) + } + if look { + l.w.EnqueuePushNotification(applied) } } +// commitHolds reports whether descendant is commit or descends from it, and whether the checkout could +// tell: it holds both commits. +func commitHolds(repo *gogit.Repository, descendant, commit string) (bool, bool) { + if descendant == "" || commit == "" { + return false, false + } + if strings.EqualFold(descendant, commit) { + return true, true + } + if repo == nil { + return false, false + } + descendantID, ok := plumbing.FromHex(descendant) + if !ok { + return false, false + } + commitID, ok := plumbing.FromHex(commit) + if !ok { + return false, false + } + later, err := repo.CommitObject(descendantID) + if err != nil { + return false, false + } + earlier, err := repo.CommitObject(commitID) + if err != nil { + return false, false + } + held, err := earlier.IsAncestor(later) + if err != nil { + return false, false + } + return held, true +} + // retrySaves records the saves an attempt that did not succeed gave back as waiting again, with why. -func (l *branchWorkerEventLoop) retrySaves(saves map[commitRequestID]bool, why string) { +func (l *branchWorkerEventLoop) retrySaves(saves map[commitRequestID]string, why string) { for id := range saves { l.w.recordSaveApply(id, SaveApply{Reason: SaveApplyRetrying, Message: "the apply did not complete, and is asked again: " + why}) } } -// dropSaves settles the saves of an applier no GitTarget names any more: nobody asks it for anything. +// dropSaves settles the saves of an applier no GitTarget names any more as applied: nothing applies the +// branch for them now, and the cluster already holds the saved objects. func (l *branchWorkerEventLoop) dropSaves(state *applierState) { - saves := state.waitingSaves + saves := maps.Clone(state.waitingSaves) if state.inflight != nil { - for id := range state.inflight.saves { - if saves == nil { - saves = map[commitRequestID]bool{} - } - saves[id] = true + if saves == nil { + saves = map[commitRequestID]string{} } + maps.Copy(saves, state.inflight.saves) } for id := range saves { - l.w.recordSaveApply(id, SaveApply{Done: true, Reason: SaveApplyApplierGone, - Message: "applier " + state.ref.String() + " is no longer named on this branch"}) + l.w.recordSaveApply(id, SaveApply{Done: true, Applied: true, Reason: SaveApplyApplierGone, + Message: "applier " + state.ref.String() + " is no longer named on this branch, so nothing applies " + + "it; the cluster already holds the saved objects"}) } } diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index 926bb977..8a85d720 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -364,7 +364,7 @@ data: const configMap, request = "applier-converge-edit", "applier-save-catch-up" setFluxClaimCatchUpAfter(testNs, fluxName, "0s") - By("saving an API edit with catchUp: ImmediatelyAfterPush") + By("saving an API edit with catchUp: AfterPush") _, err := kubectlRunWithStdin(testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 kind: CommitRequest metadata: @@ -374,7 +374,7 @@ spec: gitTargetRef: name: %s message: "e2e: save and apply" - catchUp: ImmediatelyAfterPush + catchUp: AfterPush window: attachTimeout: "4s" maxDuration: "4s" From c5afe20b6dfed46b4f167d8b60d5eca518ec2a8e Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 15:15:03 +0000 Subject: [PATCH 07/33] fix(commitrequest): a save that needed no apply is Applied=True, NothingToApply A request that pushed no commit has its content in Git already and on the cluster, so it is Applied=True with reason NothingToApply, as one whose GitTarget names no applier is. Applied=False is left for something wrong: CommitDropped, a commit the branch no longer holds. A client waiting for Applied=True never sees False for a save that went fine. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/commitrequest_types.go | 10 ++++---- .../bases/configbutler.ai_commitrequests.yaml | 10 ++++---- docs/configuration.md | 5 ++-- docs/design/apply-trigger.md | 5 ++-- .../controller/commitrequest_apply_test.go | 11 +++++---- internal/git/apply_converge_test.go | 8 +++---- internal/git/save_apply.go | 24 +++++++++---------- 7 files changed, 37 insertions(+), 36 deletions(-) diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 9f086d55..3409f2c9 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -218,12 +218,12 @@ type CommitRequestStatus struct { // complete and is asked again; the message says why. // - True, reason Applied: the applier applied a revision that holds the // commit, named in status.appliedRevision. - // - True, reason NoApplierClaim or ApplierGone: the GitTarget names no - // applier claim, or stopped naming the one asked, and the cluster - // already holds the saved objects. - // - False, reason NothingPushed: there is no commit to apply. + // - True, reason NothingToApply, NoApplierClaim or ApplierGone: no apply + // was needed. The request pushed no commit, or the GitTarget names no + // applier claim, or stopped naming the one asked; the cluster already + // holds the saved objects. // - False, reason CommitDropped: the branch no longer holds the commit; - // its history was rewritten after the push. + // its history was rewritten after the push. Only this is False. // - Unknown, reason Lost: GitOps Reverser restarted before it knew. // +optional // +listType=map diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index e373fccb..69cf73ce 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -267,12 +267,12 @@ spec: complete and is asked again; the message says why. - True, reason Applied: the applier applied a revision that holds the commit, named in status.appliedRevision. - - True, reason NoApplierClaim or ApplierGone: the GitTarget names no - applier claim, or stopped naming the one asked, and the cluster - already holds the saved objects. - - False, reason NothingPushed: there is no commit to apply. + - True, reason NothingToApply, NoApplierClaim or ApplierGone: no apply + was needed. The request pushed no commit, or the GitTarget names no + applier claim, or stopped naming the one asked; the cluster already + holds the saved objects. - False, reason CommitDropped: the branch no longer holds the commit; - its history was rewritten after the push. + its history was rewritten after the push. Only this is False. - Unknown, reason Lost: GitOps Reverser restarted before it knew. items: description: Condition contains details for one aspect of the current diff --git a/docs/configuration.md b/docs/configuration.md index 6fafc0df..754f7a99 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -3141,9 +3141,8 @@ already at the tip needs none. The request reports the outcome in its `Applied` | `Unknown` | `Applying` | the apply is asked for or runs | | `Unknown` | `Retrying` | it did not complete and is asked again; the message says why | | `True` | `Applied` | the applier applied a revision that holds the commit; `status.appliedRevision` names it, as the applier reported it | -| `True` | `NoApplierClaim`, `ApplierGone` | the `GitTarget` names no applier claim, or stopped naming the one asked: nothing applies the branch, and the cluster already holds what was saved | -| `False` | `NothingPushed` | the request pushed no commit, so there is nothing to apply | -| `False` | `CommitDropped` | the branch no longer holds the commit: its history was rewritten after the push | +| `True` | `NothingToApply`, `NoApplierClaim`, `ApplierGone` | no apply was needed: the request pushed no commit, or the `GitTarget` names no applier claim, or stopped naming the one asked. The cluster already holds what was saved | +| `False` | `CommitDropped` | the branch no longer holds the commit: its history was rewritten after the push. The only `False` | | `Unknown` | `Lost` | GitOps Reverser restarted before it learnt how the apply went | Wait for `Applied=True` for "saved and applied"; `Ready=True` stays "saved". A finished request is diff --git a/docs/design/apply-trigger.md b/docs/design/apply-trigger.md index d5d4b438..4cdf32ca 100644 --- a/docs/design/apply-trigger.md +++ b/docs/design/apply-trigger.md @@ -284,8 +284,9 @@ deletes its file the applier leaves it on the cluster, and a later resync writes carries it succeeds, and is settled applied only when the applied revision holds that commit: a force-push that dropped it settles it not applied, and a revision the checkout does not hold yet keeps it waiting while the remote is looked at. The request's `Applied` condition reports it: - retried with the applier's backoff, settled applied, settled applied when no applier is named - (the cluster already holds the save), or settled not applied when nothing was pushed + retried with the applier's backoff, or settled applied, also when no apply was needed (nothing + was pushed, or no applier is named: the cluster already holds the save). `False` is kept for + something wrong, a dropped commit, so a client never sees it for a save that went fine (`save_apply.go`). - **No apply before every initial snapshot is published.** An applier is asked for nothing, for any reason, until every collection of every `GitTarget` it covers has a settled snapshot (applied, diff --git a/internal/controller/commitrequest_apply_test.go b/internal/controller/commitrequest_apply_test.go index 470d4ec5..b8adbd8c 100644 --- a/internal/controller/commitrequest_apply_test.go +++ b/internal/controller/commitrequest_apply_test.go @@ -55,15 +55,16 @@ func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { assert.Equal(t, lookups, f.applyLookups, "settled: not looked up again") } -// TestCommitRequestReconcile_ASettledApplyWithoutAnApplier: a request that pushed nothing is -// Applied=False; one whose GitTarget names no applier is Applied=True, since the cluster already holds -// what it saved and nothing applies the branch. Neither touches Ready. -func TestCommitRequestReconcile_ASettledApplyWithoutAnApplier(t *testing.T) { +// TestCommitRequestReconcile_ASettledApplyWithoutAnApply: a request that pushed nothing, or whose +// GitTarget names no applier, is Applied=True: no apply was needed, and the cluster already holds what +// it saved. One whose commit the branch lost is Applied=False. None touches Ready. +func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { for _, tc := range []struct { apply git.SaveApply want metav1.ConditionStatus }{ - {git.SaveApply{Done: true, Reason: git.SaveApplyNothingPushed}, metav1.ConditionFalse}, + {git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyNothingToApply}, metav1.ConditionTrue}, + {git.SaveApply{Done: true, Reason: git.SaveApplyCommitDropped}, metav1.ConditionFalse}, {git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyNoApplier}, metav1.ConditionTrue}, } { t.Run(tc.apply.Reason, func(t *testing.T) { diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index 8c33b6e6..26e74e47 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -232,8 +232,8 @@ func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { // TestSaveApply_FollowsTheAttemptThatCarriesIt: a save that asked to catch up waits on its applier, // is carried by the attempt sent next, is retried when that attempt does not succeed, and is -// settled applied by the one that does. One that pushed nothing is settled at once as not applied, -// and one whose GitTarget names no applier as applied: nothing applies the branch. +// settled applied by the one that does. One that pushed nothing, or whose GitTarget names no applier, +// is settled at once as applied: no apply is needed. func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true), applierTestTarget("loner", false)) convergedAt(t, loop, requester) @@ -253,8 +253,8 @@ func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { } nothing := save("nothing", "alpha", "") - assert.Equal(t, SaveApply{Done: true, Reason: SaveApplyNothingPushed, Message: lookup(nothing).Message}, - lookup(nothing)) + assert.Equal(t, SaveApply{Done: true, Applied: true, Reason: SaveApplyNothingToApply, + Message: lookup(nothing).Message}, lookup(nothing)) loop.w.inbox.observe(RemoteObservation{Commit: "bbb", By: ObservedByPush}) pushed := save("pushed", "alpha", "bbb") diff --git a/internal/git/save_apply.go b/internal/git/save_apply.go index bcc9e22b..e9149957 100644 --- a/internal/git/save_apply.go +++ b/internal/git/save_apply.go @@ -20,19 +20,19 @@ import ( // The worker keeps, per applier, the saves waiting for an apply, each with the commit it pushed. The // attempt that goes out carries them, and the one that succeeds settles each save whose commit the // applied revision holds (settleSaves). An attempt that does not succeed gives them back, and the next -// carries them again. A save that pushed no commit is settled as not applied. One whose GitTarget names -// no applier, or whose applier stops being named, is settled as applied: nothing applies the branch, -// and the cluster, where the saved objects came from, already holds them. +// carries them again. A save that pushed no commit, whose GitTarget names no applier, or whose applier +// stops being named is settled as applied: no apply is needed, and the cluster, where the saved objects +// came from, already holds them. Only a save whose commit the branch lost is settled as not applied. // Why a save's apply was settled, or why it still waits: the Applied condition's reasons. const ( - SaveApplyApplying = "Applying" - SaveApplyRetrying = "Retrying" - SaveApplyApplied = "Applied" - SaveApplyNothingPushed = "NothingPushed" - SaveApplyNoApplier = "NoApplierClaim" - SaveApplyApplierGone = "ApplierGone" - SaveApplyCommitDropped = "CommitDropped" + SaveApplyApplying = "Applying" + SaveApplyRetrying = "Retrying" + SaveApplyApplied = "Applied" + SaveApplyNothingToApply = "NothingToApply" + SaveApplyNoApplier = "NoApplierClaim" + SaveApplyApplierGone = "ApplierGone" + SaveApplyCommitDropped = "CommitDropped" ) // SaveApply is how the apply a CommitRequest asked for stands. @@ -88,8 +88,8 @@ func (w *BranchWorker) collectSaveApplies(now time.Time) { // this wake (takeSavedCatchUps); with none, there is nothing to apply. func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, commit string) { if commit == "" { - l.w.recordSaveApply(pcr.id, SaveApply{Done: true, Reason: SaveApplyNothingPushed, - Message: "the request pushed no commit, so there is nothing to apply"}) + l.w.recordSaveApply(pcr.id, SaveApply{Done: true, Applied: true, Reason: SaveApplyNothingToApply, + Message: "the request pushed no commit: what it saved was in Git already, and the cluster holds it"}) return } if l.savedCatchUps == nil { From 2a2e1c7bd75299ac9666ae5bb7839a5a3c24e0b1 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 16:08:25 +0000 Subject: [PATCH 08/33] test(e2e): Flux applying into a kcp workspace the GitTarget mirrors Scenario 10 of the source-cluster corner is gitops-api's production topology (ask #34): a Kustomization in flux-system applies into a kcp workspace through spec.kubeConfig, the GitTarget mirrors that workspace through a secretRef KubeProvider, and the FluxClaim holds the Kustomization through the install's own provider. It shows the startup converge landing in the workspace, an edit made in the workspace during an apply withheld from the apply's revert (E5's sequence, now shared as echoRun.editDuringApply, plus the withheld apply_window_collisions_total going up), and two saves with catchUp AfterPush both reaching Applied=True. Co-Authored-By: Claude Opus 5.5 (1M context) --- test/e2e/bi_directional_echoes_e2e_test.go | 51 ++-- test/e2e/bi_directional_timing_e2e_test.go | 36 +-- test/e2e/source_cluster_e2e_test.go | 258 ++++++++++++++++++++- 3 files changed, 312 insertions(+), 33 deletions(-) diff --git a/test/e2e/bi_directional_echoes_e2e_test.go b/test/e2e/bi_directional_echoes_e2e_test.go index c65c1f63..3e1b6e4a 100644 --- a/test/e2e/bi_directional_echoes_e2e_test.go +++ b/test/e2e/bi_directional_echoes_e2e_test.go @@ -93,6 +93,16 @@ type echoRun struct { *timingPipeline applier echoApplier + // kubectl runs kubectl in testNs on the cluster the GitTarget mirrors; nil is the operator's own. + kubectl func(args ...string) (string, error) +} + +// liveKubectl runs kubectl in testNs on the cluster the GitTarget mirrors. +func (r *echoRun) liveKubectl(args ...string) (string, error) { + if r.kubectl != nil { + return r.kubectl(args...) + } + return kubectlRunInNamespace(r.testNs, args...) } // cmFile is where the GitTarget writes the ConfigMap name. @@ -124,7 +134,7 @@ func (r *echoRun) createThroughAPI(name, value string) { // editThroughAPI sets data.value of the ConfigMap name through the Kubernetes API. func (r *echoRun) editThroughAPI(name, value string) { GinkgoHelper() - _, err := kubectlRunInNamespace(r.testNs, "patch", "configmap", name, "--type=merge", "-p", + _, err := r.liveKubectl("patch", "configmap", name, "--type=merge", "-p", fmt.Sprintf(`{"data":{"value":%q}}`, value)) Expect(err).NotTo(HaveOccurred(), "failed to edit ConfigMap %s through the API", name) } @@ -144,7 +154,7 @@ func (r *echoRun) committedValue(g Gomega, name string) (string, bool) { // liveValue is data.value of the live ConfigMap name, and whether it exists. func (r *echoRun) liveValue(g Gomega, name string) (string, bool) { - out, err := kubectlRunInNamespace(r.testNs, "get", "configmap", name, "--ignore-not-found", + out, err := r.liveKubectl("get", "configmap", name, "--ignore-not-found", "-o", "jsonpath={.metadata.name}|{.data.value}") g.Expect(err).NotTo(HaveOccurred()) gotName, value, _ := strings.Cut(strings.TrimSpace(out), "|") @@ -220,6 +230,27 @@ func (r *echoRun) eventuallyPruned(name string) { }, echoHold, 2*time.Second).Should(Succeed()) } +// editDuringApply pushes a Git-side change to one object and edits another through the API while +// that apply runs: the apply reverts the edit on the cluster, and the window withholds that write. +// The edit is in Git at once, and on the cluster again after the Repair (E5). +func (r *echoRun) editDuringApply() { + GinkgoHelper() + r.pushFromOutside("during", "d0") + r.eventuallyEverywhere("during", "d0", echoApplyBudget) + + By("pushing a Git-side change to another object, and editing this one while that apply runs") + before := r.applier.operation(Default).Token + r.pushFromOutside("during-other", "o1") + Eventually(func(g Gomega) { + g.Expect(r.applier.operation(g).Token).NotTo(Equal(before), "the apply is asked for") + }, echoApplyBudget, 250*time.Millisecond).Should(Succeed()) + r.editThroughAPI("during", "d-edit") + + r.eventuallyEverywhere("during", "d-edit", echoApplyBudget) + r.eventuallyEverywhere("during-other", "o1", echoApplyBudget) + r.consistentlyEverywhere("during", "d-edit") +} + // echoScenarios are the scenarios that run the same for both appliers, against run. func echoScenarios(run func() *echoRun) { It("adopts an object the API created, so a Git-side delete prunes it (E2)", func() { @@ -250,21 +281,7 @@ func echoScenarios(run func() *echoRun) { }) It("keeps an edit made during an apply, in Git at once and on the cluster after the Repair (E5)", func() { - r := run() - r.pushFromOutside("during", "d0") - r.eventuallyEverywhere("during", "d0", echoApplyBudget) - - By("pushing a Git-side change to another object, and editing this one while that apply runs") - before := r.applier.operation(Default).Token - r.pushFromOutside("during-other", "o1") - Eventually(func(g Gomega) { - g.Expect(r.applier.operation(g).Token).NotTo(Equal(before), "the apply is asked for") - }, echoApplyBudget, 250*time.Millisecond).Should(Succeed()) - r.editThroughAPI("during", "d-edit") - - r.eventuallyEverywhere("during", "d-edit", echoApplyBudget) - r.eventuallyEverywhere("during-other", "o1", echoApplyBudget) - r.consistentlyEverywhere("during", "d-edit") + run().editDuringApply() }) It("applies a Git-side push while edits keep arriving, and keeps every edit (E6)", func() { diff --git a/test/e2e/bi_directional_timing_e2e_test.go b/test/e2e/bi_directional_timing_e2e_test.go index e85f3558..c4604cba 100644 --- a/test/e2e/bi_directional_timing_e2e_test.go +++ b/test/e2e/bi_directional_timing_e2e_test.go @@ -129,21 +129,7 @@ func (p *timingPipeline) start() { p.hookID = registerOperatorReceiver(p.testNs, p.receiverName, p.repo.RepoName) createGitTarget(p.targetName, p.testNs, p.providerName, p.gitPath, "main") - _, err = kubectlRunWithStdin(p.testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: WatchRule -metadata: - name: %s - namespace: %s -spec: - gitTargetRef: - name: %s - rules: - - resources: ["configmaps"] - objectSelector: - matchLabels: - %s: "true" -`, p.ruleName, p.testNs, p.targetName, timingLabelKey), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply the WatchRule") + p.applyWatchRule() verifyResourceStatus("gittarget", p.targetName, p.testNs, "True", "Succeeded", "") waitForStreamsRunning(p.targetName, p.testNs) @@ -163,6 +149,26 @@ data: }, timingCommitBudget, timingPoll).Should(Succeed(), "the ConfigMap must be in Git before an applier applies it") } +// applyWatchRule applies the WatchRule that mirrors the labeled ConfigMaps of testNs to the GitTarget. +func (p *timingPipeline) applyWatchRule() { + GinkgoHelper() + _, err := kubectlRunWithStdin(p.testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: %s + namespace: %s +spec: + gitTargetRef: + name: %s + rules: + - resources: ["configmaps"] + objectSelector: + matchLabels: + %s: "true" +`, p.ruleName, p.testNs, p.targetName, timingLabelKey), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to apply the WatchRule") +} + func (p *timingPipeline) cleanup() { deleteRepoHook(p.repo.RepoName, p.hookID) cleanupWatchRule(p.ruleName, p.testNs) diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index a1f6c524..d8b50637 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -19,7 +19,7 @@ import ( // the provider's OWN namespace, or spec.inCluster where the installation allows it), for which // other namespaces may use it (spec.allowedGitTargets), and for its audit route. // -// Three kinds of spec live here: +// Four kinds of spec live here: // - Input-validation specs (Scenario 1) assert the KubeProvider reconciler's Validated verdict on // bad kubeconfigs and on an inCluster provider the installation does not list, without a // remote cluster and without a dial. @@ -31,6 +31,12 @@ import ( // Scenario 8 holds the SAME namespace + resource with different content in three workspaces, // proving state is keyed by source cluster, not (namespace, GVR); Scenario 9 gives two // providers ONE name in two namespaces, and recreates one of them. +// - The applier spec (Scenario 10) is an applier in this cluster applying into a remote one, as +// gitops-api runs: Flux applies a folder into a workspace through Kustomization.spec.kubeConfig, +// the GitTarget mirrors that workspace, and its FluxClaim holds the Kustomization through the +// install's own provider. It shows the startup converge landing in the workspace, an edit made +// there during an apply withheld from the apply's revert by the REMOTE object's managedFields, +// and two saves that ask to catch up reporting the apply. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), // and the kcp specs additionally Skip when kcp is not installed — so a default `task test-e2e` @@ -635,4 +641,254 @@ spec: expectMirroredValue(g, repo.CheckoutDir, "clusters/twin-a", "twin", "alpha") }).WithTimeout(420 * time.Second).WithPolling(5 * time.Second).Should(Succeed()) }) + + // Scenario 10 — an applier in this cluster applying into the remote one: Flux applies a folder + // into a workspace through Kustomization.spec.kubeConfig, the GitTarget mirrors that workspace + // through a secretRef KubeProvider, and the FluxClaim holds the Kustomization through the + // install's own provider. So the apply window judges the managedFields of the REMOTE object. It + // has its own repository, so no GitTarget of the scenarios above shares its branch. + // + // The e2e kcp has no audit stream, so no write in the workspace names its author: everything is + // mirrored from the watch, and every event waits out the attribution grace first. + Describe("Flux applying into a workspace the GitTarget mirrors", Ordered, func() { + const ws = "sc-flux" + var ( + p *timingPipeline + r *echoRun + ksName string + ) + + BeforeAll(func() { + if kcp == nil { + Skip("kcp is not installed; run this corner via `task test-e2e-source-cluster`") + } + grantAppliersInAnyNamespace() + ensurePrometheusClient() + fluxRepo := SetupRepo(resolveE2EContext(), testNs, + fmt.Sprintf("e2e-source-cluster-flux-%d", GinkgoRandomSeed())) + p = newTimingPipeline(testNs, ws, fluxRepo) + ksName = ws + "-" + testNs + _, err := kubectlRunInNamespace(testNs, "apply", "-f", fluxRepo.SecretsYAML) + Expect(err).NotTo(HaveOccurred(), "failed to apply repo secrets") + configureRepoOriginWithCredentials(fluxRepo, testNs) + + By("provisioning the workspace, and the namespace the folder holds inside it") + DeferCleanup(kcp.deleteWorkspace, ws) + hash := kcp.createWorkspace(ws) + _, err = kcp.wsKubectl(hash, "create", "namespace", testNs) + Expect(err).NotTo(HaveOccurred(), "create the folder's namespace in %s", ws) + + By("mirroring the workspace: KubeProvider, GitProvider with a receiver, GitTarget, WatchRule") + DeferCleanup(func() { + deleteKubeProvider(testNs, ws) + cleanupNamespacedResource(testNs, "secret", ws+"-kubeconfig") + }) + writeKubeConfigSecret(testNs, ws+"-kubeconfig", "value", kcp.operatorKubeConfig(hash)) + _, err = applyKubeProvider(testNs, ws, ws+"-kubeconfig", "") + Expect(err).NotTo(HaveOccurred()) + DeferCleanup(p.cleanup) + createReadyGitProvider(p.providerName, testNs, fluxRepo.GitSecretHTTP, fluxRepo.RepoURLHTTP) + p.hookID = registerOperatorReceiver(testNs, p.receiverName, fluxRepo.RepoName) + _, err = applyGitTargetWithKubeProvider(testNs, p.targetName, p.providerName, "", p.gitPath, testNs, ws) + Expect(err).NotTo(HaveOccurred()) + verifyResourceCondition("gittarget", p.targetName, testNs, "SourceClusterReachable", "True", "", "", "180s") + p.applyWatchRule() + waitForStreamsRunning(p.targetName, testNs) + + r = &echoRun{ + timingPipeline: p, + applier: echoApplier{ + name: "Flux", + owns: func(g Gomega, ns, configMap string) bool { + out, err := kcp.wsKubectl(hash, "-n", ns, "get", "configmap", configMap, + `-o=jsonpath={.metadata.labels.kustomize\.toolkit\.fluxcd\.io/name}`) + g.Expect(err).NotTo(HaveOccurred()) + return strings.TrimSpace(out) == ksName + }, + operation: func(g Gomega) claimOperation { + return readClaimOperation(g, testNs, fluxClaimResource, ksName) + }, + }, + kubectl: func(args ...string) (string, error) { + return kcp.wsKubectl(hash, append([]string{"-n", testNs}, args...)...) + }, + } + + By("creating a ConfigMap in the workspace, and waiting until the GitTarget mirrored it") + _, err = r.liveKubectl("create", "configmap", timingConfigMap, "--from-literal=value=seed") + Expect(err).NotTo(HaveOccurred()) + _, err = r.liveKubectl("label", "configmap", timingConfigMap, timingLabelKey+"=true") + Expect(err).NotTo(HaveOccurred()) + r.eventuallyEverywhere(timingConfigMap, "seed", echoApplyBudget) + + By("creating the Kustomization, which applies the folder into the workspace once") + DeferCleanup(func() { + cleanupNamespacedResource("flux-system", "kustomization", ksName) + cleanupNamespacedResource("flux-system", "gitrepository", ksName) + cleanupNamespacedResource("flux-system", "secret", ksName+"-git") + cleanupNamespacedResource("flux-system", "secret", ksName+"-kubeconfig") + }) + applyFluxIntoWorkspace(testNs, fluxRepo, ksName, kcp.operatorKubeConfig(hash), "./"+p.gitPath) + + By("pushing a Git-side change the Kustomization has not fetched, then holding it with a FluxClaim") + r.pushFromOutside("converged", "c1") + DeferCleanup(cleanupNamespacedResource, testNs, fluxClaimResource, ksName) + createFluxClaim(testNs, ksName, ksName) + nameApplierClaim(testNs, p.targetName, "FluxClaim", ksName, "") + }) + + // Naming the FluxClaim converges it once: a CatchUp of the tip, which holds a ConfigMap only + // Git has, since the GitRepository's 24h interval never fetched it. So the ConfigMap reaches + // the workspace through the FluxClaim's apply alone, and Flux's labels on it say who applied it. + It("applies its startup converge into the workspace, and mirrors what it applied", func() { + verifyResourceCondition(fluxClaimResource, ksName, testNs, "Configured", "True", "", "") + verifyResourceCondition(fluxClaimResource, ksName, testNs, "Ready", "True", "Succeeded", "") + verifyResourceCondition("gittarget", p.targetName, testNs, "ApplierClaimReady", "True", "Succeeded", "") + Eventually(func(g Gomega) { + head := p.remoteBranchHead(g) + g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(head)) + done := r.applier.operation(g) + g.Expect(done.Reason).To(Equal("CatchUp")) + g.Expect(done.Commit).To(Equal(head)) + g.Expect(done.Phase).To(Equal("Succeeded")) + live, exists := r.liveValue(g, "converged") + g.Expect(exists).To(BeTrue(), "the converge applies the Git-side change into the workspace") + g.Expect(live).To(Equal("c1")) + g.Expect(r.applier.owns(g, testNs, "converged")).To(BeTrue(), "Flux applied it") + g.Expect(r.applier.owns(g, testNs, timingConfigMap)).To(BeTrue(), "Flux adopted the mirrored one") + }, echoApplyBudget, 2*time.Second).Should(Succeed()) + verifyResourceCondition(fluxClaimResource, ksName, testNs, "InControl", "True", "Suspended", "") + r.consistentlyEverywhere("converged", "c1") + r.consistentlyEverywhere(timingConfigMap, "seed") + }) + + // The collision the apply window exists for, as E5 provokes it: an edit through the workspace's + // API while an apply runs. The apply reverts it in the workspace; the window withholds that + // write, because the object's managedFields name kustomize-controller alone, after the edit. + It("withholds an apply that reverts an edit made in the workspace during it", func() { + withheld := fmt.Sprintf(`sum(gitopsreverser_apply_window_collisions_total{gitbranch_namespace=%q,`+ + `gitbranch_name=%q,outcome="withheld"}) or vector(0)`, testNs, gitBranchName(p.providerName, "main")) + before, err := queryPrometheus(withheld) + Expect(err).NotTo(HaveOccurred()) + + r.editDuringApply() + waitForMetricWithTimeout(withheld, func(v float64) bool { return v > before }, + "the apply over the workspace edit is counted as a withheld collision", time.Minute) + }) + + // A save on the workspace's GitTarget that asks to catch up is applied once it is pushed, and + // each request reports the apply: Applied=True, with the revision Flux applied. Two saves made + // together may share one apply. + // + // The saves record empty commits (whenNothingToCommit: CommitEmpty). A request's author is + // captured at admission in this cluster, and with no audit stream from kcp a write in the + // workspace names no author, so a request never attaches to that write's window. + It("applies two saves that ask to catch up, and reports the apply on each", func() { + names := []string{"sc-flux-save-1", "sc-flux-save-2"} + for _, name := range names { + applyCatchUpSave(testNs, name, p.targetName) + } + commits := map[string]string{} + for _, name := range names { + Eventually(func(g Gomega) { + g.Expect(commitRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal("NoWindow")) + g.Expect(commitRequestCondition(g, testNs, name, "Pushed")).To(Equal("True")) + commits[name] = commitRequestField(g, testNs, name, "{.status.commit}") + g.Expect(commits[name]).NotTo(BeEmpty()) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + } + + last := commits[names[len(names)-1]] + Eventually(func(g Gomega) { + for _, name := range names { + applied := commitRequestCondition(g, testNs, name, "Applied") + reason := commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) + if applied == "False" { + StopTrying(fmt.Sprintf("CommitRequest %s is Applied=False (%s): only a commit the branch "+ + "lost may be", name, reason)).Now() + } + g.Expect(applied).To(Equal("True"), "CommitRequest %s Applied (%s)", name, reason) + g.Expect(reason).To(Equal("Applied")) + // A later save's commit holds the earlier one's, so one apply may settle both. + g.Expect(commitRequestField(g, testNs, name, "{.status.appliedRevision}")). + To(Or(ContainSubstring(commits[name]), ContainSubstring(last))) + } + g.Expect(commitRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). + To(ContainSubstring(last)) + g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) + }, echoApplyBudget, 2*time.Second).Should(Succeed()) + }) + }) }) + +// applyFluxIntoWorkspace creates, in flux-system, a GitRepository following main and a Kustomization +// applying path from it into a kcp workspace: its spec.kubeConfig names a Secret holding kubeconfig +// under the key "value", which kustomize-controller, in this cluster, dials. Both are on a 24h +// interval, so a later commit is applied only when the FluxClaim asks. It waits for the first apply, +// made before any applier claim holds the Kustomization. +func applyFluxIntoWorkspace(testNs string, repo *RepoArtifacts, name, kubeconfig, path string) { + GinkgoHelper() + writeKubeConfigSecret("flux-system", name+"-kubeconfig", "value", kubeconfig) + fields := struct { + Namespace, SecretName, Name, RepoURL, Branch, Interval, Username, Password string + }{ + Namespace: "flux-system", SecretName: name + "-git", Name: name, RepoURL: repo.RepoURLHTTP, + Branch: "main", Interval: "24h", + } + checkout := gitCheckout{secretNamespace: testNs, gitSecretName: repo.GitSecretHTTP} + fields.Username, fields.Password = checkout.readGitCredentialSecretDataBase64() + Expect(applyFromTemplate("test/e2e/templates/bi-directional/flux-gitrepository-http.tmpl", fields, + "flux-system")).To(Succeed(), "failed to apply the Flux GitRepository") + _, err := kubectlRunWithStdin("flux-system", fmt.Sprintf(`apiVersion: kustomize.toolkit.fluxcd.io/v1 +kind: Kustomization +metadata: + name: %s + namespace: flux-system +spec: + interval: 24h + timeout: 2m + path: %s + prune: true + sourceRef: + kind: GitRepository + name: %s + kubeConfig: + secretRef: + name: %s-kubeconfig + key: value +`, name, path, name, name), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to apply the Flux Kustomization") + // Ready's message is where an unreachable workspace shows: a TLS or DNS failure to the front-proxy. + Eventually(func(g Gomega) { + out, err := kubectlRunInNamespace("flux-system", "get", "kustomization", name, "-o", + `jsonpath={.status.conditions[?(@.type=="Ready")].status}|`+ + `{.status.conditions[?(@.type=="Ready")].message}`) + g.Expect(err).NotTo(HaveOccurred()) + status, message, _ := strings.Cut(strings.TrimSpace(out), "|") + g.Expect(status).To(Equal("True"), "Kustomization %s is not Ready: %s", name, message) + }, 3*time.Minute, 2*time.Second).Should(Succeed(), "the Kustomization must apply its folder into the workspace") +} + +// applyCatchUpSave creates the CommitRequest name on gitTarget, asking its applier to catch up once +// it is pushed. It attaches only to a window already open, and records its message in an empty +// commit when there is none. +func applyCatchUpSave(namespace, name, gitTarget string) { + GinkgoHelper() + _, err := kubectlRunWithStdin(namespace, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 +kind: CommitRequest +metadata: + name: %s + namespace: %s +spec: + gitTargetRef: + name: %s + message: "e2e: save %s and apply it" + catchUp: AfterPush + whenNothingToCommit: CommitEmpty + window: + attachTimeout: "0s" +`, name, namespace, gitTarget, name), "apply", "-f", "-") + Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s", name) +} From 5ccaf54a2dc2963c675d0f6cc02d0450a96560dc Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 16:29:44 +0000 Subject: [PATCH 09/33] feat(commitrequest): take over a window whose writes name no author, from a cluster that sends no audit facts A CommitRequest attached only to a window whose author matched its submitter's. A write on a cluster with no audit stream names no author, so a save there could never attach: it ended WindowMismatch. When the GitTarget's KubeProvider has never received an audit fact (AuditFactsReceived is not True, read from its status), a request that names its submitter now also takes over a window on its GitTarget whose writes name no author. The submitter authors the commit, and the message ends with "Author-Source: CommitRequest /; no audit record names who made these changes". Once the provider has received a fact, a write naming no author means its fact went missing, and its window stays another's. The source-cluster Flux spec now saves real edits in the kcp workspace this way, and accepts ForeignCommit as well as CatchUp as the reason for the apply that follows naming the FluxClaim. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/commitrequest_types.go | 11 +++- .../bases/configbutler.ai_commitrequests.yaml | 8 ++- docs/architecture.md | 10 +++- .../controller/commitrequest_controller.go | 56 ++++++++++++++----- .../commitrequest_controller_unit_test.go | 44 +++++++++++++++ internal/git/attribution_outcome_test.go | 39 +++++++++++++ internal/git/branch_worker.go | 4 ++ internal/git/commit_executor.go | 16 ++++++ internal/git/commit_request_attach.go | 23 ++++++-- internal/git/commit_request_attach_loop.go | 8 +++ internal/git/commit_request_attach_test.go | 38 +++++++++++++ internal/git/open_window.go | 3 + internal/git/pending_writes.go | 4 +- internal/git/types.go | 4 ++ test/e2e/source_cluster_e2e_test.go | 55 +++++++++++------- 15 files changed, 275 insertions(+), 48 deletions(-) diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 3409f2c9..60eef6f6 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -191,7 +191,8 @@ type CommitRequestWindow struct { // admission record exists, or False (AuthorCaptureDisabled) when capture // is disabled. In either False case the request claims no actor. False is // not a failure and does not affect Ready; the final Git author remains -// the matching watch window's author. +// the matching watch window's author, unless the request took over a +// window whose writes name no author. // - Pushed (domain): True once the commit is in the remote repository. // - Applied (domain): with spec.catchUp "AfterPush", whether the applier // caught up with the commit. It never affects Ready. @@ -263,8 +264,12 @@ type CommitRequestStatus struct { // CommitRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and -// gives the commit its message. Its spec is immutable, and status.commit -// reports the commit once it is on the remote. +// gives the commit its message. When the GitTarget's KubeProvider has never +// received an audit fact (AuditFactsReceived is not True), it also takes +// over a window whose writes name no author: its creator, recorded at +// admission, authors the commit, and the message ends with an Author-Source +// line. Its spec is immutable, and status.commit reports the commit once it +// is on the remote. type CommitRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 69cf73ce..13820520 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -56,8 +56,12 @@ spec: description: |- CommitRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and - gives the commit its message. Its spec is immutable, and status.commit - reports the commit once it is on the remote. + gives the commit its message. When the GitTarget's KubeProvider has never + received an audit fact (AuditFactsReceived is not True), it also takes + over a window whose writes name no author: its creator, recorded at + admission, authors the commit, and the message ends with an Author-Source + line. Its spec is immutable, and status.commit reports the commit once it + is on the remote. properties: apiVersion: description: |- diff --git a/docs/architecture.md b/docs/architecture.md index 50c1e687..b084f300 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -757,7 +757,8 @@ flowchart TD The important boundary is the open window: it accepts only the same author and `GitTarget`. A `CommitRequest` from another author never closes someone else's window; it resolves as a benign no-commit -instead. +instead. The one exception is a cluster that has never sent an audit fact: there no write can name its +author, so a request takes over a window whose writes name none (step 2 of the `CommitRequest` flow). ### Remote moved while we were writing @@ -1777,7 +1778,12 @@ immediately. That is not a failure: 2. The controller eagerly **attaches** the request to the worker (`AttachCommitRequest`), which waits for a window until `registration + attachTimeout`. The worker binds it to an open window only when the author state and GitTarget match, first come first served, and replaces the window's timers - with the request's own, so time spent waiting never shortens the collection. It **never finalizes + with the request's own, so time spent waiting never shortens the collection. A request that names its + submitter also takes over a window on its `GitTarget` whose writes name no author, when the + `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`): + no write from that cluster can name one. The submitter then authors the commit, and its message ends + with an `Author-Source: CommitRequest /` line. Once the provider has received a + fact, a write that names no author means its fact went missing, and its window stays another's. It **never finalizes another author's window**; a window carries at most one request. The controller reports the phase the worker reports: `WaitingForWindow`, `CollectingWindow`, `WaitingForPush`. 3. The window finalizes when the request's timers close it (or when it closes for any other reason). diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index 33d58e52..7a3c1be7 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -9,6 +9,7 @@ import ( "github.com/go-logr/logr" apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" @@ -248,20 +249,21 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques // and poll the outcome. window := resolveRequestWindow(commitRequest.Spec) attach := git.AttachCommitRequest{ - Namespace: commitRequest.Namespace, - Name: commitRequest.Name, - UID: string(commitRequest.UID), - Author: author.Author, - Attribution: attribution.gitOutcome(), - GitTargetName: commitRequest.Spec.GitTargetRef.Name, - GitTargetNamespace: commitRequest.Namespace, - Message: commitRequest.Spec.Message, - Attach: window.attach, - AttachTimeout: window.attachTimeout, - IdleTimeout: window.idleTimeout, - MaxDuration: window.maxDuration, - CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + Namespace: commitRequest.Namespace, + Name: commitRequest.Name, + UID: string(commitRequest.UID), + Author: author.Author, + Attribution: attribution.gitOutcome(), + GitTargetName: commitRequest.Spec.GitTargetRef.Name, + GitTargetNamespace: commitRequest.Namespace, + Message: commitRequest.Spec.Message, + Attach: window.attach, + AttachTimeout: window.attachTimeout, + IdleTimeout: window.idleTimeout, + MaxDuration: window.maxDuration, + CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + TakeOverUnattributed: r.sourceSendsNoAuditFacts(ctx, commitRequest), } if time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout() { // Past the bound the controller only withdraws: an attach sent now would sit behind the @@ -271,6 +273,32 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques return r.attachCommitRequest(ctx, log, req, commitRequest, attribution, attach) } +// sourceSendsNoAuditFacts reports whether the KubeProvider of the request's GitTarget has never +// received an audit fact: its AuditFactsReceived condition is not True. No write watched on that +// cluster can then name its author, so the request may take over a window whose writes name none +// (git.AttachCommitRequest.TakeOverUnattributed). A read that fails says no, and the request attaches +// by author alone. +func (r *CommitRequestReconciler) sourceSendsNoAuditFacts( + ctx context.Context, + commitRequest *configbutleraiv1alpha3.CommitRequest, +) bool { + var target configbutleraiv1alpha3.GitTarget + if err := r.Get(ctx, client.ObjectKey{ + Namespace: commitRequest.Namespace, Name: commitRequest.Spec.GitTargetRef.Name, + }, &target); err != nil { + return false + } + key := target.KubeProviderKey() + if key.Name == "" { + return false + } + var provider configbutleraiv1alpha3.KubeProvider + if err := r.Get(ctx, key, &provider); err != nil { + return false + } + return !apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) +} + // attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then // it records the worker's phase and polls; a service error is a not-yet-serviceable worker, so it // polls on the same terms. diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index 6b7ac9e0..81c97a67 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -878,3 +878,47 @@ func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { }) } } + +// A request may take over a window whose writes name no author only when the GitTarget's +// KubeProvider has never received an audit fact. Any read that fails keeps the author rule. +func TestCommitRequestAttach_TakesOverUnattributedWindowsOnlyFromASourceWithNoAuditFacts(t *testing.T) { + target := &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, + }, + } + provider := func(facts metav1.ConditionStatus) *configv1alpha3.KubeProvider { + p := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} + if facts != "" { + p.Status.Conditions = []metav1.Condition{{ + Type: KubeProviderConditionAuditFactsReceived, Status: facts, Reason: "Test", + }} + } + return p + } + cases := []struct { + name string + objects []client.Object + want bool + }{ + {"audit facts received", []client.Object{target, provider(metav1.ConditionTrue)}, false}, + {"no audit fact yet", []client.Object{target, provider(metav1.ConditionUnknown)}, true}, + {"author attribution off", []client.Object{target, provider("")}, true}, + {"KubeProvider missing", []client.Object{target}, false}, + {"GitTarget missing", nil, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + objects := append([]client.Object{newCommitRequest("save")}, tc.objects...) + c := newCommitRequestClient(t, nil, objects...) + f := &fakeFinalizer{} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} + + reconcileCommitRequest(t, r, "save") + + require.NotEmpty(t, f.calls, "the request must be attached") + assert.Equal(t, tc.want, f.calls[0].TakeOverUnattributed) + }) + } +} diff --git a/internal/git/attribution_outcome_test.go b/internal/git/attribution_outcome_test.go index a7a581df..7e98dd96 100644 --- a/internal/git/attribution_outcome_test.go +++ b/internal/git/attribution_outcome_test.go @@ -257,3 +257,42 @@ func TestRenderEventCommitMessage_UnresolvedUsernameStaysEmptyInTemplates(t *tes require.NotNil(t, options.Author) assert.Equal(t, UnresolvedAuthorDisplayName, options.Author.Name) } + +// A request that names its submitter takes over a window whose writes name no author only when it +// is allowed to, and never a window whose writes name an actor, nor one on another GitTarget. +func TestPendingCommitRequest_TakesOverOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { + unattributed := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) + named := newOpenWindow(attributedEvent("bob@acme.com", AttributionResolved), nil) + request := func(allowed bool, target string) *pendingCommitRequest { + return &pendingCommitRequest{ + author: "jane@acme.com", + attribution: AttributionResolved, + gitTargetName: target, + gitTargetNamespace: "default", + takeOverUnattributed: allowed, + } + } + + assert.False(t, request(false, "target").matchesWindow(unattributed), + "without the allowance a named request keeps the author rule") + assert.True(t, request(true, "target").matchesWindow(unattributed)) + assert.True(t, request(true, "target").takesOver(unattributed)) + assert.False(t, request(true, "target").matchesWindow(named), + "a window whose writes name an actor is never taken over") + assert.False(t, request(true, "other-target").matchesWindow(unattributed), + "the allowance does not widen attachment across GitTargets") + + own := newOpenWindow(attributedEvent("jane@acme.com", AttributionResolved), nil) + assert.True(t, request(true, "target").matchesWindow(own)) + assert.False(t, request(true, "target").takesOver(own), "the submitter's own window is not a takeover") +} + +func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { + id := &commitRequestID{Namespace: "team-a", Name: "save-1"} + assert.Equal(t, + "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes\n", + withTakeOverNote("save: edit\n", id), "the message's trailing newline is kept") + assert.Equal(t, + "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes", + withTakeOverNote("save: edit", id)) +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 41acca1a..15ab561a 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1538,6 +1538,10 @@ func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinali return } pendingWrite.CommitMessage = effectiveMessage + if takenOverBy := l.openWindow.takenOverBy; takenOverBy != nil { + pendingWrite.RequestAuthor, pendingWrite.RequestAttribution = *takenOverBy, AttributionResolved + pendingWrite.TakenOver = true + } pendingCR, proceed := l.gateBuiltWindow(pendingCR, pendingWrite.Target(), reason) if !proceed { return diff --git a/internal/git/commit_executor.go b/internal/git/commit_executor.go index f892c207..73363d39 100644 --- a/internal/git/commit_executor.go +++ b/internal/git/commit_executor.go @@ -8,6 +8,7 @@ import ( "fmt" "path/filepath" "slices" + "strings" "time" gogit "github.com/go-git/go-git/v6" @@ -312,10 +313,25 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes if err != nil { return "", nil, messageResolutionUnsupported, err } + if p.TakenOver { + message = withTakeOverNote(message, p.CommitRequest) + } log.Log.V(1).Info("Selected commit message", "source", resolution.label()) return message, commitOptionsFor(p, p.CommitConfig, p.Signer, time.Now()), resolution, nil } +// withTakeOverNote ends the message of a commit whose CommitRequest took over a window with no author +// named (PendingWrite.TakenOver) with a trailer that says the author is the request's submitter. +func withTakeOverNote(message string, request *commitRequestID) string { + name := "a CommitRequest" + if request != nil { + name = "CommitRequest " + request.Namespace + "/" + request.Name + } + body := strings.TrimRight(message, "\n") + return body + "\n\nAuthor-Source: " + name + "; no audit record names who made these changes" + + message[len(body):] +} + // framedRequestMessage renders the target's requestTemplate around an attached CommitRequest's // message, falling back to that message verbatim if the render fails. // diff --git a/internal/git/commit_request_attach.go b/internal/git/commit_request_attach.go index c247d746..bec92e8f 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -149,6 +149,11 @@ type AttachCommitRequest struct { // CatchUpNow asks the GitTarget's applier to apply the tip as soon as the request's commit is // pushed: catchUp: AfterPush. CatchUpNow bool + // TakeOverUnattributed lets a request that names its submitter take over a window on its + // GitTarget whose writes name no author, and author its commit. It is set when the GitTarget's + // KubeProvider has never received an audit fact, so no write from its cluster can name one. Once + // it has, a write naming no author means its fact went missing, and that window stays another's. + TakeOverUnattributed bool } // commitRequestID is the worker-local key for a CommitRequest: its namespaced @@ -189,6 +194,9 @@ type pendingCommitRequest struct { commitEmpty bool // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. catchUpNow bool + // takeOverUnattributed lets the request take over a window whose writes name no author + // (AttachCommitRequest.TakeOverUnattributed). + takeOverUnattributed bool // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. @@ -251,10 +259,17 @@ func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { if p == nil || w == nil { return false } - return p.author == w.Author && - p.attribution.NamesActor() == w.Attribution.NamesActor() && - p.gitTargetName == w.GitTarget && - p.gitTargetNamespace == w.GitTargetNamespace + if p.gitTargetName != w.GitTarget || p.gitTargetNamespace != w.GitTargetNamespace { + return false + } + return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.takesOver(w) +} + +// takesOver reports whether the request matches a window on its GitTarget whose writes name no +// author only because it may take such a window over: it names its submitter, who then authors the +// commit (AttachCommitRequest.TakeOverUnattributed). +func (p *pendingCommitRequest) takesOver(w *openWindow) bool { + return p.takeOverUnattributed && p.attribution.NamesActor() && !w.Attribution.NamesActor() } // commitRequestOutcomeEntry is a resolved outcome retained for the controller to diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/commit_request_attach_loop.go index b088bb53..c0be06e5 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -215,6 +215,8 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque maxDuration: req.MaxDuration, commitEmpty: req.CommitEmpty, catchUpNow: req.CatchUpNow, + + takeOverUnattributed: req.TakeOverUnattributed, } l.pendingCRs[id] = pcr l.w.setCommitRequestPhase(id, PhaseWaitingForWindow) @@ -345,6 +347,12 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { } l.openWindow.timers = timers l.openWindow.lastWriteAt = now + if pcr.takesOver(l.openWindow) { + l.openWindow.takenOverBy = &UserInfo{Username: pcr.author} + l.w.Log.Info("CommitRequest took over a window whose writes name no author; its submitter authors the commit", + "request", id.Namespace+"/"+id.Name, "author", pcr.author, + "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) + } l.w.setCommitRequestPhase(id, PhaseCollectingWindow) l.w.Log.Info("CommitRequest attached to open window", "request", id.Namespace+"/"+id.Name, diff --git a/internal/git/commit_request_attach_test.go b/internal/git/commit_request_attach_test.go index dabbc2e0..5563524f 100644 --- a/internal/git/commit_request_attach_test.go +++ b/internal/git/commit_request_attach_test.go @@ -340,6 +340,44 @@ func TestAttach_WritesAfterTheClaimJoinOneCommit(t *testing.T) { }, "both writes must land in the one commit carrying the message") } +// TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit pins the takeover from a source that +// sends no audit facts: the request attaches to a window whose writes name no author, its submitter +// authors the commit, and the message ends with a note that says so. +func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + + loop := newBranchWorkerEventLoop(worker, time.Hour) + defer loop.stopTimers() + + event := configMapTargetEvent("cm", "", "team-a") + event.Attribution = AttributionUnresolved + loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{event}}}) + require.NotNil(t, loop.openWindow) + + req := attachReq("jane@acme.com", 60*time.Second) + req.Attribution = AttributionResolved + req.Message = "save: from the editor" + req.TakeOverUnattributed = true + serviceAttach(loop, req) + require.NotNil(t, loop.openWindow.pendingCR, "the request takes the window over") + + forceDue(loop) + loop.endWake(0) + res, ok := outcome(t, worker) + require.True(t, ok) + require.NoError(t, res.Err) + require.Equal(t, FinalizeCommitted, res.Outcome) + + repo, err := gogit.PlainOpen(worker.repoPath()) + require.NoError(t, err) + commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) + require.NoError(t, err) + assert.Equal(t, "jane@acme.com", commit.Author.Name, "the submitter authors the commit") + assert.Equal(t, "save: from the editor\n\nAuthor-Source: CommitRequest default/save; "+ + "no audit record names who made these changes", commit.Message) +} + // TestAttach_ForeignWindowIsNotStolen verifies an attach for a different author // parks (never finalizes another author's window) and resolves WindowMismatch once // its grace elapses, leaving the foreign window open. diff --git a/internal/git/open_window.go b/internal/git/open_window.go index 8679f247..9cbc7d13 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -46,6 +46,9 @@ type openWindow struct { // pendingCR identifies the CommitRequest attached to this window; at most one. On // finalize its outcome is resolved (Committed once the carrying write pushes). pendingCR *commitRequestID + // takenOverBy is the submitter of the attached CommitRequest when it took the window over: its + // writes name no author, so the submitter authors the commit (pendingCommitRequest.takesOver). + takenOverBy *UserInfo // timers decide when the window closes. They are the GitTarget's, snapshotted when the window // opened, until a CommitRequest attaches and replaces them with its own. diff --git a/internal/git/pending_writes.go b/internal/git/pending_writes.go index f3abce4e..71cebe44 100644 --- a/internal/git/pending_writes.go +++ b/internal/git/pending_writes.go @@ -250,7 +250,7 @@ func (p PendingWrite) Author() string { // writes, including any OIDC display name and email. Empty writes have no // per-user author and return the zero value. func (p PendingWrite) AuthorUserInfo() UserInfo { - if p.Kind == PendingWriteRequestRecord { + if p.Kind == PendingWriteRequestRecord || p.TakenOver { return p.RequestAuthor } if len(p.Events) == 0 { @@ -303,7 +303,7 @@ const ( // mirrors AuthorUserInfo: the window is single-author, so the first event's outcome describes // the whole write. Empty writes never attempt attribution. func (p PendingWrite) AttributionOutcome() AttributionOutcome { - if p.Kind == PendingWriteRequestRecord { + if p.Kind == PendingWriteRequestRecord || p.TakenOver { return p.RequestAttribution } if len(p.Events) == 0 { diff --git a/internal/git/types.go b/internal/git/types.go index 47059fa8..a979ac0a 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -384,6 +384,10 @@ type PendingWrite struct { // window's author is. RequestAuthor UserInfo RequestAttribution AttributionOutcome + // TakenOver says the CommitRequest attached to this grouped write took over a window whose + // writes named no author: RequestAuthor and RequestAttribution, the submitter's, author the + // commit, and its message ends with a note that says so (withTakeOverNote). + TakenOver bool // emptyCommitted is stamped by executePendingWrites, the way CommitSHA is: the commit this // write made changed no file. A request riding it resolves with the cause (AlreadyPresent or // NoOpenWindow) and the commit, so a caller can tell "saved your changes" from "recorded your diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index d8b50637..081685f4 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -36,7 +36,8 @@ import ( // the GitTarget mirrors that workspace, and its FluxClaim holds the Kustomization through the // install's own provider. It shows the startup converge landing in the workspace, an edit made // there during an apply withheld from the apply's revert by the REMOTE object's managedFields, -// and two saves that ask to catch up reporting the apply. +// and two saves that take over their edits' windows (the workspace sends no audit facts) and +// report the apply. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), // and the kcp specs additionally Skip when kcp is not installed — so a default `task test-e2e` @@ -737,9 +738,11 @@ spec: nameApplierClaim(testNs, p.targetName, "FluxClaim", ksName, "") }) - // Naming the FluxClaim converges it once: a CatchUp of the tip, which holds a ConfigMap only - // Git has, since the GitRepository's 24h interval never fetched it. So the ConfigMap reaches - // the workspace through the FluxClaim's apply alone, and Flux's labels on it say who applied it. + // Naming the FluxClaim applies the tip, which holds a ConfigMap only Git has, since the + // GitRepository's 24h interval never fetched it. The reason is the ForeignCommit the receiver + // reported, which the worker kept while no applier was named, or the startup CatchUp. So the + // ConfigMap reaches the workspace through the FluxClaim's apply alone, and Flux's labels on it + // say who applied it. It("applies its startup converge into the workspace, and mirrors what it applied", func() { verifyResourceCondition(fluxClaimResource, ksName, testNs, "Configured", "True", "", "") verifyResourceCondition(fluxClaimResource, ksName, testNs, "Ready", "True", "Succeeded", "") @@ -748,7 +751,7 @@ spec: head := p.remoteBranchHead(g) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(head)) done := r.applier.operation(g) - g.Expect(done.Reason).To(Equal("CatchUp")) + g.Expect(done.Reason).To(BeElementOf("ForeignCommit", "CatchUp")) g.Expect(done.Commit).To(Equal(head)) g.Expect(done.Phase).To(Equal("Succeeded")) live, exists := r.liveValue(g, "converged") @@ -777,27 +780,36 @@ spec: }) // A save on the workspace's GitTarget that asks to catch up is applied once it is pushed, and - // each request reports the apply: Applied=True, with the revision Flux applied. Two saves made - // together may share one apply. + // each request reports the apply: Applied=True, with the revision Flux applied. The second save + // is made as soon as the first is pushed, so the two may share one apply. // - // The saves record empty commits (whenNothingToCommit: CommitEmpty). A request's author is - // captured at admission in this cluster, and with no audit stream from kcp a write in the - // workspace names no author, so a request never attaches to that write's window. - It("applies two saves that ask to catch up, and reports the apply on each", func() { + // No write in the workspace names its author, since kcp sends no audit facts, and the request's + // submitter is captured at admission in this cluster. The workspace's KubeProvider has never + // received a fact, so each request takes over the window of its edit, its submitter authors the + // commit, and the message ends with a note that says so. Every event from the workspace waits + // out the attribution grace first, so the requests wait longer than that for their window. + It("takes over the window of an edit in the workspace, and reports the apply on each of two saves", func() { + verifyResourceCondition("kubeprovider", ws, testNs, "AuditFactsReceived", "Unknown", "", "") names := []string{"sc-flux-save-1", "sc-flux-save-2"} - for _, name := range names { - applyCatchUpSave(testNs, name, p.targetName) - } commits := map[string]string{} - for _, name := range names { + for i, name := range names { + value := fmt.Sprintf("saved-%d", i+1) + applyCatchUpSave(testNs, name, p.targetName) + r.editThroughAPI(timingConfigMap, value) Eventually(func(g Gomega) { g.Expect(commitRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) g.Expect(commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(Equal("NoWindow")) - g.Expect(commitRequestCondition(g, testNs, name, "Pushed")).To(Equal("True")) + To(Equal(commitRequestReasonCommitted)) commits[name] = commitRequestField(g, testNs, name, "{.status.commit}") g.Expect(commits[name]).NotTo(BeEmpty()) }, 2*time.Minute, 2*time.Second).Should(Succeed()) + committed, inGit := r.committedValue(Default, timingConfigMap) + Expect(inGit).To(BeTrue()) + Expect(committed).To(Equal(value), "the save's commit holds its edit") + message, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commits[name]) + Expect(err).NotTo(HaveOccurred()) + Expect(message).To(ContainSubstring("Author-Source: CommitRequest "+testNs+"/"+name), + "the commit says its author is the request's submitter") } last := commits[names[len(names)-1]] @@ -819,6 +831,7 @@ spec: To(ContainSubstring(last)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) + r.eventuallyEverywhere(timingConfigMap, "saved-2", echoApplyBudget) }) }) }) @@ -872,8 +885,8 @@ spec: } // applyCatchUpSave creates the CommitRequest name on gitTarget, asking its applier to catch up once -// it is pushed. It attaches only to a window already open, and records its message in an empty -// commit when there is none. +// it is pushed. It waits 45s for its window: a write from a cluster with no audit stream reaches the +// window only after the attribution grace. func applyCatchUpSave(namespace, name, gitTarget string) { GinkgoHelper() _, err := kubectlRunWithStdin(namespace, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 @@ -886,9 +899,9 @@ spec: name: %s message: "e2e: save %s and apply it" catchUp: AfterPush - whenNothingToCommit: CommitEmpty window: - attachTimeout: "0s" + attachTimeout: "45s" + maxDuration: "4s" `, name, namespace, gitTarget, name), "apply", "-f", "-") Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s", name) } From 33498b9feb8f28fb4d57348df5254c3996bef9b5 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:31:37 +0000 Subject: [PATCH 10/33] fix(commitrequest): a save whose GitTarget is gone is Applied=True LookupCommitRequestApply answered a request whose GitTarget is gone with Done and reason ApplierGone, but left Applied false, so the controller wrote Applied=False. ApplierGone means no apply was needed, and the contract says only CommitDropped is Applied=False; dropSaves already set Applied for the same reason. Co-Authored-By: Claude Opus 5.5 (1M context) --- internal/watch/event_router.go | 2 +- internal/watch/event_router_test.go | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index 277b39cb..f6b2187a 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -169,7 +169,7 @@ func (r *EventRouter) LookupCommitRequestApply( ) (git.SaveApply, bool, error) { worker, _, err := r.commitRequestWorker(ctx, attach) if errors.Is(err, errGitTargetGone) { - return git.SaveApply{Done: true, Reason: git.SaveApplyApplierGone, + return git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyApplierGone, Message: "the GitTarget is gone, so its applier is asked for nothing"}, true, nil } if err != nil || worker == nil { diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 513d492b..03ad9df5 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -61,6 +61,27 @@ func TestServiceCommitRequest_GitTargetNotFound(t *testing.T) { require.ErrorIs(t, err, errGitTargetGone) } +// A save whose GitTarget is gone needs no apply: Applied=True with ApplierGone. Only a commit the +// branch lost is ever Applied=False. +func TestLookupCommitRequestApply_GitTargetGoneIsApplied(t *testing.T) { + scheme := eventRouterScheme(t) + client := fake.NewClientBuilder().WithScheme(scheme).Build() + workerManager := git.NewWorkerManager( + client, + logr.Discard(), + git.BranchWorkerLimits{}, + types.SensitiveResourcePolicy{}, + ) + router := NewEventRouter(workerManager, nil, client, logr.Discard()) + + apply, known, err := router.LookupCommitRequestApply(context.Background(), saveAttach("missing", "team-a")) + require.NoError(t, err) + require.True(t, known) + assert.True(t, apply.Done) + assert.True(t, apply.Applied, "a save whose GitTarget is gone needs no apply; it is not Applied=False") + assert.Equal(t, git.SaveApplyApplierGone, apply.Reason) +} + func TestGitTargetEventStreamRegistry(t *testing.T) { scheme := eventRouterScheme(t) client := fake.NewClientBuilder().WithScheme(scheme).Build() From ae00b25cbf5390d7b0dfa6dc5899b4b571f8bf29 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:31:45 +0000 Subject: [PATCH 11/33] feat(commitrequest): claiming a window whose writes name no author is opt-in, as SavedBy or TakeOver The takeover in the previous commit claimed such a window automatically whenever the GitTarget's KubeProvider had never received an audit fact. Every write that names no author shares one window, whoever made it, so that named a save's submitter as the author of other people's edits. Claiming is now asked for and allowed explicitly: - CommitRequest spec.window.author: "Own" (default), "SavedBy" or "TakeOver" says what a save asks for. - GitTarget spec.commit.unattributedWindows: "Keep" (default), "SavedBy" or "TakeOver" bounds what a save on it may claim. - "SavedBy" gives the window the request's message and names no author; the message ends with "Saved-By: ". "TakeOver" makes the submitter the author, with the Author-Source trailer. - A request that asks for more than its GitTarget allows fails at first sight: Ready=False, Stalled=True, reason TakeOverNotAllowed. - Both act only while the KubeProvider's AuditFactsReceived is not True. Co-Authored-By: Claude Opus 5.5 (1M context) --- api/v1alpha3/commitrequest_types.go | 41 +++++-- api/v1alpha3/gittarget_types.go | 46 ++++++++ .../bases/configbutler.ai_commitrequests.yaml | 34 ++++-- .../crd/bases/configbutler.ai_gittargets.yaml | 25 ++++ docs/architecture.md | 19 +-- docs/configuration.md | 38 +++++- hack/crdfields/fields.yaml | 9 ++ .../controller/commitrequest_controller.go | 90 +++++++++----- .../commitrequest_controller_unit_test.go | 110 +++++++++++++++--- internal/controller/commitrequest_finalize.go | 8 +- internal/git/attribution_outcome_test.go | 48 +++++--- internal/git/branch_worker.go | 8 +- internal/git/commit_executor.go | 20 +++- internal/git/commit_request_attach.go | 43 ++++--- internal/git/commit_request_attach_loop.go | 10 +- internal/git/commit_request_attach_test.go | 39 ++++++- internal/git/open_window.go | 8 +- internal/git/types.go | 6 +- test/e2e/source_cluster_e2e_test.go | 13 ++- 19 files changed, 492 insertions(+), 123 deletions(-) diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 60eef6f6..73068df8 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -57,7 +57,7 @@ type CommitRequestSpec struct { // write does. Omitted, the request attaches to the author's current or next // window within 2s and closes it 2s later. // +optional - // +kubebuilder:default={attach: "CurrentOrNext", attachTimeout: "2s", maxDuration: "2s"} + // +kubebuilder:default={attach: "CurrentOrNext", author: "Own", attachTimeout: "2s", maxDuration: "2s"} Window *CommitRequestWindow `json:"window,omitempty"` // WhenNothingToCommit decides what happens when the request ends with nothing @@ -121,6 +121,20 @@ const ( AttachNext AttachPolicy = "Next" ) +// WindowAuthor is whose commit window a CommitRequest may attach to. +type WindowAuthor string + +const ( + // WindowAuthorOwn attaches only to the creator's own window. + WindowAuthorOwn WindowAuthor = "Own" + // WindowAuthorSavedBy may also give the request's message to a window whose writes name no + // author, naming no author. + WindowAuthorSavedBy WindowAuthor = "SavedBy" + // WindowAuthorTakeOver may also take a window whose writes name no author over, and the + // creator authors its commit. + WindowAuthorTakeOver WindowAuthor = "TakeOver" +) + // CommitRequestWindow is how a CommitRequest attaches to a commit window and // when that window closes. A window is only ever opened by a write; a request // waits for one and never opens one. @@ -128,7 +142,7 @@ const ( // Every duration shares the shape GitTargetCommitSpec documents: a Go duration // string behind a pattern and a CEL bound. The immutability rule compares them // as durations, for the reason CommitRequestSpec gives. -// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" // +kubebuilder:validation:XValidation:rule="self.attach != 'Next' || duration(self.attachTimeout) > duration('0s')",message="attach: Next needs a positive attachTimeout; with 0s it would close the author's window and give up in the same instant" type CommitRequestWindow struct { @@ -142,6 +156,17 @@ type CommitRequestWindow struct { // +kubebuilder:default=CurrentOrNext Attach AttachPolicy `json:"attach,omitempty"` + // Author says whose window the request may attach to. "Own" attaches only + // to the creator's own window. "SavedBy" may also give its message to a + // window whose writes name no author, naming no author. "TakeOver" may + // also take such a window over, and the creator authors the commit. The + // GitTarget's spec.commit.unattributedWindows must allow what is asked, or + // the request fails with reason TakeOverNotAllowed. Defaults to "Own". + // +optional + // +kubebuilder:validation:Enum=Own;SavedBy;TakeOver + // +kubebuilder:default=Own + Author WindowAuthor `json:"author,omitempty"` + // AttachTimeout is how long the request waits for a window, as a Go duration, // from when GitOps Reverser picks the request up. "0s" attaches only to a // window already open. A request whose wait ran out never takes a later @@ -192,7 +217,7 @@ type CommitRequestWindow struct { // is disabled. In either False case the request claims no actor. False is // not a failure and does not affect Ready; the final Git author remains // the matching watch window's author, unless the request took over a -// window whose writes name no author. +// window whose writes name no author (spec.window.author "TakeOver"). // - Pushed (domain): True once the commit is in the remote repository. // - Applied (domain): with spec.catchUp "AfterPush", whether the applier // caught up with the commit. It never affects Ready. @@ -264,12 +289,10 @@ type CommitRequestStatus struct { // CommitRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and -// gives the commit its message. When the GitTarget's KubeProvider has never -// received an audit fact (AuditFactsReceived is not True), it also takes -// over a window whose writes name no author: its creator, recorded at -// admission, authors the commit, and the message ends with an Author-Source -// line. Its spec is immutable, and status.commit reports the commit once it -// is on the remote. +// gives the commit its message. With spec.window.author, and where the +// GitTarget allows it, it may also claim a window whose writes name no +// author. Its spec is immutable, and status.commit reports the commit once +// it is on the remote. type CommitRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 6d0c41ad..06779d1f 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -3,6 +3,8 @@ package v1alpha3 import ( + "slices" + meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" @@ -235,6 +237,50 @@ type GitTargetCommitSpec struct { // built-in one. // +optional Message *CommitMessageSpec `json:"message,omitempty"` + + // UnattributedWindows says how much a CommitRequest on this GitTarget may + // claim of a commit window whose writes name no author. It acts only while + // the GitTarget's KubeProvider has never received an audit fact; on a + // cluster that has, a write naming no author is never claimed. Defaults to + // "Keep". + // + // - "Keep": a request attaches only to its creator's own window. + // - "SavedBy": a request that asks may give such a window its message. The + // commit names no author, and its message ends with a Saved-By line + // naming the request's creator. + // - "TakeOver": a request that asks may also take such a window over. Its + // creator authors the commit, and the message ends with an Author-Source + // line. + // + // Every write that names no author shares one window, whoever made it, so + // a takeover names the creator as the author of all of them. Allow it only + // where one person edits the mirrored namespaces at a time. + // +optional + // +kubebuilder:validation:Enum=Keep;SavedBy;TakeOver + // +kubebuilder:default=Keep + UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` +} + +// UnattributedWindowPolicy is how much a CommitRequest may claim of a commit +// window whose writes name no author. +type UnattributedWindowPolicy string + +const ( + // UnattributedWindowsKeep lets a request attach only to its creator's own window. + UnattributedWindowsKeep UnattributedWindowPolicy = "Keep" + // UnattributedWindowsSavedBy lets a request give such a window its message, naming no author. + UnattributedWindowsSavedBy UnattributedWindowPolicy = "SavedBy" + // UnattributedWindowsTakeOver also lets a request take such a window over and author its commit. + UnattributedWindowsTakeOver UnattributedWindowPolicy = "TakeOver" +) + +// Allows reports whether a CommitRequest asking for author may attach as it asks under this policy: +// "TakeOver" allows every ask, "SavedBy" allows "SavedBy" and "Own", and "Keep" (or empty) allows only +// "Own". An unknown value allows only "Own". +func (p UnattributedWindowPolicy) Allows(author WindowAuthor) bool { + // Weakest first; an unknown value ranks with the weakest (-1 + 1 = 0). + order := []string{string(UnattributedWindowsSavedBy), string(UnattributedWindowsTakeOver)} + return slices.Index(order, string(author)) <= slices.Index(order, string(p)) } // CommitWindow holds the two timers that close a GitTarget's commit windows. A diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 13820520..37cf6e54 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -56,12 +56,10 @@ spec: description: |- CommitRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and - gives the commit its message. When the GitTarget's KubeProvider has never - received an audit fact (AuditFactsReceived is not True), it also takes - over a window whose writes name no author: its creator, recorded at - admission, authors the commit, and the message ends with an Author-Source - line. Its spec is immutable, and status.commit reports the commit once it - is on the remote. + gives the commit its message. With spec.window.author, and where the + GitTarget allows it, it may also claim a window whose writes name no + author. Its spec is immutable, and status.commit reports the commit once + it is on the remote. properties: apiVersion: description: |- @@ -151,6 +149,7 @@ spec: default: attach: CurrentOrNext attachTimeout: 2s + author: Own maxDuration: 2s description: |- Window says which commit window this request attaches to, how long it waits @@ -184,6 +183,20 @@ spec: x-kubernetes-validations: - message: spec.window.attachTimeout must not exceed 5m rule: duration(self) <= duration('5m') + author: + default: Own + description: |- + Author says whose window the request may attach to. "Own" attaches only + to the creator's own window. "SavedBy" may also give its message to a + window whose writes name no author, naming no author. "TakeOver" may + also take such a window over, and the creator authors the commit. The + GitTarget's spec.commit.unattributedWindows must allow what is asked, or + the request fails with reason TakeOverNotAllowed. Defaults to "Own". + enum: + - Own + - SavedBy + - TakeOver + type: string idleTimeout: description: |- IdleTimeout closes the attached window this long after its last write, as a @@ -208,10 +221,11 @@ spec: type: object x-kubernetes-validations: - message: CommitRequest spec is immutable after creation - rule: self.attach == oldSelf.attach && duration(self.attachTimeout) - == duration(oldSelf.attachTimeout) && duration(self.maxDuration) - == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) - && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) + rule: self.attach == oldSelf.attach && self.author == oldSelf.author + && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) + && duration(self.maxDuration) == duration(oldSelf.maxDuration) + && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) + || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) - message: spec.window.idleTimeout must not exceed maxDuration rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' - message: 'attach: Next needs a positive attachTimeout; with 0s it diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index f1c2572c..c61d4911 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -191,6 +191,31 @@ spec: one that fails at commit time commits the message as written. type: string type: object + unattributedWindows: + default: Keep + description: |- + UnattributedWindows says how much a CommitRequest on this GitTarget may + claim of a commit window whose writes name no author. It acts only while + the GitTarget's KubeProvider has never received an audit fact; on a + cluster that has, a write naming no author is never claimed. Defaults to + "Keep". + + - "Keep": a request attaches only to its creator's own window. + - "SavedBy": a request that asks may give such a window its message. The + commit names no author, and its message ends with a Saved-By line + naming the request's creator. + - "TakeOver": a request that asks may also take such a window over. Its + creator authors the commit, and the message ends with an Author-Source + line. + + Every write that names no author shares one window, whoever made it, so + a takeover names the creator as the author of all of them. Allow it only + where one person edits the mirrored namespaces at a time. + enum: + - Keep + - SavedBy + - TakeOver + type: string window: description: |- Window sets when this GitTarget's commit windows close. A window opens on an diff --git a/docs/architecture.md b/docs/architecture.md index b084f300..36eb5a0b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -757,8 +757,9 @@ flowchart TD The important boundary is the open window: it accepts only the same author and `GitTarget`. A `CommitRequest` from another author never closes someone else's window; it resolves as a benign no-commit -instead. The one exception is a cluster that has never sent an audit fact: there no write can name its -author, so a request takes over a window whose writes name none (step 2 of the `CommitRequest` flow). +instead. The one exception is opt-in, on a cluster that has never sent an audit fact: there no write +can name its author, and a request may claim a window whose writes name none (step 2 of the +`CommitRequest` flow). ### Remote moved while we were writing @@ -1779,11 +1780,15 @@ immediately. That is not a failure: waits for a window until `registration + attachTimeout`. The worker binds it to an open window only when the author state and GitTarget match, first come first served, and replaces the window's timers with the request's own, so time spent waiting never shortens the collection. A request that names its - submitter also takes over a window on its `GitTarget` whose writes name no author, when the - `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`): - no write from that cluster can name one. The submitter then authors the commit, and its message ends - with an `Author-Source: CommitRequest /` line. Once the provider has received a - fact, a write that names no author means its fact went missing, and its window stays another's. It **never finalizes + submitter may also claim a window on its `GitTarget` whose writes name no author, when it asks + (`spec.window.author`), the `GitTarget` allows it (`spec.commit.unattributedWindows`), and the + `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`). + With `SavedBy` the window takes the request's message, the commit names no author, and the message + ends with a `Saved-By` trailer; with `TakeOver` the submitter authors the commit, and the message ends + with an `Author-Source: CommitRequest /` trailer. Once the provider has received a + fact, a write that names no author means its fact went missing, and its window stays another's. A + request that asks for more than its `GitTarget` allows fails at first sight with + `TakeOverNotAllowed`. It **never finalizes another author's window**; a window carries at most one request. The controller reports the phase the worker reports: `WaitingForWindow`, `CollectingWindow`, `WaitingForPush`. 3. The window finalizes when the request's timers close it (or when it closes for any other reason). diff --git a/docs/configuration.md b/docs/configuration.md index 754f7a99..404094dc 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -118,6 +118,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | +| `commit.unattributedWindows` | `Keep` | How much a `CommitRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | | `placement.useKustomize` | off | Keep the folder a kustomize folder. See [keeping the folder a kustomize folder](#keeping-the-folder-a-kustomize-folder-specplacementusekustomize) | @@ -178,6 +179,7 @@ The references are immutable: to hold another `Application`, delete the applier | `gitTargetRef` | **required** | The target whose commit window the request attaches to | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | +| `window.author` | `Own` | Whose window to attach to: `Own`, or also claim a window whose writes name no author with `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | | `window.attachTimeout` | `2s` | How long to wait for a window to attach to | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | @@ -3002,6 +3004,7 @@ replaces that window's timers with its own, shorter or longer: | Field | Meaning | Default | |---|---|---| | `attach` | `CurrentOrNext` attaches to the author's window if one is open when the worker registers the request, and otherwise to the next one a write opens. `Next` first closes the author's open window, under the message it already carries, and attaches to the next one | `CurrentOrNext` | +| `author` | `Own` attaches only to the author's window. `SavedBy` and `TakeOver` may also claim a window whose writes name no author; see [saving where writes name no author](#saving-where-writes-name-no-author) | `Own` | | `attachTimeout` | stop waiting for a window this long after the worker registers the request | `2s` | | `idleTimeout` | close the attached window after this much silence; every write restarts it | no idle close | | `maxDuration` | close the attached window this long after the attach, however much keeps arriving | `2s` | @@ -3050,7 +3053,40 @@ push follows, with its cooldown and any retries. Do not size either against `--author-attribution-grace`. When that grace expires with no fact the write still ships, as a window that names no actor, and a request naming a submitter can never -attach to it: the outcome is `WindowMismatch` no matter how long the request waits. +attach to it: the outcome is `WindowMismatch` no matter how long the request waits. On a cluster +that sends no audit events at all, see +[saving where writes name no author](#saving-where-writes-name-no-author). + +### Saving where writes name no author + +On a source cluster that sends no audit events, no write names its author, so every write lands in +a window that names no actor. A request whose submitter was captured at admission cannot attach to +such a window, and its save ends `WindowMismatch`. Two settings let it claim one, and both default to +claiming nothing: + +- the `GitTarget`'s `spec.commit.unattributedWindows` says how much a request on it may claim: + `Keep` (the default), `SavedBy`, or `TakeOver`; +- the request's `spec.window.author` says what this save asks for: `Own` (the default), `SavedBy`, + or `TakeOver`. + +| `spec.window.author` | What the request does with a window whose writes name no author | +|---|---| +| `Own` | nothing: it attaches only to its submitter's own window | +| `SavedBy` | gives the window its message; the commit names no author, and its message ends with `Saved-By: ` | +| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: CommitRequest /; no audit record names who made these changes` | + +`TakeOver` on the `GitTarget` allows both claims, and `SavedBy` allows only `SavedBy`. A request that +asks for more than its `GitTarget` allows fails when it is first seen, before it attaches: +`Ready=False`, `Stalled=True`, reason `TakeOverNotAllowed`. + +Both claims act only while the `GitTarget`'s `KubeProvider` has never received an audit fact: its +`AuditFactsReceived` condition is not `True`. On a cluster that has sent one, a write that names no +author means its fact went missing, so no request claims it, and a request that asked attaches to +its own windows only. + +Every write that names no author shares one window, whoever made it. A takeover therefore names the +submitter as the author of all of them, and `SavedBy` gives all of them the submitter's message. +Allow `TakeOver` only where one person edits the `GitTarget`'s namespaces at a time. The two directions are not symmetric. Overshooting costs a few seconds of latency on the commit; undershooting `attachTimeout` resolves the request `Ready=True` with reason `NoWindow` while the diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 5405d3b8..a3c0d9c9 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -154,6 +154,11 @@ kinds: - path: commit.message default: the built-in templates doc: How commits are phrased. See [commit messages](commit-messages.md) + - path: commit.unattributedWindows + doc: >- + How much a `CommitRequest` may claim of a window whose writes name no author, on a cluster + that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See + [saving where writes name no author](#saving-where-writes-name-no-author) - path: placement.byType default: the built-in path doc: >- @@ -299,6 +304,10 @@ kinds: doc: >- Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) + - path: window.author + doc: >- + Whose window to attach to: `Own`, or also claim a window whose writes name no author with + `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) - path: window.attachTimeout doc: How long to wait for a window to attach to - path: window.idleTimeout diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index 7a3c1be7..76352653 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -5,6 +5,7 @@ package controller import ( "context" "errors" + "fmt" "time" "github.com/go-logr/logr" @@ -229,6 +230,15 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques // decision is final — there is no wait and no requeue for the author. author, attribution := r.attributeAuthor(ctx, commitRequest) + // Refused only at first sight, before any attach was sent: a GitTarget's policy lowered later + // does not fail a request the worker may already hold, which keeps what it was first sent. + claim, claimErr := r.unattributedClaim(ctx, commitRequest) + if claimErr != nil && findCondition(commitRequest.Status.Conditions, ConditionTypeReady) == nil { + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, claimErr, attribution, + resolvedCommitRequestTarget(commitRequest)) + return ctrl.Result{}, nil + } + if err := r.stampFirstSightConditions(ctx, commitRequest, attribution); err != nil { return ctrl.Result{}, err } @@ -249,21 +259,21 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques // and poll the outcome. window := resolveRequestWindow(commitRequest.Spec) attach := git.AttachCommitRequest{ - Namespace: commitRequest.Namespace, - Name: commitRequest.Name, - UID: string(commitRequest.UID), - Author: author.Author, - Attribution: attribution.gitOutcome(), - GitTargetName: commitRequest.Spec.GitTargetRef.Name, - GitTargetNamespace: commitRequest.Namespace, - Message: commitRequest.Spec.Message, - Attach: window.attach, - AttachTimeout: window.attachTimeout, - IdleTimeout: window.idleTimeout, - MaxDuration: window.maxDuration, - CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, - TakeOverUnattributed: r.sourceSendsNoAuditFacts(ctx, commitRequest), + Namespace: commitRequest.Namespace, + Name: commitRequest.Name, + UID: string(commitRequest.UID), + Author: author.Author, + Attribution: attribution.gitOutcome(), + GitTargetName: commitRequest.Spec.GitTargetRef.Name, + GitTargetNamespace: commitRequest.Namespace, + Message: commitRequest.Spec.Message, + Attach: window.attach, + AttachTimeout: window.attachTimeout, + IdleTimeout: window.idleTimeout, + MaxDuration: window.maxDuration, + CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + Unattributed: claim, } if time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout() { // Past the bound the controller only withdraws: an attach sent now would sit behind the @@ -273,30 +283,58 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques return r.attachCommitRequest(ctx, log, req, commitRequest, attribution, attach) } -// sourceSendsNoAuditFacts reports whether the KubeProvider of the request's GitTarget has never -// received an audit fact: its AuditFactsReceived condition is not True. No write watched on that -// cluster can then name its author, so the request may take over a window whose writes name none -// (git.AttachCommitRequest.TakeOverUnattributed). A read that fails says no, and the request attaches -// by author alone. -func (r *CommitRequestReconciler) sourceSendsNoAuditFacts( +// errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author +// than its GitTarget's spec.commit.unattributedWindows allows. It is terminal: the spec is immutable. +var errTakeOverNotAllowed = errors.New("spec.window.author asks for more than the GitTarget allows") + +// unattributedClaim decides what the request may claim of a window on its GitTarget whose writes +// name no author (git.AttachCommitRequest.Unattributed). It claims what spec.window.author asks for, +// and only while the GitTarget's KubeProvider has never received an audit fact: its +// AuditFactsReceived condition is not True, so no write watched on that cluster can name its author. +// On a cluster that has sent one, a write naming no author means its fact went missing, and nothing +// is claimed. A request that asks for more than the GitTarget's spec.commit.unattributedWindows +// allows is errTakeOverNotAllowed. A GitTarget or KubeProvider that cannot be read claims nothing, +// and the request attaches by author alone. +func (r *CommitRequestReconciler) unattributedClaim( ctx context.Context, commitRequest *configbutleraiv1alpha3.CommitRequest, -) bool { +) (git.UnattributedClaim, error) { + ask := configbutleraiv1alpha3.WindowAuthorOwn + if window := commitRequest.Spec.Window; window != nil && window.Author != "" { + ask = window.Author + } + if ask == configbutleraiv1alpha3.WindowAuthorOwn { + return git.UnattributedClaimNone, nil + } var target configbutleraiv1alpha3.GitTarget if err := r.Get(ctx, client.ObjectKey{ Namespace: commitRequest.Namespace, Name: commitRequest.Spec.GitTargetRef.Name, }, &target); err != nil { - return false + return git.UnattributedClaimNone, nil //nolint:nilerr // a missing GitTarget is the attach path's to report + } + policy := configbutleraiv1alpha3.UnattributedWindowsKeep + if target.Spec.Commit != nil && target.Spec.Commit.UnattributedWindows != "" { + policy = target.Spec.Commit.UnattributedWindows + } + if !policy.Allows(ask) { + return git.UnattributedClaimNone, fmt.Errorf("%w: spec.window.author is %q, and GitTarget %s has "+ + "spec.commit.unattributedWindows %q", errTakeOverNotAllowed, ask, target.Name, policy) } key := target.KubeProviderKey() if key.Name == "" { - return false + return git.UnattributedClaimNone, nil } var provider configbutleraiv1alpha3.KubeProvider if err := r.Get(ctx, key, &provider); err != nil { - return false + return git.UnattributedClaimNone, nil //nolint:nilerr // an unread provider claims nothing + } + if apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) { + return git.UnattributedClaimNone, nil + } + if ask == configbutleraiv1alpha3.WindowAuthorTakeOver { + return git.UnattributedClaimTakeOver, nil } - return !apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) + return git.UnattributedClaimSavedBy, nil } // attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index 81c97a67..48cce252 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -879,14 +879,18 @@ func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { } } -// A request may take over a window whose writes name no author only when the GitTarget's -// KubeProvider has never received an audit fact. Any read that fails keeps the author rule. -func TestCommitRequestAttach_TakesOverUnattributedWindowsOnlyFromASourceWithNoAuditFacts(t *testing.T) { - target := &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, - Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, - }, +// A request claims a window whose writes name no author only when it asked, its GitTarget allows +// what it asked, and the GitTarget's KubeProvider has never received an audit fact. A request that +// asks for more than its GitTarget allows fails at once, before anything is attached. +func TestCommitRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { + target := func(policy configv1alpha3.UnattributedWindowPolicy) *configv1alpha3.GitTarget { + return &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, + Commit: &configv1alpha3.GitTargetCommitSpec{UnattributedWindows: policy}, + }, + } } provider := func(facts metav1.ConditionStatus) *configv1alpha3.KubeProvider { p := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} @@ -897,28 +901,100 @@ func TestCommitRequestAttach_TakesOverUnattributedWindowsOnlyFromASourceWithNoAu } return p } + unaudited := provider(metav1.ConditionUnknown) cases := []struct { name string + ask configv1alpha3.WindowAuthor objects []client.Object - want bool + want git.UnattributedClaim + refused bool }{ - {"audit facts received", []client.Object{target, provider(metav1.ConditionTrue)}, false}, - {"no audit fact yet", []client.Object{target, provider(metav1.ConditionUnknown)}, true}, - {"author attribution off", []client.Object{target, provider("")}, true}, - {"KubeProvider missing", []client.Object{target}, false}, - {"GitTarget missing", nil, false}, + { + "own asks nothing", + configv1alpha3.WindowAuthorOwn, + []client.Object{ + target(configv1alpha3.UnattributedWindowsTakeOver), + unaudited, + }, + git.UnattributedClaimNone, + false, + }, + { + "takeover allowed", + configv1alpha3.WindowAuthorTakeOver, + []client.Object{ + target(configv1alpha3.UnattributedWindowsTakeOver), + unaudited, + }, + git.UnattributedClaimTakeOver, + false, + }, + { + "saved-by under a takeover policy", + configv1alpha3.WindowAuthorSavedBy, + []client.Object{ + target(configv1alpha3.UnattributedWindowsTakeOver), + unaudited, + }, + git.UnattributedClaimSavedBy, + false, + }, + { + "saved-by allowed", + configv1alpha3.WindowAuthorSavedBy, + []client.Object{ + target(configv1alpha3.UnattributedWindowsSavedBy), + unaudited, + }, + git.UnattributedClaimSavedBy, + false, + }, + { + "author attribution off", + configv1alpha3.WindowAuthorTakeOver, + []client.Object{ + target(configv1alpha3.UnattributedWindowsTakeOver), + provider(""), + }, + git.UnattributedClaimTakeOver, + false, + }, + {"audit facts received", configv1alpha3.WindowAuthorTakeOver, + []client.Object{target(configv1alpha3.UnattributedWindowsTakeOver), provider(metav1.ConditionTrue)}, + git.UnattributedClaimNone, false}, + {"KubeProvider missing", configv1alpha3.WindowAuthorTakeOver, + []client.Object{target(configv1alpha3.UnattributedWindowsTakeOver)}, git.UnattributedClaimNone, false}, + {"takeover under a saved-by policy", configv1alpha3.WindowAuthorTakeOver, + []client.Object{target(configv1alpha3.UnattributedWindowsSavedBy), unaudited}, "", true}, + {"saved-by under the default policy", configv1alpha3.WindowAuthorSavedBy, + []client.Object{target(""), unaudited}, "", true}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - objects := append([]client.Object{newCommitRequest("save")}, tc.objects...) - c := newCommitRequestClient(t, nil, objects...) + cr := newCommitRequest("save") + cr.Spec.Window = &configv1alpha3.CommitRequestWindow{Author: tc.ask} + c := newCommitRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) f := &fakeFinalizer{} r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save") + if tc.refused { + assert.Empty(t, f.calls, "a refused request is never attached") + var got configv1alpha3.CommitRequest + require.NoError( + t, + c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got), + ) + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionFalse, ready.Status) + assert.Equal(t, crReasonTakeOverNotAllowed, ready.Reason) + assert.True(t, apimeta.IsStatusConditionTrue(got.Status.Conditions, ConditionTypeStalled)) + return + } require.NotEmpty(t, f.calls, "the request must be attached") - assert.Equal(t, tc.want, f.calls[0].TakeOverUnattributed) + assert.Equal(t, tc.want, f.calls[0].Unattributed) }) } } diff --git a/internal/controller/commitrequest_finalize.go b/internal/controller/commitrequest_finalize.go index 313a31ae..9c3e1db4 100644 --- a/internal/controller/commitrequest_finalize.go +++ b/internal/controller/commitrequest_finalize.go @@ -4,6 +4,7 @@ package controller import ( "context" + "errors" "go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/metric" @@ -27,6 +28,7 @@ const ( crReasonCommitterFallback = "CommitterFallback" crReasonAuthorCaptureDisabled = "AuthorCaptureDisabled" crReasonGitTargetRefPruned = "GitTargetRefPruned" + crReasonTakeOverNotAllowed = "TakeOverNotAllowed" ) // The bounded `outcome` values on gitopsreverser_commit_requests_total. @@ -279,7 +281,11 @@ func applyFinalizeResultToStatus( setCommitRequestAttributed(cr, attribution) if finalizeErr != nil { - failCommitRequest(cr, crReasonFinalizeFailed, finalizeErr.Error()) + reason := crReasonFinalizeFailed + if errors.Is(finalizeErr, errTakeOverNotAllowed) { + reason = crReasonTakeOverNotAllowed + } + failCommitRequest(cr, reason, finalizeErr.Error()) return } diff --git a/internal/git/attribution_outcome_test.go b/internal/git/attribution_outcome_test.go index 7e98dd96..53117d3e 100644 --- a/internal/git/attribution_outcome_test.go +++ b/internal/git/attribution_outcome_test.go @@ -258,33 +258,36 @@ func TestRenderEventCommitMessage_UnresolvedUsernameStaysEmptyInTemplates(t *tes assert.Equal(t, UnresolvedAuthorDisplayName, options.Author.Name) } -// A request that names its submitter takes over a window whose writes name no author only when it -// is allowed to, and never a window whose writes name an actor, nor one on another GitTarget. -func TestPendingCommitRequest_TakesOverOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { +// A request that names its submitter claims a window whose writes name no author only when it was +// allowed to, and never a window whose writes name an actor, nor one on another GitTarget. +func TestPendingCommitRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { unattributed := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) named := newOpenWindow(attributedEvent("bob@acme.com", AttributionResolved), nil) - request := func(allowed bool, target string) *pendingCommitRequest { + request := func(claim UnattributedClaim, target string) *pendingCommitRequest { return &pendingCommitRequest{ - author: "jane@acme.com", - attribution: AttributionResolved, - gitTargetName: target, - gitTargetNamespace: "default", - takeOverUnattributed: allowed, + author: "jane@acme.com", + attribution: AttributionResolved, + gitTargetName: target, + gitTargetNamespace: "default", + unattributed: claim, } } - assert.False(t, request(false, "target").matchesWindow(unattributed), - "without the allowance a named request keeps the author rule") - assert.True(t, request(true, "target").matchesWindow(unattributed)) - assert.True(t, request(true, "target").takesOver(unattributed)) - assert.False(t, request(true, "target").matchesWindow(named), - "a window whose writes name an actor is never taken over") - assert.False(t, request(true, "other-target").matchesWindow(unattributed), - "the allowance does not widen attachment across GitTargets") + assert.False(t, request(UnattributedClaimNone, "target").matchesWindow(unattributed), + "without a claim a named request keeps the author rule") + for _, claim := range []UnattributedClaim{UnattributedClaimSavedBy, UnattributedClaimTakeOver} { + assert.True(t, request(claim, "target").matchesWindow(unattributed), string(claim)) + assert.True(t, request(claim, "target").claims(unattributed), string(claim)) + assert.False(t, request(claim, "target").matchesWindow(named), + "a window whose writes name an actor is never claimed: %s", claim) + assert.False(t, request(claim, "other-target").matchesWindow(unattributed), + "a claim does not widen attachment across GitTargets: %s", claim) + } own := newOpenWindow(attributedEvent("jane@acme.com", AttributionResolved), nil) - assert.True(t, request(true, "target").matchesWindow(own)) - assert.False(t, request(true, "target").takesOver(own), "the submitter's own window is not a takeover") + assert.True(t, request(UnattributedClaimTakeOver, "target").matchesWindow(own)) + assert.False(t, request(UnattributedClaimTakeOver, "target").claims(own), + "the submitter's own window is not a claim") } func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { @@ -296,3 +299,10 @@ func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes", withTakeOverNote("save: edit", id)) } + +func TestWithSavedByNote_NamesTheSubmitterLikeACoAuthor(t *testing.T) { + assert.Equal(t, "save: edit\n\nSaved-By: jane@acme.com ", + withSavedByNote("save: edit", UserInfo{Username: "jane@acme.com"})) + assert.Equal(t, "save: edit\n\nSaved-By: Jane Doe \n", + withSavedByNote("save: edit\n", UserInfo{Username: "jane", DisplayName: "Jane Doe", Email: "jane@acme.com"})) +} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 15ab561a..c0ca3698 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1538,9 +1538,13 @@ func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinali return } pendingWrite.CommitMessage = effectiveMessage - if takenOverBy := l.openWindow.takenOverBy; takenOverBy != nil { - pendingWrite.RequestAuthor, pendingWrite.RequestAttribution = *takenOverBy, AttributionResolved + switch l.openWindow.claim { + case UnattributedClaimTakeOver: + pendingWrite.RequestAuthor, pendingWrite.RequestAttribution = l.openWindow.claimedBy, AttributionResolved pendingWrite.TakenOver = true + case UnattributedClaimSavedBy: + pendingWrite.SavedBy = l.openWindow.claimedBy + case UnattributedClaimNone: } pendingCR, proceed := l.gateBuiltWindow(pendingCR, pendingWrite.Target(), reason) if !proceed { diff --git a/internal/git/commit_executor.go b/internal/git/commit_executor.go index 73363d39..df76d030 100644 --- a/internal/git/commit_executor.go +++ b/internal/git/commit_executor.go @@ -313,8 +313,11 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes if err != nil { return "", nil, messageResolutionUnsupported, err } - if p.TakenOver { + switch { + case p.TakenOver: message = withTakeOverNote(message, p.CommitRequest) + case p.SavedBy.Username != "": + message = withSavedByNote(message, p.SavedBy) } log.Log.V(1).Info("Selected commit message", "source", resolution.label()) return message, commitOptionsFor(p, p.CommitConfig, p.Signer, time.Now()), resolution, nil @@ -327,9 +330,20 @@ func withTakeOverNote(message string, request *commitRequestID) string { if request != nil { name = "CommitRequest " + request.Namespace + "/" + request.Name } + return withTrailer(message, "Author-Source: "+name+"; no audit record names who made these changes") +} + +// withSavedByNote ends the message of a commit whose CommitRequest gave a window with no author named +// its message (PendingWrite.SavedBy) with a trailer naming the request's submitter, in the shape of a +// Co-authored-by trailer. +func withSavedByNote(message string, savedBy UserInfo) string { + return withTrailer(message, "Saved-By: "+authorName(savedBy)+" <"+authorEmail(savedBy)+">") +} + +// withTrailer adds trailer as the message's last paragraph, keeping the message's trailing newline. +func withTrailer(message, trailer string) string { body := strings.TrimRight(message, "\n") - return body + "\n\nAuthor-Source: " + name + "; no audit record names who made these changes" + - message[len(body):] + return body + "\n\n" + trailer + message[len(body):] } // framedRequestMessage renders the target's requestTemplate around an attached CommitRequest's diff --git a/internal/git/commit_request_attach.go b/internal/git/commit_request_attach.go index bec92e8f..15c8352f 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -149,13 +149,28 @@ type AttachCommitRequest struct { // CatchUpNow asks the GitTarget's applier to apply the tip as soon as the request's commit is // pushed: catchUp: AfterPush. CatchUpNow bool - // TakeOverUnattributed lets a request that names its submitter take over a window on its - // GitTarget whose writes name no author, and author its commit. It is set when the GitTarget's - // KubeProvider has never received an audit fact, so no write from its cluster can name one. Once - // it has, a write naming no author means its fact went missing, and that window stays another's. - TakeOverUnattributed bool + // Unattributed is what a request that names its submitter may claim of a window on its GitTarget + // whose writes name no author: nothing, its message (SavedBy), or the window (TakeOver). The + // controller sets more than nothing only when the request asked, its GitTarget allows it, and + // the GitTarget's KubeProvider has never received an audit fact. Once it has, a write naming no + // author means its fact went missing, and that window stays another's. + Unattributed UnattributedClaim } +// UnattributedClaim is what a CommitRequest may claim of a commit window whose writes name no author. +type UnattributedClaim string + +const ( + // UnattributedClaimNone attaches only to the submitter's own window. + UnattributedClaimNone UnattributedClaim = "" + // UnattributedClaimSavedBy gives the window the request's message and names no author: the + // commit stays unattributed, and its message ends with a Saved-By trailer. + UnattributedClaimSavedBy UnattributedClaim = "SavedBy" + // UnattributedClaimTakeOver takes the window over: the submitter authors the commit, and its + // message ends with an Author-Source trailer. + UnattributedClaimTakeOver UnattributedClaim = "TakeOver" +) + // commitRequestID is the worker-local key for a CommitRequest: its namespaced // name plus UID when available. Two CommitRequests with the same name but // different UIDs (a delete-and-recreate) are distinct. @@ -194,9 +209,9 @@ type pendingCommitRequest struct { commitEmpty bool // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. catchUpNow bool - // takeOverUnattributed lets the request take over a window whose writes name no author - // (AttachCommitRequest.TakeOverUnattributed). - takeOverUnattributed bool + // unattributed is what the request may claim of a window whose writes name no author + // (AttachCommitRequest.Unattributed). + unattributed UnattributedClaim // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. @@ -262,14 +277,14 @@ func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { if p.gitTargetName != w.GitTarget || p.gitTargetNamespace != w.GitTargetNamespace { return false } - return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.takesOver(w) + return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.claims(w) } -// takesOver reports whether the request matches a window on its GitTarget whose writes name no -// author only because it may take such a window over: it names its submitter, who then authors the -// commit (AttachCommitRequest.TakeOverUnattributed). -func (p *pendingCommitRequest) takesOver(w *openWindow) bool { - return p.takeOverUnattributed && p.attribution.NamesActor() && !w.Attribution.NamesActor() +// claims reports whether the request matches a window on its GitTarget whose writes name no author +// only because it may claim such a window: it names its submitter, and asked for more than its own +// windows (AttachCommitRequest.Unattributed). +func (p *pendingCommitRequest) claims(w *openWindow) bool { + return p.unattributed != UnattributedClaimNone && p.attribution.NamesActor() && !w.Attribution.NamesActor() } // commitRequestOutcomeEntry is a resolved outcome retained for the controller to diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/commit_request_attach_loop.go index c0be06e5..4f7c7d7d 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -216,7 +216,7 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque commitEmpty: req.CommitEmpty, catchUpNow: req.CatchUpNow, - takeOverUnattributed: req.TakeOverUnattributed, + unattributed: req.Unattributed, } l.pendingCRs[id] = pcr l.w.setCommitRequestPhase(id, PhaseWaitingForWindow) @@ -347,10 +347,10 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { } l.openWindow.timers = timers l.openWindow.lastWriteAt = now - if pcr.takesOver(l.openWindow) { - l.openWindow.takenOverBy = &UserInfo{Username: pcr.author} - l.w.Log.Info("CommitRequest took over a window whose writes name no author; its submitter authors the commit", - "request", id.Namespace+"/"+id.Name, "author", pcr.author, + if pcr.claims(l.openWindow) { + l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} + l.w.Log.Info("CommitRequest claimed a window whose writes name no author", + "request", id.Namespace+"/"+id.Name, "author", pcr.author, "claim", string(pcr.unattributed), "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) } l.w.setCommitRequestPhase(id, PhaseCollectingWindow) diff --git a/internal/git/commit_request_attach_test.go b/internal/git/commit_request_attach_test.go index 5563524f..75f09341 100644 --- a/internal/git/commit_request_attach_test.go +++ b/internal/git/commit_request_attach_test.go @@ -358,7 +358,7 @@ func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { req := attachReq("jane@acme.com", 60*time.Second) req.Attribution = AttributionResolved req.Message = "save: from the editor" - req.TakeOverUnattributed = true + req.Unattributed = UnattributedClaimTakeOver serviceAttach(loop, req) require.NotNil(t, loop.openWindow.pendingCR, "the request takes the window over") @@ -378,6 +378,43 @@ func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { "no audit record names who made these changes", commit.Message) } +// TestAttach_SavedByGivesTheMessageAndNamesNoAuthor pins the milder claim: the request gives a window +// whose writes name no author its message, the commit stays authored as unresolved, and the message +// ends with a Saved-By trailer naming the submitter. +func TestAttach_SavedByGivesTheMessageAndNamesNoAuthor(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + + loop := newBranchWorkerEventLoop(worker, time.Hour) + defer loop.stopTimers() + + event := configMapTargetEvent("cm", "", "team-a") + event.Attribution = AttributionUnresolved + loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{event}}}) + require.NotNil(t, loop.openWindow) + + req := attachReq("jane@acme.com", 60*time.Second) + req.Attribution = AttributionResolved + req.Message = "save: from the editor" + req.Unattributed = UnattributedClaimSavedBy + serviceAttach(loop, req) + require.NotNil(t, loop.openWindow.pendingCR, "the request gives the window its message") + + forceDue(loop) + loop.endWake(0) + res, ok := outcome(t, worker) + require.True(t, ok) + require.NoError(t, res.Err) + require.Equal(t, FinalizeCommitted, res.Outcome) + + repo, err := gogit.PlainOpen(worker.repoPath()) + require.NoError(t, err) + commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) + require.NoError(t, err) + assert.Equal(t, UnresolvedAuthorDisplayName, commit.Author.Name, "nobody is named as the author") + assert.Equal(t, "save: from the editor\n\nSaved-By: jane@acme.com ", commit.Message) +} + // TestAttach_ForeignWindowIsNotStolen verifies an attach for a different author // parks (never finalizes another author's window) and resolves WindowMismatch once // its grace elapses, leaving the foreign window open. diff --git a/internal/git/open_window.go b/internal/git/open_window.go index 9cbc7d13..a1fe5299 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -46,9 +46,11 @@ type openWindow struct { // pendingCR identifies the CommitRequest attached to this window; at most one. On // finalize its outcome is resolved (Committed once the carrying write pushes). pendingCR *commitRequestID - // takenOverBy is the submitter of the attached CommitRequest when it took the window over: its - // writes name no author, so the submitter authors the commit (pendingCommitRequest.takesOver). - takenOverBy *UserInfo + // claim and claimedBy say what the attached CommitRequest claimed of the window, whose writes + // name no author, and who submitted it (pendingCommitRequest.claims): its message (SavedBy), or + // the window, which the submitter then authors (TakeOver). Empty when it claimed nothing. + claim UnattributedClaim + claimedBy UserInfo // timers decide when the window closes. They are the GitTarget's, snapshotted when the window // opened, until a CommitRequest attaches and replaces them with its own. diff --git a/internal/git/types.go b/internal/git/types.go index a979ac0a..f54a59f3 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -386,8 +386,12 @@ type PendingWrite struct { RequestAttribution AttributionOutcome // TakenOver says the CommitRequest attached to this grouped write took over a window whose // writes named no author: RequestAuthor and RequestAttribution, the submitter's, author the - // commit, and its message ends with a note that says so (withTakeOverNote). + // commit, and its message ends with an Author-Source trailer (withTakeOverNote). TakenOver bool + // SavedBy is the submitter of the CommitRequest that gave this grouped write its message without + // claiming its writes, which named no author: the commit stays unattributed, and its message ends + // with a Saved-By trailer naming them (withSavedByNote). Zero when no request did. + SavedBy UserInfo // emptyCommitted is stamped by executePendingWrites, the way CommitSHA is: the commit this // write made changed no file. A request riding it resolves with the cause (AlreadyPresent or // NoOpenWindow) and the commit, so a caller can tell "saved your changes" from "recorded your diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 081685f4..134247d6 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -784,12 +784,16 @@ spec: // is made as soon as the first is pushed, so the two may share one apply. // // No write in the workspace names its author, since kcp sends no audit facts, and the request's - // submitter is captured at admission in this cluster. The workspace's KubeProvider has never - // received a fact, so each request takes over the window of its edit, its submitter authors the - // commit, and the message ends with a note that says so. Every event from the workspace waits - // out the attribution grace first, so the requests wait longer than that for their window. + // submitter is captured at admission in this cluster. The GitTarget allows a takeover, each + // request asks for one, and the workspace's KubeProvider has never received a fact, so each + // request takes over the window of its edit: its submitter authors the commit, and the message + // ends with a trailer that says so. Every event from the workspace waits out the attribution + // grace first, so the requests wait longer than that for their window. It("takes over the window of an edit in the workspace, and reports the apply on each of two saves", func() { verifyResourceCondition("kubeprovider", ws, testNs, "AuditFactsReceived", "Unknown", "", "") + _, err := kubectlRunInNamespace(testNs, "patch", "gittarget", p.targetName, "--type=merge", "-p", + `{"spec":{"commit":{"unattributedWindows":"TakeOver"}}}`) + Expect(err).NotTo(HaveOccurred()) names := []string{"sc-flux-save-1", "sc-flux-save-2"} commits := map[string]string{} for i, name := range names { @@ -900,6 +904,7 @@ spec: message: "e2e: save %s and apply it" catchUp: AfterPush window: + author: TakeOver attachTimeout: "45s" maxDuration: "4s" `, name, namespace, gitTarget, name), "apply", "-f", "-") From 1a45946a86ec20b77186d4da4fc4062b997a0427 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:39:54 +0000 Subject: [PATCH 12/33] fix(applier): an apply window waits until the watch delivered the applier's writes An apply window judges a write by when its event reaches the branch worker. It closed 10s after the operation completed. Every event waits for its audit fact in turn, and on a source cluster that sends none each waits the whole attribution grace, so the applier's writes reached the worker after the window had closed. Mirrored then, a write that reverted an edit made during the apply put the revert in Git and lost the edit; the source-cluster e2e showed Flux's writes arriving 12s after the window closed. Now, once the operation completes, the window takes a barrier: for each GitTarget the applier covers, the watch lists every stream's collection and takes the newest object resourceVersion. Each stream answers once it has handled an event or bookmark at or past it, on the worker's queue behind every event it routed first. The window stays open for its grace and until every stream answered, for at most two minutes. apply_window_barriers_total{outcome} counts met, timed_out and failed barriers. A source whose resourceVersions are not numbers fails the barrier and keeps the grace alone, as before. Co-Authored-By: Claude Opus 5.5 (1M context) --- cmd/main.go | 3 + docs/bi-directional.md | 6 + docs/interpreting-metrics.md | 3 +- .../20261026-apply-window-late-events.md | 264 ++++++++++++++++++ internal/git/applier_claim.go | 22 +- internal/git/apply_barrier.go | 195 +++++++++++++ internal/git/apply_barrier_test.go | 148 ++++++++++ internal/git/apply_window.go | 25 +- internal/git/branch_worker.go | 9 + internal/git/types.go | 3 + internal/git/worker_manager.go | 13 + internal/telemetry/apply_window.go | 16 ++ internal/telemetry/exporter.go | 1 + internal/watch/apply_barrier.go | 180 ++++++++++++ internal/watch/apply_barrier_test.go | 113 ++++++++ internal/watch/manager.go | 5 + internal/watch/target_watch.go | 5 + 17 files changed, 1000 insertions(+), 11 deletions(-) create mode 100644 docs/review/20261026-apply-window-late-events.md create mode 100644 internal/git/apply_barrier.go create mode 100644 internal/git/apply_barrier_test.go create mode 100644 internal/watch/apply_barrier.go create mode 100644 internal/watch/apply_barrier_test.go diff --git a/cmd/main.go b/cmd/main.go index 0395a8f1..2397dbe5 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -184,6 +184,9 @@ func main() { // A worker discards a snapshot an apply made stale and asks for a fresh one, and asks no applier // for anything before a GitTarget's initial snapshots are in Git: both read the streams. workerManager.SetSnapshotHooks(watchMgr.InitialSnapshotsSettled, watchMgr.RequestRecheckForGitTarget) + // An apply window waits, after its operation completed, until the streams delivered every write + // made before then, however far behind attribution held them. + workerManager.SetApplyBarrier(watchMgr.ApplyBarrier) // The branch workers decide when a GitTarget's applier may apply; the dispatcher delivers it, // off their queues, to the applier its applier claim holds, in the applier claim's cluster. The status // writer records each operation on the applier claim, also off their queues. diff --git a/docs/bi-directional.md b/docs/bi-directional.md index 81ec57ba..7fdafe03 100644 --- a/docs/bi-directional.md +++ b/docs/bi-directional.md @@ -281,6 +281,12 @@ nothing else. `KubeProvider`s, and, for a `Refusal`, still consents; and the applier claim still holds the applier. When any of that changed, it asks nothing and decides again, after a new flush. It never asks for something weaker than what it was told. +- **The apply window waits for the watch.** After the applier reports the apply finished, the window + stays open for a 10-second grace, and until the watch has delivered every write the source cluster + held at that moment, for at most two minutes. Each event waits for its audit fact first, and on a + cluster that sends none, for the whole attribution grace, so the applier's writes can reach the + branch worker long after the apply finished. They are still judged inside the window + (`apply_window_barriers_total`). - **The apply's own writes are not mirrored.** Inside the apply window, a write whose newest field managers are the applier's alone, made since the request was sent, to an object nobody edited in the window, is the apply landing. It is kept out of Git and counted in diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index 34d5ac43..d127e008 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -867,7 +867,8 @@ else pushed. See [who applies this folder](configuration.md#who-applies-this-fol | `apply_revision_mismatches_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `applier_kind`, `claim_namespace`, `claim_name` | One per apply operation that succeeded on another revision than the commit GitOps Reverser checked. A `Kustomization` applies whatever its `GitRepository` artifact holds when it reconciles; the artifact is read again just before the resume, so this is somebody fetching the source in between (a person, or the source's own interval). The apply window still withholds a late write over a newer edit; a steady rate means something else keeps fetching the source | | `apply_claimed_resource_reconciled_total` | counter | `applier_kind`, `claim_namespace`, `claim_name` | One per apply an applier claim did not start: a held `Kustomization` resumed (`flux resume`, or a controller writing its spec without `spec.suspend`), or a held `Application`'s automated sync switched on, by somebody else. From then until the claim takes it back, on its next look within a minute, the applier applies whatever its source holds, unchecked against the branch. The claim records a `ClaimedResourceReconciled` Event for each as well, but the API server folds identical Events into one and expires them, so alert on this counter. A `Kustomization` resumed so also shows the apply as an `Unresolved` operation on the claim | | `apply_window_collisions_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `outcome` | One per write by the applier, while an apply it was asked for runs, to an object the API side edited after the flush before that request: the apply arriving late over a newer edit. `outcome` is `withheld` (proven to be the applier's write, by its field manager alone, with an entry as new as the edit or newer: `kustomize-controller`, or `argocd-controller` / `argocd-application-controller`; so not mirrored: Git keeps the API's value, and a `Repair` of the applier follows, which restores it on the cluster) or `ambiguous` (the applier and another field manager wrote in the same second, so it was mirrored as ever; if it was the apply, the revert is in Git as the applier's commit). An applier entry older than the edit is not counted: it is the API's undo of an edit, mirrored | -| `apply_echoes_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `outcome` | One per write by the applier's own field managers alone, judged by the apply window: an apply GitOps Reverser asked for landing, object by object. `outcome` is `not_mirrored` (the applier wrote the object last, at or after the earliest send of the apply windows still open, and nobody edited it in the window: the apply itself, whose content is the commit Git already holds, so it is not mirrored) or `late` (the applier's managers alone wrote the object last with no apply window open: an echo arriving more than 10 seconds after its operation completed, or an apply somebody else started; it is mirrored as an edit). `not_mirrored` rises with every apply and needs no action. A steady `late` rate means applies keep landing outside the windows, and such a write can revert a newer Git-side change to the object: look for a person or a tool applying the folder by hand (`flux reconcile`, a manual sync), or for watch events reaching the worker long after the apply | +| `apply_echoes_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `outcome` | One per write by the applier's own field managers alone, judged by the apply window: an apply GitOps Reverser asked for landing, object by object. `outcome` is `not_mirrored` (the applier wrote the object last, at or after the earliest send of the apply windows still open, and nobody edited it in the window: the apply itself, whose content is the commit Git already holds, so it is not mirrored) or `late` (the applier's managers alone wrote the object last with no apply window open: an echo arriving after its window closed, or an apply somebody else started; it is mirrored as an edit). `not_mirrored` rises with every apply and needs no action. A steady `late` rate means applies keep landing outside the windows, and such a write can revert a newer Git-side change to the object: look for a person or a tool applying the folder by hand (`flux reconcile`, a manual sync), or for watch events reaching the worker long after the apply | +| `apply_window_barriers_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `outcome` | One per apply window whose operation completed: the window waits until the watch has delivered every write its GitTargets' source clusters held at that moment, so the applier's own writes are judged inside it however far behind the watch is (each event waits for its audit fact, and on a cluster that sends none, for the whole `--author-attribution-grace`). `outcome` is `met` (every stream delivered them; the window then closed at the end of its 10-second grace), `timed_out` (the window closed at the 2-minute limit with a stream still behind: a later write of the applier's is mirrored as an edit, and counted as `apply_echoes_total{outcome="late"}`), or `failed` (the source cluster could not be listed, or its resourceVersions are not numbers; the window closed after its grace alone). A steady `timed_out` or `failed` rate means the window cannot see every apply it asked for land | | `apply_ownership_conflicts_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | One per object another applier owns that the branch worker kept out of a coordinated folder: a live write not mirrored, or a snapshot object neither upserted nor swept. The owner is read from Flux's owner labels (`kustomize.toolkit.fluxcd.io/name` and `/namespace`) or Argo CD's tracking annotation (`argocd.argoproj.io/tracking-id`). Any non-zero rate is a misconfiguration: another `Kustomization` or `Application` applies objects into a namespace the folder mirrors, and two appliers applying one object fight each other. The worker logs each object once, naming both appliers; narrow the `WatchRule`, or move the object to one applier | | `apply_stale_snapshots_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `outcome` | One per resync snapshot the branch worker found stale: collected while Git was ahead of the cluster (a commit pushed and not yet applied), collected across an apply, or arriving while Git is ahead. `outcome` is `discarded` (not decided; a fresh snapshot of its `GitTarget` is asked for once Git is no longer ahead, and the `GitTarget`'s live writes are held back by the render-fidelity gate until it lands) or `used` (the `GitTarget`'s snapshots had been held for 5 minutes while an apply kept failing, so this one was decided anyway, and whatever the unapplied commit changed in its objects is reverted, as the API-wins rule says). `discarded` comes in short bursts around applies and needs no action; a discard is not a resync failure and is not counted in `git_resync_failures_total`. Any `used` means an applier is failing for minutes at a time: read the applier claim's `status.operation` and the `ApplyFailed` Events. See [stale snapshots](spec/reconcile-via-watchlist-mark-and-sweep.md#stale-snapshots-are-discarded) | | `apply_operations_unresolved` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | Apply operations on the branch that have not completed 5 minutes after they were requested. Each is still followed, holds its applier's apply window open, and holds back the applier's next request until it completes, so a value that stays above zero is an applier stuck mid-apply: read its status, and the `ApplyUnresolved` Event on the `GitTarget`s. Read at scrape time | diff --git a/docs/review/20261026-apply-window-late-events.md b/docs/review/20261026-apply-window-late-events.md new file mode 100644 index 00000000..1a213740 --- /dev/null +++ b/docs/review/20261026-apply-window-late-events.md @@ -0,0 +1,264 @@ +# The apply window misses late events from a cluster with no audit stream + +On a source cluster that sends no audit facts, every event the watch delivers waits the full +attribution grace (10s in e2e) before it reaches the branch worker. The stream resolves events one at +a time, so the waits add up. The apply window closes on a wall clock, 10s after the applier reports +the apply finished, so the applier's own writes can reach the worker after the window has closed. +A write that arrives then is mirrored as an edit. If that write was an apply reverting a newer API +edit, the revert lands in Git and the edit is lost. + +This was found by the source-cluster e2e for gitops-api's production topology (ask #34): Flux in the +operator's cluster applies into a kcp workspace, and the GitTarget mirrors that workspace through a +`secretRef` KubeProvider. The e2e kcp has no audit stream. The run is at commit `cf3b97be`. + +Status: fix 2 below is implemented (`internal/git/apply_barrier.go`, `internal/watch/apply_barrier.go`), +with one change from the first draft: the mark is the newest resourceVersion among the collection's +objects, not the list's own, so a quiet stream does not wait for a bookmark. + +## The apply window, in short + +When the branch worker asks an applier to apply a commit, it opens an **apply window** for that +applier. Every live event for a GitTarget the applier covers is judged against it: + +- A write by the applier's field manager alone, to an object nobody edited since the window opened, + is **the apply landing**. Its content is the commit's, which Git already holds, so it is not + mirrored (`apply_echoes_total{outcome="not_mirrored"}`). +- A write by the applier's field manager alone, to an object the API side edited after the window + opened, is **the apply arriving late over that edit**. It is withheld, and a Repair puts the edit + back on the cluster (`apply_window_collisions_total{outcome="withheld"}`). +- Any other write is an edit, and is mirrored as usual. + +The window opens when the attempt is sent (`internal/git/applier_claim.go`, `openApplyWindow`) and +closes `applyWindowGrace` (10s) after the operation completed (`settleApplyWindow`). With no window +open, a write by the applier's manager alone is mirrored as an edit, and counted as +`apply_echoes_total{outcome="late"}`. + +```mermaid +flowchart LR + Send["Worker sends the attempt"] -->|window opens| Open(["Apply window open"]) + Open --> Done["Applier reports the operation completed"] + Done -->|"+ 10s grace"| Closed(["Apply window closed"]) + + Event["A live event arrives at the worker"] --> When{"Is a window open
when it arrives?"} + When -->|yes| Judge["Judged: the apply landing,
withheld, or an edit"] + When -->|no| Mirror["Mirrored as an edit
(late, if Flux wrote it)"] +``` + +The window is judged by **when the event arrives at the worker**, not by when the write happened on +the cluster. That gap is the whole problem. + +## Why events from the workspace arrive late + +Between the watch and the worker, each event waits for its author: an audit fact that names who made +the write. On a cluster with audit, the fact usually arrives within a second or two. On a cluster +with no audit stream, no fact ever arrives, so every event waits the full grace +(`--author-attribution-grace`, 10s in e2e) and then goes on unattributed. In this run, the median of +`attribution_resolution_wait_seconds` was 10s. + +A watch stream handles its events in order, one at a time (`internal/watch/target_watch.go`, +`attachAuthor` is called inline). So an event queued behind others waits for their waits as well. + +```mermaid +sequenceDiagram + participant K as kcp workspace + participant S as Watch stream + participant A as Attribution + participant W as Branch worker + + K->>S: write 1 (Flux) + K->>S: write 2 (an API edit), 1s later + S->>A: who wrote 1? + Note over A: no audit fact ever comes
waits the full 10s + A-->>S: unattributed + S->>W: event 1, 10s after write 1 + S->>A: who wrote 2? + Note over A: waits another 10s + A-->>S: unattributed + S->>W: event 2, 19s after write 2 +``` + +## What this run observed + +The collision spec pushed a Git-side change (`o1` to `during-other`), waited until the FluxClaim +showed the apply had been asked for, and then edited `during` through the workspace's API. These +are the worker's log lines, with the times the writes happened inferred from the spec's log. + +```mermaid +sequenceDiagram + participant T as Test + participant W as Branch worker + participant F as Flux + participant K as kcp workspace + + T->>W: push o1 (commit a9cb18d) + W->>F: 16:36:16 apply a9cb18d (ForeignCommit) + Note over W: window opens + F->>K: about 16:36:16, writes during-other + T->>K: about 16:36:17, edits during + F-->>W: about 16:36:20, operation completed + Note over W: 16:36:30 window closes
(completion + 10s) + K-->>W: 16:36:32, Flux's write to during-other arrives + Note over W: no window open:
mirrored as an edit, late echo counted + K-->>W: 16:36:42, the edit of during arrives
(25s after it was made) + Note over W: mirrored and committed +``` + +Two things happened here: + +- **Flux was faster than the edit.** It wrote before the spec's edit landed, so it never reverted the + edit. There was nothing to withhold, the edit reached Git and stayed on the workspace, and + `apply_window_collisions_total` stayed at 0. That is why the spec failed: it asserts that one + collision is counted. Waiting for the operation token and then editing does not guarantee that + the edit lands before the applier writes. The local E5 spec uses the same technique and passes + whichever side wins, which is why it never asserts the counter. +- **One of Flux's writes arrived after the window closed.** `apply_echoes_total{outcome="late"}` is 1 + on this branch. Here it was harmless, because the write held what Git already held. + +## What goes wrong when Flux does revert an edit + +Had the edit landed before Flux's write, Flux would have applied the commit it was asked for, which +does not hold the edit yet, and so reverted it on the workspace. The window exists to catch that +write. With the delays above, the write reaches the worker after the window has closed: + +```mermaid +sequenceDiagram + participant U as Someone editing + participant W as Branch worker + participant F as Flux + participant K as kcp workspace + participant G as Git + + W->>F: t+0 apply commit C (does not hold the edit) + Note over W: window opens + U->>K: t+1 edit: value = "edit" + F->>K: t+2 apply C: value = "old" (reverts the edit) + F-->>W: t+4 operation completed + K-->>W: t+11 the edit arrives + Note over W: window open: recorded as an edit,
mirrored + W->>G: commit value = "edit" + Note over W: t+14 window closes + K-->>W: t+21 Flux's revert arrives + Note over W: no window open: mirrored as an edit + W->>G: commit value = "old" + Note over G,K: Git and the workspace both hold "old".
No Repair is asked for. The edit survives
only in Git history. +``` + +This is the loss the apply window was built to prevent (#29), and from such a cluster nothing in the +window can catch it. + +A longer grace does not fix it. Events wait in a queue, so the delay grows with the number of events +ahead: the edit in this run waited 25s, more than twice the grace, because it queued behind Flux's +write. No fixed grace covers every burst. + +## Fix 1: do not wait for audit facts that never come + +The KubeProvider's `AuditFactsReceived` condition says whether an audit fact has ever arrived on its +route. When it is not `True`, waiting for one is pointless, so attribution could skip the wait for +events from that provider. Events would then reach the worker within moments of the write, inside +the window. + +```mermaid +flowchart LR + E["Event from the watch"] --> Q{"Has this KubeProvider ever
received an audit fact?"} + Q -->|yes| Wait["Wait up to the grace
for this write's fact"] + Q -->|no| Skip["Go on unattributed
at once"] + Wait --> W["Branch worker"] + Skip --> W +``` + +It is small, and it also removes the 10s or more of delay on every edit mirrored from such a cluster. +It has two limits: + +- On a newly added audited provider, writes made before its first fact arrives are recorded with no + author. With the CommitRequest takeover (`cf3b97be`), a request could then take over their window. +- It does nothing for a cluster that does send audit facts but sends them slowly. The window still + closes on the clock. + +## Fix 2: close the window by how far the watch has got + +The idea in one sentence: instead of closing the window "10 seconds after Flux said it was done", close +it "once the watch has delivered everything that happened on the cluster up to the moment Flux said it +was done". + +Every write to a Kubernetes object gets a **resourceVersion**, a position in the cluster's log of +changes, and a watch delivers events in that order. So the worker can mark a position and wait until +the watch has passed it: + +1. **Take a mark when the apply completes.** The worker lists the watched collection and takes the + newest resourceVersion among its objects, say `1284`: the position of the latest write to anything + in it. Every write Flux made for this apply happened before the apply completed, so each has a + resourceVersion of at most `1284`. +2. **Keep the window open until the watch passes the mark.** The stream records how far it has got + each time it records its cursor: after every event it handled, routed or not, and every + **bookmark**, a progress-only event in which the API server says "you have everything up to + resourceVersion X". Once it is at or past `1284`, it puts an answer on the worker's queue. +3. **Then close.** The answer rides the worker's queue behind every event the stream routed before + it, so once every stream has answered, every write Flux made has reached the worker, however long + attribution held each one. The window then closes at the end of its grace, or at once if that has + passed. + +The list's own resourceVersion would be the cluster's current position, which a quiet stream passes +only on a bookmark, and a bookmark can be a minute away. An object's resourceVersion is a write the +stream has seen or will see. + +```mermaid +sequenceDiagram + participant F as Flux + participant K as kcp workspace + participant S as Watch stream + participant W as Branch worker + + W->>F: apply commit C + Note over W: window opens + F->>K: writes (resourceVersions 1281, 1284) + F-->>W: operation completed + W->>K: list the collection + K-->>W: newest object resourceVersion 1284, the mark + Note over W: window stays open until the
stream passes 1284 + S->>W: event at 1281 (after its attribution wait) + Note over W: judged inside the window + S->>W: event at 1284 (after its attribution wait) + Note over W: judged inside the window + S->>W: the stream's answer: at 1284 + Note over W: past the mark: everything Flux
wrote has arrived. Window closes. +``` + +The decision the worker makes, for each window: + +```mermaid +flowchart TB + Done["Operation completed:
mark = the newest object resourceVersion now"] --> Check{"Has every stream of the
applier's GitTargets handed over
an event or bookmark at or past the mark?"} + Check -->|yes| Close(["Close the window"]) + Check -->|"not yet"| Bound{"Held longer than
the upper bound?"} + Bound -->|no| Check + Bound -->|yes| Give(["Close anyway, and log and count it"]) +``` + +What it costs: + +- **resourceVersions are opaque in the API contract.** Clients are not promised that they compare as + numbers, though the API server's own watch cache compares them that way on etcd. A source whose + resourceVersions do not parse as a number fails the barrier, and its window closes after its grace + alone, as before. +- **A write Flux deleted is not in the list**, so its event can still arrive after the window. A + DELETE is never withheld anyway, and the object it removes is one Git no longer holds. +- **A list per stream per apply.** It is paged, and made once when the operation completes. +- **A GitTarget can have several streams**, one per watched collection, and the applier can cover + several GitTargets. The window waits for all of them. +- **An upper bound** keeps a stream that is disconnected or stuck from holding the window open + forever. It is two minutes. Past it the window closes, and the close is logged and counted + (`apply_window_barriers_total{outcome="timed_out"}`), so the miss is visible. +- **A window held open longer** judges more writes. A write by Flux alone to an object nobody edited + is treated as the apply landing and not mirrored. That is still correct for an apply nobody asked + for, because its content is what Git holds. + +Fix 2 is the general fix: it covers a cluster with no audit stream, a cluster with slow audit, and a +burst of events of any size, because it waits for the watch itself rather than guessing a delay. +Fix 1 can come on top of it, to cut the delay on every mirrored edit from a cluster with no audit +stream. + +## The spec, separately + +Neither fix makes the collision spec deterministic. It needs the edit to land after the apply was +asked for and before Flux writes the object. Flux wrote within about a second of the resume here, +and nothing in the spec holds the apply open longer than that. diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 678904da..91348781 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -882,6 +882,22 @@ func (l *branchWorkerEventLoop) sendAttempt(state *applierState, tip string, ref }) } +// settleWindowOf ends a completed attempt's part in its applier's apply window: a window whose +// operation ran stays open for its grace, and until the watch delivered the applier's writes +// (apply_barrier.go). +func (l *branchWorkerEventLoop) settleWindowOf( + state *applierState, attempt *inflightAttempt, outcome ApplyOutcome, now time.Time, +) { + state.settleApplyWindow(attempt.id, outcome.Operated(), now) + if outcome.Operated() { + l.awaitApplyBarrier(state, now) + } + if attempt.phase == ApplyUnresolved { + l.w.Log.Info("An unresolved apply completed; its apply window closes after the grace", + "branch", l.w.Branch, "applier", state.ref.String(), "outcome", string(outcome)) + } +} + // settleApplyResults reads the reports that came back. A report of an operation still out records // where it stands, and holds the attempt. An operation that succeeded is done. Any other ending // gives back what the attempt carried and, unless it was only stale, waits out its applier's @@ -903,11 +919,7 @@ func (l *branchWorkerEventLoop) settleApplyResults(now time.Time) { state.inflight = nil attempt.abandon() l.w.raiseApplyEpoch() - state.settleApplyWindow(attempt.id, res.Outcome.Operated(), now) - if attempt.phase == ApplyUnresolved { - l.w.Log.Info("An unresolved apply completed; its apply window closes after the grace", - "branch", l.w.Branch, "applier", state.ref.String(), "outcome", string(res.Outcome)) - } + l.settleWindowOf(state, attempt, res.Outcome, now) switch res.Outcome { case ApplySucceeded: state.failures, state.retryAt = 0, time.Time{} diff --git a/internal/git/apply_barrier.go b/internal/git/apply_barrier.go new file mode 100644 index 00000000..2813657a --- /dev/null +++ b/internal/git/apply_barrier.go @@ -0,0 +1,195 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "context" + "time" + + "go.opentelemetry.io/otel/attribute" + "go.opentelemetry.io/otel/metric" + + "github.com/ConfigButler/gitops-reverser/internal/telemetry" + itypes "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// An apply window judges a write by when its event reaches the worker, not by when the write was +// made. A grace after the operation completed covers a watch that delivers within seconds. It does +// not cover one that is behind: every event waits for its audit fact in turn, and on a source cluster +// that sends none each waits the whole attribution grace, so the applier's writes can arrive long +// after the window closed. Mirrored then, a write that reverted an edit made during the apply put the +// revert in Git and lost the edit (docs/review/20261026-apply-window-late-events.md). +// +// So a window whose operation completed also waits for the watch. For each GitTarget the applier +// covers, the watch side lists every stream's collection and takes the newest resourceVersion among +// its objects: the position of the latest write to anything in it, the applier's writes included. +// Each stream answers once it has handled an event or a bookmark at or past that position. Its answer +// rides this worker's queue, behind every event the stream routed before it, so when every stream +// has answered, every write the applier made has reached the window. The window then closes at the +// end of its grace, or at once if that has passed. +// +// A write the applier deleted is not in the list, so its event can still arrive after the window: a +// DELETE is never withheld anyway, and the object it removes is one Git no longer holds. A barrier +// that cannot be taken (the source cannot be listed, or its resourceVersions are not numbers) leaves +// the grace alone, as before. A stream that stops, or never gets that far, holds the window for at +// most applyWindowBarrierLimit. + +// applyWindowBarrierLimit bounds how long a window waits for the watch after its operation completed. +// Past it the window closes anyway, and the miss is logged and counted (barrierTimedOut). +const applyWindowBarrierLimit = 2 * time.Minute + +// ApplyBarrierFunc asks the watch side to call deliver, once per running stream of target, when that +// stream has handled every write its collection held when the barrier was taken, and returns how many +// streams will. deliver must enqueue on the worker's queue, so it lands behind every event the stream +// routed before it. It is called off the worker's loop, and may list the source cluster. +type ApplyBarrierFunc func(ctx context.Context, target itypes.ResourceReference, deliver func()) (int, error) + +// ApplyBarrier is one answer to a barrier an apply window took: a stream that got there (Passed), the +// number of streams that will answer (Registered), or the barrier failing (Err). +type ApplyBarrier struct { + Applier ApplierRef + Token uint64 + + Passed bool + Registered bool + Expected int + Err error +} + +// applyBarrierWait is what a window waits for after its operation completed. +type applyBarrierWait struct { + token uint64 + completedAt time.Time + // until bounds the wait: completedAt plus applyWindowBarrierLimit. + until time.Time + // expected is how many streams will answer, -1 until the registration arrived; passed is how many + // have. + expected int + passed int +} + +// holds reports whether the wait still keeps its window open at now. +func (b *applyBarrierWait) holds(now time.Time) bool { + return b != nil && now.Before(b.until) +} + +// Apply-window barrier outcomes, the outcome label of apply_window_barriers_total. +const ( + barrierMet = "met" + barrierTimedOut = "timed_out" + barrierFailed = "failed" +) + +// awaitApplyBarrier makes a window whose operation completed wait for the watch, when the worker has +// a barrier to take. The barrier is taken off the loop: listing a source cluster can take a while. +func (l *branchWorkerEventLoop) awaitApplyBarrier(state *applierState, now time.Time) { + window := state.window + w := l.w + if w.applyBarrier == nil || window == nil || window.closesAt.IsZero() || len(state.targets) == 0 { + return + } + l.barrierSeq++ + token := l.barrierSeq + window.barrier = &applyBarrierWait{ + token: token, completedAt: now, until: now.Add(applyWindowBarrierLimit), expected: -1, + } + ref := state.ref + targets := make([]itypes.ResourceReference, 0, len(state.targets)) + for _, target := range state.targets { + targets = append(targets, target.ref) + } + ctx := w.ctx + if ctx == nil { + ctx = context.Background() + } + go func() { + expected := 0 + for _, target := range targets { + n, err := w.applyBarrier(ctx, target, func() { + w.enqueueApplyBarrier(&ApplyBarrier{Applier: ref, Token: token, Passed: true}) + }) + if err != nil { + w.enqueueApplyBarrier(&ApplyBarrier{Applier: ref, Token: token, Err: err}) + return + } + expected += n + } + w.enqueueApplyBarrier(&ApplyBarrier{Applier: ref, Token: token, Registered: true, Expected: expected}) + }() +} + +// handleApplyBarrier counts one answer to the barrier its window waits for. An answer to a barrier +// the window no longer waits for (met, replaced by a newer attempt's window, or timed out) is dropped. +func (l *branchWorkerEventLoop) handleApplyBarrier(answer *ApplyBarrier) { + state := l.appliers[answer.Applier] + if state == nil || state.window == nil || state.window.barrier == nil || + state.window.barrier.token != answer.Token { + return + } + window := state.window + wait := window.barrier + switch { + case answer.Err != nil: + window.barrier = nil + l.w.Log.Info("Could not take the apply window's barrier; the window closes after its grace alone", + "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, "err", answer.Err.Error()) + l.recordApplyBarrier(barrierFailed) + return + case answer.Registered: + wait.expected = answer.Expected + case answer.Passed: + wait.passed++ + } + if wait.expected < 0 || wait.passed < wait.expected { + return + } + window.barrier = nil + l.w.Log.V(1).Info("The watch delivered every write made before the apply completed", + "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, + "waited", time.Since(wait.completedAt).Round(time.Millisecond).String(), "streams", wait.expected) + l.recordApplyBarrier(barrierMet) +} + +// barrierTimedOut logs and counts a window closing while it still waited for the watch. +func (l *branchWorkerEventLoop) barrierTimedOut(state *applierState) { + window := state.window + wait := window.barrier + if wait == nil { + return + } + window.barrier = nil + l.w.Log.Info("The apply window closed before the watch delivered every write made before the apply "+ + "completed: a write of the applier's arriving later is mirrored as an edit", + "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, + "waited", time.Since(wait.completedAt).Round(time.Second).String(), + "streams", wait.expected, "delivered", wait.passed) + l.recordApplyBarrier(barrierTimedOut) +} + +// recordApplyBarrier counts one apply window barrier, by outcome. +func (l *branchWorkerEventLoop) recordApplyBarrier(outcome string) { + if telemetry.ApplyWindowBarriersTotal == nil { + return + } + telemetry.ApplyWindowBarriersTotal.Add(context.WithoutCancel(l.w.ctx), 1, + metric.WithAttributes(l.w.gitBranchAttrs(attribute.String("outcome", outcome))...)) +} + +// enqueueApplyBarrier puts one barrier answer on the worker's queue. It never blocks: an answer the +// full queue drops leaves its window waiting until applyWindowBarrierLimit. +func (w *BranchWorker) enqueueApplyBarrier(answer *ApplyBarrier) { + w.inflightItems.Add(1) + w.pendingResyncsMu.Lock() + defer w.pendingResyncsMu.Unlock() + if w.stoppingLocked() { + w.releaseInflightItem() + return + } + select { + case w.eventQueue <- WorkItem{ApplyBarrier: answer}: + default: + w.releaseInflightItem() + w.Log.V(1).Info("Event queue full, apply window barrier answer dropped; the window closes at its limit", + "branch", w.Branch, "applier", answer.Applier.String()) + } +} diff --git a/internal/git/apply_barrier_test.go b/internal/git/apply_barrier_test.go new file mode 100644 index 00000000..9f56aafd --- /dev/null +++ b/internal/git/apply_barrier_test.go @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "context" + "errors" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + sdkmetric "go.opentelemetry.io/otel/sdk/metric" + + "github.com/ConfigButler/gitops-reverser/internal/telemetry" + itypes "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// fakeBarrier is the watch side of a barrier with one stream per GitTarget: it keeps each stream's +// deliver for the test to call, or fails. +type fakeBarrier struct { + mu sync.Mutex + delivers []func() + err error +} + +func (f *fakeBarrier) take(_ context.Context, _ itypes.ResourceReference, deliver func()) (int, error) { + if f.err != nil { + return 0, f.err + } + f.mu.Lock() + defer f.mu.Unlock() + f.delivers = append(f.delivers, deliver) + return 1, nil +} + +func (f *fakeBarrier) deliverAll() { + f.mu.Lock() + delivers := f.delivers + f.delivers = nil + f.mu.Unlock() + for _, deliver := range delivers { + deliver() + } +} + +// dispatchQueued hands every item on the worker's queue to the loop, as run() would. +func dispatchQueued(loop *branchWorkerEventLoop) { + for { + select { + case item := <-loop.w.eventQueue: + loop.dispatch(item) + loop.w.releaseInflightItem() + default: + return + } + } +} + +// completedWithBarrier is alpha's applier with an attempt that completed, its window waiting for a +// barrier whose streams have registered and not yet delivered. +func completedWithBarrier(t *testing.T, barrier *fakeBarrier) (*branchWorkerEventLoop, *applierState) { + t.Helper() + loop, req := sentAttempt(t) + loop.w.applyBarrier = barrier.take + report(req, ApplySucceeded) + loop.serviceAppliers() + state := loop.appliers[applierRef("alpha")] + require.NotNil(t, state.window) + require.Eventually(t, func() bool { + dispatchQueued(loop) + return state.window.barrier == nil || state.window.barrier.expected >= 0 + }, 5*time.Second, 10*time.Millisecond, "the barrier's registration reaches the loop") + return loop, state +} + +func barriers(t *testing.T, reader *sdkmetric.ManualReader, outcome string) int64 { + t.Helper() + got, _ := telemetry.CollectInt64Sum(reader, "gitopsreverser_apply_window_barriers_total", + map[string]string{"outcome": outcome}) + return got +} + +// A window whose operation completed stays open past its grace while the watch has not delivered +// every write made before the apply completed, so the applier's write is still judged inside it; once +// every stream delivered, it closes. +func TestApplyBarrier_HoldsTheWindowUntilEveryStreamDelivered(t *testing.T) { + reader, err := telemetry.InitTestExporter() + require.NoError(t, err) + barrier := &fakeBarrier{} + loop, state := completedWithBarrier(t, barrier) + require.NotNil(t, state.window.barrier, "the window waits for the watch") + assert.Equal(t, 1, state.window.barrier.expected) + + state.window.closesAt = time.Now().Add(-time.Millisecond) + assert.False(t, loop.withholdLateApply(windowEvent("alpha", "3", "kubectl-edit")), "an API edit is mirrored") + assert.True(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), + "past its grace, the window still judges the applier's write: withheld over the edit") + + barrier.deliverAll() + dispatchQueued(loop) + assert.Nil(t, state.window.barrier, "every stream delivered") + assert.Equal(t, int64(1), barriers(t, reader, barrierMet)) + assert.False(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), + "with its grace over and the barrier met, the window is closed") +} + +// A window waits for the watch at most applyWindowBarrierLimit. Past it, it closes, and the miss is +// counted. +func TestApplyBarrier_TheWindowClosesAtTheLimit(t *testing.T) { + reader, err := telemetry.InitTestExporter() + require.NoError(t, err) + loop, state := completedWithBarrier(t, &fakeBarrier{}) + require.NotNil(t, state.window.barrier) + + past := time.Now().Add(-time.Millisecond) + state.window.closesAt, state.window.barrier.until = past, past + loop.reportApplyWindows(time.Now()) + assert.Nil(t, state.window.barrier) + assert.Equal(t, int64(1), barriers(t, reader, barrierTimedOut)) + assert.False(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), "the window is closed") +} + +// A barrier that cannot be taken leaves the window its grace alone. +func TestApplyBarrier_AFailedBarrierLeavesTheGrace(t *testing.T) { + reader, err := telemetry.InitTestExporter() + require.NoError(t, err) + loop, state := completedWithBarrier(t, &fakeBarrier{err: errors.New("the source cannot be listed")}) + + assert.Nil(t, state.window.barrier) + assert.Equal(t, int64(1), barriers(t, reader, barrierFailed)) + assert.True(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), "within the grace") + state.window.closesAt = time.Now().Add(-time.Millisecond) + assert.False(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), "after it") +} + +// An answer for a barrier the window no longer waits for changes nothing. +func TestApplyBarrier_AnAnswerForAnotherBarrierIsDropped(t *testing.T) { + barrier := &fakeBarrier{} + loop, state := completedWithBarrier(t, barrier) + wait := state.window.barrier + require.NotNil(t, wait) + + loop.handleApplyBarrier(&ApplyBarrier{Applier: state.ref, Token: wait.token + 1, Passed: true}) + assert.Same(t, wait, state.window.barrier) + assert.Zero(t, wait.passed) +} diff --git a/internal/git/apply_window.go b/internal/git/apply_window.go index 565d36da..c7809a5d 100644 --- a/internal/git/apply_window.go +++ b/internal/git/apply_window.go @@ -136,6 +136,10 @@ type applyWindow struct { // object. reported is set once that count was logged (reportApplyWindows). echoes int reported bool + // barrier is what the window waits for once its operation completed: the watch delivering every + // write the GitTargets' source clusters held by then (apply_barrier.go). Nil when it waits for + // none, or once it was met. + barrier *applyBarrierWait } // earliestSend is the lower bound of the applier's own writes at now: the earliest send of this @@ -148,9 +152,19 @@ func (a *applyWindow) earliestSend(now time.Time) time.Time { return a.sentAt } -// open reports whether the window is open at now. +// open reports whether the window is open at now: until its grace has passed, and for as long as +// it still waits for the watch (barrier). func (a *applyWindow) open(now time.Time) bool { - return a != nil && (a.closesAt.IsZero() || now.Before(a.closesAt)) + return a != nil && (a.closesAt.IsZero() || now.Before(a.closesAt) || a.barrier.holds(now)) +} + +// closesBy is when the window closes at the latest: zero while the attempt is out, and past the +// grace while it waits for the watch. +func (a *applyWindow) closesBy() time.Time { + if a.closesAt.IsZero() || a.barrier == nil { + return a.closesAt + } + return latest(a.closesAt, a.barrier.until) } // objectKey is an event's object without its API version: a preferred-version bump during the @@ -190,7 +204,7 @@ func (state *applierState) openApplyWindow(attempt uint64, commit string, now ti if now.Before(before.carriedUntil) { maps.Copy(next.carried, before.carried) } - next.carriedUntil = before.closesAt + next.carriedUntil = before.closesBy() next.carriedSentAt = before.earliestSend(now) } state.window = next @@ -341,9 +355,10 @@ func (l *branchWorkerEventLoop) reportApplyWindows(now time.Time) time.Time { switch { case window == nil || window.reported: case !window.open(now): + l.barrierTimedOut(state) l.reportApplyWindow(state) - case !window.closesAt.IsZero() && (next.IsZero() || window.closesAt.Before(next)): - next = window.closesAt + case !window.closesAt.IsZero() && (next.IsZero() || window.closesBy().Before(next)): + next = window.closesBy() } } return next diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index c0ca3698..f3aa556b 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -176,6 +176,10 @@ type BranchWorker struct { // admits every GitTarget. applierAdmitted ApplierAdmission resyncRecheck func(target itypes.ResourceReference) + // applyBarrier lets an apply window wait for the watch after its operation completed + // (apply_barrier.go). Set by the WorkerManager; nil in the CLI and in tests, where a window closes + // after its grace alone. + applyBarrier ApplyBarrierFunc // replacementPending reports whether this worker replaced one whose GitTargets have not been told // yet (WorkerManager.ReplacementPending): until they are, their snapshot record is the old // worker's, and no applier is asked for anything. Nil in the CLI and in tests. @@ -1006,6 +1010,9 @@ func (w *BranchWorker) processEvents() { type branchWorkerEventLoop struct { w *BranchWorker + // barrierSeq numbers the barriers this loop's apply windows take (apply_barrier.go). + barrierSeq uint64 + // defaultWindow is the operator-level default for a GitTarget's commit window timers, used // for any target that declares no spec.commit.window field and for one that cannot be read. defaultWindow commitWindowDefaults @@ -1255,6 +1262,8 @@ func (l *branchWorkerEventLoop) dispatch(item WorkItem) int64 { l.handleAttachCommitRequest(item.Attach) case item.Withdraw != nil: l.handleWithdrawCommitRequest(item.Withdraw) + case item.ApplyBarrier != nil: + l.handleApplyBarrier(item.ApplyBarrier) case item.Resync != nil: // Take the CURRENT request for this scope: a newer one may have replaced it while this // marker waited in the FIFO. Running it here keeps the original queue position, so a resync diff --git a/internal/git/types.go b/internal/git/types.go index f54a59f3..0c2edde0 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -466,6 +466,9 @@ type WorkItem struct { // Withdraw cancels a CommitRequest the worker has not acted on yet. It rides the same FIFO as // the attach, so it is always handled after every attach sent before it. Withdraw *AttachCommitRequest + // ApplyBarrier is one answer to the barrier an apply window took (apply_barrier.go). A stream's + // answer rides the FIFO behind every event that stream routed before it. + ApplyBarrier *ApplyBarrier // charge is what a write or an attach was charged against the intake budget at enqueue, and is // released once the loop has handled it. A resync's is on its marker, because coalescing can diff --git a/internal/git/worker_manager.go b/internal/git/worker_manager.go index 848ef12b..16d49f00 100644 --- a/internal/git/worker_manager.go +++ b/internal/git/worker_manager.go @@ -185,6 +185,9 @@ type WorkerManager struct { // tests. initialSnapshots func(target types.ResourceReference) bool resyncRecheck func(target types.ResourceReference) + // applyBarrier lets every worker's apply windows wait for the watch (apply_barrier.go). Set once + // at startup (SetApplyBarrier); nil in the CLI and in tests. + applyBarrier ApplyBarrierFunc // applyRequester delivers every worker's apply requests. Set once at startup // (SetApplyRequester) before any worker is created; nil in the CLI and in tests, which switches @@ -279,6 +282,15 @@ func (m *WorkerManager) SetSnapshotHooks( m.initialSnapshots, m.resyncRecheck = settled, recheck } +// SetApplyBarrier injects the watch side's barrier, which lets an apply window wait until the watch +// delivered every write made before its apply completed (apply_barrier.go). Like SetMapper, it is +// called once at startup before any worker is created. +func (m *WorkerManager) SetApplyBarrier(barrier ApplyBarrierFunc) { + m.mu.Lock() + defer m.mu.Unlock() + m.applyBarrier = barrier +} + // SetSSHHostKeyConfig injects the SSH host-key resolution config used by every worker's credential // reads. Like SetMapper, it is called once at startup before any worker is created. func (m *WorkerManager) SetSSHHostKeyConfig(cfg SSHHostKeyConfig) { @@ -522,6 +534,7 @@ func (m *WorkerManager) startWorkerLocked(key BranchKey, req WorkerRequest, gitB worker.claimStatus = m.claimStatus worker.initialSnapshots = m.initialSnapshots worker.resyncRecheck = m.resyncRecheck + worker.applyBarrier = m.applyBarrier worker.replacementPending = func() bool { return m.ReplacementPending(key) } worker.applierAdmitted = m.applierAdmitted worker.renderFidelityGate = m.renderFidelityGate diff --git a/internal/telemetry/apply_window.go b/internal/telemetry/apply_window.go index 65c3c2e7..92192fb7 100644 --- a/internal/telemetry/apply_window.go +++ b/internal/telemetry/apply_window.go @@ -22,6 +22,22 @@ import "go.opentelemetry.io/otel/metric" // Registered in registerCounters (exporter.go) with every other counter. var ApplyWindowCollisionsTotal metric.Int64Counter +// ApplyWindowBarriersTotal counts the barriers apply windows took after their operation completed: a +// wait until the watch had delivered every write the source clusters held by then, so the applier's +// own writes are judged inside the window however far behind the watch was. Labelled by +// {gitbranch_*, provider_*, outcome}. outcome is a frozen enum: +// +// met — every stream delivered every such write; the window then closed at the end of its grace. +// timed_out — the window closed at the limit (2 minutes) with a stream still behind: a write of the +// applier's arriving later is mirrored as an edit (apply_echoes_total{outcome="late"}). +// failed — the barrier could not be taken (the source cluster could not be listed, or its +// resourceVersions are not numbers); the window closed after its grace alone. +// +// A steady timed_out or failed rate means the window cannot see every apply it asked for land. +// +// Registered in registerCounters (exporter.go) with every other counter. +var ApplyWindowBarriersTotal metric.Int64Counter + // ApplyEchoesTotal counts the applier's own writes the apply window judged: an apply GitOps Reverser // asked for landing, object by object. Labelled by {gitbranch_*, provider_*, // outcome}. outcome is a frozen enum: diff --git a/internal/telemetry/exporter.go b/internal/telemetry/exporter.go index 10cfc1ab..7bde4072 100644 --- a/internal/telemetry/exporter.go +++ b/internal/telemetry/exporter.go @@ -622,6 +622,7 @@ func registerCounters() error { {"gitopsreverser_apply_claimed_resource_reconciled_total", &ClaimedResourceReconciledTotal}, {"gitopsreverser_apply_window_collisions_total", &ApplyWindowCollisionsTotal}, {"gitopsreverser_apply_echoes_total", &ApplyEchoesTotal}, + {"gitopsreverser_apply_window_barriers_total", &ApplyWindowBarriersTotal}, {"gitopsreverser_apply_stale_snapshots_total", &StaleSnapshotsTotal}, {"gitopsreverser_apply_ownership_conflicts_total", &ApplyOwnershipConflictsTotal}, {"gitopsreverser_receiver_deliveries_total", &ReceiverDeliveriesTotal}, diff --git a/internal/watch/apply_barrier.go b/internal/watch/apply_barrier.go new file mode 100644 index 00000000..f65d504a --- /dev/null +++ b/internal/watch/apply_barrier.go @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "context" + "fmt" + "strconv" + "sync" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// The watch side of an apply window's barrier (internal/git/apply_barrier.go). A window whose +// operation completed asks, for each GitTarget its applier covers, to hear from every running stream +// once that stream has handled every write its collection held at that moment. The mark is the +// newest resourceVersion among the collection's objects, from a list made now: the position of the +// latest write to anything in it. A stream records how far it has got each time it records its +// cursor, after the event was routed, so the answer it sends from there rides the worker's queue +// behind every event it routed first. +// +// The mark is an object's resourceVersion, not the list's. The list's is the source cluster's +// current position, which a quiet stream only passes on a bookmark, and a bookmark can be a minute +// away. An object's is a write the stream will see, or has seen. +// +// resourceVersions are compared as numbers. The Kubernetes API leaves them opaque, and a source +// whose resourceVersions do not parse as one fails the barrier, so its window closes after its grace +// alone. + +// barrierListPage is the page size of the list a barrier takes of a stream's collection. +const barrierListPage = 500 + +// streamProgress is how far one running target stream has got: the newest resourceVersion it has +// handled, an event or a bookmark, and the barriers waiting for it to get further. +type streamProgress struct { + key targetWatchKey + + mu sync.Mutex + seen uint64 + waiting []progressWaiter +} + +// progressWaiter is one barrier waiting for a stream to reach mark. +type progressWaiter struct { + mark uint64 + deliver func() +} + +// advance records that the stream handled rv, and answers every barrier it has now reached. A +// resourceVersion that is not a number advances nothing. +func (p *streamProgress) advance(rv string) { + n, err := strconv.ParseUint(rv, 10, 64) + if err != nil { + return + } + p.mu.Lock() + p.seen = max(p.seen, n) + var due []func() + kept := p.waiting[:0] + for _, waiter := range p.waiting { + if waiter.mark <= p.seen { + due = append(due, waiter.deliver) + } else { + kept = append(kept, waiter) + } + } + p.waiting = kept + p.mu.Unlock() + for _, deliver := range due { + deliver() + } +} + +// await answers deliver once the stream has handled mark: at once when it already has. +func (p *streamProgress) await(mark uint64, deliver func()) { + p.mu.Lock() + if mark <= p.seen { + p.mu.Unlock() + deliver() + return + } + p.waiting = append(p.waiting, progressWaiter{mark: mark, deliver: deliver}) + p.mu.Unlock() +} + +// trackStreamProgress registers a running stream's progress, replacing a previous stream's for the +// same collection. +func (m *Manager) trackStreamProgress(gitDest types.ResourceReference, key targetWatchKey) *streamProgress { + progress := &streamProgress{key: key} + m.streamProgressMu.Lock() + defer m.streamProgressMu.Unlock() + if m.streamProgress == nil { + m.streamProgress = map[string]map[types.CollectionKey]*streamProgress{} + } + streams := m.streamProgress[gitDest.Key()] + if streams == nil { + streams = map[types.CollectionKey]*streamProgress{} + m.streamProgress[gitDest.Key()] = streams + } + streams[key.Collection()] = progress + return progress +} + +// untrackStreamProgress removes a stopped stream's progress, unless a newer stream replaced it. Its +// waiting barriers are never answered; their windows close at their limit. +func (m *Manager) untrackStreamProgress(gitDest types.ResourceReference, key targetWatchKey, progress *streamProgress) { + m.streamProgressMu.Lock() + defer m.streamProgressMu.Unlock() + streams := m.streamProgress[gitDest.Key()] + if streams[key.Collection()] != progress { + return + } + delete(streams, key.Collection()) + if len(streams) == 0 { + delete(m.streamProgress, gitDest.Key()) + } +} + +// advanceStreamProgress records that the stream of gitDest's collection handled rv. +func (m *Manager) advanceStreamProgress(gitDest types.ResourceReference, key targetWatchKey, rv string) { + if rv == "" { + return + } + m.streamProgressMu.Lock() + progress := m.streamProgress[gitDest.Key()][key.Collection()] + m.streamProgressMu.Unlock() + if progress != nil { + progress.advance(rv) + } +} + +// ApplyBarrier is the watch side of an apply window's barrier (git.ApplyBarrierFunc): it marks every +// running stream of gitDest at the newest write its collection holds now, has each call deliver once +// it has handled that write, and returns how many streams will. It lists the source cluster, so it +// runs off the branch worker's loop. +func (m *Manager) ApplyBarrier(ctx context.Context, gitDest types.ResourceReference, deliver func()) (int, error) { + m.streamProgressMu.Lock() + streams := make([]*streamProgress, 0, len(m.streamProgress[gitDest.Key()])) + for _, progress := range m.streamProgress[gitDest.Key()] { + streams = append(streams, progress) + } + m.streamProgressMu.Unlock() + + clusterID := m.clusterIDForGitTarget(gitDest) + for _, progress := range streams { + mark, err := m.collectionMark(ctx, clusterID, progress.key) + if err != nil { + return 0, fmt.Errorf("mark %s for %s: %w", progress.key.Collection().String(), gitDest.String(), err) + } + progress.await(mark, deliver) + } + return len(streams), nil +} + +// collectionMark is the newest resourceVersion among the objects of a stream's collection: the +// position of the latest write to any of them. An empty collection is zero. +func (m *Manager) collectionMark(ctx context.Context, clusterID string, key targetWatchKey) (uint64, error) { + var mark uint64 + opts := metav1.ListOptions{Limit: barrierListPage} + for { + list, err := m.openTargetList(ctx, clusterID, key, opts) + if err != nil { + return 0, err + } + for i := range list.Items { + rv := list.Items[i].GetResourceVersion() + n, err := strconv.ParseUint(rv, 10, 64) + if err != nil { + return 0, fmt.Errorf("resourceVersion %q of %s is not a number", rv, list.Items[i].GetName()) + } + mark = max(mark, n) + } + if list.GetContinue() == "" { + return mark, nil + } + opts.Continue = list.GetContinue() + } +} diff --git a/internal/watch/apply_barrier_test.go b/internal/watch/apply_barrier_test.go new file mode 100644 index 00000000..d54b46f4 --- /dev/null +++ b/internal/watch/apply_barrier_test.go @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "context" + "sync/atomic" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// barrierList lists objects with the given resourceVersions, two to a page. +func barrierList(rvs ...string) func( + context.Context, schema.GroupVersionResource, string, metav1.ListOptions, +) (*unstructured.UnstructuredList, error) { + return func( + _ context.Context, _ schema.GroupVersionResource, _ string, opts metav1.ListOptions, + ) (*unstructured.UnstructuredList, error) { + start := 0 + if opts.Continue != "" { + start = 2 + } + list := &unstructured.UnstructuredList{} + for i := start; i < len(rvs) && i < start+2; i++ { + item := unstructured.Unstructured{} + item.SetName("cm") + item.SetResourceVersion(rvs[i]) + list.Items = append(list.Items, item) + } + if start == 0 && len(rvs) > 2 { + list.SetContinue("next") + } + return list, nil + } +} + +// A stream answers a barrier once it has handled the newest write its collection held when the +// barrier was taken, an event or a bookmark, and not before. +func TestApplyBarrier_AStreamAnswersOnceItHandledTheNewestWrite(t *testing.T) { + gitDest := types.NewResourceReference("target", "team-a") + key := targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "team-a"} + manager := &Manager{targetWatchList: barrierList("12", "40", "7")} + manager.trackStreamProgress(gitDest, key) + manager.advanceStreamProgress(gitDest, key, "30") + + var delivered atomic.Int32 + streams, err := manager.ApplyBarrier(context.Background(), gitDest, func() { delivered.Add(1) }) + require.NoError(t, err) + assert.Equal(t, 1, streams) + assert.Zero(t, delivered.Load(), "the stream has handled 30, and the newest write, on the second page, is 40") + + manager.advanceStreamProgress(gitDest, key, "39") + assert.Zero(t, delivered.Load()) + manager.advanceStreamProgress(gitDest, key, "41") + assert.Equal(t, int32(1), delivered.Load(), "a bookmark past the mark answers it") + manager.advanceStreamProgress(gitDest, key, "50") + assert.Equal(t, int32(1), delivered.Load(), "once") +} + +// A stream that already handled the newest write answers at once, and a collection with no objects +// asks for nothing. +func TestApplyBarrier_AStreamThatIsCaughtUpAnswersAtOnce(t *testing.T) { + gitDest := types.NewResourceReference("target", "team-a") + key := targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "team-a"} + for _, rvs := range [][]string{{"12", "40"}, {}} { + manager := &Manager{targetWatchList: barrierList(rvs...)} + manager.trackStreamProgress(gitDest, key) + manager.advanceStreamProgress(gitDest, key, "40") + + var delivered atomic.Int32 + streams, err := manager.ApplyBarrier(context.Background(), gitDest, func() { delivered.Add(1) }) + require.NoError(t, err) + assert.Equal(t, 1, streams) + assert.Equal(t, int32(1), delivered.Load(), "%v", rvs) + } +} + +// A source whose resourceVersions are not numbers fails the barrier, and a GitTarget with no +// running stream has nobody to answer. +func TestApplyBarrier_NonNumericResourceVersionsFailAndNoStreamsAnswerNothing(t *testing.T) { + gitDest := types.NewResourceReference("target", "team-a") + key := targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "team-a"} + manager := &Manager{targetWatchList: barrierList("abc")} + + streams, err := manager.ApplyBarrier(context.Background(), gitDest, func() {}) + require.NoError(t, err) + assert.Zero(t, streams, "no stream runs") + + manager.trackStreamProgress(gitDest, key) + _, err = manager.ApplyBarrier(context.Background(), gitDest, func() {}) + require.Error(t, err) +} + +// A stream replaced by a newer one for the same collection keeps the newer one's progress when it +// stops. +func TestApplyBarrier_AStoppedStreamDoesNotRemoveItsReplacement(t *testing.T) { + gitDest := types.NewResourceReference("target", "team-a") + key := targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "team-a"} + manager := &Manager{} + old := manager.trackStreamProgress(gitDest, key) + newer := manager.trackStreamProgress(gitDest, key) + manager.untrackStreamProgress(gitDest, key, old) + assert.Same(t, newer, manager.streamProgress[gitDest.Key()][key.Collection()]) + manager.untrackStreamProgress(gitDest, key, newer) + assert.Empty(t, manager.streamProgress) +} diff --git a/internal/watch/manager.go b/internal/watch/manager.go index fe836ea3..cc137b63 100644 --- a/internal/watch/manager.go +++ b/internal/watch/manager.go @@ -172,6 +172,11 @@ type Manager struct { openWatchesMu sync.Mutex openWatches map[openWatchKey]int + // streamProgress is how far each running target stream has got, by GitTarget key and + // collection, for the barriers apply windows take (apply_barrier.go). + streamProgressMu sync.Mutex + streamProgress map[string]map[types.CollectionKey]*streamProgress + // targetWatches is the data plane: one raw watch per (GitTarget, GVR, namespace scope), and // the only source of live object state. Its initial desired set comes from the replay, or from // the buffered LIST fallback when sendInitialEvents is unsupported. diff --git a/internal/watch/target_watch.go b/internal/watch/target_watch.go index c7368452..f64a74c1 100644 --- a/internal/watch/target_watch.go +++ b/internal/watch/target_watch.go @@ -572,6 +572,8 @@ func (m *Manager) runTargetWatch( // sweep, and leave the scope pending all the same. stream.replayed = &atomic.Bool{} stream.changes = newDesiredStateChangeFilter() + progress := m.trackStreamProgress(gitDest, stream.key) + defer m.untrackStreamProgress(gitDest, stream.key, progress) for ctx.Err() == nil { err := m.targetWatchReplayAndStream(ctx, log, gitDest, stream, stream.hasReplayed()) recordWatchSessionEnded(ctx, m.clusterIDForGitTarget(gitDest), stream.key.GVR, sessionEndReason(ctx, err)) @@ -1474,6 +1476,9 @@ func (m *Manager) recordTargetWatchCursor( key targetWatchKey, rv string, ) error { + // Every event the stream handled, routed or not, and every bookmark passes through here after it + // was routed, which makes it where an apply window's barrier learns how far the stream has got. + m.advanceStreamProgress(gitDest, key, rv) uid := m.resolveGitTargetUID(gitDest) if m.WatchCursorStore == nil || rv == "" || uid == "" { return nil From df42e46c3da8d1873d5155430e1c00a353fcda67 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:42:12 +0000 Subject: [PATCH 13/33] test(e2e): provoke the workspace collision by editing before its event arrives E5 edits once the apply was asked for and races Flux, which often wrote first, so the source-cluster spec counted no collision. On a kcp workspace every event waits out the attribution grace, so an edit made just before a Git-side push reaches the worker after the push's apply was asked for and applied: Flux reverts it, and the apply window, which now waits for the watch, withholds that write. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../20261026-apply-window-late-events.md | 11 +++++-- test/e2e/source_cluster_e2e_test.go | 32 ++++++++++++++++--- 2 files changed, 35 insertions(+), 8 deletions(-) diff --git a/docs/review/20261026-apply-window-late-events.md b/docs/review/20261026-apply-window-late-events.md index 1a213740..aaed75da 100644 --- a/docs/review/20261026-apply-window-late-events.md +++ b/docs/review/20261026-apply-window-late-events.md @@ -259,6 +259,11 @@ stream. ## The spec, separately -Neither fix makes the collision spec deterministic. It needs the edit to land after the apply was -asked for and before Flux writes the object. Flux wrote within about a second of the resume here, -and nothing in the spec holds the apply open longer than that. +Neither fix makes E5's way of provoking a collision deterministic: it edits once the apply was asked +for, and Flux often writes within a second of the resume, before the edit lands. The source-cluster +spec uses the delay this document is about instead. It edits first and pushes a Git-side change +right after. The edit's event waits out the attribution grace, so the push is heard, its apply asked +for and applied before the edit reaches the worker. The applied commit does not hold the edit, so +Flux reverts it, and with fix 2 the window still judges that write when it arrives, and withholds +it. That holds while the attribution grace (10s in e2e) outlasts the time from the push to the +apply request. diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 134247d6..cf456799 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -765,16 +765,38 @@ spec: r.consistentlyEverywhere(timingConfigMap, "seed") }) - // The collision the apply window exists for, as E5 provokes it: an edit through the workspace's - // API while an apply runs. The apply reverts it in the workspace; the window withholds that - // write, because the object's managedFields name kustomize-controller alone, after the edit. - It("withholds an apply that reverts an edit made in the workspace during it", func() { + // The collision the apply window exists for: an edit the apply reverts. E5 edits once the apply + // was asked for, and races Flux, which often writes first. Here every event from the workspace + // waits out the attribution grace before it reaches the worker (kcp sends no audit facts), and + // that makes the collision certain: an edit made just before a Git-side push reaches the worker + // only after the push was heard, its apply asked for, and Flux applied it. The commit Flux + // applied does not hold the edit, so Flux reverts it in the workspace. Its write reaches the + // worker after the edit, later still, and the window, which waits until the watch delivered every + // write made before the apply completed, withholds it: the object's managedFields name + // kustomize-controller alone, after the edit. Git keeps the edit, and a Repair puts it back. + // + // It holds while the attribution grace (10s in e2e) outlasts the time from the push to the apply + // request, which the receiver makes a second or two. + It("withholds an apply that reverts an edit made in the workspace", func() { withheld := fmt.Sprintf(`sum(gitopsreverser_apply_window_collisions_total{gitbranch_namespace=%q,`+ `gitbranch_name=%q,outcome="withheld"}) or vector(0)`, testNs, gitBranchName(p.providerName, "main")) before, err := queryPrometheus(withheld) Expect(err).NotTo(HaveOccurred()) - r.editDuringApply() + r.pushFromOutside("during", "d0") + r.eventuallyEverywhere("during", "d0", echoApplyBudget) + + By("editing an object in the workspace, and pushing a Git-side change to another before its event arrives") + token := r.applier.operation(Default).Token + r.editThroughAPI("during", "d-edit") + r.pushFromOutside("during-other", "o1") + Eventually(func(g Gomega) { + g.Expect(r.applier.operation(g).Token).NotTo(Equal(token), "the apply is asked for") + }, echoApplyBudget, 250*time.Millisecond).Should(Succeed()) + + r.eventuallyEverywhere("during", "d-edit", echoApplyBudget) + r.eventuallyEverywhere("during-other", "o1", echoApplyBudget) + r.consistentlyEverywhere("during", "d-edit") waitForMetricWithTimeout(withheld, func(v float64) bool { return v > before }, "the apply over the workspace edit is counted as a withheld collision", time.Minute) }) From 2e2b0b5d0f71a2b3cf4acc302306474a04dfa642 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:54:15 +0000 Subject: [PATCH 14/33] feat(saverequest): save the edits made on a source cluster with a SaveRequest created there A CommitRequest lives in the cluster GitOps Reverser runs in, so the people who edit a source cluster, such as a kcp workspace, need access to that cluster to save, and the save's author comes from its admission webhook rather than from the audit stream their edits are attributed from. SaveRequest (source.configbutler.ai/v1alpha1) is created on the source cluster itself. It names no GitTarget: it saves the commit window of every GitTarget that mirrors its namespace from that cluster, with its message and timers, and reports each commit and, with catchUp AfterPush, each apply in status.targets and the Ready and Applied conditions. - A KubeProvider serves its cluster's SaveRequests only with spec.saveRequests: Enabled; that is GitOps Reverser's one write on a source cluster (status, and delete for the finished-request TTL). - The request's author is found from the source cluster's audit facts, as its edits' authors are. With no audit it names nobody and attaches to a window that names nobody. On a provider that has received an audit fact, a request no fact names is refused with AuthorUnresolved. - SaveRequests are never mirrored: the default watch policy excludes them. Co-Authored-By: Claude Opus 5.5 (1M context) --- .golangci.yml | 4 +- Taskfile-build.yml | 3 + api/source/v1alpha1/groupversion_info.go | 25 + api/source/v1alpha1/saverequest_types.go | 225 ++++++ api/source/v1alpha1/zz_generated.deepcopy.go | 163 +++++ api/v1alpha3/kubeprovider_types.go | 24 + cmd/main.go | 9 + .../bases/configbutler.ai_kubeproviders.yaml | 16 + .../source.configbutler.ai_saverequests.yaml | 292 ++++++++ config/crd/kustomization.yaml | 1 + config/rbac/role.yaml | 16 + docs/configuration.md | 79 ++ docs/future/source-side-save-plan.md | 277 +++++++ docs/rbac.md | 10 + hack/crdfields/fields.yaml | 22 + internal/watch/resource_policy.go | 4 +- internal/watch/save_requests.go | 681 ++++++++++++++++++ internal/watch/save_requests_test.go | 313 ++++++++ test/e2e/source_cluster_e2e_test.go | 92 ++- 19 files changed, 2249 insertions(+), 7 deletions(-) create mode 100644 api/source/v1alpha1/groupversion_info.go create mode 100644 api/source/v1alpha1/saverequest_types.go create mode 100644 api/source/v1alpha1/zz_generated.deepcopy.go create mode 100644 config/crd/bases/source.configbutler.ai_saverequests.yaml create mode 100644 docs/future/source-side-save-plan.md create mode 100644 internal/watch/save_requests.go create mode 100644 internal/watch/save_requests_test.go diff --git a/.golangci.yml b/.golangci.yml index ca80b8a7..405888fd 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -328,7 +328,7 @@ linters: # Allow global variables in specific patterns (Kubernetes operator patterns) - text: "(scheme|setupLog|GroupVersion|SchemeBuilder|AddToScheme) is a global variable" linters: [gochecknoglobals] - - path: 'api/v1alpha3/.*\.go' + - path: 'api/(v1alpha3|source/v1alpha1)/.*\.go' linters: [gochecknoglobals, gochecknoinits] - path: 'cmd/main\.go' linters: [gochecknoglobals, cyclop, gochecknoinits, gocognit, funlen] @@ -361,7 +361,7 @@ linters: linters: [staticcheck] # Kubernetes controller-runtime still uses this pattern in API packages. - text: "SA1019:.*scheme\\.Builder is deprecated" - path: 'api/v1alpha3/groupversion_info\.go' + path: 'api/(v1alpha3|source/v1alpha1)/groupversion_info\.go' linters: [staticcheck] # mgr.GetEventRecorderFor returns the record.EventRecorder our shared status writer emits on # (a single Event per persisted Ready transition). It is the ubiquitous kubebuilder idiom; the diff --git a/Taskfile-build.yml b/Taskfile-build.yml index 676902cb..44353062 100644 --- a/Taskfile-build.yml +++ b/Taskfile-build.yml @@ -55,6 +55,7 @@ tasks: - exclude: cmd/**/zz_generated.deepcopy.go generates: - config/crd/bases/configbutler.ai*.yaml + - config/crd/bases/source.configbutler.ai*.yaml - config/rbac/role.yaml cmds: - | @@ -93,6 +94,7 @@ tasks: - exclude: cmd/**/zz_generated.deepcopy.go generates: - charts/gitops-reverser/crds/configbutler.*.yaml + - charts/gitops-reverser/crds/source.configbutler.*.yaml - charts/gitops-reverser/config/role.yaml cmds: - | @@ -110,6 +112,7 @@ tasks: - exclude: api/**/zz_generated.deepcopy.go generates: - api/v1alpha3/zz_generated.deepcopy.go + - api/source/v1alpha1/zz_generated.deepcopy.go cmds: - '{{.CONTROLLER_GEN}} object:headerFile="hack/boilerplate.go.txt" paths=./api/...' diff --git a/api/source/v1alpha1/groupversion_info.go b/api/source/v1alpha1/groupversion_info.go new file mode 100644 index 00000000..227a1dfc --- /dev/null +++ b/api/source/v1alpha1/groupversion_info.go @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package v1alpha1 contains API Schema definitions for the source.configbutler.ai v1alpha1 API group: +// the kinds GitOps Reverser serves on a source cluster, where the edits it mirrors are made. It is +// installed there on its own, apart from the configbutler.ai kinds of the cluster the operator runs +// in. +// +kubebuilder:object:generate=true +// +groupName=source.configbutler.ai +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects. + GroupVersion = schema.GroupVersion{Group: "source.configbutler.ai", Version: "v1alpha1"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme. + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/api/source/v1alpha1/saverequest_types.go b/api/source/v1alpha1/saverequest_types.go new file mode 100644 index 00000000..5eda3f94 --- /dev/null +++ b/api/source/v1alpha1/saverequest_types.go @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: Apache-2.0 + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// A SaveRequest is the source-cluster twin of a configbutler.ai CommitRequest, for the people who edit +// a source cluster and cannot reach the cluster GitOps Reverser runs in. It is its own kind, in its own +// group, rather than a CommitRequest with another schema: it names no GitTarget, because its creator +// cannot see them, its author comes from the source cluster's audit events rather than from admission, +// and it is installed alone. docs/future/source-side-save-plan.md has the reasoning. +// +// The spec is immutable, compared as CommitRequestSpec compares it: durations as durations, so a typed +// client's re-spelling of "1m" as "1m0s" is not a change. + +// SaveRequestSpec defines the desired state of SaveRequest. It is immutable. +// +kubebuilder:validation:XValidation:rule="has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="SaveRequest spec is immutable after creation" +type SaveRequestSpec struct { + // Message is the commit message this request asks for. Omitted, each + // GitTarget's spec.commit.message.liveTemplate words the commit. + // + // The text is never evaluated as a template. If a GitTarget sets + // spec.commit.message.requestTemplate, that template wraps this text; + // otherwise the text is committed as written. + // + // It is 1–1024 characters and must contain a non-whitespace character. + // Newlines are allowed, for a subject and body; tabs and other control + // characters are rejected. + // +optional + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=1024 + // +kubebuilder:validation:Pattern=`^[^\x00-\x09\x0B-\x1F\x7F]*$` + // +kubebuilder:validation:XValidation:rule="self.matches(r'[^\\s\\x{0085}\\x{00A0}\\x{1680}\\x{2000}-\\x{200A}\\x{2028}\\x{2029}\\x{202F}\\x{205F}\\x{3000}]')",message="message must contain a non-whitespace character" + Message string `json:"message,omitempty"` + + // The default is spelled out rather than {} because the API server checks a default against + // the block's CEL rules without applying the nested field defaults first. + + // Window says how long the request waits for a commit window with the + // edits it saves, and when that window closes once it has. Omitted, the + // request waits 5s and closes the window 2s after it attached. + // +optional + // +kubebuilder:default={attachTimeout: "5s", maxDuration: "2s"} + Window *SaveRequestWindow `json:"window,omitempty"` + + // CatchUp decides when the applier of each GitTarget is asked to apply + // the branch after this request pushes a commit. "AsDefinedOnClaim" + // leaves it to the applier claim's spec.catchUpAfter. "AfterPush" asks + // right after the push, and status.targets reports how it went. The + // cluster already holds the saved objects. Defaults to + // "AsDefinedOnClaim". + // +optional + // +kubebuilder:validation:Enum=AsDefinedOnClaim;AfterPush + // +kubebuilder:default=AsDefinedOnClaim + CatchUp CatchUpPolicy `json:"catchUp,omitempty"` +} + +// CatchUpPolicy is when a SaveRequest's commit is applied by a GitTarget's +// applier. +type CatchUpPolicy string + +const ( + // CatchUpAsDefinedOnClaim leaves the apply to the applier claim's spec.catchUpAfter. + CatchUpAsDefinedOnClaim CatchUpPolicy = "AsDefinedOnClaim" + // CatchUpAfterPush asks the applier to apply the tip right after the commit is pushed. + CatchUpAfterPush CatchUpPolicy = "AfterPush" +) + +// SaveRequestWindow is how long a SaveRequest waits for a commit window, and +// when that window closes. A window is only ever opened by an edit; a +// request waits for one and never opens one. +// +kubebuilder:validation:XValidation:rule="duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="SaveRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" +type SaveRequestWindow struct { + // AttachTimeout is how long the request waits for a commit window, as a + // Go duration, from when GitOps Reverser picks the request up, which is + // after its own author was found. A request whose wait ran out never + // takes a later window. At most "5m". Defaults to "5s". + // +optional + // +kubebuilder:validation:Type=string + // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" + // +kubebuilder:default="5s" + // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.attachTimeout must not exceed 5m" + AttachTimeout *metav1.Duration `json:"attachTimeout,omitempty"` + + // IdleTimeout closes the attached window this long after its last edit, + // as a Go duration. Omitted, only maxDuration closes it. At most "5m", + // and not more than maxDuration. + // +optional + // +kubebuilder:validation:Type=string + // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" + // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.idleTimeout must not exceed 5m" + IdleTimeout *metav1.Duration `json:"idleTimeout,omitempty"` + + // MaxDuration closes the attached window this long after the request + // attached, however many edits keep arriving, as a Go duration. At most + // "5m". Defaults to "2s". + // +optional + // +kubebuilder:validation:Type=string + // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" + // +kubebuilder:default="2s" + // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.maxDuration must not exceed 5m" + MaxDuration *metav1.Duration `json:"maxDuration,omitempty"` +} + +// SaveRequestStatus defines the observed state of SaveRequest. +type SaveRequestStatus struct { + // ObservedGeneration is the latest generation observed by GitOps Reverser. + // +optional + ObservedGeneration int64 `json:"observedGeneration,omitempty"` + + // Author is who created this request, as this cluster's audit events name + // them. Empty when they name nobody: then the request saves only edits + // that name nobody either. + // +optional + Author string `json:"author,omitempty"` + + // Targets is the save in each GitTarget that mirrors this namespace. + // +optional + // +listType=atomic + // +kubebuilder:validation:MaxItems=64 + Targets []SaveRequestTarget `json:"targets,omitempty"` + + // Conditions are the latest observations of this SaveRequest's state. + // Ready is True once the request ended without error in every GitTarget, + // with reason "Committed" when one of them committed, and otherwise the + // reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", + // "NoGitTarget" when no GitTarget mirrors this namespace, or + // "AuthorUnresolved" when this cluster sends audit events and none named + // the request's creator. Ready=False with Stalled=True is a failure. + // + // Applied, with spec.catchUp "AfterPush", is True once every GitTarget's + // applier applied a revision that holds its commit, or needed none, and + // False only when a branch lost the commit. It never changes Ready. + // +optional + // +listType=map + // +listMapKey=type + // +patchMergeKey=type + // +patchStrategy=merge + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` +} + +// SaveRequestTarget is the save in one GitTarget. +type SaveRequestTarget struct { + // GitTarget is the GitTarget, as namespace/name on the cluster GitOps + // Reverser runs in. + // +required + GitTarget string `json:"gitTarget"` + + // Reason is how the save ended in this GitTarget, as the Ready reasons + // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or + // "Failed". Empty while it runs. + // +optional + Reason string `json:"reason,omitempty"` + + // Message says more about the reason. + // +optional + Message string `json:"message,omitempty"` + + // Branch is the branch the save was committed on. + // +optional + Branch string `json:"branch,omitempty"` + + // Commit is the commit the save made, once it is on the remote. + // +optional + Commit string `json:"commit,omitempty"` + + // Applied is how the apply the save asked for went, with spec.catchUp + // "AfterPush": "Applying", "Retrying", "Applied", "NothingToApply", + // "NoApplierClaim", "ApplierGone", "CommitDropped" or "Lost". + // +optional + Applied string `json:"applied,omitempty"` + + // AppliedRevision is the revision the applier applied. + // +optional + AppliedRevision string `json:"appliedRevision,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` +// +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` +// +kubebuilder:printcolumn:name="Author",type=string,JSONPath=`.status.author` +// +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 +// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` + +// SaveRequest asks GitOps Reverser to save the edits made in its namespace: +// the commit window of each GitTarget that mirrors this namespace gets its +// message and closes on its timers. Create it on the cluster the edits were +// made on, as JSON. Its creator is found from this cluster's audit events, +// as the edits' authors are, so it saves only its creator's edits; on a +// cluster with no audit it names nobody, and saves edits that name nobody. +// Its spec is immutable, and status reports each GitTarget's commit. +type SaveRequest struct { + metav1.TypeMeta `json:",inline"` + + // metadata is a standard object metadata + // +optional + metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` + + // spec defines the desired state of SaveRequest. It is immutable after + // creation. + // +required + Spec SaveRequestSpec `json:"spec"` + + // status defines the observed state of SaveRequest. + // +optional + Status SaveRequestStatus `json:"status,omitempty,omitzero"` +} + +// +kubebuilder:object:root=true + +// SaveRequestList contains a list of SaveRequest. +type SaveRequestList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + + Items []SaveRequest `json:"items"` +} + +func init() { + SchemeBuilder.Register(&SaveRequest{}, &SaveRequestList{}) +} diff --git a/api/source/v1alpha1/zz_generated.deepcopy.go b/api/source/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 00000000..6ec0b27f --- /dev/null +++ b/api/source/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,163 @@ +//go:build !ignore_autogenerated + +// SPDX-License-Identifier: Apache-2.0 + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequest) DeepCopyInto(out *SaveRequest) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequest. +func (in *SaveRequest) DeepCopy() *SaveRequest { + if in == nil { + return nil + } + out := new(SaveRequest) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SaveRequest) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestList) DeepCopyInto(out *SaveRequestList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]SaveRequest, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestList. +func (in *SaveRequestList) DeepCopy() *SaveRequestList { + if in == nil { + return nil + } + out := new(SaveRequestList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SaveRequestList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestSpec) DeepCopyInto(out *SaveRequestSpec) { + *out = *in + if in.Window != nil { + in, out := &in.Window, &out.Window + *out = new(SaveRequestWindow) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestSpec. +func (in *SaveRequestSpec) DeepCopy() *SaveRequestSpec { + if in == nil { + return nil + } + out := new(SaveRequestSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestStatus) DeepCopyInto(out *SaveRequestStatus) { + *out = *in + if in.Targets != nil { + in, out := &in.Targets, &out.Targets + *out = make([]SaveRequestTarget, len(*in)) + copy(*out, *in) + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestStatus. +func (in *SaveRequestStatus) DeepCopy() *SaveRequestStatus { + if in == nil { + return nil + } + out := new(SaveRequestStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestTarget) DeepCopyInto(out *SaveRequestTarget) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestTarget. +func (in *SaveRequestTarget) DeepCopy() *SaveRequestTarget { + if in == nil { + return nil + } + out := new(SaveRequestTarget) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestWindow) DeepCopyInto(out *SaveRequestWindow) { + *out = *in + if in.AttachTimeout != nil { + in, out := &in.AttachTimeout, &out.AttachTimeout + *out = new(v1.Duration) + **out = **in + } + if in.IdleTimeout != nil { + in, out := &in.IdleTimeout, &out.IdleTimeout + *out = new(v1.Duration) + **out = **in + } + if in.MaxDuration != nil { + in, out := &in.MaxDuration, &out.MaxDuration + *out = new(v1.Duration) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestWindow. +func (in *SaveRequestWindow) DeepCopy() *SaveRequestWindow { + if in == nil { + return nil + } + out := new(SaveRequestWindow) + in.DeepCopyInto(out) + return out +} diff --git a/api/v1alpha3/kubeprovider_types.go b/api/v1alpha3/kubeprovider_types.go index fe31a03e..fb1d1ce2 100644 --- a/api/v1alpha3/kubeprovider_types.go +++ b/api/v1alpha3/kubeprovider_types.go @@ -99,8 +99,32 @@ type KubeProviderSpec struct { // audit events. // +optional Attribution *KubeProviderAttribution `json:"attribution,omitempty"` + + // SaveRequests says whether GitOps Reverser serves the SaveRequests + // (source.configbutler.ai) created on this cluster. "Enabled" watches + // them, saves the commit windows of the GitTargets that mirror each one's + // namespace through this KubeProvider, and writes each one's status back. + // Defaults to "Disabled". + // + // "Enabled" writes to this cluster: the credential needs get, list, watch + // and delete on saverequests, and update on saverequests/status. The + // SaveRequest CRD must be installed on the cluster. + // +optional + // +kubebuilder:validation:Enum=Disabled;Enabled + // +kubebuilder:default=Disabled + SaveRequests SaveRequestsMode `json:"saveRequests,omitempty"` } +// SaveRequestsMode is whether a KubeProvider's SaveRequests are served. +type SaveRequestsMode string + +const ( + // SaveRequestsDisabled leaves the cluster's SaveRequests alone. + SaveRequestsDisabled SaveRequestsMode = "Disabled" + // SaveRequestsEnabled serves the cluster's SaveRequests. + SaveRequestsEnabled SaveRequestsMode = "Enabled" +) + // KubeProviderInCluster selects the operator's own identity. It has no fields: // the arm being set is the whole statement. type KubeProviderInCluster struct{} diff --git a/cmd/main.go b/cmd/main.go index 2397dbe5..59407426 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -331,6 +331,15 @@ func main() { // Setup watch manager (must be after controllers are set up) fatalIfErr(watchMgr.SetupWithManager(mgr), "unable to setup watch ingestion manager") fatalIfErr(mgr.Add(watchMgr), "unable to add watch ingestion manager") + // SaveRequests are saves made on a source cluster, next to the edits they save: served on every + // KubeProvider whose spec.saveRequests is Enabled, attributed from that cluster's audit facts. + fatalIfErr(mgr.Add(&watch.SaveRequestServer{ + Manager: watchMgr, + Client: mgr.GetClient(), + Finalizer: eventRouter, + TTL: cfg.commitRequestTTL, + Log: ctrl.Log.WithName("saverequests"), + }), "unable to add the SaveRequest server") // The GitBranch, GitTarget and applier claim controllers find a GitProvider's GitBranches, in any // namespace, through this index. diff --git a/config/crd/bases/configbutler.ai_kubeproviders.yaml b/config/crd/bases/configbutler.ai_kubeproviders.yaml index d32aa10b..69044e36 100644 --- a/config/crd/bases/configbutler.ai_kubeproviders.yaml +++ b/config/crd/bases/configbutler.ai_kubeproviders.yaml @@ -509,6 +509,22 @@ spec: InClusterNotAuthorized, and is never used. Set exactly one of secretRef and inCluster. Immutable: delete and recreate the KubeProvider to change it. type: object + saveRequests: + default: Disabled + description: |- + SaveRequests says whether GitOps Reverser serves the SaveRequests + (source.configbutler.ai) created on this cluster. "Enabled" watches + them, saves the commit windows of the GitTargets that mirror each one's + namespace through this KubeProvider, and writes each one's status back. + Defaults to "Disabled". + + "Enabled" writes to this cluster: the credential needs get, list, watch + and delete on saverequests, and update on saverequests/status. The + SaveRequest CRD must be installed on the cluster. + enum: + - Disabled + - Enabled + type: string secretRef: description: |- SecretRef names the Secret, in this namespace, that holds a kubeconfig for diff --git a/config/crd/bases/source.configbutler.ai_saverequests.yaml b/config/crd/bases/source.configbutler.ai_saverequests.yaml new file mode 100644 index 00000000..04c4feef --- /dev/null +++ b/config/crd/bases/source.configbutler.ai_saverequests.yaml @@ -0,0 +1,292 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.23.0 + name: saverequests.source.configbutler.ai +spec: + group: source.configbutler.ai + names: + kind: SaveRequest + listKind: SaveRequestList + plural: saverequests + singular: saverequest + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.conditions[?(@.type=="Ready")].status + name: Ready + type: string + - jsonPath: .status.conditions[?(@.type=="Ready")].reason + name: Reason + type: string + - jsonPath: .status.author + name: Author + type: string + - jsonPath: .status.conditions[?(@.type=="Applied")].status + name: Applied + priority: 1 + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + SaveRequest asks GitOps Reverser to save the edits made in its namespace: + the commit window of each GitTarget that mirrors this namespace gets its + message and closes on its timers. Create it on the cluster the edits were + made on, as JSON. Its creator is found from this cluster's audit events, + as the edits' authors are, so it saves only its creator's edits; on a + cluster with no audit it names nobody, and saves edits that name nobody. + Its spec is immutable, and status reports each GitTarget's commit. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: |- + spec defines the desired state of SaveRequest. It is immutable after + creation. + properties: + catchUp: + default: AsDefinedOnClaim + description: |- + CatchUp decides when the applier of each GitTarget is asked to apply + the branch after this request pushes a commit. "AsDefinedOnClaim" + leaves it to the applier claim's spec.catchUpAfter. "AfterPush" asks + right after the push, and status.targets reports how it went. The + cluster already holds the saved objects. Defaults to + "AsDefinedOnClaim". + enum: + - AsDefinedOnClaim + - AfterPush + type: string + message: + description: |- + Message is the commit message this request asks for. Omitted, each + GitTarget's spec.commit.message.liveTemplate words the commit. + + The text is never evaluated as a template. If a GitTarget sets + spec.commit.message.requestTemplate, that template wraps this text; + otherwise the text is committed as written. + + It is 1–1024 characters and must contain a non-whitespace character. + Newlines are allowed, for a subject and body; tabs and other control + characters are rejected. + maxLength: 1024 + minLength: 1 + pattern: ^[^\x00-\x09\x0B-\x1F\x7F]*$ + type: string + x-kubernetes-validations: + - message: message must contain a non-whitespace character + rule: self.matches(r'[^\s\x{0085}\x{00A0}\x{1680}\x{2000}-\x{200A}\x{2028}\x{2029}\x{202F}\x{205F}\x{3000}]') + window: + default: + attachTimeout: 5s + maxDuration: 2s + description: |- + Window says how long the request waits for a commit window with the + edits it saves, and when that window closes once it has. Omitted, the + request waits 5s and closes the window 2s after it attached. + properties: + attachTimeout: + default: 5s + description: |- + AttachTimeout is how long the request waits for a commit window, as a + Go duration, from when GitOps Reverser picks the request up, which is + after its own author was found. A request whose wait ran out never + takes a later window. At most "5m". Defaults to "5s". + pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ + type: string + x-kubernetes-validations: + - message: spec.window.attachTimeout must not exceed 5m + rule: duration(self) <= duration('5m') + idleTimeout: + description: |- + IdleTimeout closes the attached window this long after its last edit, + as a Go duration. Omitted, only maxDuration closes it. At most "5m", + and not more than maxDuration. + pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ + type: string + x-kubernetes-validations: + - message: spec.window.idleTimeout must not exceed 5m + rule: duration(self) <= duration('5m') + maxDuration: + default: 2s + description: |- + MaxDuration closes the attached window this long after the request + attached, however many edits keep arriving, as a Go duration. At most + "5m". Defaults to "2s". + pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ + type: string + x-kubernetes-validations: + - message: spec.window.maxDuration must not exceed 5m + rule: duration(self) <= duration('5m') + type: object + x-kubernetes-validations: + - message: SaveRequest spec is immutable after creation + rule: duration(self.attachTimeout) == duration(oldSelf.attachTimeout) + && duration(self.maxDuration) == duration(oldSelf.maxDuration) + && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) + || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) + - message: spec.window.idleTimeout must not exceed maxDuration + rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' + type: object + x-kubernetes-validations: + - message: SaveRequest spec is immutable after creation + rule: has(self.message) == has(oldSelf.message) && (!has(self.message) + || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) + && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) + || self.catchUp == oldSelf.catchUp) + status: + description: status defines the observed state of SaveRequest. + properties: + author: + description: |- + Author is who created this request, as this cluster's audit events name + them. Empty when they name nobody: then the request saves only edits + that name nobody either. + type: string + conditions: + description: |- + Conditions are the latest observations of this SaveRequest's state. + Ready is True once the request ended without error in every GitTarget, + with reason "Committed" when one of them committed, and otherwise the + reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", + "NoGitTarget" when no GitTarget mirrors this namespace, or + "AuthorUnresolved" when this cluster sends audit events and none named + the request's creator. Ready=False with Stalled=True is a failure. + + Applied, with spec.catchUp "AfterPush", is True once every GitTarget's + applier applied a revision that holds its commit, or needed none, and + False only when a branch lost the commit. It never changes Ready. + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + observedGeneration: + description: ObservedGeneration is the latest generation observed + by GitOps Reverser. + format: int64 + type: integer + targets: + description: Targets is the save in each GitTarget that mirrors this + namespace. + items: + description: SaveRequestTarget is the save in one GitTarget. + properties: + applied: + description: |- + Applied is how the apply the save asked for went, with spec.catchUp + "AfterPush": "Applying", "Retrying", "Applied", "NothingToApply", + "NoApplierClaim", "ApplierGone", "CommitDropped" or "Lost". + type: string + appliedRevision: + description: AppliedRevision is the revision the applier applied. + type: string + branch: + description: Branch is the branch the save was committed on. + type: string + commit: + description: Commit is the commit the save made, once it is + on the remote. + type: string + gitTarget: + description: |- + GitTarget is the GitTarget, as namespace/name on the cluster GitOps + Reverser runs in. + type: string + message: + description: Message says more about the reason. + type: string + reason: + description: |- + Reason is how the save ended in this GitTarget, as the Ready reasons + above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or + "Failed". Empty while it runs. + type: string + required: + - gitTarget + type: object + maxItems: 64 + type: array + x-kubernetes-list-type: atomic + type: object + required: + - spec + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index ce7cb62c..a34418c5 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -11,3 +11,4 @@ resources: - bases/configbutler.ai_gittargets.yaml - bases/configbutler.ai_kubeproviders.yaml - bases/configbutler.ai_watchrules.yaml + - bases/source.configbutler.ai_saverequests.yaml diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 5baf7a21..54c540e5 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -115,3 +115,19 @@ rules: - get - list - watch +- apiGroups: + - source.configbutler.ai + resources: + - saverequests + verbs: + - delete + - get + - list + - watch +- apiGroups: + - source.configbutler.ai + resources: + - saverequests/status + verbs: + - get + - update diff --git a/docs/configuration.md b/docs/configuration.md index 404094dc..97ff1a07 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -105,6 +105,7 @@ Set exactly one of `secretRef` and `inCluster`. Both are immutable: delete and r | `client.qps` | `20` | Client rate limit for this source cluster. Refused beside `inCluster` | | `client.burst` | `30` | Client burst for this source cluster. Refused beside `inCluster` | | `attribution.auditRoute` | `.` | Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) | +| `saveRequests` | `Disabled` | `Enabled` serves the `SaveRequest`s created on this cluster, which writes to it. See [`SaveRequest`](#saverequest) | ### `GitTarget` (namespaced): which branch and folder @@ -186,6 +187,16 @@ The references are immutable: to hold another `Application`, delete the applier | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | | `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it | +### `SaveRequest` (namespaced, on a source cluster): save the edits made here + +| Field | Default | What it does | +|---|---|---| +| `message` | each target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | +| `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`SaveRequest`](#saverequest) | +| `window.idleTimeout` | no idle close | Close the attached window after this much silence | +| `window.maxDuration` | `2s` | Close the attached window this long after the attach | +| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit is pushed; `status.targets` reports it | + ## Sharing connections across namespaces @@ -3209,6 +3220,74 @@ The period comes from `--commit-request-ttl` (Helm: `controllerManager.commitReq requests are kept. A request that finished before this feature existed has no annotation and is never deleted. +## `SaveRequest` + +A `SaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next to the +edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It names no +`GitTarget`: it saves the commit window of every `GitTarget` that mirrors its own namespace from that +cluster, and each window gets its message and closes on its timers, as with a +[`CommitRequest`](#commitrequest). + +```yaml +apiVersion: source.configbutler.ai/v1alpha1 +kind: SaveRequest +metadata: + name: save-now + namespace: shop +spec: + message: "fix(shop): raise the cart limit" + catchUp: AfterPush +``` + +### Serving the `SaveRequest`s of a cluster + +Two things turn it on, both on purpose: + +1. **Install the CRD on the source cluster.** It is one small CRD, + `config/crd/bases/source.configbutler.ai_saverequests.yaml`, apart from the `configbutler.ai` kinds + of the cluster GitOps Reverser runs in. The Helm chart and the installer install it there too, for + the clusters the operator mirrors with its own identity. +2. **Set `spec.saveRequests: Enabled` on the `KubeProvider`.** GitOps Reverser then watches the + cluster's `SaveRequest`s through that provider, and writes each one's status back. It is the one + write GitOps Reverser makes on a source cluster: the provider's credential needs `get`, `list`, + `watch` and `delete` on `saverequests`, and `update` on `saverequests/status`, in the + `source.configbutler.ai` group. A `SaveRequest` is never mirrored, whatever a `WatchRule` selects. + +### Who the save is from + +A `SaveRequest`'s creator is found from the source cluster's audit events, as the edits' authors are, +so a save attaches only to its creator's windows, by the ordinary author rule. Create it as JSON: +`kubectl` and browsers send JSON, and an API server that drops the audit event of a YAML body loses +the save's author without an error. + +| The source cluster | The save | +| --- | --- | +| sends audit events, and one names the creator | attaches to the creator's windows; `status.author` names them | +| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window whose edits name nobody; the commit's author stays unresolved | +| sends audit events, but none names the creator | is refused, `Ready=True` with reason `AuthorUnresolved`: its event went missing, and attaching it would claim the window that should stay unattributed. Save again | + +`spec.window.attachTimeout` (default `5s`) counts from when GitOps Reverser picks the request up, +which is after the request's own audit event arrived, so it only covers the spread between the +request's event and its edits'. `maxDuration` (default `2s`) and `idleTimeout` close the attached +window as for a `CommitRequest`. + +### Its status + +`status.targets` lists each `GitTarget` that mirrors the namespace, as `namespace/name` on the cluster +GitOps Reverser runs in, with the save's `reason` there (`Committed`, `NoWindow`, `AlreadyPresent`, +`WindowMismatch` or `Failed`), its `branch` and `commit`, and, with `catchUp: AfterPush`, its +`applied` reason and `appliedRevision`. + +- **Ready** is `True` once the save ended without error in every `GitTarget`, with reason `Committed` + when one of them committed, and otherwise the first one's reason, `NoGitTarget` when no `GitTarget` + mirrors the namespace, or `AuthorUnresolved`. `Ready=False` with `Stalled=True` is a failure. +- **Applied**, with `catchUp: AfterPush`, is `True` once every applier applied a revision that holds + the commit, or needed none, `False` only when a branch lost the commit, and `Unknown` with reason + `Lost` when GitOps Reverser restarted before it learned how an apply went. + +A finished `SaveRequest` is deleted `--commit-request-ttl` (default `48h`) after it finished; `0` +keeps it. + ## Audit ingestion settings Object state comes from Kubernetes **watch**, not from audit. Audit is an optional attribution lookup: diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md new file mode 100644 index 00000000..dd4dd25f --- /dev/null +++ b/docs/future/source-side-save-plan.md @@ -0,0 +1,277 @@ +# Plan: saving from the source cluster + +A `CommitRequest` saves the writes in its author's open commit window. It lives in the control +cluster, where the operator runs, while the edits it saves happen on the source cluster. This plan +adds a second way to save: a small request created **on the source cluster**, next to the edits, +attributed from that cluster's audit stream like the edits are. + +It also plans the second step: letting a save take over a window whose writes name no author, as an +opt-in, for a source cluster that cannot send audit events at all. There it is the only way a save can +put its author's name on the commit. + +Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The source-side save is `SaveRequest` +(`source.configbutler.ai/v1alpha1`, `internal/watch/save_requests.go`), served on a KubeProvider with +`spec.saveRequests: Enabled`. The takeover is opt-in, as described below: the GitTarget's +`spec.commit.unattributedWindows` and the CommitRequest's `spec.window.author`, each `SavedBy` or +`TakeOver`. The open questions are answered as proposed: `SaveRequest` in its own group, a default +`attachTimeout` of `5s`, the reason `AuthorUnresolved`, and no extra queue for a second save on a +GitTarget beyond the worker's own (a window carries one request at a time). + +This revision takes in gitops-api's review of the first draft. + +## Decisions + +1. **Build the source-side save first.** It gives the right author wherever the source cluster has + audit, and an honestly unresolved one where it does not. +2. **It is a new kind, defined by GitOps Reverser.** A `CommitRequest` with another schema, or a + kind gitops-api defines and translates, were both considered. See + [Why a new kind](#why-a-new-kind). +3. **The takeover is the second step.** Some people cannot configure their API server's audit, so + for them it is the only way to get their name on a commit. It is opt-in on the GitTarget, which + chooses how much a save may claim; a request that asks without permission fails loudly. See + [The takeover](#the-takeover). +4. **`cf3b97be` does not merge as it is.** Its automatic takeover becomes the opt-in, or comes out. + +## Why: two identity domains + +Today the save and the edits are attributed in two places: the `CommitRequest`'s author comes from +the control cluster's admission webhook, the edits' authors from the source cluster's audit events. +They match only when one person has the same username in both. + +gitops-api makes them match. Its server impersonates the human when it creates the `CommitRequest`, +so the admission webhook records the same Dex email that kcp's audit records for the edits, and the +ordinary author rule attaches. gitops-api does not need a takeover. What it pays for that is +elsewhere: + +- **A grant into the control cluster.** Every tenant's identity may `create commitrequests` in its + `tenant-` namespace there. It is the one place a tenant reaches outside its kcp + workspace. +- **A server on the save path.** The browser cannot reach that namespace, so every save goes through + gitops-api's `/editor/commit`, which impersonates and creates the request. + +A save created in the workspace removes both. "A customer's only API surface is their kcp workspace" +then holds with no exceptions, and the browser creates the save itself. That is the production +benefit this plan is for. It also helps any setup where people can reach only the source cluster. + +```mermaid +flowchart LR + subgraph Source["Source cluster (kcp workspace)"] + Edit["Alice edits cm/a"] + Save["Alice creates a save"] + end + subgraph Control["Control cluster"] + Op["Operator"] + end + Edit -->|"watched, attributed
from audit"| Op + Save -->|"watched, attributed
from the same audit"| Op + Op -->|"status written back
(a new write on the source cluster)"| Save +``` + +## The source-side save + +### What it does + +The operator watches the new kind on each source cluster through its KubeProvider. It attributes +each request exactly as it attributes the edits, from the source cluster's audit facts. It then +attaches the request to the matching commit window, and writes the request's status back where its +author can read it. + +- **On a source with audit**, Alice's save and Alice's edits resolve to the same user, so the save + attaches under the ordinary author rule. No takeover and no trust are involved, and Git names the + person who made the edits. +- **On a source with no audit**, the save names nobody, like the edits. A request that names no actor + may attach to a window that names none, which is today's rule + (`TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow`). The save gets its message + and its timers, and the author in Git stays unresolved. The window still holds everyone's + unattributed writes, but nobody is named as their author. + +### Why a new kind + +gitops-api could define its own kind and translate it into a `CommitRequest`, but it cannot learn who +created the object: watching the workspace shows the object and nothing about its creator. Only the audit stream +says that, and GitOps Reverser is what reads it. A translation would also spend part of the attach +wait on an extra hop, and add a second status to keep in step. + +Reusing `CommitRequest` with a different schema would be worse than a separate kind: + +- **Another shape.** It names no GitTarget, and its status lists one or more of them. +- **Another trust model.** Its author comes from audit; a `CommitRequest`'s comes from admission. +- **One name for two things** invites the confusion that `tenant-` already costs gitops-api. +- **Installed alone.** A workspace gets one small CRD instead of the whole `configbutler.ai` set, so + nobody is offered GitTargets that nothing reads. gitops-api installs CRDs into workspaces through its + `workspaceCRDs` list, so no `APIExport` is needed there. +- **Easy to never mirror.** In its own API group, or at least as its own kind, keeping it out of Git + whatever a `WatchRule` selects is one check, like `kube-root-ca.crt`. + +### Which GitTarget it saves + +The people creating it cannot see GitTargets, so it names none. It saves for the GitTargets that +mirror its own namespace from that KubeProvider. Usually that is one: in gitops-api, each +`repo-` namespace maps to exactly one GitTarget. When several mirror the namespace, into +several folders, it saves each one's window, and its status lists each GitTarget with its outcome. + +### When its wait starts + +On a source with audit, a request is attributed only once its audit fact has joined its watch event, +and the edits it saves wait in the same audit pipeline (kcp batches audit with a `batchMaxWait` of +1s). The request's wait for a window starts when the branch worker registers it, which is after the +request itself was attributed. That is how a `CommitRequest`'s `attachTimeout` is anchored today +(`handleAttachCommitRequest` in `internal/git/commit_request_attach_loop.go`). The timeout then covers +only the spread between the request's stream and the edits' streams. Its +default can still be set for a source with audit, rather than copied from `CommitRequest`'s `2s`. + +### An unresolved save on a source that has audit + +On a provider whose `AuditFactsReceived` latch is `True`, a save that resolves no author means its +audit fact went missing. Attaching it to a window that names no author would claim exactly the +window that should stay unattributed. So the latch applies here too: + +```mermaid +flowchart TB + S["A source-side save is attributed"] --> N{"Did it resolve
an author?"} + N -->|yes| Own["Attach to that author's window"] + N -->|no| L{"Has its KubeProvider ever
received an audit fact?"} + L -->|no| None["Attach to a window that names no author"] + L -->|yes| Refuse["Refused: its fact went missing.
Ready=True with reason AuthorUnresolved"] +``` + +Refusing rather than waiting keeps the outcome visible, and the person saves again. + +### What it writes on the source cluster + +This is GitOps Reverser's first write on a source cluster. Today a source credential only lists and +watches the mirrored types ([rbac.md](../rbac.md)). The new access is scoped to the new kind: list, +watch, status update, and delete (for the finished-request TTL). It is off unless the KubeProvider +turns it on, and the KubeProvider's docs say what it adds. + +### Notes for clients + +- **Create it as JSON.** gitops-api found that kcp drops audit events whose request body is YAML. A + save sent as YAML would lose its author without any error. A browser sends JSON anyway; the docs + must say it for everyone else. +- **One save at a time per GitTarget.** gitops-api queues saves in its server (`commitGate`). If the + browser creates them directly, that queue is gone. The worker already lets a window carry at most + one request, and a second waits for the next window (`attachWaitingCommitRequests`). Whether that + is enough, or the operator should hold a save until the one before it has resolved, is open. + +## The takeover + +### Who needs it + +A source-side save names its author only when the source cluster sends audit events. Many people +cannot make it: a managed Kubernetes service usually does not let the customer point the API +server's audit at a webhook. On such a cluster no write and no save can name who made it, and Git +records every save as attribution-unresolved. A takeover is the only way to put a name on those +commits, and that name is a claim on trust by definition: the cluster cannot confirm it. + +### The risk + +On a cluster with no audit, every write names no author, so all of them share one window on the +GitTarget, whoever made them. A save that takes the window over names its submitter as the author of +everyone's writes. + +```mermaid +flowchart LR + subgraph Source["Source cluster with no audit stream"] + A["Alice edits cm/a"] + B["Bob edits cm/b"] + end + A --> Win["One window on the GitTarget
author: none"] + B --> Win + CR["Alice's save"] -.->|"a takeover would name Alice
as the author of Bob's edit"| Win +``` + +That is harmless when one person edits the GitTarget's namespaces at a time, and wrong as soon as two +do. Only the GitTarget's owner knows which it is, so the GitTarget decides. + +### What the GitTarget allows + +One field on the GitTarget, an enum, because there are two levels of trust worth offering. Proposed: +`spec.commit.unattributedWindows`. + +- **`"Keep"`** (the default): a save attaches only to its author's window, as today. On a cluster + with no audit, a save from the control cluster finds no window of its own. +- **`"SavedBy"`**: a save may attach its message to a window whose writes name no author, without + claiming them. The author stays unresolved, and the commit records who saved it: + + ```text + Saved-By: alice@example.com + ``` + + The message may still describe Bob's edit, but nobody is named as the author of something they + did not write. A misleading message is a much smaller harm than a false author. +- **`"TakeOver"`**: a save may take such a window over. Its submitter authors the commit, and the + message ends with a trailer that says the name was taken on trust: + + ```text + Author-Source: CommitRequest team-a/save-1; no audit record names who made these changes + ``` + +A request's `spec.gitTargetRef` is a local reference, so the request is always in its GitTarget's +namespace, and a plain field is enough. A grant on the KubeProvider was considered and set aside: +the provider's owner usually does not know who edits which namespace, and the grant convention's +default ("omitted means this namespace") is the wrong one here. + +### Three conditions, all required + +```mermaid +flowchart TB + Start["A save finds an open window on its GitTarget
whose writes name no author"] --> Q1{"1. Did the save ask
for more than its own window?"} + Q1 -->|no| Own["Its own windows only"] + Q1 -->|yes| Q2{"2. Does the GitTarget's
unattributedWindows allow what it asked?"} + Q2 -->|no| Refuse["Refused: Ready=False, Stalled=True
reason TakeOverNotAllowed"] + Q2 -->|yes| Q3{"3. Has the KubeProvider ever
received an audit fact?"} + Q3 -->|yes| Own + Q3 -->|no| Take["SavedBy: attach the message, author unresolved
TakeOver: the submitter authors the commit"] +``` + +1. **The save asks for it.** An enum under `spec.window` beside `attach`, matching the GitTarget's: + `spec.window.author`, `"Own"` (the default), `"SavedBy"` or `"TakeOver"`. A save asks for what it + wants each time, and the GitTarget bounds it: `"TakeOver"` allows a save that asks for + `"SavedBy"`, and not the other way round. +2. **The GitTarget allows it.** A save that asks for more than the GitTarget allows fails at once, + when it is first reconciled: `Ready=False`, `Stalled=True`, reason `TakeOverNotAllowed`. The spec + is immutable, so it cannot recover by itself. +3. **The KubeProvider has never received an audit fact** (`AuditFactsReceived` is not `True`). On a + cluster that has sent audit, a write that names no author means its fact went missing, so no + save claims it, whatever 1 and 2 say. The save then attaches to its own windows only. + +### How it relates to the source-side save + +The two cover different clusters: + +| Source cluster | Source-side save | Control-cluster save with `"TakeOver"` | +| --- | --- | --- | +| Sends audit | The saver's name, from audit | Not needed; the latch blocks it | +| Cannot send audit | Message only, author unresolved | The saver's name, on trust | + +A source-side save cannot offer `"TakeOver"`: on a cluster with no audit, nothing records who created +it either, so it has no name to give. The name has to come from the control cluster's admission +webhook, which is why a takeover is always made by a control-cluster `CommitRequest`. + +## What happens to `cf3b97be` + +It takes over automatically. Most of it is the worker side of `"TakeOver"`: matching a window whose +writes name no author, authoring the commit as the submitter, and the `Author-Source` trailer. Two +ways forward before the branch merges: + +- **Turn it into the opt-in here.** Add `spec.commit.unattributedWindows` on the GitTarget and + `spec.window.author` on the `CommitRequest`, the controller's check and refusal, `"SavedBy"`, and + tests for each row above. The source-cluster e2e's save spec then sets both. +- **Take it out here**, and build the takeover in its own PR after the source-side save. + +## Open questions + +1. The source-side kind's name and API group. Proposed: `SaveRequest` in + `source.configbutler.ai/v1alpha1`. "Save" is the word its user means and the one the project + already uses for what a request asks; `SimpleCommitRequest` was considered and set aside, since + it is a different kind with another trust model, not a simpler `CommitRequest`. Its spec carries + no Git terms (a message, a wait, a catch-up), and its status still reports each GitTarget's + commit, so a save stays traceable. Its own group makes "never mirror it" and "install it alone" + one check each. +2. The default of its attach wait on a source with audit. +3. The reason name for an unresolved save on a source that has audit (`AuthorUnresolved` above). +4. Whether the operator should hold a second save on a GitTarget until the first has resolved. +5. Names for the takeover: `spec.commit.unattributedWindows` on the GitTarget and + `spec.window.author` on the `CommitRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. +6. `cf3b97be`: turn it into the opt-in on this branch, or take it out and build the takeover later? diff --git a/docs/rbac.md b/docs/rbac.md index 76e37092..2d684d30 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -86,6 +86,16 @@ evaluated against `Namespace`s in the *source* cluster, so a remote provider's i grant it exactly the namespaces and types the rules name — a `sourceNamespace: "*"` item asks for a cluster-wide `list`/`watch` of its types, and gets a clean 403 if the credential cannot serve it. +### `saverequests` on a source cluster + +A source credential only reads, with one opt-in exception. A `KubeProvider` with +`spec.saveRequests: Enabled` serves the [`SaveRequest`s](configuration.md#saverequest) created on its +cluster, and writes their status back there. Its credential then needs, in the +`source.configbutler.ai` group, `get`, `list`, `watch` and `delete` on `saverequests` (`delete` for +the finished-request TTL), and `update` on `saverequests/status`. Nothing else on a source cluster is +written. The manager role carries the same rules for the cluster the operator runs in, for an +`inCluster` provider. + ## The operator does not read Secrets wholesale It holds **no Secret informer**: Secrets are excluded from the manager's cache diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index a3c0d9c9..7a492f82 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -128,6 +128,10 @@ kinds: doc: >- Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) + - path: saveRequests + doc: >- + `Enabled` serves the `SaveRequest`s created on this cluster, which writes to it. See + [`SaveRequest`](#saverequest) - kind: GitTarget title: "`GitTarget` (namespaced): which branch and folder" @@ -322,3 +326,21 @@ kinds: `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it + + - kind: SaveRequest + title: "`SaveRequest` (namespaced, on a source cluster): save the edits made here" + fields: + - path: message + default: each target's templates + doc: Commit message, committed verbatim unless `requestTemplate` frames it + - path: window.attachTimeout + doc: How long to wait for a window with the edits it saves. See [`SaveRequest`](#saverequest) + - path: window.idleTimeout + default: no idle close + doc: Close the attached window after this much silence + - path: window.maxDuration + doc: Close the attached window this long after the attach + - path: catchUp + doc: >- + `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit + is pushed; `status.targets` reports it diff --git a/internal/watch/resource_policy.go b/internal/watch/resource_policy.go index 837a2282..c40b1f62 100644 --- a/internal/watch/resource_policy.go +++ b/internal/watch/resource_policy.go @@ -23,7 +23,9 @@ func isDefaultResourceExcluded(group, resource string) bool { groupResourceKey("flowcontrol.apiserver.k8s.io", "flowschemas"), groupResourceKey("flowcontrol.apiserver.k8s.io", "prioritylevelconfigurations"), groupResourceKey("batch", "jobs"), - groupResourceKey("batch", "cronjobs"): + groupResourceKey("batch", "cronjobs"), + // A save is a request to GitOps Reverser, never state to mirror (save_requests.go). + groupResourceKey("source.configbutler.ai", "saverequests"): return true default: return false diff --git a/internal/watch/save_requests.go b/internal/watch/save_requests.go new file mode 100644 index 00000000..da78bba7 --- /dev/null +++ b/internal/watch/save_requests.go @@ -0,0 +1,681 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "context" + "errors" + "fmt" + "sort" + "sync" + "time" + + "github.com/go-logr/logr" + apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + k8stypes "k8s.io/apimachinery/pkg/types" + k8swatch "k8s.io/apimachinery/pkg/watch" + "k8s.io/client-go/dynamic" + "sigs.k8s.io/controller-runtime/pkg/client" + + sourcev1alpha1 "github.com/ConfigButler/gitops-reverser/api/source/v1alpha1" + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/queue" + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// A SaveRequest is a save made on a source cluster, next to the edits it saves +// (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose +// spec.saveRequests is "Enabled", through that provider's identity, and serves each one the way the +// CommitRequest controller serves a CommitRequest: it attaches the request to the commit window of +// every GitTarget that mirrors the request's namespace from that cluster, polls the branch workers +// for the outcomes, and writes them back to the request's status on the source cluster. +// +// The request's author is found as the edits' authors are, from the source cluster's audit facts, so +// a save attaches only to its creator's windows by the ordinary author rule. On a cluster that sends +// no audit events it names nobody, and attaches to a window whose edits name nobody. On a cluster +// whose KubeProvider has received an audit fact, a request that names nobody means its fact went +// missing; it is refused rather than attached, so it cannot claim the window that should stay +// unattributed. + +// +kubebuilder:rbac:groups=source.configbutler.ai,resources=saverequests,verbs=get;list;watch;delete +// +kubebuilder:rbac:groups=source.configbutler.ai,resources=saverequests/status,verbs=get;update + +// saveRequestGVR() is the served resource of a SaveRequest. +func saveRequestGVR() schema.GroupVersionResource { + return sourcev1alpha1.GroupVersion.WithResource("saverequests") +} + +const ( + // saveRequestProviderResync is how often the server looks for KubeProviders to serve. + saveRequestProviderResync = 10 * time.Second + // saveRequestReconnect is the wait before a watch of a cluster's SaveRequests opens again. + saveRequestReconnect = 10 * time.Second + // saveRequestRelist ends a watch this often, so finished requests past their TTL are deleted. + saveRequestRelist = 5 * time.Minute + // saveRequestPoll is how often a request's outcome is asked for. + saveRequestPoll = time.Second + // saveRequestPushAllowance is what a request may take beyond its own waits: the push, its + // cooldown and a retry. Past it, a request no worker holds is withdrawn. + saveRequestPushAllowance = 2 * time.Minute + // saveRequestApplyLost is how long a save's apply may stay unknown to its worker before it is + // reported lost: GitOps Reverser restarted, or the record aged out. + saveRequestApplyLost = time.Minute + // defaultSaveAttachTimeout and defaultSaveMaxDuration are the window when a request sets none. + defaultSaveAttachTimeout = 5 * time.Second + defaultSaveMaxDuration = 2 * time.Second +) + +// SaveRequest reasons, on the Ready condition and each target. +const ( + saveReasonCommitted = "Committed" + saveReasonNoWindow = "NoWindow" + saveReasonWindowMismatch = "WindowMismatch" + saveReasonAlreadyPresent = "AlreadyPresent" + saveReasonFailed = "Failed" + saveReasonNoGitTarget = "NoGitTarget" + saveReasonAuthorUnresolved = "AuthorUnresolved" + saveReasonSaving = "Saving" + saveReasonApplying = "Applying" + saveReasonApplied = "Applied" +) + +// SaveFinalizer is what serves a save once it is attached: the CommitRequest service of the event +// router, which owns the branch workers. +type SaveFinalizer interface { + ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) +} + +// SaveRequestServer serves the SaveRequests of every KubeProvider that enables them. It runs on the +// leader only, as every status writer does. +type SaveRequestServer struct { + // Manager is the watch manager: the source clusters' clients, their attribution, and which + // GitTargets mirror which namespace. + Manager *Manager + // Client reads KubeProviders and GitTargets on the cluster the operator runs in. + Client client.Client + // Finalizer serves an attached save. + Finalizer SaveFinalizer + // TTL is how long a finished request stays before it is deleted; zero keeps it. + TTL time.Duration + Log logr.Logger + + // dynamicClient overrides the source cluster's client, in tests. + dynamicClient func(ctx context.Context, provider k8stypes.NamespacedName) (dynamic.Interface, error) + + mu sync.Mutex + watchers map[k8stypes.NamespacedName]*saveRequestWatcher +} + +// saveRequestWatcher is the watch of one KubeProvider's SaveRequests. +type saveRequestWatcher struct { + provider k8stypes.NamespacedName + route string + cancel context.CancelFunc + // serving holds the UIDs of the requests being served now. + serving sync.Map +} + +// Start runs the server until ctx ends. +func (s *SaveRequestServer) Start(ctx context.Context) error { + ticker := time.NewTicker(saveRequestProviderResync) + defer ticker.Stop() + for { + s.syncProviders(ctx) + select { + case <-ctx.Done(): + s.stopAll() + return nil + case <-ticker.C: + } + } +} + +// syncProviders runs a watcher for every KubeProvider that enables SaveRequests, and stops the rest. +func (s *SaveRequestServer) syncProviders(ctx context.Context) { + var providers configv1alpha3.KubeProviderList + if err := s.Client.List(ctx, &providers); err != nil { + s.Log.V(1).Info("Cannot list KubeProviders for SaveRequests; will retry", "err", err.Error()) + return + } + want := map[k8stypes.NamespacedName]string{} + for i := range providers.Items { + provider := &providers.Items[i] + if provider.DeletionTimestamp == nil && provider.Spec.SaveRequests == configv1alpha3.SaveRequestsEnabled { + want[provider.Key()] = provider.AuditRoute() + } + } + s.mu.Lock() + defer s.mu.Unlock() + if s.watchers == nil { + s.watchers = map[k8stypes.NamespacedName]*saveRequestWatcher{} + } + for key, watcher := range s.watchers { + if route, ok := want[key]; !ok || route != watcher.route { + watcher.cancel() + delete(s.watchers, key) + } + } + for key, route := range want { + if _, running := s.watchers[key]; running { + continue + } + watchCtx, cancel := context.WithCancel(ctx) + watcher := &saveRequestWatcher{provider: key, route: route, cancel: cancel} + s.watchers[key] = watcher + go s.run(watchCtx, watcher) + } +} + +func (s *SaveRequestServer) stopAll() { + s.mu.Lock() + defer s.mu.Unlock() + for key, watcher := range s.watchers { + watcher.cancel() + delete(s.watchers, key) + } +} + +// run watches one KubeProvider's SaveRequests until its context ends, reconnecting after a failure. +func (s *SaveRequestServer) run(ctx context.Context, watcher *saveRequestWatcher) { + log := s.Log.WithValues("kubeProvider", watcher.provider.String()) + log.Info("Serving the SaveRequests of a KubeProvider") + if s.Manager != nil && s.Manager.FactStreams != nil { + release := s.Manager.FactStreams.Acquire( + queue.FactStreamKeyFor(watcher.route, saveRequestGVR().GroupResource()), + ) + defer release() + } + for ctx.Err() == nil { + if err := s.session(ctx, log, watcher); err != nil && ctx.Err() == nil { + log.Info("The watch of SaveRequests ended; reconnecting", "err", err.Error()) + } + select { + case <-ctx.Done(): + case <-time.After(saveRequestReconnect): + } + } +} + +// session lists the cluster's SaveRequests and watches them, until the watch ends. +func (s *SaveRequestServer) session(ctx context.Context, log logr.Logger, watcher *saveRequestWatcher) error { + dc, err := s.clusterClient(ctx, watcher.provider) + if err != nil { + return err + } + resource := dc.Resource(saveRequestGVR()) + list, err := resource.List(ctx, metav1.ListOptions{}) + if err != nil { + return fmt.Errorf("list SaveRequests: %w", err) + } + for i := range list.Items { + s.consider(ctx, log, watcher, resource, &list.Items[i]) + } + timeout := int64(saveRequestRelist / time.Second) + w, err := resource.Watch(ctx, metav1.ListOptions{ + ResourceVersion: list.GetResourceVersion(), AllowWatchBookmarks: true, TimeoutSeconds: &timeout, + }) + if err != nil { + return fmt.Errorf("watch SaveRequests: %w", err) + } + defer w.Stop() + for { + select { + case <-ctx.Done(): + return nil + case ev, ok := <-w.ResultChan(): + if !ok { + return nil + } + switch ev.Type { + case k8swatch.Added, k8swatch.Modified: + if u, isObject := ev.Object.(*unstructured.Unstructured); isObject { + s.consider(ctx, log, watcher, resource, u) + } + case k8swatch.Error: + return fmt.Errorf("watch SaveRequests: %w", apierrors.FromObject(ev.Object)) + case k8swatch.Deleted, k8swatch.Bookmark: + } + } + } +} + +func (s *SaveRequestServer) clusterClient( + ctx context.Context, + provider k8stypes.NamespacedName, +) (dynamic.Interface, error) { + if s.dynamicClient != nil { + return s.dynamicClient(ctx, provider) + } + return s.Manager.ApplierDynamicClient(ctx, SourceClusterID(provider)) +} + +// consider serves a request that has not finished and is not served yet, and deletes one that +// finished longer than the TTL ago. +func (s *SaveRequestServer) consider( + ctx context.Context, log logr.Logger, watcher *saveRequestWatcher, resource dynamic.NamespaceableResourceInterface, + u *unstructured.Unstructured, +) { + var request sourcev1alpha1.SaveRequest + if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, &request); err != nil { + log.Info("Cannot read a SaveRequest", "saveRequest", u.GetNamespace()+"/"+u.GetName(), "err", err.Error()) + return + } + if finishedAt, finished := saveRequestFinished(&request); finished { + if s.TTL > 0 && time.Since(finishedAt) > s.TTL && request.DeletionTimestamp == nil { + if err := resource.Namespace(request.Namespace).Delete(ctx, request.Name, metav1.DeleteOptions{ + Preconditions: &metav1.Preconditions{UID: &request.UID}, + }); err != nil && !apierrors.IsNotFound(err) { + log.V(1).Info("Cannot delete a finished SaveRequest", "saveRequest", request.Namespace+"/"+request.Name, + "err", err.Error()) + } + } + return + } + if request.DeletionTimestamp != nil { + return + } + if _, serving := watcher.serving.LoadOrStore(request.UID, struct{}{}); serving { + return + } + go func() { + defer watcher.serving.Delete(request.UID) + s.serve(ctx, log, watcher, resource.Namespace(request.Namespace), &request) + }() +} + +// saveRequestFinished reports whether a request has an outcome that changes no more, and since when: +// Ready decided, and, for one that asked to catch up and committed, Applied decided too. +func saveRequestFinished(request *sourcev1alpha1.SaveRequest) (time.Time, bool) { + ready := apimeta.FindStatusCondition(request.Status.Conditions, "Ready") + if ready == nil || ready.Status == metav1.ConditionUnknown { + return time.Time{}, false + } + finishedAt := ready.LastTransitionTime.Time + if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && ready.Reason == saveReasonCommitted { + applied := apimeta.FindStatusCondition(request.Status.Conditions, "Applied") + if applied == nil || applied.Status == metav1.ConditionUnknown { + return time.Time{}, false + } + finishedAt = applied.LastTransitionTime.Time + } + return finishedAt, true +} + +// serve takes one request from attribution to its outcome, and to its apply when it asked to catch +// up, writing each step to its status. +func (s *SaveRequestServer) serve( + ctx context.Context, log logr.Logger, watcher *saveRequestWatcher, resource dynamic.ResourceInterface, + request *sourcev1alpha1.SaveRequest, +) { + log = log.WithValues("saveRequest", request.Namespace+"/"+request.Name) + status := request.Status.DeepCopy() + write := func() { + if err := writeSaveRequestStatus(ctx, resource, request, status); err != nil { + log.Info("Cannot write the SaveRequest's status", "err", err.Error()) + } + } + if err := git.ValidateLiteralCommitMessage(request.Spec.Message); err != nil { + setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonFailed, err.Error()) + setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonFailed, err.Error()) + write() + return + } + + author, attribution := s.resolveSaveAuthor(ctx, watcher.route, request) + status.Author = author.Username + if !attribution.NamesActor() && attribution != git.AttributionNotAttempted && + s.sentAuditFacts(ctx, watcher.provider) { + setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonAuthorUnresolved, + "no audit event of this cluster names who created this request, though the cluster sends audit "+ + "events: its event went missing, so it saves nothing. Save again") + write() + log.Info("Refused a SaveRequest whose author no audit event names, on a cluster that sends them") + return + } + + targets, err := s.mirroringTargets(ctx, watcher.provider, request.Namespace) + if err != nil { + log.Info("Cannot find the GitTargets that mirror a SaveRequest's namespace; will retry", "err", err.Error()) + return + } + if len(targets) == 0 { + setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonNoGitTarget, + "no GitTarget mirrors this namespace from this cluster, so there is nothing to save") + write() + return + } + status.Targets = make([]sourcev1alpha1.SaveRequestTarget, len(targets)) + attaches := make([]git.AttachCommitRequest, len(targets)) + for i, target := range targets { + status.Targets[i] = sourcev1alpha1.SaveRequestTarget{GitTarget: target.String()} + attaches[i] = saveRequestAttach(request, target, author.Username, attribution) + } + setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, + "waiting for the edits to reach a commit window") + write() + + s.finalize(ctx, request, status, attaches) + write() + log.Info("SaveRequest saved", "reason", apimeta.FindStatusCondition(status.Conditions, "Ready").Reason, + "author", status.Author) + + if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && readyReason(status) == saveReasonCommitted { + s.followApplies(ctx, request, status, attaches, write) + } +} + +// resolveSaveAuthor finds the request's creator from the source cluster's audit facts, as a watched +// edit's author is found. +func (s *SaveRequestServer) resolveSaveAuthor( + ctx context.Context, route string, request *sourcev1alpha1.SaveRequest, +) (git.UserInfo, git.AttributionOutcome) { + if s.Manager == nil || s.Manager.AuthorResolver == nil { + return git.UserInfo{}, git.AttributionNotAttempted + } + return s.Manager.AuthorResolver.ResolveAuthor(ctx, AuthorQuery{ + AuditRoute: route, + GVR: saveRequestGVR(), + UID: request.UID, + ResourceVersion: request.ResourceVersion, + Namespace: request.Namespace, + Labels: request.Labels, + Name: request.Name, + ExactCapable: true, + }) +} + +// sentAuditFacts reports whether the KubeProvider has ever received an audit fact: its +// AuditFactsReceived condition is True. A provider that cannot be read says no. +func (s *SaveRequestServer) sentAuditFacts(ctx context.Context, provider k8stypes.NamespacedName) bool { + var kp configv1alpha3.KubeProvider + if err := s.Client.Get(ctx, provider, &kp); err != nil { + return false + } + return apimeta.IsStatusConditionTrue(kp.Status.Conditions, "AuditFactsReceived") +} + +// mirroringTargets are the GitTargets that mirror namespace from the KubeProvider: they name it, and +// one of their running streams watches that namespace, or every namespace. +func (s *SaveRequestServer) mirroringTargets( + ctx context.Context, provider k8stypes.NamespacedName, namespace string, +) ([]types.ResourceReference, error) { + var targets configv1alpha3.GitTargetList + if err := s.Client.List(ctx, &targets); err != nil { + return nil, err + } + var out []types.ResourceReference + for i := range targets.Items { + target := &targets.Items[i] + if target.DeletionTimestamp != nil || target.KubeProviderKey() != provider { + continue + } + ref := types.NewResourceReference(target.Name, target.Namespace) + if s.Manager.mirrorsNamespace(ref, namespace) { + out = append(out, ref) + } + } + sort.Slice(out, func(i, j int) bool { return out[i].String() < out[j].String() }) + return out, nil +} + +// mirrorsNamespace reports whether a running stream of gitDest watches namespace, or every namespace. +func (m *Manager) mirrorsNamespace(gitDest types.ResourceReference, namespace string) bool { + m.streamProgressMu.Lock() + defer m.streamProgressMu.Unlock() + for collection := range m.streamProgress[gitDest.Key()] { + if collection.Namespace == namespace || collection.Namespace == "" { + return true + } + } + return false +} + +// saveRequestAttach is the attach of a request to one GitTarget's window. Its identity carries the GitTarget, +// so one request saving several GitTargets on one branch worker is several requests there. +func saveRequestAttach( + request *sourcev1alpha1.SaveRequest, target types.ResourceReference, author string, + attribution git.AttributionOutcome, +) git.AttachCommitRequest { + attach := git.AttachCommitRequest{ + Namespace: request.Namespace, + Name: request.Name, + UID: string(request.UID) + "@" + target.String(), + Author: author, + Attribution: attribution, + GitTargetName: target.Name, + GitTargetNamespace: target.Namespace, + Message: request.Spec.Message, + Attach: configv1alpha3.AttachCurrentOrNext, + AttachTimeout: defaultSaveAttachTimeout, + MaxDuration: defaultSaveMaxDuration, + CatchUpNow: request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush, + } + if window := request.Spec.Window; window != nil { + if window.AttachTimeout != nil { + attach.AttachTimeout = window.AttachTimeout.Duration + } + if window.MaxDuration != nil { + attach.MaxDuration = window.MaxDuration.Duration + } + if window.IdleTimeout != nil { + idle := window.IdleTimeout.Duration + attach.IdleTimeout = &idle + } + } + return attach +} + +// finalize polls every GitTarget's outcome until each one resolved, withdrawing those still unheld past +// the request's bound, and records them on status. +func (s *SaveRequestServer) finalize( + ctx context.Context, request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, + attaches []git.AttachCommitRequest, +) { + bound := time.Now().Add(attaches[0].AttachTimeout + attaches[0].MaxDuration + saveRequestPushAllowance) + resolved := make([]bool, len(attaches)) + for remaining := len(attaches); remaining > 0; { + for i := range attaches { + if resolved[i] { + continue + } + ask := s.Finalizer.ServiceCommitRequest + if time.Now().After(bound) { + ask = s.Finalizer.WithdrawCommitRequest + } + result, done, err := ask(ctx, attaches[i]) + if err != nil || !done { + continue + } + resolved[i] = true + remaining-- + recordSaveOutcome(&status.Targets[i], result) + } + if remaining == 0 { + break + } + select { + case <-ctx.Done(): + return + case <-time.After(saveRequestPoll): + } + } + setSaveReady(request, status) +} + +// recordSaveOutcome records one GitTarget's outcome on its target status. +func recordSaveOutcome(target *sourcev1alpha1.SaveRequestTarget, result git.FinalizeResult) { + target.Branch = result.Branch + if result.Err != nil { + target.Reason, target.Message = saveReasonFailed, result.Err.Error() + if errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { + target.Message = "the save was not served in time and was withdrawn" + } + return + } + switch result.Outcome { + case git.FinalizeCommitted: + target.Reason, target.Commit = saveReasonCommitted, result.Commit + target.Message = "the edits were committed and pushed" + case git.FinalizeNoOpenWindow: + target.Reason = saveReasonNoWindow + target.Message = "no edit of this request's author reached a commit window before attachTimeout" + case git.FinalizeWindowMismatch: + target.Reason = saveReasonWindowMismatch + target.Message = "only another author's commit window was open" + case git.FinalizeAlreadyPresent: + target.Reason = saveReasonAlreadyPresent + target.Message = "the edits were in Git already" + default: + target.Reason, target.Message = saveReasonFailed, "unexpected outcome: "+string(result.Outcome) + } +} + +// setSaveReady sets Ready from the targets' outcomes: a failure in any fails the request, a commit in +// any is Committed, and otherwise the first target's reason says why nothing was committed. +func setSaveReady(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus) { + for _, target := range status.Targets { + if target.Reason == saveReasonFailed { + message := target.GitTarget + ": " + target.Message + setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonFailed, message) + setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonFailed, message) + return + } + } + for _, target := range status.Targets { + if target.Reason == saveReasonCommitted { + setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonCommitted, target.Message) + return + } + } + first := status.Targets[0] + setSaveCondition(request, status, "Ready", metav1.ConditionTrue, first.Reason, first.Message) +} + +// followApplies polls the apply each committed save asked for until every one is settled, and records +// them; a save its worker knows nothing of for saveRequestApplyLost is lost. +func (s *SaveRequestServer) followApplies( + ctx context.Context, request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, + attaches []git.AttachCommitRequest, write func(), +) { + setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, + "waiting for the appliers to apply the commit") + write() + started := time.Now() + for s.pollApplies(ctx, status, attaches, time.Since(started) > saveRequestApplyLost) { + select { + case <-ctx.Done(): + return + case <-time.After(saveRequestPoll): + } + } + setSaveApplied(request, status) + write() +} + +// pollApplies asks once for the apply of every committed target not settled yet, records it, and +// reports whether one is still pending. Once lost, a save its worker knows nothing of is lost. +func (s *SaveRequestServer) pollApplies( + ctx context.Context, status *sourcev1alpha1.SaveRequestStatus, attaches []git.AttachCommitRequest, lost bool, +) bool { + pending := false + for i := range attaches { + target := &status.Targets[i] + if target.Reason != saveReasonCommitted || saveApplySettled(target.Applied) { + continue + } + apply, known, err := s.Finalizer.LookupCommitRequestApply(ctx, attaches[i]) + switch { + case err == nil && known: + target.Applied, target.AppliedRevision = apply.Reason, apply.AppliedRevision + pending = pending || !apply.Done && !apply.Applied + case err == nil && lost: + target.Applied = "Lost" + default: + pending = true + } + } + return pending +} + +// saveApplySettled reports whether a target's apply reason changes no more. +func saveApplySettled(reason string) bool { + switch reason { + case git.SaveApplyApplied, git.SaveApplyNothingToApply, git.SaveApplyNoApplier, git.SaveApplyApplierGone, + git.SaveApplyCommitDropped, "Lost": + return true + } + return false +} + +// setSaveApplied sets Applied from the committed targets' applies: False only for a commit a branch +// lost, Unknown when one was lost track of, and True otherwise. +func setSaveApplied(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus) { + reason, condition := saveReasonApplied, metav1.ConditionTrue + message := "every applier applied a revision that holds the commit, or needed no apply" + for _, target := range status.Targets { + switch target.Applied { + case git.SaveApplyCommitDropped: + setSaveCondition(request, status, "Applied", metav1.ConditionFalse, target.Applied, + target.GitTarget+": the branch no longer holds the commit") + return + case "Lost": + reason, condition = "Lost", metav1.ConditionUnknown + message = target.GitTarget + ": GitOps Reverser does not know how the apply went" + } + } + setSaveCondition(request, status, "Applied", condition, reason, message) +} + +func readyReason(status *sourcev1alpha1.SaveRequestStatus) string { + if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil { + return ready.Reason + } + return "" +} + +func setSaveCondition( + request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, conditionType string, + condition metav1.ConditionStatus, reason, message string, +) { + status.ObservedGeneration = request.Generation + apimeta.SetStatusCondition(&status.Conditions, metav1.Condition{ + Type: conditionType, Status: condition, Reason: reason, Message: message, + ObservedGeneration: request.Generation, + }) +} + +// writeSaveRequestStatus writes status onto the request's latest version, retrying a conflict. +func writeSaveRequestStatus( + ctx context.Context, resource dynamic.ResourceInterface, request *sourcev1alpha1.SaveRequest, + status *sourcev1alpha1.SaveRequestStatus, +) error { + fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(status) + if err != nil { + return err + } + const attempts = 3 + for attempt := 1; ; attempt++ { + current, err := resource.Get(ctx, request.Name, metav1.GetOptions{}) + if err != nil { + return err + } + if current.GetUID() != request.UID { + return fmt.Errorf("SaveRequest %s/%s was replaced", request.Namespace, request.Name) + } + if err := unstructured.SetNestedMap(current.Object, fields, "status"); err != nil { + return err + } + _, err = resource.UpdateStatus(ctx, current, metav1.UpdateOptions{}) + if err == nil || !apierrors.IsConflict(err) || attempt == attempts { + return err + } + } +} diff --git a/internal/watch/save_requests_test.go b/internal/watch/save_requests_test.go new file mode 100644 index 00000000..a6f3e68f --- /dev/null +++ b/internal/watch/save_requests_test.go @@ -0,0 +1,313 @@ +// SPDX-License-Identifier: Apache-2.0 + +package watch + +import ( + "context" + "errors" + "sync" + "testing" + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + k8stypes "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/dynamic" + dynamicfake "k8s.io/client-go/dynamic/fake" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + + sourcev1alpha1 "github.com/ConfigButler/gitops-reverser/api/source/v1alpha1" + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/types" +) + +// saveFinalizer answers every attach and apply lookup the way the test sets, and records them. +type saveFinalizer struct { + mu sync.Mutex + result git.FinalizeResult + apply git.SaveApply + attaches []git.AttachCommitRequest +} + +func (f *saveFinalizer) ServiceCommitRequest( + _ context.Context, attach git.AttachCommitRequest, +) (git.FinalizeResult, bool, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.attaches = append(f.attaches, attach) + return f.result, true, nil +} + +func (f *saveFinalizer) WithdrawCommitRequest( + _ context.Context, _ git.AttachCommitRequest, +) (git.FinalizeResult, bool, error) { + return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil +} + +func (f *saveFinalizer) LookupCommitRequestApply( + _ context.Context, _ git.AttachCommitRequest, +) (git.SaveApply, bool, error) { + return f.apply, true, nil +} + +// fixedAuthor resolves every query to one user, or to none. +type fixedAuthor struct { + user git.UserInfo + outcome git.AttributionOutcome +} + +func (a fixedAuthor) ResolveAuthor(context.Context, AuthorQuery) (git.UserInfo, git.AttributionOutcome) { + return a.user, a.outcome +} + +var saveProvider = k8stypes.NamespacedName{Namespace: "providers", Name: "workspace"} + +// saveServer is a server for the KubeProvider providers/workspace, whose GitTarget team-a/app mirrors +// the source namespace "shop" when mirrored is set, with a SaveRequest shop/save on the source cluster. +func saveServer( + t *testing.T, request *sourcev1alpha1.SaveRequest, mirrored bool, auditFacts metav1.ConditionStatus, + author AuthorResolver, finalizer *saveFinalizer, +) (*SaveRequestServer, dynamic.ResourceInterface) { + t.Helper() + scheme := runtime.NewScheme() + require.NoError(t, configv1alpha3.AddToScheme(scheme)) + provider := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} + if auditFacts != "" { + provider.Status.Conditions = []metav1.Condition{ + {Type: "AuditFactsReceived", Status: auditFacts, Reason: "Test"}, + } + } + target := &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a"}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, + }, + } + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, target). + WithStatusSubresource(&configv1alpha3.KubeProvider{}).Build() + + manager := &Manager{AuthorResolver: author} + if mirrored { + manager.trackStreamProgress(types.NewResourceReference("app", "team-a"), + targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "shop"}) + } + fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(request) + require.NoError(t, err) + obj := &unstructured.Unstructured{Object: fields} + obj.SetAPIVersion(sourcev1alpha1.GroupVersion.String()) + obj.SetKind("SaveRequest") + dc := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), + map[schema.GroupVersionResource]string{saveRequestGVR(): "SaveRequestList"}, obj) + server := &SaveRequestServer{Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard()} + return server, dc.Resource(saveRequestGVR()).Namespace("shop") +} + +func newSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1.SaveRequest { + return &sourcev1alpha1.SaveRequest{ + ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: "shop", UID: "uid-save", Generation: 1}, + Spec: sourcev1alpha1.SaveRequestSpec{ + Message: "save: from the editor", + CatchUp: catchUp, + Window: &sourcev1alpha1.SaveRequestWindow{ + AttachTimeout: &metav1.Duration{Duration: 3 * time.Second}, + MaxDuration: &metav1.Duration{Duration: time.Second}, + }, + }, + } +} + +// serveOnce serves the request and returns its status as written to the source cluster. +func serveOnce( + t *testing.T, server *SaveRequestServer, resource dynamic.ResourceInterface, request *sourcev1alpha1.SaveRequest, +) sourcev1alpha1.SaveRequestStatus { + t.Helper() + watcher := &saveRequestWatcher{provider: saveProvider, route: "providers.workspace"} + server.serve(context.Background(), logr.Discard(), watcher, resource, request) + got, err := resource.Get(context.Background(), request.Name, metav1.GetOptions{}) + require.NoError(t, err) + var written sourcev1alpha1.SaveRequest + require.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(got.Object, &written)) + return written.Status +} + +func condition(status sourcev1alpha1.SaveRequestStatus, conditionType string) *metav1.Condition { + return apimeta.FindStatusCondition(status.Conditions, conditionType) +} + +// A save attaches to the window of every GitTarget that mirrors its namespace, as its author found +// from the source cluster's audit, and reports each commit and the apply it asked for. +func TestSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) { + finalizer := &saveFinalizer{ + result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, + apply: git.SaveApply{ + Done: true, + Applied: true, + Reason: git.SaveApplyApplied, + AppliedRevision: "main@sha1:c0ffee", + }, + } + author := fixedAuthor{user: git.UserInfo{Username: "alice@example.com"}, outcome: git.AttributionResolved} + request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) + server, resource := saveServer(t, request, true, metav1.ConditionTrue, author, finalizer) + + status := serveOnce(t, server, resource, request) + + require.Len(t, finalizer.attaches, 1) + attach := finalizer.attaches[0] + assert.Equal(t, "alice@example.com", attach.Author) + assert.Equal(t, git.AttributionResolved, attach.Attribution) + assert.Equal(t, "app", attach.GitTargetName) + assert.Equal(t, "team-a", attach.GitTargetNamespace) + assert.Equal(t, "save: from the editor", attach.Message) + assert.Equal(t, 3*time.Second, attach.AttachTimeout) + assert.True(t, attach.CatchUpNow) + assert.Equal(t, "uid-save@team-a/app", attach.UID, "one identity per GitTarget") + + assert.Equal(t, "alice@example.com", status.Author) + require.Len(t, status.Targets, 1) + assert.Equal(t, sourcev1alpha1.SaveRequestTarget{ + GitTarget: "team-a/app", Reason: saveReasonCommitted, Message: "the edits were committed and pushed", + Branch: "main", Commit: "c0ffee", Applied: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", + }, status.Targets[0]) + require.NotNil(t, condition(status, "Ready")) + assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) + assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) + require.NotNil(t, condition(status, "Applied")) + assert.Equal(t, metav1.ConditionTrue, condition(status, "Applied").Status) +} + +// On a cluster with no audit stream a save names nobody, and attaches as such: to a window whose +// edits name nobody. +func TestSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}} + request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, true, metav1.ConditionUnknown, + fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) + + status := serveOnce(t, server, resource, request) + + require.Len(t, finalizer.attaches, 1) + assert.Empty(t, finalizer.attaches[0].Author) + assert.Equal(t, git.AttributionUnresolved, finalizer.attaches[0].Attribution) + assert.Equal(t, saveReasonNoWindow, condition(status, "Ready").Reason) + assert.Nil(t, condition(status, "Applied"), "no catch-up was asked for") +} + +// On a cluster that sends audit events, a save no event names lost its fact: it is refused rather +// than attached to the window that should stay unattributed. +func TestSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { + finalizer := &saveFinalizer{} + request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, true, metav1.ConditionTrue, + fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Empty(t, finalizer.attaches, "nothing is attached") + assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) + assert.Equal(t, saveReasonAuthorUnresolved, condition(status, "Ready").Reason) +} + +// A save in a namespace no GitTarget mirrors has nothing to save. +func TestSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { + finalizer := &saveFinalizer{} + request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, false, "", nil, finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Empty(t, finalizer.attaches) + assert.Equal(t, saveReasonNoGitTarget, condition(status, "Ready").Reason) +} + +// A save that failed in a GitTarget fails, and says where. +func TestSaveRequest_AFailureFailsTheRequest(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} + request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) + server, resource := saveServer(t, request, true, "", nil, finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) + assert.Equal(t, saveReasonFailed, condition(status, "Ready").Reason) + assert.Contains(t, condition(status, "Ready").Message, "team-a/app: the GitTarget is suspended") + assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) + assert.Nil(t, condition(status, "Applied"), "nothing was committed to apply") +} + +// A request is finished once Ready is decided, and, when it committed and asked to catch up, once +// Applied is decided too. +func TestSaveRequest_Finished(t *testing.T) { + at := metav1.NewTime(time.Now().Add(-time.Hour)) + ready := func(status metav1.ConditionStatus, reason string) metav1.Condition { + return metav1.Condition{Type: "Ready", Status: status, Reason: reason, LastTransitionTime: at} + } + request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) + _, finished := saveRequestFinished(request) + assert.False(t, finished, "no Ready yet") + + request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionUnknown, saveReasonSaving)} + _, finished = saveRequestFinished(request) + assert.False(t, finished, "still saving") + + request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonCommitted)} + _, finished = saveRequestFinished(request) + assert.False(t, finished, "committed, its apply not decided") + + request.Status.Conditions = append( + request.Status.Conditions, + metav1.Condition{ + Type: "Applied", + Status: metav1.ConditionTrue, + Reason: saveReasonApplied, + LastTransitionTime: at, + }, + ) + finishedAt, finished := saveRequestFinished(request) + assert.True(t, finished) + assert.Equal(t, at.Time, finishedAt) + + request.Spec.CatchUp = sourcev1alpha1.CatchUpAsDefinedOnClaim + request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonNoWindow)} + _, finished = saveRequestFinished(request) + assert.True(t, finished) +} + +// The server serves the KubeProviders that enable SaveRequests, and stops serving one that no longer +// does. +func TestSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T) { + scheme := runtime.NewScheme() + require.NoError(t, configv1alpha3.AddToScheme(scheme)) + enabled := &configv1alpha3.KubeProvider{ + ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}, + Spec: configv1alpha3.KubeProviderSpec{SaveRequests: configv1alpha3.SaveRequestsEnabled}, + } + disabled := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "other", Namespace: "providers"}} + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(enabled, disabled).Build() + server := &SaveRequestServer{ + Client: c, Log: logr.Discard(), + dynamicClient: func(context.Context, k8stypes.NamespacedName) (dynamic.Interface, error) { + return nil, errors.New("no cluster in this test") + }, + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + server.syncProviders(ctx) + assert.Len(t, server.watchers, 1) + assert.Contains(t, server.watchers, saveProvider) + + enabled.Spec.SaveRequests = configv1alpha3.SaveRequestsDisabled + require.NoError(t, c.Update(ctx, enabled)) + server.syncProviders(ctx) + assert.Empty(t, server.watchers) + server.stopAll() +} diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index cf456799..d3754817 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -35,9 +35,9 @@ import ( // gitops-api runs: Flux applies a folder into a workspace through Kustomization.spec.kubeConfig, // the GitTarget mirrors that workspace, and its FluxClaim holds the Kustomization through the // install's own provider. It shows the startup converge landing in the workspace, an edit made -// there during an apply withheld from the apply's revert by the REMOTE object's managedFields, -// and two saves that take over their edits' windows (the workspace sends no audit facts) and -// report the apply. +// there withheld from the apply's revert by the REMOTE object's managedFields, two +// CommitRequests that take over their edits' windows (the workspace sends no audit facts) and +// report the apply, and a SaveRequest created in the workspace itself. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), // and the kcp specs additionally Skip when kcp is not installed — so a default `task test-e2e` @@ -657,6 +657,8 @@ spec: p *timingPipeline r *echoRun ksName string + // hash is the workspace's logical cluster. + hash string ) BeforeAll(func() { @@ -675,7 +677,7 @@ spec: By("provisioning the workspace, and the namespace the folder holds inside it") DeferCleanup(kcp.deleteWorkspace, ws) - hash := kcp.createWorkspace(ws) + hash = kcp.createWorkspace(ws) _, err = kcp.wsKubectl(hash, "create", "namespace", testNs) Expect(err).NotTo(HaveOccurred(), "create the folder's namespace in %s", ws) @@ -859,9 +861,91 @@ spec: }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.eventuallyEverywhere(timingConfigMap, "saved-2", echoApplyBudget) }) + + // A save made in the workspace itself: a SaveRequest, served because the workspace's + // KubeProvider enables them. It names no GitTarget; it saves the window of every GitTarget that + // mirrors its namespace. kcp sends no audit facts, so the request names nobody, like the edit, + // and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is + // looked for first, for the whole attribution grace, so the edit is made once the request is + // waiting for a window. + It("saves an edit made in the workspace with a SaveRequest created there", func() { + By("installing the SaveRequest CRD in the workspace, and serving the workspace's SaveRequests") + _, err := kcp.wsKubectl(hash, "apply", "-f", "config/crd/bases/source.configbutler.ai_saverequests.yaml") + Expect(err).NotTo(HaveOccurred()) + _, err = kcp.wsKubectl(hash, "wait", "--for=condition=Established", + "crd/saverequests.source.configbutler.ai", "--timeout=60s") + Expect(err).NotTo(HaveOccurred()) + _, err = kubectlRunInNamespace(testNs, "patch", "kubeprovider", ws, "--type=merge", "-p", + `{"spec":{"saveRequests":"Enabled"}}`) + Expect(err).NotTo(HaveOccurred()) + + By("creating a SaveRequest in the workspace, and editing once it waits for a window") + const name, message = "sc-flux-workspace-save", "e2e: save from the workspace" + createWorkspaceSaveRequest(kcp, hash, testNs, name, message) + saveField := func(g Gomega, jsonPath string) string { + out, getErr := kcp.wsKubectl(hash, "-n", testNs, "get", "saverequest", name, "-o", "jsonpath="+jsonPath) + g.Expect(getErr).NotTo(HaveOccurred()) + return strings.TrimSpace(out) + } + Eventually(func(g Gomega) { + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Saving")) + }, 2*time.Minute, time.Second).Should(Succeed(), "the server picks the request up and waits for a window") + r.editThroughAPI(timingConfigMap, "saved-3") + + By("the save commits the edit, and reports the commit and the apply on the request") + var commit string + Eventually(func(g Gomega) { + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("True")) + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Committed")) + g.Expect(saveField(g, "{.status.targets[0].gitTarget}")).To(Equal(testNs + "/" + p.targetName)) + commit = saveField(g, "{.status.targets[0].commit}") + g.Expect(commit).NotTo(BeEmpty()) + g.Expect(saveField(g, "{.status.author}")).To(BeEmpty(), "kcp sends no audit facts") + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + committed, inGit := r.committedValue(Default, timingConfigMap) + Expect(inGit).To(BeTrue()) + Expect(committed).To(Equal("saved-3")) + body, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commit) + Expect(err).NotTo(HaveOccurred()) + Expect(strings.TrimSpace(body)).To(Equal(message)) + + Eventually(func(g Gomega) { + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Applied")].status}`)).To(Equal("True")) + g.Expect(saveField(g, "{.status.targets[0].applied}")).To(Equal("Applied")) + g.Expect(saveField(g, "{.status.targets[0].appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(commit)) + }, echoApplyBudget, 2*time.Second).Should(Succeed()) + r.consistentlyEverywhere(timingConfigMap, "saved-3") + }) }) }) +// createWorkspaceSaveRequest creates the SaveRequest name in namespace of a workspace, asking to catch +// up. It waits up to 30s for a window: an edit from a cluster with no audit stream reaches its window +// only after the attribution grace. +func createWorkspaceSaveRequest(kcp *kcpTunnel, hash, namespace, name, message string) { + GinkgoHelper() + f, err := os.CreateTemp("", "e2e-saverequest-*.yaml") + Expect(err).NotTo(HaveOccurred()) + defer func() { _ = os.Remove(f.Name()) }() + _, err = fmt.Fprintf(f, `apiVersion: source.configbutler.ai/v1alpha1 +kind: SaveRequest +metadata: + name: %s + namespace: %s +spec: + message: %q + catchUp: AfterPush + window: + attachTimeout: "30s" + maxDuration: "4s" +`, name, namespace, message) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Close()).To(Succeed()) + _, err = kcp.wsKubectl(hash, "create", "-f", f.Name()) + Expect(err).NotTo(HaveOccurred(), "failed to create SaveRequest %s/%s in the workspace", namespace, name) +} + // applyFluxIntoWorkspace creates, in flux-system, a GitRepository following main and a Kustomization // applying path from it into a kcp workspace: its spec.kubeConfig names a Secret holding kubeconfig // under the key "value", which kustomize-controller, in this cluster, dials. Both are on a 24h From aaed46f3d8f7bff072e9e5874ddfe9189296dec7 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 17:57:37 +0000 Subject: [PATCH 15/33] fix(applier): carry a pending apply-window barrier as a wait, and bound its setup A window opened while the one before still waited for the watch took that barrier's two-minute limit as its carried close time, so it stayed open until then even after both barriers were met. It now carries the pending barrier itself: the earlier apply's writes are judged until that barrier is met or lapses, its answers still count after the replacement, and the new window closes after its own grace and barrier. Taking a barrier lists the source clusters on the worker's lifetime context; it now runs under the window's limit, so a hanging list cannot outlive the wait it serves. Co-Authored-By: Claude Opus 5.5 (1M context) --- internal/git/apply_barrier.go | 67 +++++++++++++++++--------- internal/git/apply_barrier_test.go | 29 ++++++++++++ internal/git/apply_window.go | 75 ++++++++++++++++++++++-------- 3 files changed, 131 insertions(+), 40 deletions(-) diff --git a/internal/git/apply_barrier.go b/internal/git/apply_barrier.go index 2813657a..44dbf66c 100644 --- a/internal/git/apply_barrier.go +++ b/internal/git/apply_barrier.go @@ -4,6 +4,7 @@ package git import ( "context" + "slices" "time" "go.opentelemetry.io/otel/attribute" @@ -98,11 +99,15 @@ func (l *branchWorkerEventLoop) awaitApplyBarrier(state *applierState, now time. for _, target := range state.targets { targets = append(targets, target.ref) } - ctx := w.ctx - if ctx == nil { - ctx = context.Background() + base := w.ctx + if base == nil { + base = context.Background() } + until := window.barrier.until go func() { + // Taking the barrier lists the source clusters; past the window's limit nothing waits for it. + ctx, cancel := context.WithDeadline(base, until) + defer cancel() expected := 0 for _, target := range targets { n, err := w.applyBarrier(ctx, target, func() { @@ -118,19 +123,22 @@ func (l *branchWorkerEventLoop) awaitApplyBarrier(state *applierState, now time. }() } -// handleApplyBarrier counts one answer to the barrier its window waits for. An answer to a barrier -// the window no longer waits for (met, replaced by a newer attempt's window, or timed out) is dropped. +// handleApplyBarrier counts one answer to a barrier the window waits for: its own, or one it carries +// from the window it replaced. An answer to a barrier nothing waits for any more (met, or timed out) +// is dropped. func (l *branchWorkerEventLoop) handleApplyBarrier(answer *ApplyBarrier) { state := l.appliers[answer.Applier] - if state == nil || state.window == nil || state.window.barrier == nil || - state.window.barrier.token != answer.Token { + if state == nil || state.window == nil { return } window := state.window - wait := window.barrier + wait := window.waitFor(answer.Token) + if wait == nil { + return + } switch { case answer.Err != nil: - window.barrier = nil + window.dropWait(wait) l.w.Log.Info("Could not take the apply window's barrier; the window closes after its grace alone", "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, "err", answer.Err.Error()) l.recordApplyBarrier(barrierFailed) @@ -143,27 +151,44 @@ func (l *branchWorkerEventLoop) handleApplyBarrier(answer *ApplyBarrier) { if wait.expected < 0 || wait.passed < wait.expected { return } - window.barrier = nil + window.dropWait(wait) l.w.Log.V(1).Info("The watch delivered every write made before the apply completed", "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, "waited", time.Since(wait.completedAt).Round(time.Millisecond).String(), "streams", wait.expected) l.recordApplyBarrier(barrierMet) } -// barrierTimedOut logs and counts a window closing while it still waited for the watch. +// waitFor is the barrier with token that the window waits for, nil when none. +func (a *applyWindow) waitFor(token uint64) *applyBarrierWait { + for _, wait := range a.pendingWaits() { + if wait.token == token { + return wait + } + } + return nil +} + +// dropWait stops the window waiting for a barrier. +func (a *applyWindow) dropWait(wait *applyBarrierWait) { + if a.barrier == wait { + a.barrier = nil + return + } + a.carriedWaits = slices.DeleteFunc(a.carriedWaits, func(w *applyBarrierWait) bool { return w == wait }) +} + +// barrierTimedOut logs and counts the barriers a window still waited for when it closed. func (l *branchWorkerEventLoop) barrierTimedOut(state *applierState) { window := state.window - wait := window.barrier - if wait == nil { - return + for _, wait := range window.pendingWaits() { + window.dropWait(wait) + l.w.Log.Info("The apply window closed before the watch delivered every write made before the apply "+ + "completed: a write of the applier's arriving later is mirrored as an edit", + "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, + "waited", time.Since(wait.completedAt).Round(time.Second).String(), + "streams", wait.expected, "delivered", wait.passed) + l.recordApplyBarrier(barrierTimedOut) } - window.barrier = nil - l.w.Log.Info("The apply window closed before the watch delivered every write made before the apply "+ - "completed: a write of the applier's arriving later is mirrored as an edit", - "branch", l.w.Branch, "applier", state.ref.String(), "commit", window.commit, - "waited", time.Since(wait.completedAt).Round(time.Second).String(), - "streams", wait.expected, "delivered", wait.passed) - l.recordApplyBarrier(barrierTimedOut) } // recordApplyBarrier counts one apply window barrier, by outcome. diff --git a/internal/git/apply_barrier_test.go b/internal/git/apply_barrier_test.go index 9f56aafd..ea200eb5 100644 --- a/internal/git/apply_barrier_test.go +++ b/internal/git/apply_barrier_test.go @@ -146,3 +146,32 @@ func TestApplyBarrier_AnAnswerForAnotherBarrierIsDropped(t *testing.T) { assert.Same(t, wait, state.window.barrier) assert.Zero(t, wait.passed) } + +// A window opened while the one before still waits for the watch carries that wait as a wait, not as +// its limit: the earlier apply's writes are judged until its barrier is met, and once it is, the new +// window closes after its own grace. The carried barrier's answers still count after the replacement. +func TestApplyBarrier_ANewerWindowCarriesThePendingWaitNotItsLimit(t *testing.T) { + barrier := &fakeBarrier{} + loop, state := completedWithBarrier(t, barrier) + before := state.window + wait := before.barrier + require.NotNil(t, wait) + + now := time.Now() + state.openApplyWindow(before.attempt+1, "ccc", now) + next := state.window + require.Len(t, next.carriedWaits, 1, "the pending barrier is carried") + assert.Same(t, wait, next.carriedWaits[0]) + assert.True(t, next.carriedUntil.Before(wait.until), "the limit is not carried as a close time") + + state.settleApplyWindow(next.attempt, true, now) + next.closesAt = now.Add(-time.Millisecond) + assert.True(t, next.open(time.Now()), "the earlier apply's writes can still arrive") + assert.True(t, loop.withholdLateApply(windowEvent("alpha", "2", FluxFieldManager)), + "the earlier apply's write is still judged inside the window") + + barrier.deliverAll() + dispatchQueued(loop) + assert.Empty(t, next.carriedWaits, "the carried barrier is met") + assert.False(t, next.open(time.Now()), "and the window closes after its own grace") +} diff --git a/internal/git/apply_window.go b/internal/git/apply_window.go index c7809a5d..dba110fa 100644 --- a/internal/git/apply_window.go +++ b/internal/git/apply_window.go @@ -126,11 +126,13 @@ type applyWindow struct { // time of the latest such write's managedFields entries (zero when the event carried none). edited map[string]time.Time // carried holds the objects the window before this one had seen edited, when that window was - // still open as this attempt was sent, and carriedUntil is when it would have closed: the - // earlier apply's writes can arrive until then (openApplyWindow). carriedSentAt is the earliest - // send of the windows carried, zero when none is. + // still open as this attempt was sent, and carriedUntil is when its grace would have ended; + // carriedWaits are its barriers still waiting for the watch. The earlier apply's writes can arrive + // until both have passed (openApplyWindow). carriedSentAt is the earliest send of the windows + // carried, zero when none is. carried map[string]time.Time carriedUntil time.Time + carriedWaits []*applyBarrierWait carriedSentAt time.Time // echoes counts the applier's writes the window kept out of Git: the apply landing, object by // object. reported is set once that count was logged (reportApplyWindows). @@ -146,25 +148,54 @@ type applyWindow struct { // window and of the windows it still carries. A delayed echo of the attempt before, arriving after // this one was sent and inside that one's grace, is still that apply. func (a *applyWindow) earliestSend(now time.Time) time.Time { - if !a.carriedSentAt.IsZero() && now.Before(a.carriedUntil) { + if !a.carriedSentAt.IsZero() && a.carrying(now) { return a.carriedSentAt } return a.sentAt } -// open reports whether the window is open at now: until its grace has passed, and for as long as -// it still waits for the watch (barrier). +// carrying reports whether the windows this one carries could still see their applies' writes arrive +// at now: until their grace has passed, and while one of their barriers still waits for the watch. +func (a *applyWindow) carrying(now time.Time) bool { + return now.Before(a.carriedUntil) || a.carriedWaitHolds(now) +} + +// carriedWaitHolds reports whether a barrier carried from the window before still waits at now. +func (a *applyWindow) carriedWaitHolds(now time.Time) bool { + return slices.ContainsFunc(a.carriedWaits, func(wait *applyBarrierWait) bool { return wait.holds(now) }) +} + +// open reports whether the window is open at now: until its grace has passed, which settling the +// window extends to the grace it carries, and for as long as it, or a window it carries, still waits +// for the watch (barrier). func (a *applyWindow) open(now time.Time) bool { - return a != nil && (a.closesAt.IsZero() || now.Before(a.closesAt) || a.barrier.holds(now)) + return a != nil && + (a.closesAt.IsZero() || now.Before(a.closesAt) || a.barrier.holds(now) || a.carriedWaitHolds(now)) } -// closesBy is when the window closes at the latest: zero while the attempt is out, and past the -// grace while it waits for the watch. +// closesBy is when the window closes at the latest: zero while the attempt is out, and past the grace +// while a barrier waits for the watch. func (a *applyWindow) closesBy() time.Time { - if a.closesAt.IsZero() || a.barrier == nil { + if a.closesAt.IsZero() { return a.closesAt } - return latest(a.closesAt, a.barrier.until) + by := latest(a.closesAt, a.carriedUntil) + if a.barrier != nil { + by = latest(by, a.barrier.until) + } + for _, wait := range a.carriedWaits { + by = latest(by, wait.until) + } + return by +} + +// pendingWaits are the barriers that hold the window: its own and the ones it carries. +func (a *applyWindow) pendingWaits() []*applyBarrierWait { + waits := slices.Clone(a.carriedWaits) + if a.barrier != nil { + waits = append(waits, a.barrier) + } + return waits } // objectKey is an event's object without its API version: a preferred-version bump during the @@ -180,7 +211,7 @@ func objectKey(id itypes.ResourceIdentifier) string { // attempt's apply of a commit that holds every edit published before it (S1). func (a *applyWindow) editedSince(key string, now, written time.Time) (time.Time, bool) { at, edited := a.edited[key] - if carriedAt, carried := a.carried[key]; carried && now.Before(a.carriedUntil) && + if carriedAt, carried := a.carried[key]; carried && a.carrying(now) && written.Before(a.sentAt.Truncate(time.Second)) { return latest(at, carriedAt), true } @@ -193,18 +224,24 @@ func (a *applyWindow) recordEdit(key string, at time.Time) { } // openApplyWindow starts an applier's window as its attempt is sent. The window before it can still -// be open: an attempt may follow one that completed within its grace, and that apply's writes are -// still arriving. Its edits are carried until it would have closed, so a late write of the earlier -// apply over one of them is still withheld. Replacing the window lost them, and the revert was -// mirrored. +// be open: an attempt may follow one that completed within its grace, or whose barrier still waits for +// the watch, and that apply's writes are still arriving. Its edits are carried until its grace ends +// and its barriers are met, so a late write of the earlier apply over one of them is still withheld. +// Replacing the window lost them, and the revert was mirrored. Its barriers are carried as they are, +// not as their limits: one that is met stops holding the new window. func (state *applierState) openApplyWindow(attempt uint64, commit string, now time.Time) { next := &applyWindow{attempt: attempt, commit: commit, sentAt: now, edited: map[string]time.Time{}} if before := state.window; before.open(now) { next.carried = maps.Clone(before.edited) - if now.Before(before.carriedUntil) { + if before.carrying(now) { maps.Copy(next.carried, before.carried) } - next.carriedUntil = before.closesBy() + next.carriedUntil = latest(before.closesAt, before.carriedUntil) + for _, wait := range before.pendingWaits() { + if wait.holds(now) { + next.carriedWaits = append(next.carriedWaits, wait) + } + } next.carriedSentAt = before.earliestSend(now) } state.window = next @@ -221,7 +258,7 @@ func (state *applierState) settleApplyWindow(attempt uint64, operated bool, now switch { case operated: window.closesAt = latest(now.Add(applyWindowGrace), window.carriedUntil) - case now.Before(window.carriedUntil): + case window.carrying(now): window.closesAt = window.carriedUntil default: state.window = nil From f9289bf8d3ae1bce743c97785ee768993ecc0d2c Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 18:33:32 +0000 Subject: [PATCH 16/33] fix(applier): an attempt that carries a save is never a lag-only converge onlyLag did not look at the saves waiting for an apply. Today every save also clears lag, so an attempt never carried both, but a lag-only attempt that goes stale is not given back, and its saves would have been lost with it. onlyLag now answers no when a save is waiting. Co-Authored-By: Claude Opus 5.5 (1M context) --- internal/git/applier_claim.go | 6 ++++-- internal/git/apply_converge_test.go | 9 +++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 91348781..d956e233 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -303,9 +303,11 @@ func (s *applierState) restoreLag(attempt *inflightAttempt) { } // onlyLag reports whether scheduleConverges' converge is all that is pending for the applier, given its -// unsent refusals: an attempt carrying nothing else yields to an edit (convergeHolds). +// unsent refusals: an attempt carrying nothing else yields to an edit (convergeHolds). One that carries +// a save is never only a lag, whatever lag says: a save asked for its apply, and a lag-only attempt that +// went stale would not give it back. func (s *applierState) onlyLag(refusals map[refusalKey]unsentRefusal) bool { - return s.lag && !s.foreign && !s.adopt && !s.repair && len(refusals) == 0 + return s.lag && !s.foreign && !s.adopt && !s.repair && len(refusals) == 0 && len(s.waitingSaves) == 0 } // repairPending reports whether a Repair is pending: not yet carried, or carried by the attempt out. diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index 26e74e47..edc3973c 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -429,3 +429,12 @@ func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { }) } } + +// An attempt that carries a save is never a lag-only converge, so it never yields to an edit and a +// stale one gives its saves back. +func TestApplierState_ASaveIsNeverOnlyALag(t *testing.T) { + state := &applierState{lag: true} + assert.True(t, state.onlyLag(nil)) + state.waitSave(commitRequestID{Namespace: "team-a", Name: "save", UID: "uid"}, "ccc") + assert.False(t, state.onlyLag(nil)) +} From abf55709b2db89bbac1f194b80c2f7c4bc1e7afc Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 19:08:26 +0000 Subject: [PATCH 17/33] feat(api)!: CommitRequest is SaveRequest; the source-cluster save is RemoteSaveRequest The two request kinds are named as one pair. SaveRequest (configbutler.ai/v1alpha3, the former CommitRequest) is created on the cluster GitOps Reverser runs in and names its GitTarget; its author comes from admission. RemoteSaveRequest (source.configbutler.ai/v1alpha1, the source-cluster kind this branch added as SaveRequest) is created on a source cluster, names no GitTarget, and is attributed from that cluster's audit events. KubeProvider.spec.saveRequests is spec.remoteSaveRequests. Everything named after CommitRequest follows it: the RBAC resource, the --save-request-ttl flag and controllerManager.saveRequestTTL value, the save_requests_total metric, and the save_request values of message_source and timer_source. BREAKING CHANGE: CommitRequest is SaveRequest. Grant saverequests instead of commitrequests, rename --commit-request-ttl to --save-request-ttl and controllerManager.commitRequestTTL to saveRequestTTL, read gitopsreverser_save_requests_total, and delete the commitrequests.configbutler.ai CRD after upgrading. Co-Authored-By: Claude Opus 5.5 (1M context) --- .vale/styles/HouseStyle/Headings.yml | 3 +- README.md | 4 +- ...st_types.go => remotesaverequest_types.go} | 54 +-- api/source/v1alpha1/zz_generated.deepcopy.go | 58 +-- api/v1alpha3/argocdclaim_types.go | 2 +- api/v1alpha3/examples_test.go | 4 +- api/v1alpha3/fluxclaim_types.go | 2 +- api/v1alpha3/gitprovider_types.go | 8 +- api/v1alpha3/gittarget_types.go | 8 +- api/v1alpha3/kubeprovider_types.go | 20 +- ...trequest_types.go => saverequest_types.go} | 52 +-- api/v1alpha3/status_accessors.go | 2 +- api/v1alpha3/zz_generated.deepcopy.go | 264 +++++------ charts/gitops-reverser/README.md | 16 +- .../gitops-reverser/templates/deployment.yaml | 2 +- .../validate-operator-types-webhook.yaml | 8 +- charts/gitops-reverser/values.schema.json | 4 +- charts/gitops-reverser/values.yaml | 12 +- cmd/main.go | 38 +- .../bases/configbutler.ai_argocdclaims.yaml | 2 +- .../crd/bases/configbutler.ai_fluxclaims.yaml | 2 +- .../crd/bases/configbutler.ai_gittargets.yaml | 10 +- .../bases/configbutler.ai_kubeproviders.yaml | 8 +- ...yaml => configbutler.ai_saverequests.yaml} | 22 +- ...e.configbutler.ai_remotesaverequests.yaml} | 24 +- config/crd/kustomization.yaml | 4 +- config/rbac/role.yaml | 16 +- config/samples/README.md | 2 +- .../{commitrequest.yaml => saverequest.yaml} | 4 +- config/webhook/validating-webhook.yaml | 10 +- docs/INDEX.md | 10 +- docs/README.md | 2 +- docs/TODO.md | 10 +- docs/UPGRADING.md | 23 +- docs/api-first-publication.md | 16 +- docs/architecture.md | 78 ++-- docs/attribution-setup-guide.md | 2 +- docs/bi-directional.md | 4 +- docs/commit-messages.md | 8 +- docs/components.md | 6 +- docs/configuration.md | 152 +++--- docs/definitions.md | 7 +- docs/demo/commit-window.md | 12 +- docs/design/applier-echoes.md | 4 +- docs/design/branch-worker-event-model.md | 4 +- .../design/commitrequest-save-wait-options.md | 22 +- ...e2e-coverage-gaps-and-improvements-plan.md | 2 +- docs/design/e2e-suite-trim.md | 2 +- docs/design/event-pipeline-overview.md | 2 +- docs/design/git-write-preflight.md | 2 +- .../gittarget-branch-worker-pending-writes.md | 4 +- docs/design/gittarget-parent-hardening.md | 2 +- docs/design/gittarget-state-of-affairs.md | 4 +- docs/design/reconcile-triggering.md | 6 +- .../crd-descriptions-in-kubectl-explain.md | 2 +- docs/facts/generated-name-support.md | 20 +- docs/facts/subresources.md | 14 +- docs/finished/redis-key-schema-v3.md | 4 +- ...fig-surface-for-a-structured-repository.md | 6 +- .../direction-and-configuration-surface.md | 4 +- docs/future/ha-gittarget-distribution-plan.md | 6 +- docs/future/source-side-save-plan.md | 48 +- docs/future/task-oriented-docs-and-website.md | 4 +- docs/interpreting-metrics.md | 42 +- docs/proud.md | 2 +- docs/rbac.md | 8 +- docs/review/20261025-crd-text-proposal.md | 2 +- docs/spec/README.md | 4 +- docs/spec/attribution.md | 10 +- docs/spec/commit-window-refactor.md | 12 +- docs/spec/current-manifest-support-review.md | 2 +- docs/spec/e2e-serial-registry.md | 4 +- docs/spec/e2e-test-design.md | 8 +- ...md => saverequest-admission-authorship.md} | 14 +- ...equest-design.md => saverequest-design.md} | 12 +- docs/spec/status-conditions-guide.md | 6 +- docs/spec/where-validation-lives.md | 4 +- hack/crdfields/fields.yaml | 18 +- internal/auditutil/identity_test.go | 30 +- internal/controller/constants.go | 12 +- .../duration_fields_admission_test.go | 42 +- internal/controller/gitbranch_controller.go | 4 +- .../controller/gitbranch_controller_test.go | 2 +- .../controller/gittarget_publication_test.go | 8 +- .../controller/resource_condition_metrics.go | 6 +- .../resource_condition_metrics_test.go | 4 +- ...trequest_apply.go => saverequest_apply.go} | 30 +- ...pply_test.go => saverequest_apply_test.go} | 68 +-- ...saverequest_attribution_agreement_test.go} | 26 +- ...ontroller.go => saverequest_controller.go} | 424 ++++++++--------- ...test.go => saverequest_controller_test.go} | 38 +- ...go => saverequest_controller_unit_test.go} | 436 +++++++++--------- ...st_finalize.go => saverequest_finalize.go} | 162 +++---- ...us_test.go => saverequest_kstatus_test.go} | 10 +- ....go => saverequest_outcome_metric_test.go} | 124 ++--- ...st_ttl_test.go => saverequest_ttl_test.go} | 120 ++--- internal/controller/status.go | 4 +- internal/controller/suite_test.go | 2 +- internal/git/applier_claim.go | 12 +- internal/git/applier_claim_test.go | 2 +- internal/git/apply_attempt_test.go | 4 +- internal/git/apply_converge_test.go | 48 +- internal/git/attribution_outcome_test.go | 36 +- internal/git/branch_worker.go | 132 +++--- internal/git/branch_worker_empty_repo_test.go | 2 +- internal/git/branch_worker_metrics_test.go | 14 +- .../git/branch_worker_queue_depth_test.go | 2 +- internal/git/commit.go | 4 +- internal/git/commit_executor.go | 50 +- internal/git/commit_executor_test.go | 6 +- internal/git/commit_window_metrics_test.go | 2 +- internal/git/commit_window_timers_test.go | 8 +- internal/git/decided_write_test.go | 4 +- internal/git/dirty_worktree_recovery_test.go | 2 +- internal/git/intake_test.go | 8 +- internal/git/network_bound_test.go | 4 +- internal/git/open_window.go | 26 +- internal/git/parent_recovery_test.go | 10 +- internal/git/pending_writes.go | 16 +- internal/git/pending_writes_loop.go | 12 +- internal/git/render_fidelity_gate.go | 2 +- internal/git/render_fidelity_gate_test.go | 4 +- internal/git/replay_failure_test.go | 12 +- internal/git/replay_refusal_test.go | 6 +- internal/git/request_template_test.go | 14 +- internal/git/resync_push_test.go | 6 +- internal/git/retry.go | 2 +- internal/git/retry_test.go | 8 +- internal/git/save_apply.go | 40 +- ...quest_attach.go => save_request_attach.go} | 70 +-- ...ch_loop.go => save_request_attach_loop.go} | 158 +++---- ...ch_test.go => save_request_attach_test.go} | 44 +- ...pty_test.go => save_request_empty_test.go} | 4 +- ...quest_owners.go => save_request_owners.go} | 14 +- ..._test.go => save_request_withdraw_test.go} | 60 +-- internal/git/types.go | 30 +- internal/git/worker_manager.go | 20 +- internal/git/worker_repo_identity_test.go | 2 +- internal/git/write_gate.go | 6 +- internal/git/write_gate_test.go | 36 +- internal/telemetry/exporter.go | 24 +- internal/watch/event_router.go | 52 +-- internal/watch/event_router_test.go | 66 +-- ...ve_requests.go => remote_save_requests.go} | 222 ++++----- ...s_test.go => remote_save_requests_test.go} | 94 ++-- internal/watch/resource_policy.go | 2 +- internal/watch/target_watch.go | 2 +- .../validate_operator_types_handler.go | 10 +- .../validate_operator_types_handler_test.go | 18 +- test/e2e/Taskfile-e2e.yml | 2 +- test/e2e/bi_directional_echoes_e2e_test.go | 10 +- test/e2e/cluster/audit/policy.yaml | 4 +- test/e2e/e2e_suite_test.go | 20 +- test/e2e/flux_applier_e2e_test.go | 18 +- test/e2e/helpers.go | 10 +- ...t_e2e_test.go => save_request_e2e_test.go} | 222 ++++----- test/e2e/source_cluster_e2e_test.go | 60 +-- 157 files changed, 2346 insertions(+), 2289 deletions(-) rename api/source/v1alpha1/{saverequest_types.go => remotesaverequest_types.go} (83%) rename api/v1alpha3/{commitrequest_types.go => saverequest_types.go} (91%) rename config/crd/bases/{configbutler.ai_commitrequests.yaml => configbutler.ai_saverequests.yaml} (96%) rename config/crd/bases/{source.configbutler.ai_saverequests.yaml => source.configbutler.ai_remotesaverequests.yaml} (95%) rename config/samples/{commitrequest.yaml => saverequest.yaml} (94%) rename docs/spec/{commitrequest-admission-authorship.md => saverequest-admission-authorship.md} (87%) rename docs/spec/{commitrequest-design.md => saverequest-design.md} (96%) rename internal/controller/{commitrequest_apply.go => saverequest_apply.go} (77%) rename internal/controller/{commitrequest_apply_test.go => saverequest_apply_test.go} (59%) rename internal/controller/{commitrequest_attribution_agreement_test.go => saverequest_attribution_agreement_test.go} (86%) rename internal/controller/{commitrequest_controller.go => saverequest_controller.go} (62%) rename internal/controller/{commitrequest_controller_test.go => saverequest_controller_test.go} (81%) rename internal/controller/{commitrequest_controller_unit_test.go => saverequest_controller_unit_test.go} (69%) rename internal/controller/{commitrequest_finalize.go => saverequest_finalize.go} (66%) rename internal/controller/{commitrequest_kstatus_test.go => saverequest_kstatus_test.go} (92%) rename internal/controller/{commitrequest_outcome_metric_test.go => saverequest_outcome_metric_test.go} (74%) rename internal/controller/{commitrequest_ttl_test.go => saverequest_ttl_test.go} (51%) rename internal/git/{commit_request_attach.go => save_request_attach.go} (85%) rename internal/git/{commit_request_attach_loop.go => save_request_attach_loop.go} (74%) rename internal/git/{commit_request_attach_test.go => save_request_attach_test.go} (96%) rename internal/git/{commit_request_empty_test.go => save_request_empty_test.go} (99%) rename internal/git/{commit_request_owners.go => save_request_owners.go} (80%) rename internal/git/{commit_request_withdraw_test.go => save_request_withdraw_test.go} (83%) rename internal/watch/{save_requests.go => remote_save_requests.go} (68%) rename internal/watch/{save_requests_test.go => remote_save_requests_test.go} (76%) rename test/e2e/{commit_request_e2e_test.go => save_request_e2e_test.go} (72%) diff --git a/.vale/styles/HouseStyle/Headings.yml b/.vale/styles/HouseStyle/Headings.yml index a42f4758..e9967cbe 100644 --- a/.vale/styles/HouseStyle/Headings.yml +++ b/.vale/styles/HouseStyle/Headings.yml @@ -54,7 +54,8 @@ exceptions: - WatchRule - ClusterWatchRule - KubeProvider - - CommitRequest + - SaveRequest + - RemoteSaveRequest - APIResourceCatalog - HelmRelease - Kustomization diff --git a/README.md b/README.md index 71582845..c77e69e4 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ What sets it apart: a commit names the identity the API server authenticated, and no field lets anyone type a name in. It needs kube-apiserver flags, so managed control planes that hide them cannot use it. See [attribution](docs/spec/attribution.md#authorship-is-established-never-asserted). -- **A commit can say why.** Changes are grouped into commit windows, and a `CommitRequest` lets the +- **A commit can say why.** Changes are grouped into commit windows, and a `SaveRequest` lets the person making a change explain **why** it was necessary. See [commit windows](#commit-windows). ## How it works @@ -94,7 +94,7 @@ renders: the operator cannot tell authored resources from generated ones. See ### Commit windows Changes for the same target and author share a commit window, and each window becomes one commit. -When a window closes is yours to tune: its timers are set per `GitTarget`, and a `CommitRequest` can +When a window closes is yours to tune: its timers are set per `GitTarget`, and a `SaveRequest` can bring its own timers along with the commit message. ![Kubernetes resource changes flow through GitOps Reverser's commit window into Git](docs/images/commit-window.excalidraw.svg) diff --git a/api/source/v1alpha1/saverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go similarity index 83% rename from api/source/v1alpha1/saverequest_types.go rename to api/source/v1alpha1/remotesaverequest_types.go index 5eda3f94..40d388ae 100644 --- a/api/source/v1alpha1/saverequest_types.go +++ b/api/source/v1alpha1/remotesaverequest_types.go @@ -6,18 +6,18 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -// A SaveRequest is the source-cluster twin of a configbutler.ai CommitRequest, for the people who edit +// A RemoteSaveRequest is the source-cluster twin of a configbutler.ai SaveRequest, for the people who edit // a source cluster and cannot reach the cluster GitOps Reverser runs in. It is its own kind, in its own -// group, rather than a CommitRequest with another schema: it names no GitTarget, because its creator +// group, rather than a SaveRequest with another schema: it names no GitTarget, because its creator // cannot see them, its author comes from the source cluster's audit events rather than from admission, // and it is installed alone. docs/future/source-side-save-plan.md has the reasoning. // -// The spec is immutable, compared as CommitRequestSpec compares it: durations as durations, so a typed +// The spec is immutable, compared as SaveRequestSpec compares it: durations as durations, so a typed // client's re-spelling of "1m" as "1m0s" is not a change. -// SaveRequestSpec defines the desired state of SaveRequest. It is immutable. -// +kubebuilder:validation:XValidation:rule="has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="SaveRequest spec is immutable after creation" -type SaveRequestSpec struct { +// RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It is immutable. +// +kubebuilder:validation:XValidation:rule="has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="RemoteSaveRequest spec is immutable after creation" +type RemoteSaveRequestSpec struct { // Message is the commit message this request asks for. Omitted, each // GitTarget's spec.commit.message.liveTemplate words the commit. // @@ -43,7 +43,7 @@ type SaveRequestSpec struct { // request waits 5s and closes the window 2s after it attached. // +optional // +kubebuilder:default={attachTimeout: "5s", maxDuration: "2s"} - Window *SaveRequestWindow `json:"window,omitempty"` + Window *RemoteSaveRequestWindow `json:"window,omitempty"` // CatchUp decides when the applier of each GitTarget is asked to apply // the branch after this request pushes a commit. "AsDefinedOnClaim" @@ -57,7 +57,7 @@ type SaveRequestSpec struct { CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } -// CatchUpPolicy is when a SaveRequest's commit is applied by a GitTarget's +// CatchUpPolicy is when a RemoteSaveRequest's commit is applied by a GitTarget's // applier. type CatchUpPolicy string @@ -68,12 +68,12 @@ const ( CatchUpAfterPush CatchUpPolicy = "AfterPush" ) -// SaveRequestWindow is how long a SaveRequest waits for a commit window, and +// RemoteSaveRequestWindow is how long a RemoteSaveRequest waits for a commit window, and // when that window closes. A window is only ever opened by an edit; a // request waits for one and never opens one. -// +kubebuilder:validation:XValidation:rule="duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="SaveRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="RemoteSaveRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" -type SaveRequestWindow struct { +type RemoteSaveRequestWindow struct { // AttachTimeout is how long the request waits for a commit window, as a // Go duration, from when GitOps Reverser picks the request up, which is // after its own author was found. A request whose wait ran out never @@ -105,8 +105,8 @@ type SaveRequestWindow struct { MaxDuration *metav1.Duration `json:"maxDuration,omitempty"` } -// SaveRequestStatus defines the observed state of SaveRequest. -type SaveRequestStatus struct { +// RemoteSaveRequestStatus defines the observed state of RemoteSaveRequest. +type RemoteSaveRequestStatus struct { // ObservedGeneration is the latest generation observed by GitOps Reverser. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty"` @@ -121,9 +121,9 @@ type SaveRequestStatus struct { // +optional // +listType=atomic // +kubebuilder:validation:MaxItems=64 - Targets []SaveRequestTarget `json:"targets,omitempty"` + Targets []RemoteSaveRequestTarget `json:"targets,omitempty"` - // Conditions are the latest observations of this SaveRequest's state. + // Conditions are the latest observations of this RemoteSaveRequest's state. // Ready is True once the request ended without error in every GitTarget, // with reason "Committed" when one of them committed, and otherwise the // reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", @@ -142,8 +142,8 @@ type SaveRequestStatus struct { Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` } -// SaveRequestTarget is the save in one GitTarget. -type SaveRequestTarget struct { +// RemoteSaveRequestTarget is the save in one GitTarget. +type RemoteSaveRequestTarget struct { // GitTarget is the GitTarget, as namespace/name on the cluster GitOps // Reverser runs in. // +required @@ -186,40 +186,40 @@ type SaveRequestTarget struct { // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// SaveRequest asks GitOps Reverser to save the edits made in its namespace: +// RemoteSaveRequest asks GitOps Reverser to save the edits made in its namespace: // the commit window of each GitTarget that mirrors this namespace gets its // message and closes on its timers. Create it on the cluster the edits were // made on, as JSON. Its creator is found from this cluster's audit events, // as the edits' authors are, so it saves only its creator's edits; on a // cluster with no audit it names nobody, and saves edits that name nobody. // Its spec is immutable, and status reports each GitTarget's commit. -type SaveRequest struct { +type RemoteSaveRequest struct { metav1.TypeMeta `json:",inline"` // metadata is a standard object metadata // +optional metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` - // spec defines the desired state of SaveRequest. It is immutable after + // spec defines the desired state of RemoteSaveRequest. It is immutable after // creation. // +required - Spec SaveRequestSpec `json:"spec"` + Spec RemoteSaveRequestSpec `json:"spec"` - // status defines the observed state of SaveRequest. + // status defines the observed state of RemoteSaveRequest. // +optional - Status SaveRequestStatus `json:"status,omitempty,omitzero"` + Status RemoteSaveRequestStatus `json:"status,omitempty,omitzero"` } // +kubebuilder:object:root=true -// SaveRequestList contains a list of SaveRequest. -type SaveRequestList struct { +// RemoteSaveRequestList contains a list of RemoteSaveRequest. +type RemoteSaveRequestList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` - Items []SaveRequest `json:"items"` + Items []RemoteSaveRequest `json:"items"` } func init() { - SchemeBuilder.Register(&SaveRequest{}, &SaveRequestList{}) + SchemeBuilder.Register(&RemoteSaveRequest{}, &RemoteSaveRequestList{}) } diff --git a/api/source/v1alpha1/zz_generated.deepcopy.go b/api/source/v1alpha1/zz_generated.deepcopy.go index 6ec0b27f..36ca5bd0 100644 --- a/api/source/v1alpha1/zz_generated.deepcopy.go +++ b/api/source/v1alpha1/zz_generated.deepcopy.go @@ -12,7 +12,7 @@ import ( ) // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequest) DeepCopyInto(out *SaveRequest) { +func (in *RemoteSaveRequest) DeepCopyInto(out *RemoteSaveRequest) { *out = *in out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) @@ -20,18 +20,18 @@ func (in *SaveRequest) DeepCopyInto(out *SaveRequest) { in.Status.DeepCopyInto(&out.Status) } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequest. -func (in *SaveRequest) DeepCopy() *SaveRequest { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequest. +func (in *RemoteSaveRequest) DeepCopy() *RemoteSaveRequest { if in == nil { return nil } - out := new(SaveRequest) + out := new(RemoteSaveRequest) in.DeepCopyInto(out) return out } // DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *SaveRequest) DeepCopyObject() runtime.Object { +func (in *RemoteSaveRequest) DeepCopyObject() runtime.Object { if c := in.DeepCopy(); c != nil { return c } @@ -39,31 +39,31 @@ func (in *SaveRequest) DeepCopyObject() runtime.Object { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestList) DeepCopyInto(out *SaveRequestList) { +func (in *RemoteSaveRequestList) DeepCopyInto(out *RemoteSaveRequestList) { *out = *in out.TypeMeta = in.TypeMeta in.ListMeta.DeepCopyInto(&out.ListMeta) if in.Items != nil { in, out := &in.Items, &out.Items - *out = make([]SaveRequest, len(*in)) + *out = make([]RemoteSaveRequest, len(*in)) for i := range *in { (*in)[i].DeepCopyInto(&(*out)[i]) } } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestList. -func (in *SaveRequestList) DeepCopy() *SaveRequestList { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestList. +func (in *RemoteSaveRequestList) DeepCopy() *RemoteSaveRequestList { if in == nil { return nil } - out := new(SaveRequestList) + out := new(RemoteSaveRequestList) in.DeepCopyInto(out) return out } // DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *SaveRequestList) DeepCopyObject() runtime.Object { +func (in *RemoteSaveRequestList) DeepCopyObject() runtime.Object { if c := in.DeepCopy(); c != nil { return c } @@ -71,31 +71,31 @@ func (in *SaveRequestList) DeepCopyObject() runtime.Object { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestSpec) DeepCopyInto(out *SaveRequestSpec) { +func (in *RemoteSaveRequestSpec) DeepCopyInto(out *RemoteSaveRequestSpec) { *out = *in if in.Window != nil { in, out := &in.Window, &out.Window - *out = new(SaveRequestWindow) + *out = new(RemoteSaveRequestWindow) (*in).DeepCopyInto(*out) } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestSpec. -func (in *SaveRequestSpec) DeepCopy() *SaveRequestSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestSpec. +func (in *RemoteSaveRequestSpec) DeepCopy() *RemoteSaveRequestSpec { if in == nil { return nil } - out := new(SaveRequestSpec) + out := new(RemoteSaveRequestSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestStatus) DeepCopyInto(out *SaveRequestStatus) { +func (in *RemoteSaveRequestStatus) DeepCopyInto(out *RemoteSaveRequestStatus) { *out = *in if in.Targets != nil { in, out := &in.Targets, &out.Targets - *out = make([]SaveRequestTarget, len(*in)) + *out = make([]RemoteSaveRequestTarget, len(*in)) copy(*out, *in) } if in.Conditions != nil { @@ -107,33 +107,33 @@ func (in *SaveRequestStatus) DeepCopyInto(out *SaveRequestStatus) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestStatus. -func (in *SaveRequestStatus) DeepCopy() *SaveRequestStatus { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestStatus. +func (in *RemoteSaveRequestStatus) DeepCopy() *RemoteSaveRequestStatus { if in == nil { return nil } - out := new(SaveRequestStatus) + out := new(RemoteSaveRequestStatus) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestTarget) DeepCopyInto(out *SaveRequestTarget) { +func (in *RemoteSaveRequestTarget) DeepCopyInto(out *RemoteSaveRequestTarget) { *out = *in } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestTarget. -func (in *SaveRequestTarget) DeepCopy() *SaveRequestTarget { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestTarget. +func (in *RemoteSaveRequestTarget) DeepCopy() *RemoteSaveRequestTarget { if in == nil { return nil } - out := new(SaveRequestTarget) + out := new(RemoteSaveRequestTarget) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestWindow) DeepCopyInto(out *SaveRequestWindow) { +func (in *RemoteSaveRequestWindow) DeepCopyInto(out *RemoteSaveRequestWindow) { *out = *in if in.AttachTimeout != nil { in, out := &in.AttachTimeout, &out.AttachTimeout @@ -152,12 +152,12 @@ func (in *SaveRequestWindow) DeepCopyInto(out *SaveRequestWindow) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestWindow. -func (in *SaveRequestWindow) DeepCopy() *SaveRequestWindow { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestWindow. +func (in *RemoteSaveRequestWindow) DeepCopy() *RemoteSaveRequestWindow { if in == nil { return nil } - out := new(SaveRequestWindow) + out := new(RemoteSaveRequestWindow) in.DeepCopyInto(out) return out } diff --git a/api/v1alpha3/argocdclaim_types.go b/api/v1alpha3/argocdclaim_types.go index eeedf011..392e738e 100644 --- a/api/v1alpha3/argocdclaim_types.go +++ b/api/v1alpha3/argocdclaim_types.go @@ -186,7 +186,7 @@ type ArgoCDClaimSource struct { // - after objects were created through the API in its folders; // - once at startup, and when a GitTarget first names this ArgoCDClaim; // - once spec.catchUpAfter passed since a push of GitOps Reverser's own; -// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. +// - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type ArgoCDClaim struct { diff --git a/api/v1alpha3/examples_test.go b/api/v1alpha3/examples_test.go index d1a3c23f..f0d94ee3 100644 --- a/api/v1alpha3/examples_test.go +++ b/api/v1alpha3/examples_test.go @@ -137,8 +137,8 @@ func newExampleObject(kind string) any { return &WatchRule{} case "ClusterWatchRule": return &ClusterWatchRule{} - case "CommitRequest": - return &CommitRequest{} + case "SaveRequest": + return &SaveRequest{} case ApplierClaimKindFlux: return &FluxClaim{} case ApplierClaimKindArgoCD: diff --git a/api/v1alpha3/fluxclaim_types.go b/api/v1alpha3/fluxclaim_types.go index e50554f2..83930e73 100644 --- a/api/v1alpha3/fluxclaim_types.go +++ b/api/v1alpha3/fluxclaim_types.go @@ -192,7 +192,7 @@ type FluxClaimSource struct { // - after objects were created through the API in its folders; // - once at startup, and when a GitTarget first names this FluxClaim; // - once spec.catchUpAfter passed since a push of GitOps Reverser's own; -// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. +// - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type FluxClaim struct { diff --git a/api/v1alpha3/gitprovider_types.go b/api/v1alpha3/gitprovider_types.go index 2de1033a..01d91771 100644 --- a/api/v1alpha3/gitprovider_types.go +++ b/api/v1alpha3/gitprovider_types.go @@ -224,21 +224,21 @@ type CommitMessageSpec struct { // can be empty, and Generation is 0 for a kind without a spec, so guard // them with {{with}}. // - // A CommitRequest's message replaces this template without changing the + // A SaveRequest's message replaces this template without changing the // author; spec.commit.message.requestTemplate can wrap that message. // +optional LiveTemplate string `json:"liveTemplate,omitempty"` // Rationale, kept out of the doc comment (and so out of the CRD schema) by the blank line - // below: a CommitRequest is created by whoever holds create on it in a namespace, so its + // below: a SaveRequest is created by whoever holds create on it in a namespace, so its // author supplies the message but must not set commit-message policy. Framing the message // rather than parsing it keeps that split — the operator owns the wording, the requester owns // the words, and nothing a requester writes is ever executed. Rejecting a template that never // renders RequestMessage is what keeps the split honest in the other direction: silently // discarding a requester's stated reason would be worse than having no template at all. - // RequestTemplate is the Go template that frames a CommitRequest's message. It - // is used only for a commit whose CommitRequest supplied a message, and gets + // RequestTemplate is the Go template that frames a SaveRequest's message. It + // is used only for a commit whose SaveRequest supplied a message, and gets // liveTemplate's fields plus RequestMessage, the message as written; template // syntax inside the message stays literal. Omitted, the message is committed // as written. A template that never renders RequestMessage is rejected, and diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 06779d1f..474a01c3 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -228,7 +228,7 @@ type GitTargetCommitSpec struct { // Window sets when this GitTarget's commit windows close. A window opens on an // author's first write and collects that author's writes until idleTimeout // passes without one, or maxDuration after it opened, whichever comes first. A - // change applies to windows opened afterward. A CommitRequest attached to a + // change applies to windows opened afterward. A SaveRequest attached to a // window replaces both timers for that window. // +optional Window *CommitWindow `json:"window,omitempty"` @@ -238,7 +238,7 @@ type GitTargetCommitSpec struct { // +optional Message *CommitMessageSpec `json:"message,omitempty"` - // UnattributedWindows says how much a CommitRequest on this GitTarget may + // UnattributedWindows says how much a SaveRequest on this GitTarget may // claim of a commit window whose writes name no author. It acts only while // the GitTarget's KubeProvider has never received an audit fact; on a // cluster that has, a write naming no author is never claimed. Defaults to @@ -261,7 +261,7 @@ type GitTargetCommitSpec struct { UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` } -// UnattributedWindowPolicy is how much a CommitRequest may claim of a commit +// UnattributedWindowPolicy is how much a SaveRequest may claim of a commit // window whose writes name no author. type UnattributedWindowPolicy string @@ -274,7 +274,7 @@ const ( UnattributedWindowsTakeOver UnattributedWindowPolicy = "TakeOver" ) -// Allows reports whether a CommitRequest asking for author may attach as it asks under this policy: +// Allows reports whether a SaveRequest asking for author may attach as it asks under this policy: // "TakeOver" allows every ask, "SavedBy" allows "SavedBy" and "Own", and "Keep" (or empty) allows only // "Own". An unknown value allows only "Own". func (p UnattributedWindowPolicy) Allows(author WindowAuthor) bool { diff --git a/api/v1alpha3/kubeprovider_types.go b/api/v1alpha3/kubeprovider_types.go index fb1d1ce2..da954183 100644 --- a/api/v1alpha3/kubeprovider_types.go +++ b/api/v1alpha3/kubeprovider_types.go @@ -100,29 +100,29 @@ type KubeProviderSpec struct { // +optional Attribution *KubeProviderAttribution `json:"attribution,omitempty"` - // SaveRequests says whether GitOps Reverser serves the SaveRequests + // RemoteSaveRequests says whether GitOps Reverser serves the RemoteSaveRequests // (source.configbutler.ai) created on this cluster. "Enabled" watches // them, saves the commit windows of the GitTargets that mirror each one's // namespace through this KubeProvider, and writes each one's status back. // Defaults to "Disabled". // // "Enabled" writes to this cluster: the credential needs get, list, watch - // and delete on saverequests, and update on saverequests/status. The - // SaveRequest CRD must be installed on the cluster. + // and delete on remotesaverequests, and update on remotesaverequests/status. The + // RemoteSaveRequest CRD must be installed on the cluster. // +optional // +kubebuilder:validation:Enum=Disabled;Enabled // +kubebuilder:default=Disabled - SaveRequests SaveRequestsMode `json:"saveRequests,omitempty"` + RemoteSaveRequests RemoteSaveRequestsMode `json:"remoteSaveRequests,omitempty"` } -// SaveRequestsMode is whether a KubeProvider's SaveRequests are served. -type SaveRequestsMode string +// RemoteSaveRequestsMode is whether a KubeProvider's RemoteSaveRequests are served. +type RemoteSaveRequestsMode string const ( - // SaveRequestsDisabled leaves the cluster's SaveRequests alone. - SaveRequestsDisabled SaveRequestsMode = "Disabled" - // SaveRequestsEnabled serves the cluster's SaveRequests. - SaveRequestsEnabled SaveRequestsMode = "Enabled" + // RemoteSaveRequestsDisabled leaves the cluster's RemoteSaveRequests alone. + RemoteSaveRequestsDisabled RemoteSaveRequestsMode = "Disabled" + // RemoteSaveRequestsEnabled serves the cluster's RemoteSaveRequests. + RemoteSaveRequestsEnabled RemoteSaveRequestsMode = "Enabled" ) // KubeProviderInCluster selects the operator's own identity. It has no fields: diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/saverequest_types.go similarity index 91% rename from api/v1alpha3/commitrequest_types.go rename to api/v1alpha3/saverequest_types.go index 73068df8..f334e902 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/saverequest_types.go @@ -7,7 +7,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -// CommitRequestSpec defines the desired state of CommitRequest. The spec is +// SaveRequestSpec defines the desired state of SaveRequest. The spec is // immutable after creation: a CEL validation rule rejects any update that // changes it, so a delayed audit event always acts on the spec the object was // created with. @@ -20,11 +20,11 @@ import ( // does not round-trip as one: a request created with "1m" reads into Go as a // time.Duration and serializes back as "1m0s", so every typed update — // including one that only touches metadata — was rejected as a spec change. -// CommitRequestWindow compares its durations as durations, which compares what +// SaveRequestWindow compares its durations as durations, which compares what // the fields mean instead of how they were spelled. -// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="CommitRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="SaveRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.whenNothingToCommit) || self.whenNothingToCommit != 'CommitEmpty' || has(self.message)",message="whenNothingToCommit: CommitEmpty requires spec.message, because a message is all an empty commit records" -type CommitRequestSpec struct { +type SaveRequestSpec struct { // GitTargetRef names the GitTarget, in this namespace, whose commit window // this request attaches to. // +required @@ -58,7 +58,7 @@ type CommitRequestSpec struct { // window within 2s and closes it 2s later. // +optional // +kubebuilder:default={attach: "CurrentOrNext", author: "Own", attachTimeout: "2s", maxDuration: "2s"} - Window *CommitRequestWindow `json:"window,omitempty"` + Window *SaveRequestWindow `json:"window,omitempty"` // WhenNothingToCommit decides what happens when the request ends with nothing // to commit: attachTimeout ran out with no window, or the window closed @@ -86,7 +86,7 @@ type CommitRequestSpec struct { CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } -// CatchUpPolicy is when a CommitRequest's commit is applied by the GitTarget's +// CatchUpPolicy is when a SaveRequest's commit is applied by the GitTarget's // applier. type CatchUpPolicy string @@ -97,7 +97,7 @@ const ( CatchUpAfterPush CatchUpPolicy = "AfterPush" ) -// NothingToCommitAction is what a CommitRequest does when it ends with nothing +// NothingToCommitAction is what a SaveRequest does when it ends with nothing // to commit. type NothingToCommitAction string @@ -108,7 +108,7 @@ const ( NothingToCommitCommitEmpty NothingToCommitAction = "CommitEmpty" ) -// AttachPolicy selects the commit window a CommitRequest attaches to. +// AttachPolicy selects the commit window a SaveRequest attaches to. type AttachPolicy string const ( @@ -121,7 +121,7 @@ const ( AttachNext AttachPolicy = "Next" ) -// WindowAuthor is whose commit window a CommitRequest may attach to. +// WindowAuthor is whose commit window a SaveRequest may attach to. type WindowAuthor string const ( @@ -135,17 +135,17 @@ const ( WindowAuthorTakeOver WindowAuthor = "TakeOver" ) -// CommitRequestWindow is how a CommitRequest attaches to a commit window and +// SaveRequestWindow is how a SaveRequest attaches to a commit window and // when that window closes. A window is only ever opened by a write; a request // waits for one and never opens one. // // Every duration shares the shape GitTargetCommitSpec documents: a Go duration // string behind a pattern and a CEL bound. The immutability rule compares them -// as durations, for the reason CommitRequestSpec gives. -// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" +// as durations, for the reason SaveRequestSpec gives. +// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="SaveRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" // +kubebuilder:validation:XValidation:rule="self.attach != 'Next' || duration(self.attachTimeout) > duration('0s')",message="attach: Next needs a positive attachTimeout; with 0s it would close the author's window and give up in the same instant" -type CommitRequestWindow struct { +type SaveRequestWindow struct { // Attach selects the window. "CurrentOrNext" attaches to the author's open // window, or to the next one a write opens. "Next" closes the author's open // window under its own message first, and attaches to the next one; it @@ -199,7 +199,7 @@ type CommitRequestWindow struct { MaxDuration *metav1.Duration `json:"maxDuration,omitempty"` } -// CommitRequestStatus defines the observed state of CommitRequest. Progress +// SaveRequestStatus defines the observed state of SaveRequest. Progress // and outcome are reported entirely through conditions (kstatus-compatible), // so the object carries no lifecycle phase string: // @@ -221,12 +221,12 @@ type CommitRequestWindow struct { // - Pushed (domain): True once the commit is in the remote repository. // - Applied (domain): with spec.catchUp "AfterPush", whether the applier // caught up with the commit. It never affects Ready. -type CommitRequestStatus struct { +type SaveRequestStatus struct { // ObservedGeneration is the latest generation observed by the controller. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty"` - // Conditions are the latest observations of this CommitRequest's state. + // Conditions are the latest observations of this SaveRequest's state. // Reconciling and Stalled follow kstatus. AuthorAttributed says whether the // request's creator, recorded at admission, names the commit author; False // is not a failure and does not affect Ready. @@ -287,39 +287,39 @@ type CommitRequestStatus struct { // +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// CommitRequest asks for a save: it attaches to its author's current or next +// SaveRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and // gives the commit its message. With spec.window.author, and where the // GitTarget allows it, it may also claim a window whose writes name no // author. Its spec is immutable, and status.commit reports the commit once // it is on the remote. -type CommitRequest struct { +type SaveRequest struct { metav1.TypeMeta `json:",inline"` // metadata is a standard object metadata // +optional metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` - // spec defines the desired state of CommitRequest. It is immutable after + // spec defines the desired state of SaveRequest. It is immutable after // creation. // +required - Spec CommitRequestSpec `json:"spec"` + Spec SaveRequestSpec `json:"spec"` - // status defines the observed state of CommitRequest. + // status defines the observed state of SaveRequest. // +optional - Status CommitRequestStatus `json:"status,omitempty,omitzero"` + Status SaveRequestStatus `json:"status,omitempty,omitzero"` } // +kubebuilder:object:root=true -// CommitRequestList contains a list of CommitRequest. -type CommitRequestList struct { +// SaveRequestList contains a list of SaveRequest. +type SaveRequestList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` - Items []CommitRequest `json:"items"` + Items []SaveRequest `json:"items"` } func init() { - SchemeBuilder.Register(&CommitRequest{}, &CommitRequestList{}) + SchemeBuilder.Register(&SaveRequest{}, &SaveRequestList{}) } diff --git a/api/v1alpha3/status_accessors.go b/api/v1alpha3/status_accessors.go index e813db1a..1480409f 100644 --- a/api/v1alpha3/status_accessors.go +++ b/api/v1alpha3/status_accessors.go @@ -16,7 +16,7 @@ import ( // accessors let the session do it once. See internal/controller's statusObject, which is the // interface these satisfy. // -// CommitRequest is deliberately not here: it does not use that helper (see the comment on +// SaveRequest is deliberately not here: it does not use that helper (see the comment on // requeueAfter for why), so an accessor on it would be a method nothing calls. // SetObservedGeneration records the generation this object's status describes. diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 032972e0..cc3ec5d1 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -370,138 +370,6 @@ func (in *CommitMessageSpec) DeepCopy() *CommitMessageSpec { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CommitRequest) DeepCopyInto(out *CommitRequest) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequest. -func (in *CommitRequest) DeepCopy() *CommitRequest { - if in == nil { - return nil - } - out := new(CommitRequest) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *CommitRequest) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CommitRequestList) DeepCopyInto(out *CommitRequestList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]CommitRequest, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestList. -func (in *CommitRequestList) DeepCopy() *CommitRequestList { - if in == nil { - return nil - } - out := new(CommitRequestList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *CommitRequestList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CommitRequestSpec) DeepCopyInto(out *CommitRequestSpec) { - *out = *in - out.GitTargetRef = in.GitTargetRef - if in.Window != nil { - in, out := &in.Window, &out.Window - *out = new(CommitRequestWindow) - (*in).DeepCopyInto(*out) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestSpec. -func (in *CommitRequestSpec) DeepCopy() *CommitRequestSpec { - if in == nil { - return nil - } - out := new(CommitRequestSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CommitRequestStatus) DeepCopyInto(out *CommitRequestStatus) { - *out = *in - if in.Conditions != nil { - in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestStatus. -func (in *CommitRequestStatus) DeepCopy() *CommitRequestStatus { - if in == nil { - return nil - } - out := new(CommitRequestStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CommitRequestWindow) DeepCopyInto(out *CommitRequestWindow) { - *out = *in - if in.AttachTimeout != nil { - in, out := &in.AttachTimeout, &out.AttachTimeout - *out = new(v1.Duration) - **out = **in - } - if in.IdleTimeout != nil { - in, out := &in.IdleTimeout, &out.IdleTimeout - *out = new(v1.Duration) - **out = **in - } - if in.MaxDuration != nil { - in, out := &in.MaxDuration, &out.MaxDuration - *out = new(v1.Duration) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestWindow. -func (in *CommitRequestWindow) DeepCopy() *CommitRequestWindow { - if in == nil { - return nil - } - out := new(CommitRequestWindow) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CommitSigningSpec) DeepCopyInto(out *CommitSigningSpec) { *out = *in @@ -1701,6 +1569,138 @@ func (in *ResourceRule) DeepCopy() *ResourceRule { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequest) DeepCopyInto(out *SaveRequest) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequest. +func (in *SaveRequest) DeepCopy() *SaveRequest { + if in == nil { + return nil + } + out := new(SaveRequest) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SaveRequest) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestList) DeepCopyInto(out *SaveRequestList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]SaveRequest, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestList. +func (in *SaveRequestList) DeepCopy() *SaveRequestList { + if in == nil { + return nil + } + out := new(SaveRequestList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SaveRequestList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestSpec) DeepCopyInto(out *SaveRequestSpec) { + *out = *in + out.GitTargetRef = in.GitTargetRef + if in.Window != nil { + in, out := &in.Window, &out.Window + *out = new(SaveRequestWindow) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestSpec. +func (in *SaveRequestSpec) DeepCopy() *SaveRequestSpec { + if in == nil { + return nil + } + out := new(SaveRequestSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestStatus) DeepCopyInto(out *SaveRequestStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestStatus. +func (in *SaveRequestStatus) DeepCopy() *SaveRequestStatus { + if in == nil { + return nil + } + out := new(SaveRequestStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SaveRequestWindow) DeepCopyInto(out *SaveRequestWindow) { + *out = *in + if in.AttachTimeout != nil { + in, out := &in.AttachTimeout, &out.AttachTimeout + *out = new(v1.Duration) + **out = **in + } + if in.IdleTimeout != nil { + in, out := &in.IdleTimeout, &out.IdleTimeout + *out = new(v1.Duration) + **out = **in + } + if in.MaxDuration != nil { + in, out := &in.MaxDuration, &out.MaxDuration + *out = new(v1.Duration) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestWindow. +func (in *SaveRequestWindow) DeepCopy() *SaveRequestWindow { + if in == nil { + return nil + } + out := new(SaveRequestWindow) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *StreamsStatus) DeepCopyInto(out *StreamsStatus) { *out = *in diff --git a/charts/gitops-reverser/README.md b/charts/gitops-reverser/README.md index c9c610ce..c5e6da11 100644 --- a/charts/gitops-reverser/README.md +++ b/charts/gitops-reverser/README.md @@ -19,7 +19,7 @@ For the chart's optional starter `quickstart` block, see [`docs/configuration.md - ✅ **Git synchronization**: Push Kubernetes changes back to Git repositories - ✅ **Single-pod stability**: 1 replica by default while multi-pod support is in progress - ✅ **Automatic CRD installation**: KubeProvider, GitProvider, GitBranch, GitTarget, WatchRule, - ClusterWatchRule, CommitRequest, FluxClaim, and ArgoCDClaim CRDs installed automatically + ClusterWatchRule, SaveRequest, FluxClaim, and ArgoCDClaim CRDs installed automatically - ✅ **Watch-based ingestion**: object state comes from the Kubernetes watch API - ✅ **Optional audit attribution**: name commit authors from kube-apiserver audit events over HTTPS - ✅ **Prometheus metrics**: Built-in monitoring support @@ -30,7 +30,7 @@ For the chart's optional starter `quickstart` block, see [`docs/configuration.md - cert-manager (for TLS certificates) - **Valkey or Redis: optional but advised.** The default install runs without it in `configured-author` mode (watches cold-replay on restart). Add it to unlock warm-restart resume - cursors, CommitRequest author capture (the admission webhook), and attributed-author mode; enabling + cursors, SaveRequest author capture (the admission webhook), and attributed-author mode; enabling any of those requires a non-empty `queue.redis.addr`. - **Optional, attributed-author mode only:** kube-apiserver audit delivery, which adds mirrored-resource commit-author attribution. Without it the operator still mirrors state, authored by @@ -182,7 +182,7 @@ nodeSelector: | `controllerManager.insecureAllowGitHTTP` | Permit credentials with `http://` GitProvider URLs for trusted in-cluster development Git servers | `false` | | `controllerManager.branchWorkerQueueDepth` | Event queue depth per branch worker. A full queue refuses the write (`git_queue_refusals_total`) instead of stalling the watch path; the watch delivers it again from its cursor, and only a cursor that expires first loses that write's own commit (and, under `prune.mode: OnEvent`, a delete's removal). Costs queue depth × payload of pod memory, which `branchBufferMaxSize` does **not** cover | `1000` | | `controllerManager.branchBufferMaxSize` | Maximum in-memory event buffer per branch worker. Bounds the open commit window plus the writes retained for replay until a push succeeds. While the remote cannot be reached, retained writes that reach it close the worker to new writes, saves and resyncs until they are published; the watch delivers a refused event again | `8Mi` | -| `controllerManager.commitRequestTTL` | How long a `CommitRequest` is kept once it finishes. Written onto the request as the `configbutler.ai/delete-after` annotation; remove or edit that to keep one. `"0"` writes none | `48h` | +| `controllerManager.saveRequestTTL` | How long a `SaveRequest` is kept once it finishes. Written onto the request as the `configbutler.ai/delete-after` annotation; remove or edit that to keep one. `"0"` writes none | `48h` | | `git.refreshInterval` | How often an idle `GitTarget` re-proves where its branch is on the Git remote. `"0"` disables it | `10m` | | `applierAccess.flux` | Grant the operator `get`, `list` and `patch` on Flux `Kustomizations`, `get` and `patch` on `GitRepositories`, `list` on `HelmReleases` and `HelmCharts` (to find what else applies a source), and `create` on `SelfSubjectAccessReviews`, so it can hold the `Kustomization` a `FluxClaim` names suspended and resume it for each apply | `false` | | `applierAccess.argocd` | Grant the operator `get`, `update` and `patch` on Argo CD `Applications`, and `create` on `SelfSubjectAccessReviews`, so it can hold the automated sync of the `Application` an `ArgoCDClaim` names off and start its syncs | `false` | @@ -198,7 +198,7 @@ nodeSelector: | `quickstart.gitBranch.branch` | Branch the starter `GitBranch` writes. Must match `quickstart.gitProvider.branchPatterns` | `main` | | `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget`; set `.` only to deliberately target the repo root | `live-cluster` | | `quickstart.watchRule.rules` | Rules used by the starter `WatchRule` | `configmaps create/update/delete` | -| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record CommitRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | +| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record SaveRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | | `queue.redis.auth.existingSecret` | Name of a pre-created Secret holding the Redis password (only used when `queue.redis.addr` is set) | `""` | | `queue.redis.auth.existingSecretKey` | Key within the Secret that holds the password | `password` | | `queue.redis.auth.username` | Optional Redis ACL username | `""` | @@ -220,7 +220,7 @@ nodeSelector: | `kubeProvider.secretRef.key` | Key within that Secret. Empty reads `value` then `value.yaml` | `""` | | `kubeProvider.allowedGitTargets` | Rendered as the provider's `spec.allowedGitTargets`: which namespaces' `GitTarget`s may mirror through it, and what. Each grant has a `consumerNamespaces` block (`from`: `Same`, `Names`, `Selector`, `All`), and `sourceNamespaces`, `clusterResources` or both beside it. **Unset, the field is omitted and the CRD defaults it to the release namespace mirroring its own**; a list is the whole policy, and `[]` admits nobody, the release namespace included: a refused `GitTarget` names `spec.allowedGitTargets`. `quickstart.enabled` adds a grant for the quickstart namespace, and [`values-playground.yaml`](values-playground.yaml) grants every namespace its own. With an `inCluster` provider a grant bypasses live namespace RBAC through the operator's identity | `null` | | `kubeProvider.allowedApplierClaims` | Rendered as the provider's `spec.allowedApplierClaims`: which namespaces' `FluxClaim`s and `ArgoCDClaim`s may hold an applier through it (`consumerNamespaces`), and in which `applierNamespaces` it may live, such as `flux-system`. Admission to mirror is no admission to apply. Unset, the CRD defaults it to the release namespace holding appliers in its own; `[]` admits nobody | `null` | -| `servers.admission.enabled` | Install the validate-operator-types admission webhook that captures CommitRequest authors (a form of author attribution). Enabled by default; a no-op until `queue.redis.addr` is set | `true` | +| `servers.admission.enabled` | Install the validate-operator-types admission webhook that captures SaveRequest authors (a form of author attribution). Enabled by default; a no-op until `queue.redis.addr` is set | `true` | | `rbac.create` | Create the manager ClusterRole and its binding | `true` | | `rbac.watchTypes.mode` | Which types a `WatchRule` may read. `any` grants cluster-wide read on everything, including every Secret in the cluster. `selected` grants read on `rbac.watchTypes.selected` only, so the reverser cannot list or watch Secrets (it keeps `get` on named Secrets it is pointed at). See [`docs/rbac.md`](../../docs/rbac.md) | `any` | | `rbac.watchTypes.selected` | Types to grant when `mode: selected`, as `{apiGroups, resources}` entries (verbs are always `get,list,watch`). Required and non-empty in that mode; `namespaces`, `customresourcedefinitions` and `apiservices` come from the manager role and must not be restated | `[]` | @@ -287,7 +287,7 @@ This chart automatically manages the following CRDs: - **`gittargets.configbutler.ai`** - Branch/path and optional encryption configuration - **`watchrules.configbutler.ai`** - Namespaced watch rules - **`clusterwatchrules.configbutler.ai`** - Cluster-scoped watch rules -- **`commitrequests.configbutler.ai`** - One-shot "save now" signals that finalize the open commit window +- **`saverequests.configbutler.ai`** - One-shot "save now" signals that finalize the open commit window - **`fluxclaims.configbutler.ai`** - Applier claims holding a Flux `Kustomization` for bi-directional GitOps - **`argocdclaims.configbutler.ai`** - Applier claims holding an Argo CD `Application` for bi-directional GitOps @@ -309,7 +309,7 @@ kubectl apply -f https://github.com/ConfigButler/gitops-reverser/releases/downlo To manually remove CRDs after uninstallation: ```bash -kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai commitrequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai +kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai saverequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai ``` Delete the applier claims while the operator still runs, before this ([Uninstallation](#uninstallation)). @@ -400,7 +400,7 @@ helm uninstall gitops-reverser --namespace gitops-reverser kubectl delete namespace gitops-reverser # Delete CRDs (optional, but removes all custom resources) -kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai commitrequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai +kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai saverequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai ``` ## Troubleshooting diff --git a/charts/gitops-reverser/templates/deployment.yaml b/charts/gitops-reverser/templates/deployment.yaml index 4f74ba4f..63df3993 100644 --- a/charts/gitops-reverser/templates/deployment.yaml +++ b/charts/gitops-reverser/templates/deployment.yaml @@ -111,7 +111,7 @@ spec: - --branch-worker-queue-depth={{ int .Values.controllerManager.branchWorkerQueueDepth }} - --branch-buffer-max-size={{ .Values.controllerManager.branchBufferMaxSize }} - --git-refresh-interval={{ .Values.git.refreshInterval }} - - --commit-request-ttl={{ .Values.controllerManager.commitRequestTTL }} + - --save-request-ttl={{ .Values.controllerManager.saveRequestTTL }} {{- if .Values.servers.receiver.enabled }} - --receiver-bind-address={{ .Values.servers.receiver.bindAddress }} {{- end }} diff --git a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml b/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml index 3d382ebf..cfdf2fb0 100644 --- a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml +++ b/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml @@ -2,10 +2,10 @@ --- # Two unrelated jobs on one endpoint, both narrow by construction. # -# It captures the authenticated submitter of our own command kinds (a CommitRequest today) +# It captures the authenticated submitter of our own command kinds (a SaveRequest today) # into the command-author Redis corner the controller reads back with no wait; a miss # leaves the request without a claimed actor (AuthorAttributed=False). See -# docs/spec/commitrequest-admission-authorship.md. +# docs/spec/saverequest-admission-authorship.md. # # It also rejects a WatchRule that would bring a second source namespace to a GitTarget # declaring spec.serializeNamespace: false. That rejection is FEEDBACK, not enforcement: @@ -32,7 +32,7 @@ webhooks: path: /validate-operator-types port: {{ .Values.servers.admission.port }} # Ignore, not Fail: a missed author capture must leave the request unnamed, never - # reject a user's CommitRequest, and the WatchRule check is feedback the write path + # reject a user's SaveRequest, and the WatchRule check is feedback the write path # already enforces on its own. failurePolicy: Ignore matchPolicy: Equivalent @@ -44,7 +44,7 @@ webhooks: operations: - CREATE resources: - - commitrequests + - saverequests scope: Namespaced # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it # was created, and that edit is the same mistake. diff --git a/charts/gitops-reverser/values.schema.json b/charts/gitops-reverser/values.schema.json index 4eb69f9d..aa69f10b 100644 --- a/charts/gitops-reverser/values.schema.json +++ b/charts/gitops-reverser/values.schema.json @@ -118,9 +118,9 @@ "type": "string", "description": "Maximum in-memory event buffer per branch worker, as a Kubernetes resource quantity (e.g. 8Mi). Bounds the open commit window plus the writes retained for replay until a push succeeds." }, - "commitRequestTTL": { + "saveRequestTTL": { "type": "string", - "description": "How long a CommitRequest is kept once it finishes, as a duration (e.g. 48h). Written onto the request as the configbutler.ai/delete-after annotation when it finishes; \"0\" writes none, so requests are kept." + "description": "How long a SaveRequest is kept once it finishes, as a duration (e.g. 48h). Written onto the request as the configbutler.ai/delete-after annotation when it finishes; \"0\" writes none, so requests are kept." } } }, diff --git a/charts/gitops-reverser/values.yaml b/charts/gitops-reverser/values.yaml index e255e890..5c23b7e2 100644 --- a/charts/gitops-reverser/values.yaml +++ b/charts/gitops-reverser/values.yaml @@ -97,12 +97,12 @@ controllerManager: # included: work that would exceed it pauses the worker's intake of new writes, saves and # resyncs until a push lands, and the watch delivers what was refused once intake reopens. branchBufferMaxSize: 8Mi - # How long a CommitRequest is kept once it finishes, as a duration. Every save leaves one object + # How long a SaveRequest is kept once it finishes, as a duration. Every save leaves one object # behind, so without this they accumulate for as long as the namespace lives. The controller # writes the deletion time onto the request as the annotation configbutler.ai/delete-after when # it finishes; remove or edit that annotation to keep one request or move its deletion. Set "0" # to write no annotation, so new requests are kept. - commitRequestTTL: 48h + saveRequestTTL: 48h # How the operator treats the Git remote. git: @@ -175,16 +175,16 @@ servers: secretNameOverride: "" # Validating admission server hosting the validate-operator-types webhook, which captures - # the submitter of each CommitRequest at admission and records it as the request's named actor + # the submitter of each SaveRequest at admission and records it as the request's named actor # (read back by the controller with no wait). Independent of `attribution` below: # attribution governs *mirrored-resource* authorship (from audit), this governs - # *command* authorship (from admission). Off → CommitRequests claim no actor and report + # *command* authorship (from admission). Off → SaveRequests claim no actor and report # AuthorAttributed=False. See - # docs/spec/commitrequest-admission-authorship.md. + # docs/spec/saverequest-admission-authorship.md. # # Command-author capture is Redis-backed: the captured author is written to Redis at admission and # read back at reconcile. Enabled by default so the webhook is always installed, but it is a no-op - # without Redis: the pod runs healthy and CommitRequests claim no actor + # without Redis: the pod runs healthy and SaveRequests claim no actor # (AuthorAttributed=False) until queue.redis.addr is set. admission: enabled: true diff --git a/cmd/main.go b/cmd/main.go index 59407426..3591a21a 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -331,15 +331,15 @@ func main() { // Setup watch manager (must be after controllers are set up) fatalIfErr(watchMgr.SetupWithManager(mgr), "unable to setup watch ingestion manager") fatalIfErr(mgr.Add(watchMgr), "unable to add watch ingestion manager") - // SaveRequests are saves made on a source cluster, next to the edits they save: served on every - // KubeProvider whose spec.saveRequests is Enabled, attributed from that cluster's audit facts. - fatalIfErr(mgr.Add(&watch.SaveRequestServer{ + // RemoteSaveRequests are saves made on a source cluster, next to the edits they save: served on every + // KubeProvider whose spec.remoteSaveRequests is Enabled, attributed from that cluster's audit facts. + fatalIfErr(mgr.Add(&watch.RemoteSaveRequestServer{ Manager: watchMgr, Client: mgr.GetClient(), Finalizer: eventRouter, - TTL: cfg.commitRequestTTL, - Log: ctrl.Log.WithName("saverequests"), - }), "unable to add the SaveRequest server") + TTL: cfg.saveRequestTTL, + Log: ctrl.Log.WithName("remotesaverequests"), + }), "unable to add the RemoteSaveRequest server") // The GitBranch, GitTarget and applier claim controllers find a GitProvider's GitBranches, in any // namespace, through this index. @@ -417,7 +417,7 @@ func main() { // lives in its own Redis corner (author:v1:command), independent of // --author-attribution (which governs mirrored-resource attribution). It is wired // whenever the admission server is on; the controller reads the captured submitter - // back with no wait (docs/spec/commitrequest-admission-authorship.md). + // back with no wait (docs/spec/saverequest-admission-authorship.md). // // AuthorLookup must be a nil interface — not a non-nil interface wrapping a nil // *CommandAuthorStore — when the webhook is off, so the controller's nil check @@ -433,19 +433,19 @@ func main() { "and recorded as the request's named actor") } else { setupLog.Info("validate-operator-types webhook enabled without Redis: command author capture is a " + - "no-op; CommitRequests claim no actor (AuthorAttributed=False). " + + "no-op; SaveRequests claim no actor (AuthorAttributed=False). " + "Set --redis-addr to capture command authors.") } } - if err := (&controller.CommitRequestReconciler{ + if err := (&controller.SaveRequestReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), APIReader: mgr.GetAPIReader(), Finalizer: eventRouter, AuthorLookup: commandAuthorLookup, - TTL: cfg.commitRequestTTL, + TTL: cfg.saveRequestTTL, }).SetupWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create controller", "controller", "CommitRequest") + setupLog.Error(err, "unable to create controller", "controller", "SaveRequest") os.Exit(1) } if cfg.admissionWebhookEnabled { @@ -505,7 +505,7 @@ type appConfig struct { attributionFactTTL time.Duration attributionGrace time.Duration gitRefreshInterval time.Duration - commitRequestTTL time.Duration + saveRequestTTL time.Duration attributionMaxFactsPerType int attributionMaxFacts int attributionCollectionWindow time.Duration @@ -596,7 +596,7 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { "run without Redis: watches cold-replay on restart instead of resuming, and attribution needs "+ "--author-attribution-transport=memory. Required by --author-attribution-transport=redis when "+ "attribution is on. --admission-webhook still runs without it, but command-author capture is a "+ - "no-op: CommitRequests claim no actor.") + "no-op: SaveRequests claim no actor.") fs.BoolVar(&cfg.authorAttribution, "author-attribution", true, "Name the real actor (human or service account) who caused each change as the Git commit author, "+ "resolved from matching audit facts; this runs the audit webhook ingress (default true). When "+ @@ -657,10 +657,10 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { "Bounded per-event wait for a matching audit fact to arrive before a watch event ships unresolved, "+ "authored \"unknown (attribution unresolved)\" (duration string; default 3s). Larger values "+ "raise attribution hit-rate at the cost of commit latency.") - fs.DurationVar(&cfg.commitRequestTTL, "commit-request-ttl", controller.DefaultCommitRequestTTL, - "How long a CommitRequest is kept once it finishes (duration string; default 48h, 0 keeps "+ + fs.DurationVar(&cfg.saveRequestTTL, "save-request-ttl", controller.DefaultSaveRequestTTL, + "How long a SaveRequest is kept once it finishes (duration string; default 48h, 0 keeps "+ "them). The deletion time is written onto the request as the "+ - controller.CommitRequestDeleteAfterAnnotation+" annotation when it finishes; remove or edit "+ + controller.SaveRequestDeleteAfterAnnotation+" annotation when it finishes; remove or edit "+ "that annotation to keep the request or move its deletion.") fs.DurationVar(&cfg.gitRefreshInterval, "git-refresh-interval", controller.DefaultGitRefreshInterval, "How often an IDLE GitTarget re-proves where its branch is on the remote (duration string; "+ @@ -769,8 +769,8 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { } // Rejected rather than read as "keep everything": a negative age has no meaning, and 0 already // says that. - if cfg.commitRequestTTL < 0 { - return appConfig{}, fmt.Errorf("commit-request-ttl must be >= 0, got %s", cfg.commitRequestTTL) + if cfg.saveRequestTTL < 0 { + return appConfig{}, fmt.Errorf("save-request-ttl must be >= 0, got %s", cfg.saveRequestTTL) } bufferQuantity, err := resource.ParseQuantity(branchBufferMaxSizeFlag) @@ -939,7 +939,7 @@ func validateAdmissionWebhookConfig(cfg appConfig) error { return nil } // No redis-addr requirement here: the admission webhook stays enabled without Redis and - // simply no-ops command-author capture (CommitRequests claim no actor). Redis is + // simply no-ops command-author capture (SaveRequests claim no actor). Redis is // wired only when configured; see the commandAuthorStore setup. if _, _, err := splitBindAddress(cfg.admissionWebhookBindAddress); err != nil { return fmt.Errorf("invalid admission-webhook-bind-address %q: %w", cfg.admissionWebhookBindAddress, err) diff --git a/config/crd/bases/configbutler.ai_argocdclaims.yaml b/config/crd/bases/configbutler.ai_argocdclaims.yaml index 55d99bb1..9a538d4a 100644 --- a/config/crd/bases/configbutler.ai_argocdclaims.yaml +++ b/config/crd/bases/configbutler.ai_argocdclaims.yaml @@ -68,7 +68,7 @@ spec: - after objects were created through the API in its folders; - once at startup, and when a GitTarget first names this ArgoCDClaim; - once spec.catchUpAfter passed since a push of GitOps Reverser's own; - - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. + - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: diff --git a/config/crd/bases/configbutler.ai_fluxclaims.yaml b/config/crd/bases/configbutler.ai_fluxclaims.yaml index 385b5edb..e9a650fe 100644 --- a/config/crd/bases/configbutler.ai_fluxclaims.yaml +++ b/config/crd/bases/configbutler.ai_fluxclaims.yaml @@ -68,7 +68,7 @@ spec: - after objects were created through the API in its folders; - once at startup, and when a GitTarget first names this FluxClaim; - once spec.catchUpAfter passed since a push of GitOps Reverser's own; - - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. + - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index c61d4911..5bd40d06 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -169,7 +169,7 @@ spec: can be empty, and Generation is 0 for a kind without a spec, so guard them with {{with}}. - A CommitRequest's message replaces this template without changing the + A SaveRequest's message replaces this template without changing the author; spec.commit.message.requestTemplate can wrap that message. type: string reconcileTemplate: @@ -183,8 +183,8 @@ spec: type: string requestTemplate: description: |- - RequestTemplate is the Go template that frames a CommitRequest's message. It - is used only for a commit whose CommitRequest supplied a message, and gets + RequestTemplate is the Go template that frames a SaveRequest's message. It + is used only for a commit whose SaveRequest supplied a message, and gets liveTemplate's fields plus RequestMessage, the message as written; template syntax inside the message stays literal. Omitted, the message is committed as written. A template that never renders RequestMessage is rejected, and @@ -194,7 +194,7 @@ spec: unattributedWindows: default: Keep description: |- - UnattributedWindows says how much a CommitRequest on this GitTarget may + UnattributedWindows says how much a SaveRequest on this GitTarget may claim of a commit window whose writes name no author. It acts only while the GitTarget's KubeProvider has never received an audit fact; on a cluster that has, a write naming no author is never claimed. Defaults to @@ -221,7 +221,7 @@ spec: Window sets when this GitTarget's commit windows close. A window opens on an author's first write and collects that author's writes until idleTimeout passes without one, or maxDuration after it opened, whichever comes first. A - change applies to windows opened afterward. A CommitRequest attached to a + change applies to windows opened afterward. A SaveRequest attached to a window replaces both timers for that window. properties: idleTimeout: diff --git a/config/crd/bases/configbutler.ai_kubeproviders.yaml b/config/crd/bases/configbutler.ai_kubeproviders.yaml index 69044e36..e605af68 100644 --- a/config/crd/bases/configbutler.ai_kubeproviders.yaml +++ b/config/crd/bases/configbutler.ai_kubeproviders.yaml @@ -509,18 +509,18 @@ spec: InClusterNotAuthorized, and is never used. Set exactly one of secretRef and inCluster. Immutable: delete and recreate the KubeProvider to change it. type: object - saveRequests: + remoteSaveRequests: default: Disabled description: |- - SaveRequests says whether GitOps Reverser serves the SaveRequests + RemoteSaveRequests says whether GitOps Reverser serves the RemoteSaveRequests (source.configbutler.ai) created on this cluster. "Enabled" watches them, saves the commit windows of the GitTargets that mirror each one's namespace through this KubeProvider, and writes each one's status back. Defaults to "Disabled". "Enabled" writes to this cluster: the credential needs get, list, watch - and delete on saverequests, and update on saverequests/status. The - SaveRequest CRD must be installed on the cluster. + and delete on remotesaverequests, and update on remotesaverequests/status. The + RemoteSaveRequest CRD must be installed on the cluster. enum: - Disabled - Enabled diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_saverequests.yaml similarity index 96% rename from config/crd/bases/configbutler.ai_commitrequests.yaml rename to config/crd/bases/configbutler.ai_saverequests.yaml index 37cf6e54..dbe5a4aa 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_saverequests.yaml @@ -4,14 +4,14 @@ kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.23.0 - name: commitrequests.configbutler.ai + name: saverequests.configbutler.ai spec: group: configbutler.ai names: - kind: CommitRequest - listKind: CommitRequestList - plural: commitrequests - singular: commitrequest + kind: SaveRequest + listKind: SaveRequestList + plural: saverequests + singular: saverequest scope: Namespaced versions: - additionalPrinterColumns: @@ -54,7 +54,7 @@ spec: schema: openAPIV3Schema: description: |- - CommitRequest asks for a save: it attaches to its author's current or next + SaveRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and gives the commit its message. With spec.window.author, and where the GitTarget allows it, it may also claim a window whose writes name no @@ -80,7 +80,7 @@ spec: type: object spec: description: |- - spec defines the desired state of CommitRequest. It is immutable after + spec defines the desired state of SaveRequest. It is immutable after creation. properties: catchUp: @@ -220,7 +220,7 @@ spec: rule: duration(self) <= duration('5m') type: object x-kubernetes-validations: - - message: CommitRequest spec is immutable after creation + - message: SaveRequest spec is immutable after creation rule: self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) @@ -235,7 +235,7 @@ spec: - gitTargetRef type: object x-kubernetes-validations: - - message: CommitRequest spec is immutable after creation + - message: SaveRequest spec is immutable after creation rule: self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) @@ -248,7 +248,7 @@ spec: rule: '!has(self.whenNothingToCommit) || self.whenNothingToCommit != ''CommitEmpty'' || has(self.message)' status: - description: status defines the observed state of CommitRequest. + description: status defines the observed state of SaveRequest. properties: appliedRevision: description: |- @@ -267,7 +267,7 @@ spec: type: string conditions: description: |- - Conditions are the latest observations of this CommitRequest's state. + Conditions are the latest observations of this SaveRequest's state. Reconciling and Stalled follow kstatus. AuthorAttributed says whether the request's creator, recorded at admission, names the commit author; False is not a failure and does not affect Ready. diff --git a/config/crd/bases/source.configbutler.ai_saverequests.yaml b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml similarity index 95% rename from config/crd/bases/source.configbutler.ai_saverequests.yaml rename to config/crd/bases/source.configbutler.ai_remotesaverequests.yaml index 04c4feef..13aa37d8 100644 --- a/config/crd/bases/source.configbutler.ai_saverequests.yaml +++ b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml @@ -4,14 +4,14 @@ kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.23.0 - name: saverequests.source.configbutler.ai + name: remotesaverequests.source.configbutler.ai spec: group: source.configbutler.ai names: - kind: SaveRequest - listKind: SaveRequestList - plural: saverequests - singular: saverequest + kind: RemoteSaveRequest + listKind: RemoteSaveRequestList + plural: remotesaverequests + singular: remotesaverequest scope: Namespaced versions: - additionalPrinterColumns: @@ -35,7 +35,7 @@ spec: schema: openAPIV3Schema: description: |- - SaveRequest asks GitOps Reverser to save the edits made in its namespace: + RemoteSaveRequest asks GitOps Reverser to save the edits made in its namespace: the commit window of each GitTarget that mirrors this namespace gets its message and closes on its timers. Create it on the cluster the edits were made on, as JSON. Its creator is found from this cluster's audit events, @@ -62,7 +62,7 @@ spec: type: object spec: description: |- - spec defines the desired state of SaveRequest. It is immutable after + spec defines the desired state of RemoteSaveRequest. It is immutable after creation. properties: catchUp: @@ -141,7 +141,7 @@ spec: rule: duration(self) <= duration('5m') type: object x-kubernetes-validations: - - message: SaveRequest spec is immutable after creation + - message: RemoteSaveRequest spec is immutable after creation rule: duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) @@ -150,13 +150,13 @@ spec: rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' type: object x-kubernetes-validations: - - message: SaveRequest spec is immutable after creation + - message: RemoteSaveRequest spec is immutable after creation rule: has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp) status: - description: status defines the observed state of SaveRequest. + description: status defines the observed state of RemoteSaveRequest. properties: author: description: |- @@ -166,7 +166,7 @@ spec: type: string conditions: description: |- - Conditions are the latest observations of this SaveRequest's state. + Conditions are the latest observations of this RemoteSaveRequest's state. Ready is True once the request ended without error in every GitTarget, with reason "Committed" when one of them committed, and otherwise the reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", @@ -244,7 +244,7 @@ spec: description: Targets is the save in each GitTarget that mirrors this namespace. items: - description: SaveRequestTarget is the save in one GitTarget. + description: RemoteSaveRequestTarget is the save in one GitTarget. properties: applied: description: |- diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index a34418c5..871c183f 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -3,12 +3,12 @@ kind: Kustomization resources: - bases/configbutler.ai_argocdclaims.yaml - bases/configbutler.ai_clusterwatchrules.yaml - - bases/configbutler.ai_commitrequests.yaml - bases/configbutler.ai_fluxclaims.yaml - bases/configbutler.ai_gitbranches.yaml - bases/configbutler.ai_gitproviders.yaml - bases/configbutler.ai_gitreceivers.yaml - bases/configbutler.ai_gittargets.yaml - bases/configbutler.ai_kubeproviders.yaml + - bases/configbutler.ai_saverequests.yaml - bases/configbutler.ai_watchrules.yaml - - bases/source.configbutler.ai_saverequests.yaml + - bases/source.configbutler.ai_remotesaverequests.yaml diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 54c540e5..4b8a1e32 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -69,13 +69,13 @@ rules: resources: - argocdclaims/status - clusterwatchrules/status - - commitrequests/status - fluxclaims/status - gitbranches/status - gitproviders/status - gitreceivers/status - gittargets/status - kubeproviders/status + - saverequests/status - watchrules/status verbs: - get @@ -99,26 +99,26 @@ rules: - apiGroups: - configbutler.ai resources: - - commitrequests + - gitreceivers + - kubeproviders verbs: - - delete - get - list - - patch - watch - apiGroups: - configbutler.ai resources: - - gitreceivers - - kubeproviders + - saverequests verbs: + - delete - get - list + - patch - watch - apiGroups: - source.configbutler.ai resources: - - saverequests + - remotesaverequests verbs: - delete - get @@ -127,7 +127,7 @@ rules: - apiGroups: - source.configbutler.ai resources: - - saverequests/status + - remotesaverequests/status verbs: - get - update diff --git a/config/samples/README.md b/config/samples/README.md index 92a759d2..9df29b62 100644 --- a/config/samples/README.md +++ b/config/samples/README.md @@ -14,7 +14,7 @@ These samples are quick starting points for common GitOps Reverser setups. - `clusterwatchrule.yaml`: Minimal `ClusterWatchRule` for cluster-scoped resources. - `fluxclaim.yaml`, `argocdclaim.yaml`: An applier claim holding the Flux `Kustomization` or the Argo CD `Application` that applies `example-target`'s folder, for its `spec.applierClaimRef`. -- `commitrequest.yaml`: Minimal `CommitRequest` — an on-demand "save" signal that finalizes a +- `saverequest.yaml`: Minimal `SaveRequest` — an on-demand "save" signal that finalizes a `GitTarget`'s open commit window. It uses `metadata.generateName`, so apply it with `kubectl create -f`: `kubectl apply` refuses a generated name. diff --git a/config/samples/commitrequest.yaml b/config/samples/saverequest.yaml similarity index 94% rename from config/samples/commitrequest.yaml rename to config/samples/saverequest.yaml index 7b9ce546..8561ae72 100644 --- a/config/samples/commitrequest.yaml +++ b/config/samples/saverequest.yaml @@ -1,4 +1,4 @@ -# CommitRequest is a one-shot "save" signal. It attaches to its author's commit +# SaveRequest is a one-shot "save" signal. It attaches to its author's commit # window for the referenced GitTarget, gives that window its message, and # replaces the window's timers with its own. A window is only ever opened by a # write; a request waits for one and never opens one. The resulting commit is @@ -7,7 +7,7 @@ # Frontends typically use metadata.generateName so each "save" is a fresh # object. The referenced GitTarget must live in the same namespace. apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: generateName: save- namespace: default diff --git a/config/webhook/validating-webhook.yaml b/config/webhook/validating-webhook.yaml index 37830cbc..7cc46b49 100644 --- a/config/webhook/validating-webhook.yaml +++ b/config/webhook/validating-webhook.yaml @@ -17,10 +17,10 @@ metadata: # rejecting policy becomes product behavior, the chart must gain matching packaging # and this note must change. # 2. validate-operator-types.configbutler.ai — captures the submitter of our own command -# kinds (a CommitRequest today) at admission. This one IS product behavior and is +# kinds (a SaveRequest today) at admission. This one IS product behavior and is # packaged in the chart (charts/.../templates/validate-operator-types-webhook.yaml); the # entry here is the e2e SUT's copy. See -# docs/spec/commitrequest-admission-authorship.md. +# docs/spec/saverequest-admission-authorship.md. # # The cert-manager.io/inject-ca-from annotation injects the admission server CA bundle # for every webhook in this configuration. @@ -78,7 +78,7 @@ webhooks: path: /validate-operator-types port: 9443 # Ignore, not Fail: a missed author capture must degrade to the committer, never - # reject a user's CommitRequest. The handler also returns Allowed even when the Redis + # reject a user's SaveRequest. The handler also returns Allowed even when the Redis # write errors, so the command's success never depends on this webhook at all — # Ignore only covers the webhook being entirely unreachable. # @@ -91,7 +91,7 @@ webhooks: name: validate-operator-types.configbutler.ai # Narrow: our own command kinds (one rule per command kind), plus the one config kind # this endpoint validates. The submitter of a command is captured into the command-author - # Redis corner and read back by the CommitRequest controller with no wait; a WatchRule is + # Redis corner and read back by the SaveRequest controller with no wait; a WatchRule is # checked against the GitTarget it names and may be rejected. rules: - apiGroups: @@ -101,7 +101,7 @@ webhooks: operations: - CREATE resources: - - commitrequests + - saverequests scope: Namespaced # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it was # created, and that edit is the same mistake. diff --git a/docs/INDEX.md b/docs/INDEX.md index a91139a6..61fabdfd 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -102,8 +102,8 @@ misled. Full list in [`spec/README.md`](spec/README.md); the ones that carry a | [`type-followability.md`](spec/type-followability.md) | is a type followable, and if not, the one reason | | [`gitpath-foreign-content-stringency.md`](spec/gitpath-foreign-content-stringency.md) | refusing a path that shadows foreign content | | [`unsupported-folder-refusal-plan.md`](spec/unsupported-folder-refusal-plan.md) | `GitPathAccepted`, and refusing what we cannot own | -| [`commitrequest-design.md`](spec/commitrequest-design.md) | the CommitRequest window and its conditions | -| [`commitrequest-admission-authorship.md`](spec/commitrequest-admission-authorship.md) | how command submitters are captured and matched to commit windows | +| [`saverequest-design.md`](spec/saverequest-design.md) | the SaveRequest window and its conditions | +| [`saverequest-admission-authorship.md`](spec/saverequest-admission-authorship.md) | how command submitters are captured and matched to commit windows | | [`where-validation-lives.md`](spec/where-validation-lives.md) | schema → CEL → **the reconciler**; a webhook only for what exists solely at admission | | [`e2e-serial-registry.md`](spec/e2e-serial-registry.md) | which e2e specs must run Serial, and why | @@ -147,14 +147,14 @@ so you never have to read a proposal to find out it already shipped. | [`open-asks-priority.md`](design/open-asks-priority.md) | **the work queue.** Merges three overlapping backlogs — the downstream consumer asks, the API-surface block left unbuilt by the status and configuration-model review, and the config-surface proposal (B1–B6) — into one ordered queue under four stated tests, and says where we deliberately do **not** do what was asked. The standing caveat narrowed once the layout model reversed: a Tier 2 entry belongs to postponed [#294](https://github.com/ConfigButler/gitops-reverser/issues/294) only if it breaks a `GitTarget` field, and everything else is independently schedulable. Makes one design call against what was asked: **delete Option C sibling inference** rather than ship an off-switch for it, because it let a human's edit to the repository change operator behavior with nothing in status recording the move. That deletion has shipped, and "what the deletion taught" records what building it found. **F9 is Tier 1**: the only item whose answer is unknown rather than whose work is unscheduled, and it gates planning the enum work | | [`proving-the-metric-surface.md`](design/proving-the-metric-surface.md) | **design, partly built.** What identity an operator needs to locate a failure, and what an e2e check has to do to be worth trusting. Written after a metric value that was declared, switched on, surfaced as a condition reason and counted turned out to have no producer at all, and the check added to catch that class then skipped whenever the counter read zero — so deleting every recording call left the suite green. Two rules follow: an assertion must not take its trigger from the metric under test, and no enum value may be reachable only through an injected fake. Carries a cardinality policy that separates bounded configuration identity from per-request identity, a table of which instrument families should own `source_cluster` directly rather than by join, and the PromQL reasoning for `increase()` versus `max_over_time` (neither applies mechanically: one cannot see an event before the first sample, the other cannot reconstruct totals across a reset). Steps 1–4 are built; step 5, diagnostics export, the `e2e_run` scrape label and a pinned-seed regression are not | | [`placement-visibility-and-declared-defaults.md`](design/placement-visibility-and-declared-defaults.md) | **design, half built.** The three questions the inference deletion left, decided and then built in two goes. PR #291 shipped the deletion and none of the eight items queued behind it; the residue was filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) — **which shipped in 0.42.1 via [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) and is what reversed the layout model** — and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296), whose largest half then shipped in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326): `status.layout` became `status.placement`, and the ambiguous render root came with it as a refusal rather than a third counter outcome. The `declared` metric value has since split into `by_type`/`default` too, which was the one row with an external consumer because it changes a label a dashboard may select on. What still stands and is still unbuilt is the rest of the legibility set: the canonical path as a template constant, and `{kindLower}`, tracked on [#339](https://github.com/ConfigButler/gitops-reverser/issues/339) now that #296 is closed; the Event on the GitTarget was dropped. Its Question 2, a CRD default for `placement.default`, is **re-opened rather than superseded** and now lives as an open question in [`layout/model.md`](layout/model.md); the argument against it is legibility, since #319 removed the correctness objection | -| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **partly built**, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). One breaking wave on `GitTarget`, of which steps 2 to 7 have **shipped**: `spec.suspend`, `status.placement` and the reconcile-request annotation in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326), and B4's `commitWindow`/`commit.message` move plus the source-scope deletion in [#330](https://github.com/ConfigButler/gitops-reverser/pull/330). Organizing principle: **the folder is described on the GitTarget, the connection describes only the connection** — and this is where that became a struct boundary rather than a sentence, since grouping a field is free only in a release that is already breaking. `spec.mode` and `GitTarget.spec.interval` are both **dropped**, with re-open triggers, and `status.lastHandledReconcileAt` was dropped in the build for a stated reason. Step 8, the riders, is closed: the `CommitRequest` lifecycle hole shipped in [#400](https://github.com/ConfigButler/gitops-reverser/pull/400), the missing source-cluster provider message in [#258](https://github.com/ConfigButler/gitops-reverser/pull/258), and an asserted `CommitRequest.spec.author` is **declined** because authorship is established, never asserted ([#339](https://github.com/ConfigButler/gitops-reverser/issues/339)); #294 is closed. The reference collapse and the `scope` removal shipped in [#333](https://github.com/ConfigButler/gitops-reverser/pull/333); the `TooManyStreams` cap and the Event on a changed resolution were dropped. Also records that staying `v1alpha3` on loud rejections is a **one-consumer countdown** | +| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **partly built**, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). One breaking wave on `GitTarget`, of which steps 2 to 7 have **shipped**: `spec.suspend`, `status.placement` and the reconcile-request annotation in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326), and B4's `commitWindow`/`commit.message` move plus the source-scope deletion in [#330](https://github.com/ConfigButler/gitops-reverser/pull/330). Organizing principle: **the folder is described on the GitTarget, the connection describes only the connection** — and this is where that became a struct boundary rather than a sentence, since grouping a field is free only in a release that is already breaking. `spec.mode` and `GitTarget.spec.interval` are both **dropped**, with re-open triggers, and `status.lastHandledReconcileAt` was dropped in the build for a stated reason. Step 8, the riders, is closed: the `SaveRequest` lifecycle hole shipped in [#400](https://github.com/ConfigButler/gitops-reverser/pull/400), the missing source-cluster provider message in [#258](https://github.com/ConfigButler/gitops-reverser/pull/258), and an asserted `SaveRequest.spec.author` is **declined** because authorship is established, never asserted ([#339](https://github.com/ConfigButler/gitops-reverser/issues/339)); #294 is closed. The reference collapse and the `scope` removal shipped in [#333](https://github.com/ConfigButler/gitops-reverser/pull/333); the `TooManyStreams` cap and the Event on a changed resolution were dropped. Also records that staying `v1alpha3` on loud rejections is a **one-consumer countdown** | | [`target-watch-plan.md`](design/target-watch-plan.md) | **partly built.** The diff is built and applied; removal semantics are not. The companion to [`watch-manager-ownership.md`](design/watch-manager-ownership.md): the ownership page says WHO applies a plan, this one says WHAT a plan is and what changing it may touch. A resource collection — group, resource, namespace, deliberately no served version — is the one identity the watch stream, the render-fidelity scope and the mark-and-sweep boundary all agree on, because a key that does not round-trip to the scope it sweeps under is the class of error that deletes user data. The plan is diffed into `keep`/`start`/`restart`/`stop` and applied per collection, so adding one WatchRule stops re-initializing every unrelated collection into a queue shared with other tenants; a `restart` is a served-version change, which is why the version is spec DATA rather than identity. Readiness and the fidelity revision are per scope, so a KEPT collection holds the result its own initialization produced rather than being asked to prove itself again over an unrelated edit. `stop` never touches files — removal is a Git-side sweep under the target's existing `spec.prune.mode`, not a watch-layer delete. "Cut at the producer" is the accepted consequence: nothing fences the queue, so a deselected collection may leave a short tail of writes, bounded by the queue and converged afterwards. Still open: the `stop` classification wants a settled `TypeRemoved` from `typeset` (see TODO), and removal on INTENT is undecided. | | [`collection-terminology-rename.md`](design/collection-terminology-rename.md) | **step 1, shipped.** Replaced cells with resource collections and removed per-rule `operations` from both rule kinds. Rules observe every object event; GitTarget prune mode controls removal. Records the rename census (760 Go identifiers, 265 documentation occurrences), the operation-filter inventory, API/Helm migration, compatibility boundaries, and test cases. | | [`watches-labels-simplification.md`](design/watches-labels-simplification.md) | **step 2, implemented.** After the collection rename and operation-filter removal, native per-rule `objectSelector` (#146). Kubernetes is authoritative: `Always` compares selected identities with all managed Git identities in the type/namespace scope, without comparing Git labels. Full support includes supported Kustomize layouts from the first release, plus label-exit attribution (now selector-agnostic, see the attribution spec) and ordered producer replacement. | | [`label-selection-follow-ups.md`](design/label-selection-follow-ups.md) | **design, open.** What the label-selection review rounds left open, with the evidence for each: real deletions in a selected collection wait the full 3s attribution grace (the watch cannot tell them from a label exit, measured), the resync fence that stops overlapping collections from coalescing (kept on purpose), same-second overlap ties, and one store rebuild per object coming back. Also lists what was fixed on the branch | | [`attribution-removal-wait-options.md`](design/attribution-removal-wait-options.md) | a removal now waits for evidence about the DELETION rather than accepting the object's last write, which stopped it naming whoever last edited the object as the author of a deletion they did not perform. Enumerates the eight situations a resolution can be in and shows the cost is concentrated in exactly one: a removal for which no delete fact will ever arrive (a graceful pod delete, a status-only removal, a type the audit policy skips) spends the whole grace to return the answer it had at t=0, measured at ~3.1s against ~70ms when evidence is present. Prices five options against that, and recommends a per-route watermark — stop waiting once the fact stream has demonstrably moved past this event — over a second timeout flag whose right value lives in the API server's config rather than ours. Open: the decision, and how common the case is outside the e2e suite | -| [`commitrequest-save-wait-options.md`](design/commitrequest-save-wait-options.md) | **design, decided** — a five-phase plan, reported from a live cluster: a `CommitRequest` can expire before the write it exists to publish arrives, because the close delay defaulted to `0` while the write is still held head-of-line waiting for an audit fact, so the save resolves `Ready=True`/`NoWindowInGrace` and the edit commits seconds later under the target's `liveTemplate`. The decision is to repair the default (`spec.closeDelay`, now a duration string defaulting to `"2s"`, **shipped as phase 0**) and then let a request **name the write it is waiting for**, reusing the `(uid, resourceVersion)` identity the attribution join already keys on, including its settled `ExactCapable` rule. Records why the fact index cannot BE the wait (it sits upstream of the watch event, so a fact-keyed wait fires before the window opens) and what to take from it anyway — the register-then-check waiter discipline, and a best-effort retrospective oracle bounded by a TUNABLE TTL. Twelve situations, and the page is deliberate about the two it cannot fix cheaply: row 11 (a saturated queue is rejected in the WATCH path and never reaches the worker, so no worker-side wait can diagnose it) is an accepted, named gap rather than a promise, and row 10 is fixed only by a hook in the fidelity-drop branch that marks a wait terminally UNSATISFIABLE — satisfying before that gate would arm a finalize for a write that never commits. Carries a **preconditions chapter** for saves on today's build (audit policy first, then delay sizing, fidelity warm-up, save-button debounce, and assert on the commit rather than on `Ready`). `CommitterActivity` is deferred, not recommended: by its own arrival-not-existence rule it cannot satisfy the already-arrived case. Open: situation 12 (one actor, two saves, one window — blocks phase 2), whether the fact-index lookup earns its coupling, and how an unsatisfied wait is graded | -| [`commit-timing-surface.md`](design/commit-timing-surface.md) | **built.** One configuration surface for commit windows and saves, an in-place `v1alpha3` break with one tested upgrade procedure: only writes open a window; a `window` block (`idleTimeout`, `maxDuration`, target default `1m`) on both kinds, which a `CommitRequest` replaces whole for the window it attaches to; `attach: CurrentOrNext \| Next` with `attachTimeout`; opt-in `whenNothingToCommit: CommitEmpty` that keeps the cause in the reason. Precise clock and zero rules. Supersedes the save-wait timer model. Open: the target's `maxDuration` value | +| [`commitrequest-save-wait-options.md`](design/commitrequest-save-wait-options.md) | **design, decided** — a five-phase plan, reported from a live cluster: a `SaveRequest` can expire before the write it exists to publish arrives, because the close delay defaulted to `0` while the write is still held head-of-line waiting for an audit fact, so the save resolves `Ready=True`/`NoWindowInGrace` and the edit commits seconds later under the target's `liveTemplate`. The decision is to repair the default (`spec.closeDelay`, now a duration string defaulting to `"2s"`, **shipped as phase 0**) and then let a request **name the write it is waiting for**, reusing the `(uid, resourceVersion)` identity the attribution join already keys on, including its settled `ExactCapable` rule. Records why the fact index cannot BE the wait (it sits upstream of the watch event, so a fact-keyed wait fires before the window opens) and what to take from it anyway — the register-then-check waiter discipline, and a best-effort retrospective oracle bounded by a TUNABLE TTL. Twelve situations, and the page is deliberate about the two it cannot fix cheaply: row 11 (a saturated queue is rejected in the WATCH path and never reaches the worker, so no worker-side wait can diagnose it) is an accepted, named gap rather than a promise, and row 10 is fixed only by a hook in the fidelity-drop branch that marks a wait terminally UNSATISFIABLE — satisfying before that gate would arm a finalize for a write that never commits. Carries a **preconditions chapter** for saves on today's build (audit policy first, then delay sizing, fidelity warm-up, save-button debounce, and assert on the commit rather than on `Ready`). `CommitterActivity` is deferred, not recommended: by its own arrival-not-existence rule it cannot satisfy the already-arrived case. Open: situation 12 (one actor, two saves, one window — blocks phase 2), whether the fact-index lookup earns its coupling, and how an unsatisfied wait is graded | +| [`commit-timing-surface.md`](design/commit-timing-surface.md) | **built.** One configuration surface for commit windows and saves, an in-place `v1alpha3` break with one tested upgrade procedure: only writes open a window; a `window` block (`idleTimeout`, `maxDuration`, target default `1m`) on both kinds, which a `SaveRequest` replaces whole for the window it attaches to; `attach: CurrentOrNext \| Next` with `attachTimeout`; opt-in `whenNothingToCommit: CommitEmpty` that keeps the cause in the reason. Precise clock and zero rules. Supersedes the save-wait timer model. Open: the target's `maxDuration` value | | [`watch-and-catalog-architecture.md`](design/watch-and-catalog-architecture.md) | the target three-layer watch model — **needs a human call before building** | | [`metrics-observability-plan.md`](design/metrics-observability-plan.md) | **built.** the canonical metrics document, rewritten 2026-09 and shipped in the same wave. Reads the product as two ingestion paths — object state through watch, authorship through audit — joining at the resolver, and gives each stage one counter with a bounded `outcome` so the flow and its loss paths are drawable. Records what three review rounds corrected, including two reversals worth not re-litigating: the fact-loss counters must NOT be merged (an eviction counts a fact, a trim gap an occurrence, a decode error a whole batch), and a cumulative counter cannot replace a current count, which is why `status.streams` and `status.retainedDocuments` stayed. The instrument inventory lives in [`interpreting-metrics.md`](interpreting-metrics.md), not here. Open: the Grafana JSON and alert rules | | [`reconcile-triggering.md`](design/reconcile-triggering.md) | which controllers still fail to wake up | diff --git a/docs/README.md b/docs/README.md index fddd72d9..e091c9ed 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,7 +11,7 @@ If you only want the supported product docs, start with the files below. - [`../README.md`](../README.md): product overview and supported workflows - [`quickstart.md`](quickstart.md): installation and your first Git commit -- [`demo/commit-window.md`](demo/commit-window.md): two resources and a `CommitRequest` +- [`demo/commit-window.md`](demo/commit-window.md): two resources and a `SaveRequest` - [`configuration.md`](configuration.md): core configuration objects, every setting, and how they fit together - [`commit-messages.md`](commit-messages.md): the three commit message templates and their context diff --git a/docs/TODO.md b/docs/TODO.md index 80b41eff..d5777013 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -72,20 +72,20 @@ Their argument for each is in [gitops-api-open-asks.md](future/gitops-api-open-a - [ ] Two doc sentences: no condition detects an API server's 30s default `--audit-webhook-batch-max-wait` (say so in the `KubeProvider` docs), and a relist after a watch gap ships its changes as the configured author (say so in the operator docs). -- [ ] Optional: capture `CommitRequest` authors without Redis in single-pod mode, with an in-memory +- [ ] Optional: capture `SaveRequest` authors without Redis in single-pod mode, with an in-memory `CommandAuthorStore`. The only thing that still forces Redis on one replica. Low priority, and only useful until HA. -## Branch worker and CommitRequests +## Branch worker and SaveRequests -- [ ] `expireWaitingCommitRequests` ranges over a map, so empty commits for saves that expire in - the same pass land in random order; sort by `seq` (`internal/git/commit_request_attach_loop.go`). +- [ ] `expireWaitingSaveRequests` ranges over a map, so empty commits for saves that expire in + the same pass land in random order; sort by `seq` (`internal/git/save_request_attach_loop.go`). - [ ] `buildRequestRecordWrite` keeps only the username (`UserInfo{Username: pcr.author}`); carry the full `UserInfo` so a record commit has the same author identity as a window commit. - [ ] A record commit skips `requestTemplate`, though its doc comment says it is phrased like every other commit; fix the code or the comment. - [ ] Nits: a doubled doc comment on `commitWindowFor`; "close-delay" in the test names in - `internal/controller/commitrequest_kstatus_test.go`. + `internal/controller/saverequest_kstatus_test.go`. - [ ] **Parent observation, step 1**: keep parent availability and its timestamp after the write branch exists, from existing advertisements, with no extra connections. [gittarget-parent-observation.md](design/gittarget-parent-observation.md#implementation-and-validation). diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 75003d1e..a13b8829 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,6 +7,27 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. +## `CommitRequest` is `SaveRequest` + +`CommitRequest` is `SaveRequest`, in the same group and version (`configbutler.ai/v1alpha3`), with +the same spec and status. Nothing is migrated: a `CommitRequest` is a one-shot request, so let the +open ones finish, then upgrade and delete the old CRD, which Helm leaves installed: + +```bash +kubectl delete crd commitrequests.configbutler.ai +``` + +Everything named after the kind follows it: + +- **RBAC.** A Role that lets people `create` `commitrequests` must grant `saverequests` instead. +- **The TTL.** The flag `--commit-request-ttl` is `--save-request-ttl`, and the Helm value + `controllerManager.commitRequestTTL` is `controllerManager.saveRequestTTL`. +- **Metrics.** `gitopsreverser_commit_requests_total` is `gitopsreverser_save_requests_total`, with + the same labels. The `message_source` values `commit_request`, `commit_request_framed` and + `commit_request_fallback` of `git_commits_total` are `save_request`, `save_request_framed` and + `save_request_fallback`, and the `timer_source` value `commit_request` of + `git_commit_windows_total` is `save_request`. + ## A `FluxClaim` applies the tip a minute after GitOps Reverser's own pushes A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a @@ -14,7 +35,7 @@ A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none commit somebody else pushed. `spec.catchUpAfter` sets when that lag is caught up: once the applier has stayed behind the tip that long, an apply of the tip is asked for, one for a whole burst of saves. A `FluxClaim` defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` -defaults to `0s`. A `CommitRequest` that sets `catchUp: AfterPush` has the tip applied right after +defaults to `0s`. A `SaveRequest` that sets `catchUp: AfterPush` has the tip applied right after its commit is pushed, whatever the claim's interval, and reports the outcome in its `Applied` condition and `status.appliedRevision`. diff --git a/docs/api-first-publication.md b/docs/api-first-publication.md index 2ef82ddf..8d71d2a6 100644 --- a/docs/api-first-publication.md +++ b/docs/api-first-publication.md @@ -47,7 +47,7 @@ flowchart LR AUDIT["Optional audit facts
actor attribution"] -.-> WATCH GT["GitTarget
source, branch, path, commit window"] --> WATCH WATCH --> QUEUE["One FIFO per provider and branch"] - CR["CommitRequest
close a matching window"] --> QUEUE + CR["SaveRequest
close a matching window"] --> QUEUE QUEUE --> WINDOW["Open window
one author and one target"] QUEUE --> SNAP["Snapshot / atomic write"] WINDOW --> PENDING["Retained PendingWrite records"] @@ -68,10 +68,10 @@ flowchart LR | `BranchWorker` | Owns branch ordering, local repository operations, and publication | | Open window | Coalesces consecutive events from one author for one target | | `PendingWrite` | Retains captured content and resolved write metadata until publication | -| `CommitRequest` | Claims a matching window and reports its eventual outcome | +| `SaveRequest` | Claims a matching window and reports its eventual outcome | The watch stream supplies object state. Audit facts supply authorship when attribution is enabled. -Admission captures the submitter of a `CommitRequest`. These inputs have separate jobs; mirroring +Admission captures the submitter of a `SaveRequest`. These inputs have separate jobs; mirroring continues without actor attribution. See the [attribution contract](spec/attribution.md). The queue and retained writes are in memory. They are sufficient for replay during the worker's @@ -165,7 +165,7 @@ sequenceDiagram The idle timeout restarts on every matching edit, so continuous edits keep a window open until its `maxDuration` (`1m` by default, counted from the window opening) or another closing condition, such as an identity change or the byte threshold. A five-second idle timeout therefore does not promise -publication within five seconds of the first edit. A `CommitRequest` that attaches replaces both +publication within five seconds of the first edit. A `SaveRequest` that attaches replaces both timers with its own `idleTimeout` and `maxDuration`, both counted from the attach; an omitted `idleTimeout` leaves `maxDuration` alone to close the window. Its `attachTimeout` is separate: it runs from the worker's registration of the request and only bounds the wait for a window to attach @@ -282,7 +282,7 @@ are never held by the same worker. ## Story 3: save now, and know what reached Git -A `CommitRequest` attaches to a matching author-and-target window and closes it. It does not +A `SaveRequest` attaches to a matching author-and-target window and closes it. It does not bypass the push cooldown. Two settings with separate clocks set the timing: `attachTimeout` runs from the worker's registration of the request and bounds the wait for a window, and `maxDuration` runs from the attach and bounds the collection that follows. Both default to `2s` and accept `0s` @@ -290,7 +290,7 @@ to `5m`. A repeat attach restarts neither clock. ```mermaid sequenceDiagram - participant C as CommitRequest controller + participant C as SaveRequest controller participant W as Branch worker participant G as Git remote @@ -321,7 +321,7 @@ long a down remote takes. For a request the worker has not acted on within reports whatever the worker answers: failed when the worker cancelled it, or the real outcome when the worker resolved it first. A request reported failed is therefore never committed later. The worker keeps resolved outcomes for `15m`. -See the [request contract](spec/commitrequest-design.md). +See the [request contract](spec/saverequest-design.md). The two-second `attachTimeout` is not a guarantee that an earlier API edit has reached the worker. Attribution can wait up to three seconds per event, and queueing adds more. A request @@ -477,7 +477,7 @@ the earlier attribution and commit-window waits, cooldown, and recovery performe the cycle. It cannot by itself answer “how long from my API edit until Git accepted it?” A flat recovery counter also cannot prove progress: the worker may be retaining work while it -waits out its backoff. Check queue depth, failure logs, and `CommitRequest` outcomes alongside rates. +waits out its backoff. Check queue depth, failure logs, and `SaveRequest` outcomes alongside rates. A resync's fetch lands under `forced_recheck` whether or not the branch happened to hold retained writes, so a growing `publication` series means live publication is fetching again and nothing diff --git a/docs/architecture.md b/docs/architecture.md index 36eb5a0b..a9137d98 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -173,7 +173,7 @@ the Flux `Kustomization` applier and the Argo CD `Application` applier, its auto **Redis/Valkey is optional but advised.** The default configured-author mode runs without it: a plain `helm install` comes up healthy and watches initialize from current state on startup. When an endpoint is configured, Redis stores cursors for reconnecting running watches and the coordination records used by -CommitRequest author capture and HA. Attribution does not require it on its own: its facts travel on a +SaveRequest author capture and HA. Attribution does not require it on its own: its facts travel on a selectable transport, Redis Streams by default and an in-process ring with `--author-attribution-transport=memory`, which is refused with more than one replica. HA will require Redis as the shared store across replicas. @@ -240,7 +240,7 @@ optional and only add *who* did something: | **Discovery** (CRD/APIService) | *what types exist* | yes | the API surface: served GVRs, scope, preferred version, subresources; rules resolve against it | | **Watch** (per claimed `(GVR, scope)`) | *what changed* | yes | the object body, ordered per type; the only object-state source, with deletes reconciled via `sendInitialEvents` replay + mark-and-sweep | | **Audit webhook** (`/audit-webhook/`) | *who changed mirrored state* | no | a post-persist attribution fact, partitioned by source provider and joined to the watch event by resourceVersion ([Optional Attribution](#optional-attribution)) | -| **Validating admission webhook** (`/validate-operator-types`) | *who issued a command* | no | the submitter of a `CommitRequest`, captured at admission and keyed 1:1 by UID ([CommitRequest Finalize](#commitrequest-finalize)) | +| **Validating admission webhook** (`/validate-operator-types`) | *who issued a command* | no | the submitter of a `SaveRequest`, captured at admission and keyed 1:1 by UID ([SaveRequest Finalize](#saverequest-finalize)) | With both optional sources off, the product still mirrors state correctly, and every commit is authored by the committer. @@ -263,14 +263,14 @@ the write reaches etcd**: Audit avoids both: kube-apiserver posts a `ResponseComplete` event *after* the write persisted, carrying the resourceVersion that joins it to the watch event. -The one place admission *is* the right source is the `CommitRequest`, and only for its **command** meaning +The one place admission *is* the right source is the `SaveRequest`, and only for its **command** meaning (*who issued the save*) and not for mirroring the object. That holds because it is **our own type with a memory model we control**: the controller reconciles off its own watch, which only delivers persisted objects, so the author is captured at admission and read back 1:1 by UID: no resourceVersion join, no persistence proof from audit needed. (We still can't guarantee a create is never blocked, but it holds -across our edge cases.) If a `WatchRule` also selects CommitRequests, that *mirroring* runs through the +across our edge cases.) If a `WatchRule` also selects SaveRequests, that *mirroring* runs through the normal pipeline and is audit-attributed like any other resource. See -[CommitRequest Finalize](#commitrequest-finalize). +[SaveRequest Finalize](#saverequest-finalize). ### Proving it on new Kubernetes versions @@ -290,7 +290,7 @@ capture audit and admission with **no** resulting watch event. You configure GitOps Reverser through ten CRDs (group `configbutler.ai`, version `v1alpha3`). `WatchRule` and `ClusterWatchRule` choose which Kubernetes resources enter the pipeline. -`CommitRequest` can ask for the current window to be saved. `GitTarget` joins one source cluster to one +`SaveRequest` can ask for the current window to be saved. `GitTarget` joins one source cluster to one Git destination. `KubeProvider` supplies the source connection and authorization boundary; `GitProvider` supplies the repository, credentials, commit settings, and push policy, and a `GitBranch` holds one branch of that repository for every `GitTarget` that writes it. A @@ -304,7 +304,7 @@ graph LR subgraph CONTROL["Control plane: operator cluster"] WR[WatchRule] -->|gitTargetRef| GT[GitTarget] CWR[ClusterWatchRule] -->|gitTargetRef| GT - CR[CommitRequest] -->|gitTargetRef| GT + CR[SaveRequest] -->|gitTargetRef| GT GT -->|applierClaimRef| TR["FluxClaim /\nArgoCDClaim"] GT -->|kubeProviderRef| CP[KubeProvider] GT -->|gitBranchRef| GB[GitBranch] @@ -352,7 +352,7 @@ records the apply operation it started. |---|---|---| | `WatchRule` | namespaced | which NAMESPACED resources, in which source namespaces, route to a GitTarget | | `ClusterWatchRule` | cluster | which CLUSTER SCOPED resources route to a GitTarget | -| `CommitRequest` | namespaced | a one shot "save the open window now" signal | +| `SaveRequest` | namespaced | a one shot "save the open window now" signal | | `GitTarget` | namespaced | one materialization from a `KubeProvider` to `(GitBranch, path)` | | `KubeProvider` | namespaced | one source-cluster connection plus namespace access policy | | `GitBranch` | namespaced | one repository branch, held for every `GitTarget` that writes it | @@ -404,10 +404,10 @@ One `KubeProvider` grant answers two questions in different planes: own Kubernetes RBAC. The operator reads no `Namespace` objects in a source cluster for authorization. -### CommitRequest +### SaveRequest -- **Source**: [api/v1alpha3/commitrequest_types.go](../api/v1alpha3/commitrequest_types.go) -- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) +- **Source**: [api/v1alpha3/saverequest_types.go](../api/v1alpha3/saverequest_types.go) +- **Controller**: [internal/controller/saverequest_controller.go](../internal/controller/saverequest_controller.go) A one shot "save now" signal that finalizes the open commit window for a same namespace `GitTarget` instead of waiting for the silence timer. The **entire spec is immutable**. Key fields: @@ -431,7 +431,7 @@ instead of waiting for the silence timer. The **entire spec is immutable**. Key - `status.branch` / `status.commit`: set when the commit was pushed (`Pushed=True`). How attribution and finalization interact is described under -[CommitRequest Finalize](#commitrequest-finalize). +[SaveRequest Finalize](#saverequest-finalize). ### GitTarget @@ -704,7 +704,7 @@ Following the ConfigMap edit: [GitTargetEventStream](../internal/reconcile/git_target_event_stream.go), and the [BranchWorker](../internal/git/branch_worker.go) appends it to the open commit window for `(author, GitTarget)`. -5. **Commit + push.** When the window closes (5s of silence, a `CommitRequest`, or a buffer limit), the +5. **Commit + push.** When the window closes (5s of silence, a `SaveRequest`, or a buffer limit), the manifest aware writer patches `team-a/.../app-config.yaml` in place, commits as the resolved author, and pushes via [PushAtomic](../internal/git/git_atomic_push.go) (retrying with fetch/reset/replay if the remote moved). @@ -722,7 +722,7 @@ See [State Ingestion and Not Losing Deletes](#state-ingestion-and-not-losing-del ### Normal commit flow -Most writes are closed by the commit window timer. A `CommitRequest` is the explicit "save now" path: it +Most writes are closed by the commit window timer. A `SaveRequest` is the explicit "save now" path: it does not create new state, and it does not bypass watch. It only asks the branch worker to close a matching open window early. @@ -743,7 +743,7 @@ flowchart TD TIMER -->|silence timer| BASE TIMER -->|buffer limit or resync boundary| BASE - CR[CommitRequest persisted] --> ADMISSION[Admission submitter lookup
named or unnamed] + CR[SaveRequest persisted] --> ADMISSION[Admission submitter lookup
named or unnamed] ADMISSION --> ATTACH[Attach to matching open window] ATTACH -->|same author + GitTarget| BASE ATTACH -->|no matching window| BENIGN[Ready=True, Pushed=False] @@ -756,10 +756,10 @@ flowchart TD ``` The important boundary is the open window: it accepts only the same author and `GitTarget`. A -`CommitRequest` from another author never closes someone else's window; it resolves as a benign no-commit +`SaveRequest` from another author never closes someone else's window; it resolves as a benign no-commit instead. The one exception is opt-in, on a cluster that has never sent an audit fact: there no write can name its author, and a request may claim a window whose writes name none (step 2 of the -`CommitRequest` flow). +`SaveRequest` flow). ### Remote moved while we were writing @@ -784,7 +784,7 @@ flowchart TD CHECK -->|attempts exhausted or fetch fails| RETAIN[Keep pending writes for later retry] ``` -For a `CommitRequest`, success is reported only after the push reaches the remote. If the push had to +For a `SaveRequest`, success is reported only after the push reaches the remote. If the push had to replay on top of someone else's commit, `status.commit` is the refreshed post-replay commit, not the stale local commit from before the retry. @@ -818,7 +818,7 @@ flowchart TD A delivery is handled on the worker's own queue, so it is never judged while a push is in flight: the push has recorded its commit before the delivery is read. A rollback onto one of the worker's older commits is foreign, because the advertisement names a tip other than the last one observed. -A delivery also closes the open commit window, unless a `CommitRequest` holds it, so its +A delivery also closes the open commit window, unless a `SaveRequest` holds it, so its publication finds the move. A Git-side push usually reaches the worker through the flush itself: the push of the pending writes @@ -838,7 +838,7 @@ again is what stamps the owner labels or tracking annotation on the new object. worker's own leaves nothing pending. Three states still hold an apply back, because in them no flush can be made: a registered -`CommitRequest` (a save defines its own commit boundary, and the apply waits for it), a parent +`SaveRequest` (a save defines its own commit boundary, and the apply waits for it), a parent recovery, and a checkout a failed write left dirty. An apply also waits until every initial snapshot of the `GitTarget`s its applier covers is published (a `GitTarget` with no planned collection yet, or with a blocked watch, holds its applier the same way), and an empty tip (a @@ -1116,9 +1116,9 @@ queued behind it on the same `(GitTarget, GVR, scope)` goroutine. That cost, and is in [Watch Event Ordering](#watch-event-ordering) and [attribution-removal-wait-options.md](design/attribution-removal-wait-options.md). -The CommitRequest controller does **not** read this audit index. A CommitRequest's submitter is named by +The SaveRequest controller does **not** read this audit index. A SaveRequest's submitter is named by the `/validate-operator-types` validating admission webhook instead, captured synchronously at admission, -never joined from an audit fact (see [CommitRequest Finalize](#commitrequest-finalize)). +never joined from an audit fact (see [SaveRequest Finalize](#saverequest-finalize)). ### Author and committer identity in Git @@ -1194,7 +1194,7 @@ CommitDate: Mon Jun 30 12:00:05 2026 +0000 - **Signature construction**: [internal/git/commit.go](../internal/git/commit.go) (`authorName` / `authorEmail`) The operator never talks to the identity provider. It reads two fixed keys from the request's `user.extra` -map: the audit attribution path and the CommitRequest admission path resolve the **same** two keys: +map: the audit attribution path and the SaveRequest admission path resolve the **same** two keys: | `user.extra` key | OIDC claim it carries | Used for | |---|---|---| @@ -1513,7 +1513,7 @@ move a live worker; it replaces it, and the replacement starts with no clone, no observation. A `GitBranch` carries a finalizer. Deleting it releases the claim, then stops its worker, which -closes the worker's intake, pushes or refuses every pending write and settles every `CommitRequest` +closes the worker's intake, pushes or refuses every pending write and settles every `SaveRequest` it holds; only then does the finalizer go. Nothing deletes the branch on the remote. `GitBranch.status.remote` is a **branch** fact, and it is worth being exact about that, because the @@ -1529,7 +1529,7 @@ once a minute, and comes back when that floor lifts. ### Everything about a branch happens on that branch's queue Reading or writing a branch's repository state is the branch worker's job, and the worker does it on -one FIFO queue, on one goroutine. Live events, resyncs, `CommitRequest` attaches, periodic +one FIFO queue, on one goroutine. Live events, resyncs, `SaveRequest` attaches, periodic refreshes and push-receiver deliveries all enter the same queue and are served in order. Which writes are decided and which of them are published is therefore a fact one goroutine holds, which is what lets the worker flush before an apply request goes out. Nothing else opens a connection on a @@ -1554,9 +1554,9 @@ accumulate in one open commit window, which accepts only one `(author, GitTarget The window finalizes when its `idleTimeout` passes with no new matching event, its `maxDuration` runs out, the retained buffer reaches `--branch-buffer-max-size` (default `8Mi`), an `attach: Next` -`CommitRequest` closes it, a resync, an atomic batch, a push-receiver delivery or shutdown arrives, +`SaveRequest` closes it, a resync, an atomic batch, a push-receiver delivery or shutdown arrives, or an apply comes due for an applier on the branch. The timers -are the GitTarget's `spec.commit.window` until a `CommitRequest` attaches and replaces them. Successful local commits +are the GitTarget's `spec.commit.window` until a `SaveRequest` attaches and replaces them. Successful local commits are retained until a fixed push cooldown (`5s`) allows a push, which prevents remote push storms during bursts. The one exception is an apply's flush: the writes decided before the apply came due are pushed at once, cooldown or not, because the apply waits for them. @@ -1591,7 +1591,7 @@ the newer tip changed, and a write with no action left settles at once without a keeps the upserts that changed the tree and the documents its sweep removed: a replay plans each upsert onto the new tree and removes each document if it is still there and the prune mode still sweeps, and never sweeps again, so a document the newer tip added survives. A save is the exception: -a write carrying a `CommitRequest` keeps every object it captured, and the push decides its outcome. +a write carrying a `SaveRequest` keeps every object it captured, and the push decides its outcome. See [the pending-writes design](design/gittarget-branch-worker-pending-writes.md#a-write-carries-the-actions-decided-on-its-tree). A branch whose remote has been **deleted** takes the same path. The advertisement does not carry the @@ -1760,15 +1760,15 @@ unsafely: logged per-resource, counted in the resync summary as `placementSkippe *** -## CommitRequest finalize +## SaveRequest finalize -- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) +- **Controller**: [internal/controller/saverequest_controller.go](../internal/controller/saverequest_controller.go) - **Admission handler**: [internal/webhook/validate_operator_types_handler.go](../internal/webhook/validate_operator_types_handler.go) - **Admission author cache**: [internal/queue/command_author_store.go](../internal/queue/command_author_store.go) -A `CommitRequest` finalizes the open commit window for its GitTarget. The request author is resolved from +A `SaveRequest` finalizes the open commit window for its GitTarget. The request author is resolved from the `/validate-operator-types` validating admission webhook, which captures the authenticated submitter at admission (keyed by the object's UID) before the object is visible. The lookup is **present-or-never**: if the record is missing, the request is marked `AuthorAttributed=False`, claims no actor, and still attaches @@ -1776,7 +1776,7 @@ immediately. That is not a failure: 1. The controller stamps the in-progress conditions (`Reconciling=True`) and settles `AuthorAttributed` synchronously from the admission author cache. There is no audit wait on this path. -2. The controller eagerly **attaches** the request to the worker (`AttachCommitRequest`), which +2. The controller eagerly **attaches** the request to the worker (`AttachSaveRequest`), which waits for a window until `registration + attachTimeout`. The worker binds it to an open window only when the author state and GitTarget match, first come first served, and replaces the window's timers with the request's own, so time spent waiting never shortens the collection. A request that names its @@ -1785,7 +1785,7 @@ immediately. That is not a failure: `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`). With `SavedBy` the window takes the request's message, the commit names no author, and the message ends with a `Saved-By` trailer; with `TakeOver` the submitter authors the commit, and the message ends - with an `Author-Source: CommitRequest /` trailer. Once the provider has received a + with an `Author-Source: SaveRequest /` trailer. Once the provider has received a fact, a write that names no author means its fact went missing, and its window stays another's. A request that asks for more than its `GitTarget` allows fails at first sight with `TakeOverNotAllowed`. It **never finalizes @@ -1804,7 +1804,7 @@ immediately. That is not a failure: `AlreadyPresent` therefore means the remote confirmed there was nothing to add, not that a local plan found no diff. -The CommitRequest submitter is not recoverable from object state alone, so without an admission record the +The SaveRequest submitter is not recoverable from object state alone, so without an admission record the request cannot claim an actor. The final Git author remains the attached watch window's author: configured committer when attribution was disabled, or the explicit unresolved author when live attribution ran but could not resolve. There is no audit-fact join for the request itself: its submitter is captured at admission @@ -1841,14 +1841,14 @@ Controllers watch their dependencies so dependents reconcile quickly after spec at all (the installation's `inCluster` consent, and which namespace holds its audit route), and projects its readiness to dependent targets. A provider created, deleted or moved to another route wakes the others carrying that route, so an `AuditRouteConflict` clears without waiting for a requeue. -- `CommitRequestReconciler` runs with `MaxConcurrentReconciles=1` and attributes/attaches as above; its +- `SaveRequestReconciler` runs with `MaxConcurrentReconciles=1` and attributes/attaches as above; its optional `AuthorLookup` is the command-author cache populated by the `/validate-operator-types` webhook (wired whenever the admission webhook is enabled, independent of `--author-attribution`, and nil otherwise, so the request claims no actor). Dependency watches use narrow predicates to avoid status-only heartbeat churn: a `KubeProvider` also admits a `Ready` transition, and a `Namespace` admits only label changes. `GitProvider`, `GitBranch`, -`GitTarget`, and `CommitRequest` carry immutability constraints where a spec change would orphan a +`GitTarget`, and `SaveRequest` carry immutability constraints where a spec change would orphan a materialized subtree or invalidate an in-flight finalize. *** @@ -1879,7 +1879,7 @@ flowchart TD Hq -->|no| J[Configured-author: no fact streams or index; audit webhook skipped] I --> K[Setup + register Watch Manager] J --> K - K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + CommitRequest controllers; admission webhooks] + K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + SaveRequest controllers; admission webhooks] L --> M[Add cert watchers + health checks] M --> N[mgr.Start] ``` @@ -1917,7 +1917,7 @@ The families follow the architecture, which is why this section is a map rather |---|---|---| | `resource_condition` | the kstatus trio of every object a human declares | [Configuration state](interpreting-metrics.md#configuration-state) | | `watch_*` | ingest: events, sessions, replays, recoveries, resolved types, open streams | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push) | -| `git_*`, `commit_requests_total` | the branch worker: documents, windows, commits, pushes, fetches, refusals, losses, the retained backlog | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push), [Tuning commit windows](interpreting-metrics.md#tuning-commit-windows) | +| `git_*`, `save_requests_total` | the branch worker: documents, windows, commits, pushes, fetches, refusals, losses, the retained backlog | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push), [Tuning commit windows](interpreting-metrics.md#tuning-commit-windows) | | `placement*` | where a new document lands, and the ones refused | [New-file placement](interpreting-metrics.md#new-file-placement) | | `apply_*`, `receiver_deliveries_total` | apply requests, the apply window and the push receiver | [Apply requests](interpreting-metrics.md#apply-requests-and-the-push-receiver) | | `audit_*`, `attribution_*` | the optional author lookup: the door, the fact pipeline, the join | [Audit attribution](interpreting-metrics.md#audit-attribution-optional) | @@ -2045,7 +2045,7 @@ Deeper dives live under [docs/design/](design/): transport seam, the in-process index, and what running without Redis costs. - [Watch event ordering under the attribution grace window](spec/watch-event-ordering-and-attribution-grace.md) - [Reconcile via watchlist mark and sweep](spec/reconcile-via-watchlist-mark-and-sweep.md) -- [CommitRequest design](spec/commitrequest-design.md) +- [SaveRequest design](spec/saverequest-design.md) **Types, discovery, status, and the Git write side:** diff --git a/docs/attribution-setup-guide.md b/docs/attribution-setup-guide.md index 5e3ff512..72b74bd7 100644 --- a/docs/attribution-setup-guide.md +++ b/docs/attribution-setup-guide.md @@ -27,7 +27,7 @@ queue: Redis/Valkey holds attribution facts and watch cursors. If its data is lost, recent facts are lost and watches cold-replay. For a disposable single-pod setup, `attribution.transport: memory` needs no Redis but loses facts whenever the reverser restarts. It also leaves Redis-backed features such -as `CommitRequest` author capture inactive unless you configure Redis separately. +as `SaveRequest` author capture inactive unless you configure Redis separately. The chart's `KubeProvider`, `local` in the release namespace, admits only the release namespace by default. Grant the namespaces whose `GitTarget`s may mirror through it in diff --git a/docs/bi-directional.md b/docs/bi-directional.md index 7fdafe03..8565b08b 100644 --- a/docs/bi-directional.md +++ b/docs/bi-directional.md @@ -152,7 +152,7 @@ stateDiagram-v2 ``` Three states still hold an apply back, because in them no flush can be made: a commit window a -[`CommitRequest`](configuration.md#commitrequest) holds (a save defines its own commit boundary, and +[`SaveRequest`](configuration.md#saverequest) holds (a save defines its own commit boundary, and the apply waits for it), a missing parent branch being recovered, and a checkout a failed write left dirty. In the last two the branch worker cannot say what is published. @@ -404,7 +404,7 @@ Committing and pushing takes on the order of a second at best, and the Kubernete great many changes in a second. Treat a live edit as a change you observe arriving, not a save that returns a confirmation. Two edits inside one commit window produce one commit holding the later value, so Git history records the states the worker captured, with gaps between them. When a -specific change has to be confirmed in Git, use a [`CommitRequest`](configuration.md#commitrequest). +specific change has to be confirmed in Git, use a [`SaveRequest`](configuration.md#saverequest). ## Keep controller bookkeeping out of Git diff --git a/docs/commit-messages.md b/docs/commit-messages.md index 540c1ca3..55e89e5b 100644 --- a/docs/commit-messages.md +++ b/docs/commit-messages.md @@ -15,8 +15,8 @@ message only on some branch passes validation and falls back at commit time. | Input, in precedence order | Message source | |---|---| -| Attached [save request](configuration.md#commitrequest) **and** `requestTemplate` set | `requestTemplate`, with the request's message as `.RequestMessage` | -| Non-empty literal override, including an attached [save request](configuration.md#commitrequest) | Exact supplied text | +| Attached [save request](configuration.md#saverequest) **and** `requestTemplate` set | `requestTemplate`, with the request's message as `.RequestMessage` | +| Non-empty literal override, including an attached [save request](configuration.md#saverequest) | Exact supplied text | | Live window of any size | `liveTemplate` | | Atomic snapshot or resync | `reconcileTemplate` | @@ -61,7 +61,7 @@ Git author header when attribution ran without resolving an actor. Messages neve ## Framing a save message -By default a [save request](configuration.md#commitrequest)'s message **replaces** the template, so supplying one +By default a [save request](configuration.md#saverequest)'s message **replaces** the template, so supplying one costs you the resource body `liveTemplate` would have produced: the commit says why, but no longer says what. `requestTemplate` composes the two. @@ -95,7 +95,7 @@ than scanning the template's text. reading a label some resources do not carry is caught when the target reconciles rather than at commit time. If one does fail to render in production, the request's message is committed verbatim instead of the -window being lost, and the commit is counted under `message_source="commit_request_fallback"`. +window being lost, and the commit is counted under `message_source="save_request_fallback"`. Alert on that rate: the commit itself succeeds and no condition moves, so it is the only signal. ## Kind, scope, and labels diff --git a/docs/components.md b/docs/components.md index 3174d653..fe2e2c10 100644 --- a/docs/components.md +++ b/docs/components.md @@ -23,7 +23,7 @@ the target watches supply *state*, the branch worker owns *Git*, and audit only ```mermaid flowchart TB subgraph CFG["Config plane (operator's own cluster)"] - CRD["api/v1alpha3
GitTarget · WatchRule · ClusterWatchRule
GitProvider · GitBranch · KubeProvider · CommitRequest
FluxClaim · ArgoCDClaim"] + CRD["api/v1alpha3
GitTarget · WatchRule · ClusterWatchRule
GitProvider · GitBranch · KubeProvider · SaveRequest
FluxClaim · ArgoCDClaim"] CTRL["internal/controller
reconcilers"] ADM["internal/webhook
admission handlers"] AUTHZ["internal/authz
namespace admission"] @@ -78,7 +78,7 @@ The operator's own API surface and the reconcilers behind it. |---|---|---| | CRD types | [`api/v1alpha3/`](../api/v1alpha3/) | the ten kinds users apply | | Reconcilers | [`internal/controller/`](../internal/controller/) | one per kind, plus shared condition and status helpers | -| Admission handlers | [`internal/webhook/`](../internal/webhook/) | the always-allow observer and `validate-operator-types`, which captures a `CommitRequest` submitter | +| Admission handlers | [`internal/webhook/`](../internal/webhook/) | the always-allow observer and `validate-operator-types`, which captures a `SaveRequest` submitter | | Access grants | [`internal/authz/`](../internal/authz/) | one admission decision per act, read from the grant lists on the referenced object (`allowedGitBranches`, `allowedGitTargets`, `allowedApplierClaims`) with the scope that applies, in their own package because several call sites need the same answer | | Rule store | [`internal/rulestore/`](../internal/rulestore/) | compiled `WatchRule` and `ClusterWatchRule` cache the data plane reads | @@ -137,7 +137,7 @@ Optional, and deliberately unable to affect *what* is written. It only names the | Fact transport | [`internal/queue/fact_stream.go`](../internal/queue/fact_stream.go) | Redis Streams by default, an in-process ring with `--author-attribution-transport=memory` (which frees attribution from Redis, but not the watch resume cursors) | | Fact index and follower | [`internal/queue/fact_index.go`](../internal/queue/fact_index.go) | a bounded, TTL'd in-process index, followed per type while at least one watch needs it | | Author resolver | [`internal/watch/author_resolver.go`](../internal/watch/author_resolver.go) | joins a watch event to a fact within a bounded grace window | -| Command author store | [`internal/queue/command_author_store.go`](../internal/queue/command_author_store.go) | the `CommitRequest` submitter captured at admission | +| Command author store | [`internal/queue/command_author_store.go`](../internal/queue/command_author_store.go) | the `SaveRequest` submitter captured at admission | ### Support and tooling diff --git a/docs/configuration.md b/docs/configuration.md index 97ff1a07..4eb8054b 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -12,7 +12,7 @@ The short version: - `WatchRule` defines which namespaced resources should produce Git writes, and in which source namespaces - `ClusterWatchRule` does the same for cluster-scoped resources -- `CommitRequest` optionally saves: it closes its author's commit window on its own timers, with +- `SaveRequest` optionally saves: it closes its author's commit window on its own timers, with its own message if it has one - `FluxClaim` and `ArgoCDClaim` hold the Flux `Kustomization` or Argo CD `Application` that applies a folder back, so GitOps Reverser decides when it applies @@ -35,7 +35,7 @@ The usual flow is: 3. Create a `GitTarget` that points at the `GitBranch`, the `KubeProvider` to mirror from, and a folder in the repository. 4. Create one or more `WatchRule` or `ClusterWatchRule` objects that point at that target. -5. Create a `CommitRequest` only when you want a save: your writes in one commit, on the request's +5. Create a `SaveRequest` only when you want a save: your writes in one commit, on the request's timers rather than the target's, and optionally with your message. 6. Only when Flux or Argo CD also applies that same folder back to the cluster: hold its `Kustomization` or `Application` with an applier claim, name the applier claim on the `GitTarget`, and @@ -105,7 +105,7 @@ Set exactly one of `secretRef` and `inCluster`. Both are immutable: delete and r | `client.qps` | `20` | Client rate limit for this source cluster. Refused beside `inCluster` | | `client.burst` | `30` | Client burst for this source cluster. Refused beside `inCluster` | | `attribution.auditRoute` | `.` | Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) | -| `saveRequests` | `Disabled` | `Enabled` serves the `SaveRequest`s created on this cluster, which writes to it. See [`SaveRequest`](#saverequest) | +| `remoteSaveRequests` | `Disabled` | `Enabled` serves the `RemoteSaveRequest`s created on this cluster, which writes to it. See [`RemoteSaveRequest`](#remotesaverequest) | ### `GitTarget` (namespaced): which branch and folder @@ -119,7 +119,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | -| `commit.unattributedWindows` | `Keep` | How much a `CommitRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | +| `commit.unattributedWindows` | `Keep` | How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | | `placement.useKustomize` | off | Keep the folder a kustomize folder. See [keeping the folder a kustomize folder](#keeping-the-folder-a-kustomize-folder-specplacementusekustomize) | @@ -173,7 +173,7 @@ The references are immutable: to hold another `Application`, delete the applier | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | | `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync stays behind the branch after a push of GitOps Reverser's own before a sync of the tip is asked for. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | -### `CommitRequest` (namespaced): save now +### `SaveRequest` (namespaced): save now | Field | Default | What it does | |---|---|---| @@ -187,12 +187,12 @@ The references are immutable: to hold another `Application`, delete the applier | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | | `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it | -### `SaveRequest` (namespaced, on a source cluster): save the edits made here +### `RemoteSaveRequest` (namespaced, on a source cluster): save the edits made here | Field | Default | What it does | |---|---|---| | `message` | each target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | -| `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`SaveRequest`](#saverequest) | +| `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | | `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit is pushed; `status.targets` reports it | @@ -481,7 +481,7 @@ admits no new work under the revoked permission. Work it already held may comple - **A `GitBranch` no grant admits any more** gives up its branch the way a deleted one does, but keeps its finalizer. Its branch worker closes its intake, pushes or refuses each pending write and - settles every `CommitRequest`; an event that arrives afterwards is refused. The `GitBranch` reports + settles every `SaveRequest`; an event that arrives afterwards is refused. The `GitBranch` reports `Stalled=True` with the grant's reason. Nobody holds the branch until a waiting `GitBranch` takes it on its own reconcile, which checks its grant first, so a candidate revoked by the same edit never holds it. The new holder starts a worker of its own, with none of the old one's work. Writes @@ -552,7 +552,7 @@ kind: ClusterRole metadata: {name: gitops-reverser-tenant} rules: - apiGroups: [configbutler.ai] - resources: [gitbranches, gittargets, watchrules, fluxclaims, argocdclaims, commitrequests] + resources: [gitbranches, gittargets, watchrules, fluxclaims, argocdclaims, saverequests] verbs: [get, list, watch, create, update, patch, delete] --- apiVersion: v1 @@ -583,7 +583,7 @@ Role also needs whatever team A's own workloads require. | Identity | Manages | Cannot | |---|---|---| | The platform's reconciler | `GitProvider`, `KubeProvider` and their grants; Namespaces and their labels; tenant RBAC; platform `GitBranch`es such as `main` | | -| `team-a/tenant-reconciler` | `GitBranch`, `GitTarget`, `WatchRule`, applier claims and `CommitRequest`s in `team-a` | Create or edit a provider; label a Namespace; change RBAC | +| `team-a/tenant-reconciler` | `GitBranch`, `GitTarget`, `WatchRule`, applier claims and `SaveRequest`s in `team-a` | Create or edit a provider; label a Namespace; change RBAC | The last column is what makes the grants hold: a tenant cannot grant itself a label, and cannot create the second provider the limit describes. Flux's multi-tenancy lockdown @@ -1259,7 +1259,7 @@ at, and `Ready`; ### Deleting a `GitBranch` A `GitBranch` carries a finalizer. Deleting it stops new intake for every `GitTarget` on the -branch, pushes or refuses each pending write, gives each attached `CommitRequest` its outcome, and +branch, pushes or refuses each pending write, gives each attached `SaveRequest` its outcome, and only then removes the finalizer and lets another `GitBranch` hold the branch. **Nothing deletes the branch on the remote.** @@ -1420,7 +1420,7 @@ kubectl annotate gittarget -n \ The kstatus trio is also on Prometheus, so the same answer is alertable without a kubeconfig: `gitopsreverser_resource_condition{kind="GitTarget", type="Ready", status="False"} == 1` names every target that has not been accepted, and `reason` says which gate refused. The metric covers `Ready`, -`Reconciling` and `Stalled` for every object on this page except `CommitRequest`; the per-kind axis +`Reconciling` and `Stalled` for every object on this page except `SaveRequest`; the per-kind axis conditions above stay on the object, where `kubectl describe` reads them. See [interpreting-metrics.md](interpreting-metrics.md#configuration-state). @@ -1442,8 +1442,8 @@ spec: liveTemplate: "chore: sync {{.Count}} resource{{if ne .Count 1}}s{{end}}" ``` -Configure [automatic messages](commit-messages.md), submit a [save message](#commitrequest), -and interpret its [conditions](#commitrequest). +Configure [automatic messages](commit-messages.md), submit a [save message](#saverequest), +and interpret its [conditions](#saverequest). #### The commit window (`spec.commit.window`) @@ -1458,7 +1458,7 @@ timers, whichever comes first: `idleTimeout` may not exceed `maxDuration`. A window takes the target's timers when it opens, so editing them affects windows that open afterward, never one already running. A -[`CommitRequest`](#the-request-window-specwindow) that attaches to a window replaces both timers +[`SaveRequest`](#the-request-window-specwindow) that attaches to a window replaces both timers for that window. One branch worker holds one live window, bound to an author and target. Interleaved authors or @@ -1482,7 +1482,7 @@ ever update. Three templates decide how a commit is phrased: `liveTemplate` for a live window, `reconcileTemplate` for an atomic snapshot or resync, and `requestTemplate` for a window a -[save request](#commitrequest) attached to. Each is a Go template over a documented context, +[save request](#saverequest) attached to. Each is a Go template over a documented context, checked when the target reconciles rather than at commit time, and each has a built-in default, so a target that sets none still writes useful messages. @@ -1598,7 +1598,7 @@ commit seconds later. That is deliberate: a local commit that is never pushed wo operator's checkout and surface later, out of order, when you resume. Suspend is a valve on new work, not an undo. -A `CommitRequest` that ends on a suspended target fails with `FinalizeFailed` and the reason, rather +A `SaveRequest` that ends on a suspended target fails with `FinalizeFailed` and the reason, rather than resolving `Ready=True` with nothing written. The exception is `WindowMismatch`: a request that only saw another author's window still reports that. One whose commit was already made locally when you set `suspend` is pushed with it and resolves normally. @@ -1897,7 +1897,7 @@ spec: due, the branch worker closes the open commit window, publishes the writes decided up to that moment at once, and asks for the tip that publication produced. An edit decided after that is recorded in the apply window, so an apply that writes over it is withheld from Git and repaired. A -registered `CommitRequest` (a save sets its own commit boundary), a +registered `SaveRequest` (a save sets its own commit boundary), a parent branch being recovered, or a checkout a failed write left dirty holds the request back. No applier is asked for anything until every initial snapshot of every `GitTarget` naming its applier claim is published, and nothing is asked while the remote does not carry the branch. The rule is the same @@ -1908,7 +1908,7 @@ for a `Kustomization` and an `Application`. | `ForeignCommit` | The branch gained a commit GitOps Reverser did not push. The pending writes are replayed onto it and pushed first, so the request names that pushed commit | Every applier declared on the branch, once per applier | | `Refusal` | An eligible refused edit, with `spec.onRefusal: Reapply` | This target's applier | | `Repair` | An apply reverted an API edit made after the flush; Git kept the edit | That applier | -| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, once when a `GitTarget` first names the applier claim, once the applier claim's `catchUpAfter` has passed since a push of GitOps Reverser's own, and after a `CommitRequest` with [`catchUp: AfterPush`](#applying-a-save-at-once) pushed a commit | To adopt, that folder's applier; to converge, every applier declared on the branch at start, the newly named one, the one left behind, or the requesting `GitTarget`'s | +| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, once when a `GitTarget` first names the applier claim, once the applier claim's `catchUpAfter` has passed since a push of GitOps Reverser's own, and after a `SaveRequest` with [`catchUp: AfterPush`](#applying-a-save-at-once) pushed a commit | To adopt, that folder's applier; to converge, every applier declared on the branch at start, the newly named one, the one left behind, or the requesting `GitTarget`'s | An API edit asks for no apply, and neither does a push of GitOps Reverser's own, at once: the cluster already runs what they hold. Only the applier's own state is left behind, which `catchUpAfter` @@ -2961,9 +2961,9 @@ types in `clusterResources` do, and for those types its source credential's Kube cluster admin can create one, and no default grant, the chart's included, gives `clusterResources`. Use it sparingly. -## `CommitRequest` +## `SaveRequest` -`CommitRequest` is a one-shot "save" signal for a same-namespace `GitTarget`. It does not create or +`SaveRequest` is a one-shot "save" signal for a same-namespace `GitTarget`. It does not create or change watch rules. It attaches to its author's commit window, gives that window its message, and decides when the window closes, instead of the target's timers. @@ -2980,7 +2980,7 @@ Example: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: save-now namespace: default @@ -2997,7 +2997,7 @@ spec: maxDuration: "2s" ``` -The entire spec is immutable. Create a new `CommitRequest` for each save attempt. +The entire spec is immutable. Create a new `SaveRequest` for each save attempt. A present message accepts 1–1024 Unicode characters, including newline. All other ASCII control characters, including tab, carriage return, and DEL, are rejected, as is whitespace-only text. @@ -3084,7 +3084,7 @@ claiming nothing: |---|---| | `Own` | nothing: it attaches only to its submitter's own window | | `SavedBy` | gives the window its message; the commit names no author, and its message ends with `Saved-By: ` | -| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: CommitRequest /; no audit record names who made these changes` | +| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: SaveRequest /; no audit record names who made these changes` | `TakeOver` on the `GitTarget` allows both claims, and `SavedBy` allows only `SavedBy`. A request that asks for more than its `GitTarget` allows fails when it is first seen, before it attaches: @@ -3109,7 +3109,7 @@ finalize and the push the request is still in flight, and the controller keeps r attach until it reads an outcome; the worker recognizes the re-send as the same request, so a request never resolves `NoWindow` while its commit waits out the push cooldown. Applying resources and a request together gives no ordering guarantee between controllers and watch streams. See the -[request contract](spec/commitrequest-design.md). +[request contract](spec/saverequest-design.md). ### When there is nothing to commit @@ -3140,7 +3140,7 @@ A crash between the push and the status write can record the message twice: the repeated request only while it runs. Progress and outcome are reported through kstatus-compatible **conditions** (no `phase` string). -`kubectl get commitrequest` shows `Ready`, its `Reason` and the `Commit`; `-o wide` adds +`kubectl get saverequest` shows `Ready`, its `Reason` and the `Commit`; `-o wide` adds `AuthorAttributed` and `Pushed`. Automation must stop on either `Ready=True` or `Stalled=True`; `kubectl wait --for=condition=Ready` alone keeps waiting on terminal failures. `Ready=True` includes successful no-commit outcomes. Require `Pushed=True` and `status.commit` for evidence that the @@ -3197,7 +3197,7 @@ deleted only after its `Applied` condition is settled. ### Finished requests are deleted after 48 hours -Every save leaves one `CommitRequest` behind. When a request finishes (`Ready=True` or +Every save leaves one `SaveRequest` behind. When a request finishes (`Ready=True` or `Stalled=True`), the controller writes the time it will be deleted onto it: ```yaml @@ -3215,22 +3215,22 @@ contract, so it is also how you change the outcome for one request: - Set it yourself when you create the request to choose the time up front. The controller never replaces a value that is already there. -The period comes from `--commit-request-ttl` (Helm: `controllerManager.commitRequestTTL`, default +The period comes from `--save-request-ttl` (Helm: `controllerManager.saveRequestTTL`, default `48h`). Changing it only affects requests that finish afterwards. `0` writes no annotation, so new requests are kept. A request that finished before this feature existed has no annotation and is never deleted. -## `SaveRequest` +## `RemoteSaveRequest` -A `SaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next to the -edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It names no -`GitTarget`: it saves the commit window of every `GitTarget` that mirrors its own namespace from that -cluster, and each window gets its message and closes on its timers, as with a -[`CommitRequest`](#commitrequest). +A `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next +to the edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It +names no `GitTarget`: it saves the edits made in its own namespace, in the commit window of every +`GitTarget` that mirrors that namespace from that cluster and allows it. Each window gets its message +and closes on its timers, as with a [`SaveRequest`](#saverequest). ```yaml apiVersion: source.configbutler.ai/v1alpha1 -kind: SaveRequest +kind: RemoteSaveRequest metadata: name: save-now namespace: shop @@ -3239,54 +3239,88 @@ spec: catchUp: AfterPush ``` -### Serving the `SaveRequest`s of a cluster +### Serving the `RemoteSaveRequest`s of a cluster -Two things turn it on, both on purpose: +Three things turn it on, each on purpose: 1. **Install the CRD on the source cluster.** It is one small CRD, - `config/crd/bases/source.configbutler.ai_saverequests.yaml`, apart from the `configbutler.ai` kinds - of the cluster GitOps Reverser runs in. The Helm chart and the installer install it there too, for - the clusters the operator mirrors with its own identity. -2. **Set `spec.saveRequests: Enabled` on the `KubeProvider`.** GitOps Reverser then watches the - cluster's `SaveRequest`s through that provider, and writes each one's status back. It is the one - write GitOps Reverser makes on a source cluster: the provider's credential needs `get`, `list`, - `watch` and `delete` on `saverequests`, and `update` on `saverequests/status`, in the - `source.configbutler.ai` group. A `SaveRequest` is never mirrored, whatever a `WatchRule` selects. + `config/crd/bases/source.configbutler.ai_remotesaverequests.yaml`, apart from the + `configbutler.ai` kinds of the cluster GitOps Reverser runs in. The Helm chart and the installer + install it there too, for the clusters the operator mirrors with its own identity. +2. **Set `spec.remoteSaveRequests: Enabled` on the `KubeProvider`.** GitOps Reverser then watches the + cluster's `RemoteSaveRequest`s through that provider, and writes each one's status back. It is the + one write GitOps Reverser makes on a source cluster: the provider's credential needs `get`, + `list`, `watch` and `delete` on `remotesaverequests`, and `update` on `remotesaverequests/status`, + in the `source.configbutler.ai` group. A `RemoteSaveRequest` is never mirrored, whatever a + `WatchRule` selects. +3. **Grant it on each `GitTarget`, in `spec.allowedRemoteSaveRequests`.** The provider switch makes + the requests reachable; the `GitTarget`'s owner decides which of them may save its windows. An + omitted or empty list admits none. + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +spec: + kubeProviderRef: {name: workspace} + allowedRemoteSaveRequests: + - sourceNamespaces: [shop] + requestSelector: + matchLabels: + app.kubernetes.io/part-of: shop +``` + +A request is admitted when one grant admits both its namespace and its labels. `sourceNamespaces` are +namespaces of the cluster `spec.kubeProviderRef` names, by exact name, or `"*"` for every namespace the +`GitTarget` mirrors. `requestSelector` is optional, and narrows the grant to the requests whose labels +match: use it to pick which of several `GitTarget`s a request saves. Whoever creates a request picks +its labels, so a selector proves nothing about who made it. + +### What a save holds + +A `RemoteSaveRequest` saves only the edits made in its own namespace. It attaches to a window whose +writes are all in that namespace; while it is attached, a write anywhere else, including a +cluster-scoped one, does not join the window and opens one of its own. A window that already holds +writes from elsewhere is left alone, and the save ends `WindowOutsideNamespace`. Save from a +`SaveRequest` on the cluster GitOps Reverser runs in to commit such a window under one message. + +A `GitTarget` that mirrors only cluster-scoped types mirrors no namespace, so no `RemoteSaveRequest` +saves it. ### Who the save is from -A `SaveRequest`'s creator is found from the source cluster's audit events, as the edits' authors are, -so a save attaches only to its creator's windows, by the ordinary author rule. Create it as JSON: -`kubectl` and browsers send JSON, and an API server that drops the audit event of a YAML body loses -the save's author without an error. +A `RemoteSaveRequest`'s creator is found from the source cluster's audit events, as the edits' authors +are, so a save attaches only to its creator's windows, by the ordinary author rule. Create it as +JSON: `kubectl` and browsers send JSON, and an API server that drops the audit event of a YAML body +loses the save's author without an error. | The source cluster | The save | | --- | --- | | sends audit events, and one names the creator | attaches to the creator's windows; `status.author` names them | -| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window whose edits name nobody; the commit's author stays unresolved | +| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window in its namespace whose edits name nobody; the commit's author stays unresolved | | sends audit events, but none names the creator | is refused, `Ready=True` with reason `AuthorUnresolved`: its event went missing, and attaching it would claim the window that should stay unattributed. Save again | `spec.window.attachTimeout` (default `5s`) counts from when GitOps Reverser picks the request up, which is after the request's own audit event arrived, so it only covers the spread between the request's event and its edits'. `maxDuration` (default `2s`) and `idleTimeout` close the attached -window as for a `CommitRequest`. +window as for a `SaveRequest`. ### Its status -`status.targets` lists each `GitTarget` that mirrors the namespace, as `namespace/name` on the cluster -GitOps Reverser runs in, with the save's `reason` there (`Committed`, `NoWindow`, `AlreadyPresent`, -`WindowMismatch` or `Failed`), its `branch` and `commit`, and, with `catchUp: AfterPush`, its -`applied` reason and `appliedRevision`. +`status.gitTargets` lists each `GitTarget` the save was admitted to, by `namespace` and `name` on the +cluster GitOps Reverser runs in, with the save's `reason` there (`Committed`, `NoWindow`, +`AlreadyPresent`, `WindowMismatch`, `WindowOutsideNamespace` or `Failed`), its `branch` and `commit`, +and, with `catchUp: AfterPush`, its `applied` reason and `appliedRevision`. - **Ready** is `True` once the save ended without error in every `GitTarget`, with reason `Committed` when one of them committed, and otherwise the first one's reason, `NoGitTarget` when no `GitTarget` - mirrors the namespace, or `AuthorUnresolved`. `Ready=False` with `Stalled=True` is a failure. + mirrors the namespace, or `AuthorUnresolved`. `Ready=False` with `Stalled=True` is a failure, + reason `NotAllowed` when `GitTarget`s mirror the namespace but none of them admits the request. - **Applied**, with `catchUp: AfterPush`, is `True` once every applier applied a revision that holds the commit, or needed none, `False` only when a branch lost the commit, and `Unknown` with reason `Lost` when GitOps Reverser restarted before it learned how an apply went. -A finished `SaveRequest` is deleted `--commit-request-ttl` (default `48h`) after it finished; `0` -keeps it. +A finished `RemoteSaveRequest` is deleted `--save-request-ttl` (default `48h`) after it finished; +`0` keeps it. ## Audit ingestion settings @@ -3492,7 +3526,7 @@ Move to hand-managed resources when you want: - more than one `GitTarget` - more than one watch rule - cluster-scoped watching with `ClusterWatchRule` -- ad hoc save requests with `CommitRequest` +- ad hoc save requests with `SaveRequest` - direct control over `GitProvider.spec.commit` - direct control over encryption settings diff --git a/docs/definitions.md b/docs/definitions.md index 40444872..2790f4fd 100644 --- a/docs/definitions.md +++ b/docs/definitions.md @@ -176,10 +176,11 @@ the parent branch is not followed. **Commit window.** The interval a branch worker batches one author's writes to one target over before it commits. Only a write opens one; nothing else does. It closes on its `idleTimeout` of silence or its `maxDuration` after opening, whichever comes first, or earlier for a flush, unless a -`CommitRequest` holds it. Windows do not merge across authors, which is why attribution can be +`SaveRequest` holds it. Windows do not merge across authors, which is why attribution can be per-commit. -**Attach.** What a `CommitRequest` does to a commit window: it gives the window its message and +**Attach.** What a `SaveRequest`, or a `RemoteSaveRequest` made on a source cluster, does to a commit +window: it gives the window its message and replaces the window's timers with its own. A request attaches to at most one window and never opens one. One word for the act, in the API (`window.attach`, `attachTimeout`), the code and the docs: not "claim", which names a rule's demand on the read side. @@ -198,7 +199,7 @@ a durable journal, which this is not. `status.remote`. **Commit.** A bare Git commit hash, wherever one appears in the API: `status.remote.commit`, -`status.remote.parent.commit`, `CommitRequest.status.commit`, `status.placement.resolvedAtCommit`. +`status.remote.parent.commit`, `SaveRequest.status.commit`, `status.placement.resolvedAtCommit`. Not "sha", which names an algorithm Git is in the middle of changing, and not "revision", which Flux uses for a polymorphic identifier that may be a tag, a chart version or a composite like `main@sha1:`. If this project ever needs that composite, `revision` is the word waiting for it. diff --git a/docs/demo/commit-window.md b/docs/demo/commit-window.md index 9a0ac35c..591fb9f5 100644 --- a/docs/demo/commit-window.md +++ b/docs/demo/commit-window.md @@ -1,6 +1,6 @@ # Commit-window example -Two Kubernetes resources become one Git commit: one created, one edited. A `CommitRequest` supplies +Two Kubernetes resources become one Git commit: one created, one edited. A `SaveRequest` supplies the message and its own timers, which replace the target's for that one window. You can save after the changes or before them; the example shows both. The request itself is outside the selected resource types, so saving never commits the save button. @@ -139,7 +139,7 @@ Save as `save-now.yaml`. Use `kubectl create` rather than `kubectl apply`: the r ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: generateName: save-hello- namespace: gitops-reverser-quickstart-demo @@ -179,7 +179,7 @@ The request can come first, and wait for the changes. Save as `save-long.yaml`: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: generateName: save-hello- namespace: gitops-reverser-quickstart-demo @@ -228,13 +228,13 @@ reaches the operator after the window closed goes into the next commit. The default configured-author setup without request author capture works for this example. If actor attribution and request author capture are enabled, submit the resources and the request as the same -Kubernetes user. See [CommitRequest](../configuration.md#commitrequest) for author matching and +Kubernetes user. See [SaveRequest](../configuration.md#saverequest) for author matching and outcomes. ## Check the result ```bash -kubectl get commitrequests -n gitops-reverser-quickstart-demo -o wide +kubectl get saverequests -n gitops-reverser-quickstart-demo -o wide ``` Look for `Pushed=True` and a `status.commit`. `Pushed` is a wide column; `Ready` and the `COMMIT` @@ -257,7 +257,7 @@ With `save-long.yaml` the first line is `feat(demo): save long running`. Either | Resource | Captured by this rule? | |---|---| | ServiceAccount, ConfigMap | Yes | -| CommitRequest, GitTarget, WatchRule | No | +| SaveRequest, GitTarget, WatchRule | No | | Services, Deployments, Pods, ReplicaSets | No | Without the request, the window closes on its own five seconds after the last change, and the diff --git a/docs/design/applier-echoes.md b/docs/design/applier-echoes.md index 1ae72e9a..7d86b66a 100644 --- a/docs/design/applier-echoes.md +++ b/docs/design/applier-echoes.md @@ -95,7 +95,7 @@ built and released together. 9. **A save is the strongest API intent, and is never trimmed.** A `CommitRequest` keeps every object it captured, and its outcome is settled by the push, as the - [CommitRequest contract](../spec/commitrequest-design.md) says. Re-asserting a save over a + [CommitRequest contract](../spec/saverequest-design.md) says. Re-asserting a save over a concurrent Git-side change of the same object is the "API wins" rule. ## The race @@ -679,7 +679,7 @@ The build departs from the plan above in twelve places, decided on 2026-10-20: startup baseline. - [`gittarget-branch-worker-pending-writes.md`](gittarget-branch-worker-pending-writes.md): decide, `advance`, materialize, and replay. -- [`../spec/commitrequest-design.md`](../spec/commitrequest-design.md): the save contract F2 keeps. +- [`../spec/saverequest-design.md`](../spec/saverequest-design.md): the save contract F2 keeps. - [`../spec/reconcile-via-watchlist-mark-and-sweep.md`](../spec/reconcile-via-watchlist-mark-and-sweep.md): the resync and its sweep. - [`internal/git/apply_window.go`](../../internal/git/apply_window.go): `withholdLateApply`. diff --git a/docs/design/branch-worker-event-model.md b/docs/design/branch-worker-event-model.md index b9dc44ea..87adcfca 100644 --- a/docs/design/branch-worker-event-model.md +++ b/docs/design/branch-worker-event-model.md @@ -172,9 +172,9 @@ on the worker. For an attached save that committed locally, this left `WaitingForPush` with no time limit. After its safety window, `withdrawCommitRequest` in the -[controller](../../internal/controller/commitrequest_controller.go) sends withdrawal and continues +[controller](../../internal/controller/saverequest_controller.go) sends withdrawal and continues polling when the worker still holds the request. `handleWithdrawCommitRequest` in the -[attach loop](../../internal/git/commit_request_attach_loop.go) leaves attached or committed +[attach loop](../../internal/git/save_request_attach_loop.go) leaves attached or committed requests unchanged. Neither path retries the push. The API showed `Ready=False`, `Reconciling=True`, `Stalled=False`, and `Pushed=Unknown`, with reason `WaitingForPush`. diff --git a/docs/design/commitrequest-save-wait-options.md b/docs/design/commitrequest-save-wait-options.md index 73851628..bf005cde 100644 --- a/docs/design/commitrequest-save-wait-options.md +++ b/docs/design/commitrequest-save-wait-options.md @@ -89,7 +89,7 @@ smallest token the UI already has that can name the save it wants Git to show. Four mechanisms decide the outcome, and the failure is structural rather than a tuning miss. **The deadline is stamped from receipt.** `handleAttachCommitRequest` in -[`commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go) sets +[`save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go) sets `finalizeAt = time.Now().Add(closeDelay)` on first registration, and idempotent re-sends keep that first value. With the field omitted the deadline is the present instant, so the request is due on the next pass of the worker's event loop. @@ -105,7 +105,7 @@ audit in batches bounded by `--audit-webhook-batch-max-wait=1s`, so the floor on is bounded below by that batch plus the join. **Attach is author-bound.** `matchesWindow` in -[`commit_request_attach.go`](../../internal/git/commit_request_attach.go) admits a window only when +[`save_request_attach.go`](../../internal/git/save_request_attach.go) admits a window only when the author strings are equal, the two sides agree on whether an actor is named, and the `GitTarget` identity matches. @@ -331,10 +331,10 @@ importantly, the cluster-level default flag below is **blocked** on the pointer omitted field and an explicit `0` are the same value, so nothing downstream can tell them apart. The type is `v1alpha3` with one known consumer; this is the cheapest it will ever be. -- `api/v1alpha3/commitrequest_types.go`, then `task generate` and `task manifests` -- nil handling at the controller seam (`commitrequest_controller.go`, where the field is copied onto +- `api/v1alpha3/saverequest_types.go`, then `task generate` and `task manifests` +- nil handling at the controller seam (`saverequest_controller.go`, where the field is copied onto `AttachCommitRequest`) -- docs: `configuration.md`, `architecture.md`, `spec/commitrequest-design.md`, an `UPGRADING.md` note +- docs: `configuration.md`, `architecture.md`, `spec/saverequest-design.md`, an `UPGRADING.md` note **Half a day.** Fixes R2. Leaves rows 1, 4, 6 untouched, and moves row 3 from "always" to "when the cluster is slower than two seconds" — an improvement in reliability, not a change in kind. @@ -405,12 +405,12 @@ cluster never saw the write you named" (fact-index lookup, row 4 vs 5), "the tar accepting writes" (row 10), and "nothing was pending" (row 5). - `api/v1alpha3`, CRD, CEL cap -- `commitrequest_controller.go` plumb; `commitrequest_finalize.go` reasons and messages -- `pendingCommitRequest` satisfaction state and the two-timer model in `commit_request_attach_loop.go` +- `saverequest_controller.go` plumb; `saverequest_finalize.go` reasons and messages +- `pendingCommitRequest` satisfaction state and the two-timer model in `save_request_attach_loop.go` - `status.waitingFor` on the object while it waits (R9) -- docs: `spec/commitrequest-design.md`, `spec/status-conditions-guide.md`, `interpreting-metrics.md`, +- docs: `spec/saverequest-design.md`, `spec/status-conditions-guide.md`, `interpreting-metrics.md`, `configuration.md`, `architecture.md` -- unit tests plus a timing-sensitive e2e in `test/e2e/commit_request_e2e_test.go` +- unit tests plus a timing-sensitive e2e in `test/e2e/save_request_e2e_test.go` **Three to four days.** Fixes rows 1, 2, 3, 7, 9 outright, and row 10 via the hook above. Splits row 4 from row 5 only where a usable fact exists. Does **not** fix row 11. @@ -503,9 +503,9 @@ the participant's message on the commit. ## Related -- [`../spec/commitrequest-design.md`](../spec/commitrequest-design.md), the current contract. The +- [`../spec/saverequest-design.md`](../spec/saverequest-design.md), the current contract. The code depends on it, so each phase changes that page in the same commit. -- [`../spec/commitrequest-admission-authorship.md`](../spec/commitrequest-admission-authorship.md), +- [`../spec/saverequest-admission-authorship.md`](../spec/saverequest-admission-authorship.md), where the submitter comes from and why the lookup is present-or-never. - [`../spec/watch-event-ordering-and-attribution-grace.md`](../spec/watch-event-ordering-and-attribution-grace.md), the head-of-line wait this page proposes to wait behind rather than shorten. diff --git a/docs/design/e2e-coverage-gaps-and-improvements-plan.md b/docs/design/e2e-coverage-gaps-and-improvements-plan.md index 9dade6bd..d3841793 100644 --- a/docs/design/e2e-coverage-gaps-and-improvements-plan.md +++ b/docs/design/e2e-coverage-gaps-and-improvements-plan.md @@ -39,7 +39,7 @@ Reading it against the architecture surfaced three gaps, in increasing order of | Attribution from audit identity | [commit_author_attribution](../../test/e2e/commit_author_attribution_e2e_test.go), [impersonation](../../test/e2e/impersonation_test.go) | ✅ | | Commit signing (generated + BYOK) | [signing](../../test/e2e/signing_e2e_test.go) | ✅ | | Encryption / SOPS Secrets | [watchrule_configmap_secret](../../test/e2e/watchrule_configmap_secret_e2e_test.go), [flux_bi_directional](../../test/e2e/flux_bi_directional_e2e_test.go) | ✅ | -| Commit windows + CommitRequest | [commit_window_batching](../../test/e2e/commit_window_batching_e2e_test.go), [commit_request](../../test/e2e/commit_request_e2e_test.go) | ✅ | +| Commit windows + CommitRequest | [commit_window_batching](../../test/e2e/commit_window_batching_e2e_test.go), [commit_request](../../test/e2e/save_request_e2e_test.go) | ✅ | | Scale subresource → `spec.replicas` | [deployment_scale_subresource](../../test/e2e/deployment_scale_subresource_e2e_test.go) | ✅ | | Aggregated API server (LIST fallback) | [aggregated_apiserver](../../test/e2e/aggregated_apiserver_e2e_test.go) | ✅ | | GitTarget isolation / path overlap | [gittarget_isolation](../../test/e2e/gittarget_isolation_e2e_test.go), [gittarget_overlap](../../test/e2e/gittarget_overlap_e2e_test.go) | ✅ | diff --git a/docs/design/e2e-suite-trim.md b/docs/design/e2e-suite-trim.md index 6fb6024e..fed995a3 100644 --- a/docs/design/e2e-suite-trim.md +++ b/docs/design/e2e-suite-trim.md @@ -24,7 +24,7 @@ can go; each costs a full rebuild. This removes a 14–18 minute leg from most P These matter more than the duplicates below. -1. **[`commit_request_e2e_test.go`](../../test/e2e/commit_request_e2e_test.go)**: a comment says the +1. **[`save_request_e2e_test.go`](../../test/e2e/save_request_e2e_test.go)**: a comment says the generateName spec "is skipped in configured-author mode", but it has no `Skip`. Add it, or fix the comment. 2. **[`signing_e2e_test.go`](../../test/e2e/signing_e2e_test.go)**: its own comment says the diff --git a/docs/design/event-pipeline-overview.md b/docs/design/event-pipeline-overview.md index 501a9719..dc0c7551 100644 --- a/docs/design/event-pipeline-overview.md +++ b/docs/design/event-pipeline-overview.md @@ -571,5 +571,5 @@ remains: | Retry deadline | [`internal/git/retry.go`](../../internal/git/retry.go) | | Publication report and backlog gauges | [`internal/git/publication.go`](../../internal/git/publication.go) | | Resyncs | [`internal/git/resync_flush.go`](../../internal/git/resync_flush.go) | -| Saves | [`internal/git/commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go) | +| Saves | [`internal/git/save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go) | | Missing parent probe | [`internal/git/parent_recovery.go`](../../internal/git/parent_recovery.go) | diff --git a/docs/design/git-write-preflight.md b/docs/design/git-write-preflight.md index 033a0f16..b5aa4890 100644 --- a/docs/design/git-write-preflight.md +++ b/docs/design/git-write-preflight.md @@ -879,7 +879,7 @@ point where an author can most easily respond. [render-verification]: ../../internal/manifestanalyzer/render_verify.go [namespace-admission]: ../../internal/webhook/watchrule_source_namespace_admission.go [oracle-tests]: ../../internal/git/kustomize_oracle_test.go -[commitrequest]: ../spec/commitrequest-design.md +[commitrequest]: ../spec/saverequest-design.md [placement-tests]: ../../internal/git/placement_test.go [consent]: support-boundary/admission-consent.md [validation-placement]: ../spec/where-validation-lives.md diff --git a/docs/design/gittarget-branch-worker-pending-writes.md b/docs/design/gittarget-branch-worker-pending-writes.md index fe7ff683..d132ec80 100644 --- a/docs/design/gittarget-branch-worker-pending-writes.md +++ b/docs/design/gittarget-branch-worker-pending-writes.md @@ -492,7 +492,7 @@ Remaining limits: Updated the event model's "Next implementation" and "Problems" sections, the findings in the state-of-affairs page, the paragraph after the gate table in -[`commitrequest-design.md`](../spec/commitrequest-design.md#when-the-target-may-not-be-written), +[`saverequest-design.md`](../spec/saverequest-design.md#when-the-target-may-not-be-written), and [`architecture.md`](../architecture.md), which gained a section on a remote that cannot be reached. The effective-point comment in `write_gate.go` already said what was built and is unchanged. `UPGRADING.md` gained its entries with each step. The PR body carries the argument; this @@ -574,7 +574,7 @@ decided from is used once: the replay, under `prune.mode: Always` too. - **A save keeps every object.** A write carrying a `CommitRequest` is never trimmed: it keeps each object it captured, unchanged ones included, and the push decides its outcome (`AlreadyPresent` - or `Committed`), as the [CommitRequest contract](../spec/commitrequest-design.md) says. + or `Committed`), as the [CommitRequest contract](../spec/saverequest-design.md) says. Pinned by [`applier_echoes_test.go`](../../internal/git/applier_echoes_test.go): an unchanged object is no action, a resync replay applies its decided actions, a replayed resync obeys a tightened prune diff --git a/docs/design/gittarget-parent-hardening.md b/docs/design/gittarget-parent-hardening.md index aa4827bb..c5b25bf9 100644 --- a/docs/design/gittarget-parent-hardening.md +++ b/docs/design/gittarget-parent-hardening.md @@ -87,7 +87,7 @@ older API-wave plan as part of any of these changes. Recovery can hold work for minutes, so a request must not be failed on a controller clock while the worker still holds it. That is already fixed on this branch: the controller waits for any request the worker holds (`CollectingWindow`, `WaitingForPush`), and fails an unheld one only after -withdrawing it from the worker (`docs/spec/commitrequest-design.md`). The §4.4 tests must still +withdrawing it from the worker (`docs/spec/saverequest-design.md`). The §4.4 tests must still include a request retained through a recovery longer than `attachTimeout + maxDuration + 120s`, resolving `Committed`. diff --git a/docs/design/gittarget-state-of-affairs.md b/docs/design/gittarget-state-of-affairs.md index e4e4a984..dddbbe88 100644 --- a/docs/design/gittarget-state-of-affairs.md +++ b/docs/design/gittarget-state-of-affairs.md @@ -30,7 +30,7 @@ Restarting the Pod loses in-memory decisions and save receipts. Redis persistenc | Every call to a Git server is bounded; a lost push reply is settled by evidence | [`network_bound.go`](../../internal/git/network_bound.go) | `network_bound_test.go` | | One publication report feeds every `GitTarget` on the branch, its held saves, and the backlog metrics | [`publication.go`](../../internal/git/publication.go), [`gittarget_publication.go`](../../internal/controller/gittarget_publication.go) | `publication_test.go`, `gittarget_publication_test.go` | | Replay refusal settles only the affected entry | `pending_writes_loop.go`, `branch_worker.go` | [`replay_refusal_test.go`](../../internal/git/replay_refusal_test.go) | -| Save outcomes follow remote publication; the controller cannot time out a held save | [`commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go), [`commitrequest_controller.go`](../../internal/controller/commitrequest_controller.go) | Held-request retry and controller safety-bound tests | +| Save outcomes follow remote publication; the controller cannot time out a held save | [`save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go), [`saverequest_controller.go`](../../internal/controller/saverequest_controller.go) | Held-request retry and controller safety-bound tests | | Parent selection and compare-and-swap remain publication guards | [`git_atomic_push.go`](../../internal/git/git_atomic_push.go) | Parent-change tests and `TestGitRoundTripLedger` | | New watch streams start with a fresh replay; reconnects resume a cursor only after the stream's own replay completed | [`target_watch.go`](../../internal/watch/target_watch.go) | `runTargetWatch`, `TestRunTargetWatch_ResumesOnlyAfterItsOwnReplayCompleted` | | A live event changes the dedup baseline and the cursor only once the worker accepted it | [`target_watch.go`](../../internal/watch/target_watch.go) | `refused_admission_test.go`, `desired_state_change_filter_test.go` | @@ -144,7 +144,7 @@ and persistence and HA ([HA plan](../future/ha-gittarget-distribution-plan.md)). | `gittarget-branch-worker-pending-writes.md` | The steps as built, the pause/resume contract, and the review findings | | [`branch-worker-event-model.md`](branch-worker-event-model.md) | Transition semantics and the later boundary for reconstructing execution | | [`push-cooldown.md`](push-cooldown.md) | Healthy publication cadence; the success cooldown remains unchanged | -| [`commitrequest-design.md`](../spec/commitrequest-design.md) | Current save contract, including retained versus unaccepted requests | +| [`saverequest-design.md`](../spec/saverequest-design.md) | Current save contract, including retained versus unaccepted requests | | `ha-gittarget-distribution-plan.md` | Future durable acceptance, publication evidence, storage budgets, and HA | Parent observation, empty-repository bootstrap policy, configuration freshness, and refusal state diff --git a/docs/design/reconcile-triggering.md b/docs/design/reconcile-triggering.md index 17a89254..11ae7204 100644 --- a/docs/design/reconcile-triggering.md +++ b/docs/design/reconcile-triggering.md @@ -58,7 +58,7 @@ and out-of-band age-key updates ride the 5-minute reconcile instead. Sources: `gitprovider_controller.go`, `gittarget_controller.go`, `watchrule_controller.go`, `clusterwatchrule_controller.go`, -`commitrequest_controller.go`. +`saverequest_controller.go`. ### 1.3 Two structural consequences @@ -143,9 +143,9 @@ not reconciled until an unrelated event happens to fire. (Full treatment, incl. subtree-hash detection and Flux/Argo coordination, in the sibling doc.) **A4. CommitRequest outcomes are polled, not pushed.** The worker records a terminal -outcome (`recordCommitRequestOutcome`, `internal/git/commit_request_attach_loop.go:47`) +outcome (`recordCommitRequestOutcome`, `internal/git/save_request_attach_loop.go:47`) and the controller reads it (`LookupCommitRequestOutcome`) — but by **requeuing -every 2 s** (`commitrequest_controller.go:195`). The capture half of the recipe +every 2 s** (`saverequest_controller.go:195`). The capture half of the recipe already exists; only the push edge is missing. A nudge would replace the poll. ### Gap B — control-plane status does not propagate to dependents diff --git a/docs/facts/crd-descriptions-in-kubectl-explain.md b/docs/facts/crd-descriptions-in-kubectl-explain.md index 2254eba2..2a6cd8aa 100644 --- a/docs/facts/crd-descriptions-in-kubectl-explain.md +++ b/docs/facts/crd-descriptions-in-kubectl-explain.md @@ -75,7 +75,7 @@ of the generated CRD and point `crdDir` at the copy. These came out of the same review. Each one is behavior that a shorter or rearranged description can easily misstate. -- **A CommitRequest's message can be wrapped.** The message is never evaluated as a template. +- **A SaveRequest's message can be wrapped.** The message is never evaluated as a template. When the GitTarget sets `spec.commit.message.requestTemplate`, that template frames the message as `RequestMessage`; without one, the message is committed as written. See [`commit_executor.go`](../../internal/git/commit_executor.go). diff --git a/docs/facts/generated-name-support.md b/docs/facts/generated-name-support.md index e75c723d..3c7cb339 100644 --- a/docs/facts/generated-name-support.md +++ b/docs/facts/generated-name-support.md @@ -31,12 +31,12 @@ A client that wants the server to allocate the name sends a collection `POST` with `metadata.generateName` and without `metadata.name`: ```text -POST /apis/configbutler.ai/v1alpha3/namespaces/voter-production/commitrequests +POST /apis/configbutler.ai/v1alpha3/namespaces/voter-production/saverequests ``` ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: namespace: voter-production generateName: coffee-save- @@ -50,7 +50,7 @@ The successful API response contains the final server-allocated identity: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: namespace: voter-production generateName: coffee-save- @@ -94,16 +94,16 @@ Example audit event shape: "level": "RequestResponse", "stage": "ResponseComplete", "verb": "create", - "requestURI": "/apis/configbutler.ai/v1alpha3/namespaces/voter-production/commitrequests", + "requestURI": "/apis/configbutler.ai/v1alpha3/namespaces/voter-production/saverequests", "objectRef": { "apiGroup": "configbutler.ai", "apiVersion": "v1alpha1", - "resource": "commitrequests", + "resource": "saverequests", "namespace": "voter-production" }, "requestObject": { "apiVersion": "configbutler.ai/v1alpha3", - "kind": "CommitRequest", + "kind": "SaveRequest", "metadata": { "namespace": "voter-production", "generateName": "coffee-save-" @@ -111,7 +111,7 @@ Example audit event shape: }, "responseObject": { "apiVersion": "configbutler.ai/v1alpha3", - "kind": "CommitRequest", + "kind": "SaveRequest", "metadata": { "namespace": "voter-production", "generateName": "coffee-save-", @@ -139,14 +139,14 @@ This keeps the URL-level audit reference authoritative when it is complete, but still handles server-named objects whose final identity only appears in the response body. -## `CommitRequest` example +## `SaveRequest` example -`CommitRequest` objects are intentionally suitable for generated names because +`SaveRequest` objects are intentionally suitable for generated names because each object is a one-shot save signal: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: namespace: voter-production generateName: save- diff --git a/docs/facts/subresources.md b/docs/facts/subresources.md index 079eb357..17acfcbd 100644 --- a/docs/facts/subresources.md +++ b/docs/facts/subresources.md @@ -188,7 +188,7 @@ marker on the type: ``` Every `configbutler.ai` CRD enables `/status`: `GitProvider`, `GitTarget`, -`WatchRule`, `ClusterWatchRule`, and `CommitRequest`. A real `GitTarget`: +`WatchRule`, `ClusterWatchRule`, and `SaveRequest`. A real `GitTarget`: ```yaml apiVersion: configbutler.ai/v1alpha3 @@ -263,7 +263,7 @@ status: ``` No `configbutler.ai` resource enables `/scale`: `GitProvider`, `GitTarget`, -`WatchRule`, `ClusterWatchRule`, and `CommitRequest` are not scalable workloads, +`WatchRule`, `ClusterWatchRule`, and `SaveRequest` are not scalable workloads, so a replica count has no meaning for them. ## Audit logs and the `/scale` subresource @@ -432,14 +432,14 @@ For an imperative operation, Kubernetes API design has two options: a custom subresource (requires an aggregated API server), or a **request resource** — a normal CRD that represents "please do X", carrying its outcome in `status`. -`configbutler.ai` uses the request-resource pattern with **`CommitRequest`**. +`configbutler.ai` uses the request-resource pattern with **`SaveRequest`**. Instead of a hypothetical `POST /gittargets/{name}/commit` subresource, creating -a `CommitRequest` object finalizes the open commit window for the referenced +a `SaveRequest` object finalizes the open commit window for the referenced `GitTarget` and reports the result back in status: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: save-now namespace: default @@ -475,7 +475,7 @@ action: the request itself is a durable, inspectable object. | Durable declarative intent | CRD | | Controller reports observed state | CRD + `/status` | | HPA / `kubectl scale` compatibility | CRD + `/scale` | -| One-off declarative job/request | Request CRD (e.g. `CommitRequest`) | +| One-off declarative job/request | Request CRD (e.g. `SaveRequest`) | | Derived synchronous or streaming output | Aggregated API subresource | | Proxy / tunnel / console / session | Aggregated API subresource | | Imperative action | Request CRD, or aggregated API subresource if it must be synchronous/streaming | @@ -501,5 +501,5 @@ The pattern Kubernetes follows: `configbutler.ai` stays entirely within CRDs: every type is served by the main kube-apiserver, every type uses `/status`, none use `/scale`, and imperative -actions are modeled as request resources (`CommitRequest`) rather than custom +actions are modeled as request resources (`SaveRequest`) rather than custom subresources. diff --git a/docs/finished/redis-key-schema-v3.md b/docs/finished/redis-key-schema-v3.md index ae2ad576..1d741f66 100644 --- a/docs/finished/redis-key-schema-v3.md +++ b/docs/finished/redis-key-schema-v3.md @@ -5,7 +5,7 @@ > Status: **IMPLEMENTED** — 2026-06-29. The `v2 → v3` key-format bump landed for the live Redis families: > the audit author index, the watch-resume cursor, and the new command-author **store** (storage > mechanics only — the CommitRequest admission/controller rewrite in -> [commitrequest-admission-authorship.md](../spec/commitrequest-admission-authorship.md) is a separate round and +> [saverequest-admission-authorship.md](../spec/saverequest-admission-authorship.md) is a separate round and > is **not** wired up yet). Scope: (1) replace the opaque `attr` namespace with a top-level `author` > domain; (2) re-lay audit-sourced resource author facts as a readable, **object-grouped** hierarchy > (`:object::` + `:last`, plus a type-scoped `:rv:` escape hatch), carrying @@ -24,7 +24,7 @@ > `internal/queue/attribution_index.go`, > [`internal/queue/redis_store.go`](../../internal/queue/redis_store.go), > [`internal/watch/author_resolver.go`](../../internal/watch/author_resolver.go), -> [CommitRequest authorship from admission](../spec/commitrequest-admission-authorship.md), +> [CommitRequest authorship from admission](../spec/saverequest-admission-authorship.md), > [attribution](../spec/attribution.md), > [watch event ordering & attribution grace](../spec/watch-event-ordering-and-attribution-grace.md), > [watch-first ingestion architecture](watch-first-ingestion-architecture.md). diff --git a/docs/future/config-surface-for-a-structured-repository.md b/docs/future/config-surface-for-a-structured-repository.md index f04c15e8..55465fc2 100644 --- a/docs/future/config-surface-for-a-structured-repository.md +++ b/docs/future/config-surface-for-a-structured-repository.md @@ -4,7 +4,7 @@ > Date: 2026-07-24 > Absorbs and extends the API-surface block left unbuilt by the status and configuration-model > review — `spec.suspend`, `spec.interval` and the reconcile-request annotation, the -> `CommitRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in +> `SaveRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in > [`gittarget-api-wave.md`](../design/gittarget-api-wave.md). ## The one-sentence finding @@ -239,7 +239,7 @@ Still open from the companion review, all API-surface, all cheaper now than afte - **F6** — `spec.suspend` on GitTarget / WatchRule / ClusterWatchRule / GitProvider; `spec.interval` on GitProvider at minimum; jitter the requeue; `reconcile.configbutler.ai/requestedAt` + `status.lastHandledReconcileAt`. -- **F10** — CommitRequest lifecycle: `ttlSecondsAfterFinished` or an ownerRef, plus the +- **F10** — SaveRequest lifecycle: `ttlSecondsAfterFinished` or an ownerRef, plus the `delete` verb. Today they accumulate in etcd forever and the controller cannot reap them. - **F9** — verify the stored `scope: Namespaced` status-write path on the minimum supported Kubernetes version (one envtest). @@ -398,7 +398,7 @@ still `v1alpha3` and while there are no users. 3. **F6** — `spec.suspend` on the four reconciled kinds; `spec.interval` on GitProvider; requeue jitter; `reconcile.configbutler.ai/requestedAt` + `status.lastHandledReconcileAt`. 4. **B3** — `spec.placement.mode: Infer|Declared|Strict`. -5. **F10** — CommitRequest TTL/ownerRef + the `delete` verb. +5. **F10** — SaveRequest TTL/ownerRef + the `delete` verb. ### Wave 2 — legibility, non-breaking, ships any time after diff --git a/docs/future/direction-and-configuration-surface.md b/docs/future/direction-and-configuration-surface.md index 878f99c7..88b6b243 100644 --- a/docs/future/direction-and-configuration-surface.md +++ b/docs/future/direction-and-configuration-surface.md @@ -40,7 +40,7 @@ to make and cheap to hold. compatibility frontier. - **The hard prerequisites already shipped or are designed.** The config-plane split (inline `GitTarget.spec.kubeConfig`) points a target at a remote or ephemeral intent - cluster. `CommitRequest` carries authorship and intent. The branch-per-experiment flow + cluster. `SaveRequest` carries authorship and intent. The branch-per-experiment flow is designed in [`idea-application-editing.md`](idea-application-editing.md). - **The layout model is a direction-B feature wearing direction-A clothes.** `kind: Kustomize` with `create: true` bootstraps structure into an empty repository; @@ -295,7 +295,7 @@ environment its own dry run, status, and commit batching. | #295 placement bugs | fix now; direction-agnostic correctness | before the wave | | #296 visibility | metric split, `{kindLower}`, template constant now; `status.layout` rides the wave | split | | #293 layout model | the centerpiece; more valuable under direction B than its own text claims | the wave's core | -| #294 API wave | core four (layout, mode, suspend/interval, commitWindow move) gate nothing else; CommitRequest items ride along without gating | the wave | +| #294 API wave | core four (layout, mode, suspend/interval, commitWindow move) gate nothing else; SaveRequest items ride along without gating | the wave | | #146 per-rule objectSelector | yes; in direction B it is how an intent cluster marks what belongs in the artifact | any time | | #148 leader election | yes, scoped to active-passive; decoupled from full HA distribution | any time | | #210 conventional PR titles | implement the ~15-line workflow check or close; it is a process wish, not a bug | close out | diff --git a/docs/future/ha-gittarget-distribution-plan.md b/docs/future/ha-gittarget-distribution-plan.md index 2ee6c009..b96fb861 100644 --- a/docs/future/ha-gittarget-distribution-plan.md +++ b/docs/future/ha-gittarget-distribution-plan.md @@ -72,7 +72,7 @@ The repository has useful foundations, but it is not HA today. content under the write/prune policy, but cannot recover lost intermediate events or accepted save decisions. Persisted cursors alone do not establish lossless failover. - BranchWorker keeps open commit windows, local commits, unpushed writes, and - CommitRequest outcomes in memory. Its local clone is disposable. + SaveRequest outcomes in memory. Its local clone is disposable. - The pending-writes plan provides ordered pending writes, one materializer, one retry schedule, isolated replay refusals, and an intake budget: during a retry, admission closes on everything accepted and not yet published, queued payloads and empty records included, and producers offer @@ -223,7 +223,7 @@ accepted the push requires publication recovery before repeating an operation or Status projection progresses independently after completion. Retain checkpoints, payload references, pending effects, and receipts needed by every unfinished step. Acknowledging delivery -or deleting a `CommitRequest` must not remove the only deduplication evidence while commands can +or deleting a `SaveRequest` must not remove the only deduplication evidence while commands can still be redelivered. Receipt collection needs a stated projection and redelivery boundary. The local clone remains disposable. Recovery reconstructs the recorded workflow, uses the @@ -418,7 +418,7 @@ durations rather than relying on undocumented defaults. - On startup, claim pending records, rebuild workers from remote Git, then establish watches from their durable cursors or fresh replay. - Enforce journal ownership epochs and reject stale writers at the durable transition boundary. -- Project recorded `CommitRequest` outcomes into Kubernetes status by UID. Recover the terminal +- Project recorded `SaveRequest` outcomes into Kubernetes status by UID. Recover the terminal receipt and any pending projection after failover, without deciding the save outcome again. Exit criterion: failover and supported rolling upgrades preserve the single-active contract, diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index dd4dd25f..2de45c0a 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -1,6 +1,6 @@ # Plan: saving from the source cluster -A `CommitRequest` saves the writes in its author's open commit window. It lives in the control +A `SaveRequest` saves the writes in its author's open commit window. It lives in the control cluster, where the operator runs, while the edits it saves happen on the source cluster. This plan adds a second way to save: a small request created **on the source cluster**, next to the edits, attributed from that cluster's audit stream like the edits are. @@ -9,11 +9,11 @@ It also plans the second step: letting a save take over a window whose writes na opt-in, for a source cluster that cannot send audit events at all. There it is the only way a save can put its author's name on the commit. -Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The source-side save is `SaveRequest` +Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The source-side save is `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/save_requests.go`), served on a KubeProvider with -`spec.saveRequests: Enabled`. The takeover is opt-in, as described below: the GitTarget's -`spec.commit.unattributedWindows` and the CommitRequest's `spec.window.author`, each `SavedBy` or -`TakeOver`. The open questions are answered as proposed: `SaveRequest` in its own group, a default +`spec.remoteSaveRequests: Enabled`. The takeover is opt-in, as described below: the GitTarget's +`spec.commit.unattributedWindows` and the SaveRequest's `spec.window.author`, each `SavedBy` or +`TakeOver`. The open questions are answered as proposed: `RemoteSaveRequest` in its own group, a default `attachTimeout` of `5s`, the reason `AuthorUnresolved`, and no extra queue for a second save on a GitTarget beyond the worker's own (a window carries one request at a time). @@ -23,7 +23,7 @@ This revision takes in gitops-api's review of the first draft. 1. **Build the source-side save first.** It gives the right author wherever the source cluster has audit, and an honestly unresolved one where it does not. -2. **It is a new kind, defined by GitOps Reverser.** A `CommitRequest` with another schema, or a +2. **It is a new kind, defined by GitOps Reverser.** A `SaveRequest` with another schema, or a kind gitops-api defines and translates, were both considered. See [Why a new kind](#why-a-new-kind). 3. **The takeover is the second step.** Some people cannot configure their API server's audit, so @@ -34,16 +34,16 @@ This revision takes in gitops-api's review of the first draft. ## Why: two identity domains -Today the save and the edits are attributed in two places: the `CommitRequest`'s author comes from +Today the save and the edits are attributed in two places: the `SaveRequest`'s author comes from the control cluster's admission webhook, the edits' authors from the source cluster's audit events. They match only when one person has the same username in both. -gitops-api makes them match. Its server impersonates the human when it creates the `CommitRequest`, +gitops-api makes them match. Its server impersonates the human when it creates the `SaveRequest`, so the admission webhook records the same Dex email that kcp's audit records for the edits, and the ordinary author rule attaches. gitops-api does not need a takeover. What it pays for that is elsewhere: -- **A grant into the control cluster.** Every tenant's identity may `create commitrequests` in its +- **A grant into the control cluster.** Every tenant's identity may `create saverequests` in its `tenant-` namespace there. It is the one place a tenant reaches outside its kcp workspace. - **A server on the save path.** The browser cannot reach that namespace, so every save goes through @@ -81,21 +81,21 @@ author can read it. person who made the edits. - **On a source with no audit**, the save names nobody, like the edits. A request that names no actor may attach to a window that names none, which is today's rule - (`TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow`). The save gets its message + (`TestPendingSaveRequest_UnresolvedRequestAttachesToUnresolvedWindow`). The save gets its message and its timers, and the author in Git stays unresolved. The window still holds everyone's unattributed writes, but nobody is named as their author. ### Why a new kind -gitops-api could define its own kind and translate it into a `CommitRequest`, but it cannot learn who +gitops-api could define its own kind and translate it into a `SaveRequest`, but it cannot learn who created the object: watching the workspace shows the object and nothing about its creator. Only the audit stream says that, and GitOps Reverser is what reads it. A translation would also spend part of the attach wait on an extra hop, and add a second status to keep in step. -Reusing `CommitRequest` with a different schema would be worse than a separate kind: +Reusing `SaveRequest` with a different schema would be worse than a separate kind: - **Another shape.** It names no GitTarget, and its status lists one or more of them. -- **Another trust model.** Its author comes from audit; a `CommitRequest`'s comes from admission. +- **Another trust model.** Its author comes from audit; a `SaveRequest`'s comes from admission. - **One name for two things** invites the confusion that `tenant-` already costs gitops-api. - **Installed alone.** A workspace gets one small CRD instead of the whole `configbutler.ai` set, so nobody is offered GitTargets that nothing reads. gitops-api installs CRDs into workspaces through its @@ -115,10 +115,10 @@ several folders, it saves each one's window, and its status lists each GitTarget On a source with audit, a request is attributed only once its audit fact has joined its watch event, and the edits it saves wait in the same audit pipeline (kcp batches audit with a `batchMaxWait` of 1s). The request's wait for a window starts when the branch worker registers it, which is after the -request itself was attributed. That is how a `CommitRequest`'s `attachTimeout` is anchored today -(`handleAttachCommitRequest` in `internal/git/commit_request_attach_loop.go`). The timeout then covers +request itself was attributed. That is how a `SaveRequest`'s `attachTimeout` is anchored today +(`handleAttachSaveRequest` in `internal/git/save_request_attach_loop.go`). The timeout then covers only the spread between the request's stream and the edits' streams. Its -default can still be set for a source with audit, rather than copied from `CommitRequest`'s `2s`. +default can still be set for a source with audit, rather than copied from `SaveRequest`'s `2s`. ### An unresolved save on a source that has audit @@ -151,7 +151,7 @@ turns it on, and the KubeProvider's docs say what it adds. must say it for everyone else. - **One save at a time per GitTarget.** gitops-api queues saves in its server (`commitGate`). If the browser creates them directly, that queue is gone. The worker already lets a window carry at most - one request, and a second waits for the next window (`attachWaitingCommitRequests`). Whether that + one request, and a second waits for the next window (`attachWaitingSaveRequests`). Whether that is enough, or the operator should hold a save until the one before it has resolved, is open. ## The takeover @@ -204,7 +204,7 @@ One field on the GitTarget, an enum, because there are two levels of trust worth message ends with a trailer that says the name was taken on trust: ```text - Author-Source: CommitRequest team-a/save-1; no audit record names who made these changes + Author-Source: SaveRequest team-a/save-1; no audit record names who made these changes ``` A request's `spec.gitTargetRef` is a local reference, so the request is always in its GitTarget's @@ -247,7 +247,7 @@ The two cover different clusters: A source-side save cannot offer `"TakeOver"`: on a cluster with no audit, nothing records who created it either, so it has no name to give. The name has to come from the control cluster's admission -webhook, which is why a takeover is always made by a control-cluster `CommitRequest`. +webhook, which is why a takeover is always made by a control-cluster `SaveRequest`. ## What happens to `cf3b97be` @@ -256,16 +256,16 @@ writes name no author, authoring the commit as the submitter, and the `Author-So ways forward before the branch merges: - **Turn it into the opt-in here.** Add `spec.commit.unattributedWindows` on the GitTarget and - `spec.window.author` on the `CommitRequest`, the controller's check and refusal, `"SavedBy"`, and + `spec.window.author` on the `SaveRequest`, the controller's check and refusal, `"SavedBy"`, and tests for each row above. The source-cluster e2e's save spec then sets both. - **Take it out here**, and build the takeover in its own PR after the source-side save. ## Open questions -1. The source-side kind's name and API group. Proposed: `SaveRequest` in +1. The source-side kind's name and API group. Proposed: `RemoteSaveRequest` in `source.configbutler.ai/v1alpha1`. "Save" is the word its user means and the one the project - already uses for what a request asks; `SimpleCommitRequest` was considered and set aside, since - it is a different kind with another trust model, not a simpler `CommitRequest`. Its spec carries + already uses for what a request asks; `SimpleSaveRequest` was considered and set aside, since + it is a different kind with another trust model, not a simpler `SaveRequest`. Its spec carries no Git terms (a message, a wait, a catch-up), and its status still reports each GitTarget's commit, so a save stays traceable. Its own group makes "never mirror it" and "install it alone" one check each. @@ -273,5 +273,5 @@ ways forward before the branch merges: 3. The reason name for an unresolved save on a source that has audit (`AuthorUnresolved` above). 4. Whether the operator should hold a second save on a GitTarget until the first has resolved. 5. Names for the takeover: `spec.commit.unattributedWindows` on the GitTarget and - `spec.window.author` on the `CommitRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. + `spec.window.author` on the `SaveRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. 6. `cf3b97be`: turn it into the opt-in on this branch, or take it out and build the takeover later? diff --git a/docs/future/task-oriented-docs-and-website.md b/docs/future/task-oriented-docs-and-website.md index ca295f63..ce54feb5 100644 --- a/docs/future/task-oriented-docs-and-website.md +++ b/docs/future/task-oriented-docs-and-website.md @@ -63,7 +63,7 @@ need them, and say when each capability is required or optional. - **Make a change through Kubernetes.** For an end user: create, update, or delete a selected resource; follow the change into Git; understand timing and refusal outcomes. - **Group changes and explain why.** For an end user: collect related edits, attach intent through - a `CommitRequest`, and verify the resulting commit or no-commit outcome. Explain author matching + a `SaveRequest`, and verify the resulting commit or no-commit outcome. Explain author matching and window timing. - **Work alongside other people on the same branch.** For collaborating teams: explain how authors and targets share a branch, what splits a commit window, and what happens when edits overlap. @@ -205,7 +205,7 @@ provider-specific claim when writing it. Link this guide from normal changes and grouped changes. Keep "who made the change" and "why the change was needed" as distinct tasks. Explain the separate admission-based submitter capture used -by `CommitRequest` where it affects matching, so readers do not assume audit setup controls both. +by `SaveRequest` where it affects matching, so readers do not assume audit setup controls both. ## Guide structure diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index d127e008..19ac66b9 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -117,9 +117,9 @@ red is a panel nobody reads. Object names are labels here because these objects are configuration, bounded by how many someone wrote, as are the `GitTarget`, `GitBranch`, applier claim and `GitReceiver` names other families carry. The resources being mirrored are data, unbounded, and never a label -([metrics-observability-plan.md](design/metrics-observability-plan.md) §7.1). `CommitRequest` is +([metrics-observability-plan.md](design/metrics-observability-plan.md) §7.1). `SaveRequest` is excluded on the same reasoning: one is created per save, so it is data-shaped, and its state is -read from `commit_requests_total` in aggregate and from the object's own conditions individually. +read from `save_requests_total` in aggregate and from the object's own conditions individually. **Which configuration objects are not Ready?** The panel to start from: @@ -206,7 +206,7 @@ boundary, the commit, the push. Background: | `git_queue_refusals_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `kind` | Work refused at enqueue: the queue was full, intake was paused through an outage, or the worker was stopping. `kind` is `write` / `attach` / `resync` / `refresh`. The producer keeps a refused item and offers it again: the watch delivers a write again from its cursor, the controller re-sends a save, a resync is gathered again, and the next reconcile asks for a refresh. What it can cost, when its watch cursor expires before the write comes back, is the write's own commit and, under `prune.mode: OnEvent`, a deleted object's removal from Git. | | `git_commit_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | A live commit window that died between routing and pushing; a resync's failure is `git_resync_failures_total`. `reason` is `refused` (a Git path a human must fix) / `error`. Every increment is a window's events lost until the next resync. An unreachable remote does not count: a decided window waits for the publication retry instead. | | `git_writes_dropped_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | A live event the branch worker dropped because its `GitTarget`'s render fidelity was not established. `reason` is `render_fidelity_diverged`: a rendered value does not match the live object, and every write is dropped until a fresh replay measures the folder clean. Or it is `render_fidelity_pending`: a collection of the `GitTarget` has not finished its replay, which closes every collection of that target until it does. A live event that meets a pending gate is held, not dropped, and committed with its author once the gate opens; it is counted here only when the hold gives up, 10 minutes after the first one was held or once the branch holds more than its retained-byte budget, and the worker then asks for a fresh snapshot, which commits the current state as the operator (a deletion comes back only under prune mode `Always`). Nothing replays a dropped write, so each one is an edit missing from Git until the object changes again or its collection replays. The worker logs the first drop under each gate status at Info, naming the collections the gate waits for, and each hold, release and give-up at Info. | -| `git_commit_windows_total` | counter | `gittarget_namespace`, `gittarget_name`, `close_reason`, `timer_source` | One per closed commit window, whether or not it then produced a commit. `close_reason` is `idle_timeout` / `max_duration` / `attach_next` / `identity_change` / `buffer_limit` / `resync_before_apply` / `foreign_commit` (a Git host said the branch may have moved, so the window is published now) / `apply_due` (an apply came due for an applier on the branch, so the window's edits are published before the apply is sent) / `shutdown`; `timer_source` is `target` or `commit_request`. See [tuning commit windows](#tuning-commit-windows). | +| `git_commit_windows_total` | counter | `gittarget_namespace`, `gittarget_name`, `close_reason`, `timer_source` | One per closed commit window, whether or not it then produced a commit. `close_reason` is `idle_timeout` / `max_duration` / `attach_next` / `identity_change` / `buffer_limit` / `resync_before_apply` / `foreign_commit` (a Git host said the branch may have moved, so the window is published now) / `apply_due` (an apply came due for an applier on the branch, so the window's edits are published before the apply is sent) / `shutdown`; `timer_source` is `target` or `save_request`. See [tuning commit windows](#tuning-commit-windows). | | `git_commit_window_duration_seconds` | histogram | `gittarget_namespace`, `gittarget_name` | How long each window collected: from the write that opened it to the start of its finalize. Its count equals `git_commit_windows_total` summed over `close_reason` and `timer_source`. | | `git_queue_depth` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | Pending + in-flight + committed-but-unpushed. Read at scrape time. | | `git_retained_bytes` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | What the worker holds until a push lands, in the units of the retained-byte budget: the open window, the retained writes and waiting saves, each with a 1 KiB per-item charge. Serialized bytes, not heap. | @@ -216,7 +216,7 @@ boundary, the commit, the push. Background: | `git_next_retry_timestamp_seconds` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | When the next attempt at the pending writes is due, as a Unix timestamp. No sample while no retry is scheduled. | | `git_materialization_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | Attempts to commit decided writes that stopped before any push: `reason` is `unreachable` or `parent_unavailable`. Nothing is lost; the writes wait for the retry. A failure `git_pushes_total` cannot see, because no push cycle started. | | `git_branch_targets` | gauge | `gitbranch_namespace`, `gitbranch_name`, `gittarget_namespace`, `gittarget_name`, `source_cluster` | Always 1. The **join** between the GitTarget-labeled half of the pipeline and the GitBranch-labeled half: the `GitTarget`'s `spec.gitBranchRef`. Published per configured GitTarget, whether or not its GitBranch exists or its worker runs. | -| `commit_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `CommitRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | +| `save_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `SaveRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | | `placements_total` | counter | `source`, `disposition`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new document at a resolved path. | | `placement_refusals_total` | counter | `reason`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new resource the writer declined. Every increment is a resource **absent** from the mirror. | | `placement_kustomization_entries_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | `added` / `no_change` / `failed`. | @@ -533,9 +533,9 @@ their writes into one commit batch, so the worker is the honest attribution unit `user`, `serviceaccount`, `committer`, or `unresolved`; reconcile/resync commits and configured-author mode use `committer`. -`message_source` is `live`, `reconcile`, or one of three request values: `commit_request` (a save -request's message committed verbatim, no `requestTemplate` configured), `commit_request_framed` -(`requestTemplate` rendered), and `commit_request_fallback` (`requestTemplate` failed to render, so +`message_source` is `live`, `reconcile`, or one of three request values: `save_request` (a save +request's message committed verbatim, no `requestTemplate` configured), `save_request_framed` +(`requestTemplate` rendered), and `save_request_fallback` (`requestTemplate` failed to render, so the message was committed verbatim instead). The fallback is the one to alert on. It is a **successful** commit (nothing is refused and no @@ -544,7 +544,7 @@ looks exactly like one that was never configured: ```promql sum by (gitbranch_namespace, gitbranch_name) ( - rate(gitopsreverser_git_commits_total{message_source="commit_request_fallback"}[15m]) + rate(gitopsreverser_git_commits_total{message_source="save_request_fallback"}[15m]) ) > 0 ``` @@ -562,15 +562,15 @@ Prometheus pod scrape with `honor_labels=false` renames a bare `namespace` label selector would silently match nothing. The same reasoning applies to `watch_recovery_total` and `git_queue_depth`. -`message_source` says where each commit's message came from: `commit_request` is text a -`CommitRequest` supplied and that was used verbatim, `live` is a live window rendered through the +`message_source` says where each commit's message came from: `save_request` is text a +`SaveRequest` supplied and that was used verbatim, `live` is a live window rendered through the target's `liveTemplate`, and `reconcile` is a snapshot or resync rendered through `reconcileTemplate`. It is read from the same decision that renders the message, so it cannot report a source the renderer did not use. -`commit_request` counts commits that USED a request-supplied message, **not** CommitRequests. A +`save_request` counts commits that USED a request-supplied message, **not** SaveRequests. A request that omits `spec.message` takes the target's `liveTemplate`, so it counts as `live`, and a -low `commit_request` share does not by itself mean requests are failing. +low `save_request` share does not by itself mean requests are failing. **How much of the history is people naming their own changes?** @@ -578,7 +578,7 @@ low `commit_request` share does not by itself mean requests are failing. sum by (message_source) (rate(gitopsreverser_git_commits_total[15m])) ``` -A `commit_request` share that falls to zero after a rollout **may** mean save requests stopped +A `save_request` share that falls to zero after a rollout **may** mean save requests stopped reaching an open window, but omitted messages, saves that were no-ops, and pushes that never landed all read the same way here, because none of them produce a commit carrying a request message. Confirm against the requests themselves before changing the window: a request that @@ -586,10 +586,10 @@ attached reports `Ready=True`, one that produced a pushed commit also reports `P `status.commit`, and one that gave up reports `Stalled=True`. Only if those show requests resolving without their message reaching a commit is the request's `window` worth tuning. -`commit_requests_total` answers the same question in aggregate, which per-object conditions cannot: +`save_requests_total` answers the same question in aggregate, which per-object conditions cannot: ```promql -sum by (outcome) (rate(gitopsreverser_commit_requests_total[15m])) +sum by (outcome) (rate(gitopsreverser_save_requests_total[15m])) ``` **Which target's saves are failing?** The counter carries the GitTarget, so the fleet view above is @@ -597,7 +597,7 @@ an aggregation rather than the only view available: ```promql sum by (gittarget_namespace, gittarget_name, outcome) ( - rate(gitopsreverser_commit_requests_total[15m]) + rate(gitopsreverser_save_requests_total[15m]) ) ``` @@ -618,13 +618,13 @@ seen during its wait, so a foreign window that closes before `attachTimeout` run `no_window` as "users are being refused" over-reports; reading only `committed` under-reports. **What one increment means.** One terminal DECISION, recorded outside the status-write retry loop. -That is not one per `CommitRequest`: a terminal status that never persisted is re-decided when the +That is not one per `SaveRequest`: a terminal status that never persisted is re-decided when the request is redelivered, and a restart re-reconciles anything non-terminal. Read rates and ratios from it rather than exact request counts. ### Tuning commit windows -A GitTarget's `commit.window` timers decide when its changes become a commit, and a `CommitRequest` +A GitTarget's `commit.window` timers decide when its changes become a commit, and a `SaveRequest` can bring its own. Two metrics show what those timers actually do. **Which timer ends the windows?** Every closed window is counted once, with what closed it and whose @@ -657,7 +657,7 @@ picture toward `max_duration`. stopping. - `apply_due` closes the window because an apply came due for an applier on the branch: its edits are published at once, ahead of the push cooldown, so the apply includes them. A window a - `CommitRequest` holds is never closed this way; the apply waits for the save. Its share follows the + `SaveRequest` holds is never closed this way; the apply waits for the save. Its share follows the apply traffic (Git-side pushes, API creates, repairs), and no timer changes it. `foreign_commit` is the same early close for a push a Git host reported. @@ -1421,7 +1421,7 @@ sum by (outcome) (rate(gitopsreverser_secret_encryptions_total[5m])) | `max by (gitbranch_namespace, gitbranch_name) (gitopsreverser_apply_appliers_held) > 0`, `for: 15m` | An applier is held behind the initial-snapshot barrier: a `GitTarget` naming its claim watches nothing, or its folder is missing. | | `sum by (applier_kind, claim_namespace, claim_name, reason) (rate(gitopsreverser_apply_requests_total{outcome="failed"}[15m])) > 0`, `for: 30m` | Apply requests keep failing for the named applier claim; the worker retries at its backoff, and the cluster runs older content than the branch meanwhile. | | `sum by (receiver_namespace, receiver_name) (increase(gitopsreverser_receiver_deliveries_total{outcome="rejected"}[15m])) > 0` | A Git host is calling a `GitReceiver` with the wrong secret, so foreign pushes are found only by the periodic refresh. | -| `sum by (gitbranch_namespace, gitbranch_name) (rate(gitopsreverser_git_commits_total{message_source="commit_request_fallback"}[15m])) > 0` | A `requestTemplate` stopped rendering: saves still commit, with the message verbatim. | +| `sum by (gitbranch_namespace, gitbranch_name) (rate(gitopsreverser_git_commits_total{message_source="save_request_fallback"}[15m])) > 0` | A `requestTemplate` stopped rendering: saves still commit, with the message verbatim. | | `gitopsreverser_resource_condition{type="Ready", status="False"} == 1`, `for: 15m` | A declared object has not been accepted. The `== 1` is not optional: each condition publishes one series per status, so the selector alone matches the two zeroes too. | | `gitopsreverser_resource_condition{type="Stalled", status="True"} == 1` | Permanently wedged: `Stalled` is the kstatus "nothing will retry" signal, so this needs a human and will not clear on its own. `reason` names the gate. | | any `gitopsreverser_*` series with `otel_metric_overflow="true"` | A metric family exceeded its per-instrument cardinality cap, so the labels identifying those series are gone and every query over that family is under-reporting. Raise the cap or cut a label. | @@ -1442,7 +1442,7 @@ Listed so a missing panel is never read as a healthy zero. The plan for them is - **A relevance-filter breakdown beyond the ingest census.** `watch_events_total{outcome}` names where an event stopped; it does not attribute an `unchanged` to sanitization versus a genuine no-op diff. -- **CommitRequest timing.** `commit_requests_total` counts how saves end; how long a save waited +- **SaveRequest timing.** `save_requests_total` counts how saves end; how long a save waited to attach (including the waits that ran out), how long it took end to end, and whether a `no_window` save was recorded in an empty commit are not emitted yet. - **The reference dashboard and alert rules.** The queries in this document are the specification diff --git a/docs/proud.md b/docs/proud.md index c49fc1be..2bc8b73a 100644 --- a/docs/proud.md +++ b/docs/proud.md @@ -61,7 +61,7 @@ per process, so you can see exactly what ran concurrently and where the bottlene cluster is **reused** between runs via `.stamps`, not torn down. And when things go wrong: - **Ctrl-C, or a setup failure, preserves the entire live state** so you can poke around. -- A **failed spec auto-dumps diagnostics** — events, `CommitRequest`s, controller logs, and a pod +- A **failed spec auto-dumps diagnostics** — events, `SaveRequest`s, controller logs, and a pod describe — into the log and the Allure attachment. - The spec's **Git checkout survives** under `.stamps/repos/`, with a working remote and credentials, so you can `git log`, `git pull`, or `git push` the reversed history of a past run. diff --git a/docs/rbac.md b/docs/rbac.md index 2d684d30..eac91d1e 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -86,13 +86,13 @@ evaluated against `Namespace`s in the *source* cluster, so a remote provider's i grant it exactly the namespaces and types the rules name — a `sourceNamespace: "*"` item asks for a cluster-wide `list`/`watch` of its types, and gets a clean 403 if the credential cannot serve it. -### `saverequests` on a source cluster +### `remotesaverequests` on a source cluster A source credential only reads, with one opt-in exception. A `KubeProvider` with -`spec.saveRequests: Enabled` serves the [`SaveRequest`s](configuration.md#saverequest) created on its +`spec.remoteSaveRequests: Enabled` serves the [`RemoteSaveRequest`s](configuration.md#remotesaverequest) created on its cluster, and writes their status back there. Its credential then needs, in the -`source.configbutler.ai` group, `get`, `list`, `watch` and `delete` on `saverequests` (`delete` for -the finished-request TTL), and `update` on `saverequests/status`. Nothing else on a source cluster is +`source.configbutler.ai` group, `get`, `list`, `watch` and `delete` on `remotesaverequests` (`delete` for +the finished-request TTL), and `update` on `remotesaverequests/status`. Nothing else on a source cluster is written. The manager role carries the same rules for the cluster the operator runs in, for an `inCluster` provider. diff --git a/docs/review/20261025-crd-text-proposal.md b/docs/review/20261025-crd-text-proposal.md index e1854687..67c93cdb 100644 --- a/docs/review/20261025-crd-text-proposal.md +++ b/docs/review/20261025-crd-text-proposal.md @@ -108,7 +108,7 @@ the links track their default branches. ### 1. Explain the message override accurately -Location: [CommitRequest.spec.message](../../api/v1alpha3/commitrequest_types.go). +Location: [CommitRequest.spec.message](../../api/v1alpha3/saverequest_types.go). ```text Message supplies the requested commit message. Omitted, the GitTarget's diff --git a/docs/spec/README.md b/docs/spec/README.md index b1fc6b72..6e68086c 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -46,8 +46,8 @@ three lifecycle classes and was unreadable spread across them. The rows below st | [`sops-single-file-no-multidoc.md`](sops-single-file-no-multidoc.md) | one encrypted file is one document | | [`scale-subresource-audit-rehydration.md`](scale-subresource-audit-rehydration.md) | history: the `/scale` field-patch design; a scale is now mirrored from the watch, and the field-patch writer is what remains | | [`commit-window-refactor.md`](commit-window-refactor.md) | one grouped commit = one (author, GitTarget) | -| [`commitrequest-design.md`](commitrequest-design.md) | how a request binds to a same-actor commit window and reports its outcome | -| [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md) | a request's submitter comes from admission, present-or-never, apart from audit facts | +| [`saverequest-design.md`](saverequest-design.md) | how a request binds to a same-actor commit window and reports its outcome | +| [`saverequest-admission-authorship.md`](saverequest-admission-authorship.md) | a request's submitter comes from admission, present-or-never, apart from audit facts | | [`status-conditions-guide.md`](status-conditions-guide.md) | the conditions and reasons every kind reports | | [`gitpath-foreign-content-stringency.md`](gitpath-foreign-content-stringency.md) | a `GitTarget` owns its subtree and refuses foreign content; `.gittargetignore` | | [`unsupported-folder-refusal-plan.md`](unsupported-folder-refusal-plan.md) | history of the `GitPathAccepted` refusal of unsupported folder content | diff --git a/docs/spec/attribution.md b/docs/spec/attribution.md index 9b2b27a4..6aab537f 100644 --- a/docs/spec/attribution.md +++ b/docs/spec/attribution.md @@ -18,7 +18,7 @@ > [what attribution gives up, on purpose](#what-attribution-gives-up-on-purpose). > > Related: [`watch-event-ordering-and-attribution-grace.md`](watch-event-ordering-and-attribution-grace.md), -> [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md), +> [`saverequest-admission-authorship.md`](saverequest-admission-authorship.md), > [`../attribution-setup-guide.md`](../attribution-setup-guide.md), > [`../interpreting-metrics.md`](../interpreting-metrics.md). @@ -40,7 +40,7 @@ When no usable fact is found the commit ships with the explicit unresolved autho A commit names a person only when the Kubernetes API server **authenticated** that person for the change: the user in the audit record of a mirrored edit, or the submitter in the admission request -of a `CommitRequest`. Every other commit carries one of two identities the operator decides, never +of a `SaveRequest`. Every other commit carries one of two identities the operator decides, never one a submitter supplies: - **Attribution is off** (the chart default): the configured Git identity. @@ -49,13 +49,13 @@ one a submitter supplies: No field in any spec can name an author, and none ever will. -That is why `CommitRequest` has no `spec.author`, although one was asked for (consumer ask #5, with a +That is why `SaveRequest` has no `spec.author`, although one was asked for (consumer ask #5, with a `SubjectAccessReview` guard): - **A name in a spec is only a claim.** Anyone who can create the object can write any name. A permission check narrows who may write it, but what it records is still only what the submitter said. -- **The claim outlives the check.** A `CommitRequest` applied from Git or copied between clusters +- **The claim outlives the check.** A `SaveRequest` applied from Git or copied between clusters carries its name to wherever it lands, while the submitter who was checked is someone else. - **It would make the rule above a matter of configuration.** "A wrong author is worse than no author" only holds if nothing can put a name into the history that the API server did not @@ -64,7 +64,7 @@ That is why `CommitRequest` has no `spec.author`, although one was asked for (co To commit under your own name, authenticate as yourself. Use your identity provider (OIDC, for example) so that the API server knows who you are. A request that cannot be attributed claims no actor (`AuthorAttributed=False`); it never falls back to a name it was given. See -[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). +[`saverequest-admission-authorship.md`](saverequest-admission-authorship.md). ## 1. Deletion is attributed at intent time diff --git a/docs/spec/commit-window-refactor.md b/docs/spec/commit-window-refactor.md index a7bf0ae9..fc15a8f8 100644 --- a/docs/spec/commit-window-refactor.md +++ b/docs/spec/commit-window-refactor.md @@ -94,7 +94,7 @@ flowchart TD cooldowns, and retry scheduling. It retains `[]PendingWrite` until a push succeeds. Local commit creation does not clear pending work. A write whose first commit changes nothing in Git is the exception: it settles at once and is neither -retained nor pushed. A save (a write carrying a `CommitRequest`) is always +retained nor pushed. A save (a write carrying a `SaveRequest`) is always retained, and the push decides its outcome. Per-event writes are processed as a stream. The branch worker keeps one open @@ -106,7 +106,7 @@ closes the window so its edits are published before the apply request is sent ([the flush](../design/apply-trigger.md#flush-then-apply), `close_reason="apply_due"`). A push notice closes it so its writes are published, and replayed onto the move, at once. Neither applies while a -`CommitRequest` holds the window: no flush is made while a request is registered, +`SaveRequest` holds the window: no flush is made while a request is registered, and a push notice leaves an attached window to its save. A timer of `0s` closes the window in the same step that collected the write. Repeated writes to the same Git path inside the open window are last-write-wins while preserving first-seen @@ -134,7 +134,7 @@ between: - `PendingWriteAtomic` represents one caller-defined atomic write. - `PendingWriteResync` represents one GitTarget's snapshot, applied by mark-and-sweep. -- `PendingWriteRequestRecord` records a `CommitRequest`'s message in an empty +- `PendingWriteRequestRecord` records a `SaveRequest`'s message in an empty commit when no window reached the request and it asked for `whenNothingToCommit: CommitEmpty`. @@ -174,9 +174,9 @@ planner-time regrouping pass. Message precedence is a non-empty literal override, then `commit.message.liveTemplate` for live windows or `commit.message.reconcileTemplate` for atomic snapshots and resyncs. Every live window -uses the same context, including singleton and `0s` windows. A `CommitRequest`'s message is the +uses the same context, including singleton and `0s` windows. A `SaveRequest`'s message is the literal override, and `commit.message.requestTemplate` can frame it (see -[`commitrequest-design.md`](commitrequest-design.md)). Literal text never changes authorship. +[`saverequest-design.md`](saverequest-design.md)). Literal text never changes authorship. Resolved templates and literal overrides travel with pending writes through retries and conflict replay, so editing a target does not change an already finalized write's message configuration. @@ -229,7 +229,7 @@ OPEN WINDOW rather than once per worker, and snapshotted when the window opens: editing a target never moves a deadline already running. That is affordable because a window is already bound to exactly one `GitTarget` by construction: it finalizes the moment the target changes. An unreadable target, or one declaring -no window, takes the `5s` and `1m` defaults. A `CommitRequest` that attaches +no window, takes the `5s` and `1m` defaults. A `SaveRequest` that attaches replaces both timers for that window. The byte cap is an operator startup setting, `--branch-buffer-max-size` diff --git a/docs/spec/current-manifest-support-review.md b/docs/spec/current-manifest-support-review.md index f5d8b4d7..eab430b2 100644 --- a/docs/spec/current-manifest-support-review.md +++ b/docs/spec/current-manifest-support-review.md @@ -1147,7 +1147,7 @@ core logic. > > Running it against `config/samples` already surfaced a real constraint: > `manifestedit` derives manifest identity from a concrete `metadata.name`, so a -> `generateName`-only object (`commitrequest.yaml`) is classified non-KRM. That +> `generateName`-only object (`saverequest.yaml`) is classified non-KRM. That > is correct given today's identity rules and is exactly the kind of gap the > analyzer is meant to make visible. diff --git a/docs/spec/e2e-serial-registry.md b/docs/spec/e2e-serial-registry.md index c845e343..76a697f4 100644 --- a/docs/spec/e2e-serial-registry.md +++ b/docs/spec/e2e-serial-registry.md @@ -71,11 +71,11 @@ commit it asserts on. The two `audit-consumer`-labelled containers (`Commit Window Batching` ([commit_window_batching_e2e_test.go](../../test/e2e/commit_window_batching_e2e_test.go)), -`Commit Request` ([commit_request_e2e_test.go](../../test/e2e/commit_request_e2e_test.go))) +`Save Request` ([save_request_e2e_test.go](../../test/e2e/save_request_e2e_test.go))) were also de-serialized. Their real serial cause was **not** the shared audit *pipeline* — namespace-scoped WatchRules already route each spec's events only to its own GitTarget — but a shared *repo*: both borrowed one Gitea repo from a -`sync.Once` helper and asserted on whole-branch state (`Commit Request` reads +`sync.Once` helper and asserted on whole-branch state (`Save Request` reads `HEAD`/`status.commit`; `Commit Window Batching` reads `git rev-list --count main`), so the *other* audit spec's commits to the same `main` corrupted those reads. Each now provisions its **own** repo via `SetupRepo`, so the only writer to its diff --git a/docs/spec/e2e-test-design.md b/docs/spec/e2e-test-design.md index 0c9b111f..6479c1a4 100644 --- a/docs/spec/e2e-test-design.md +++ b/docs/spec/e2e-test-design.md @@ -213,7 +213,7 @@ These namespaces are typically named: - `-test-manager-watchrule` - `-test-signing` -- `-test-commit-request` +- `-test-save-request` - and so on Those namespaces are created in `BeforeAll` and removed in `AfterAll` for the suite. @@ -251,7 +251,7 @@ The only cluster-level variable still consumed across files is: Repo state (names, checkout paths, Secret names, optional receiver webhook info) flows through a typed `RepoArtifacts` struct returned by `SetupRepo`, not through package-global env vars. -The audit-consumer specs (`Commit Window Batching`, `Commit Request`) each provision their own repo +The audit-consumer specs (`Commit Window Batching`, `Save Request`) each provision their own repo via `SetupRepo`, like every other file. ## Repo Model: One Repo Per E2E Test File @@ -263,7 +263,7 @@ Each repo-using e2e file owns its own repo, for example: | `watchrule_configmap_secret_e2e_test.go` (one of the `manager` files) | `testNamespaceFor("manager-watchrule")` | `e2e-manager-watchrule-` | | `signing_e2e_test.go` | `testNamespaceFor("signing")` | `e2e-signing-` | | `commit_window_batching_e2e_test.go` | `testNamespaceFor("commit-window-batching")` | `e2e-commit-window-` | -| `commit_request_e2e_test.go` | `testNamespaceFor("commit-request")`, and `commit-request-bundle` for the bundle describe | `e2e-commit-request-`, `e2e-commit-request-bundle-` | +| `save_request_e2e_test.go` | `testNamespaceFor("save-request")`, and `save-request-bundle` for the bundle describe | `e2e-save-request-`, `e2e-save-request-bundle-` | | `commit_author_attribution_e2e_test.go` | `testNamespaceFor("commit-author")` | `e2e-commit-author-` | | `quickstart_framework_e2e_test.go` | `testNamespaceFor("quickstart-framework")` | `e2e-quickstart-framework-` | | `flux_bi_directional_e2e_test.go` | `testNamespaceFor("bi-directional")` | `e2e-bi-directional-` | @@ -384,7 +384,7 @@ The e2e package includes, among others: | `controller_basics_e2e_test.go` and the other `manager` files | `manager` | Main reconciliation and GitOps behavior | | `signing_e2e_test.go` | `signing` | Commit signing behavior | | `commit_window_batching_e2e_test.go` | `commit-window-batching`, `audit-consumer` | Commit-window batching through the audit pipeline | -| `commit_request_e2e_test.go` | `commit-request`, `audit-consumer` | CommitRequest finalization through the audit pipeline | +| `save_request_e2e_test.go` | `save-request`, `audit-consumer` | SaveRequest finalization through the audit pipeline | | `commit_author_attribution_e2e_test.go` | `manager` | OIDC claim to Git author attribution | | `image_refresh_test.go` | `image-refresh` | Build/load/restart invalidation chain | | `quickstart_framework_e2e_test.go` | `quickstart-framework` | Installer-level quickstart flow | diff --git a/docs/spec/commitrequest-admission-authorship.md b/docs/spec/saverequest-admission-authorship.md similarity index 87% rename from docs/spec/commitrequest-admission-authorship.md rename to docs/spec/saverequest-admission-authorship.md index 9818f09f..1341da2a 100644 --- a/docs/spec/commitrequest-admission-authorship.md +++ b/docs/spec/saverequest-admission-authorship.md @@ -1,25 +1,25 @@ -# CommitRequest authorship from admission +# SaveRequest authorship from admission > **spec** — current behaviour. The code depends on this document; change one, change the other. > Index: [`../INDEX.md`](../INDEX.md) -A `CommitRequest` is a command to close an existing GitTarget commit window. Its submitter is captured +A `SaveRequest` is a command to close an existing GitTarget commit window. Its submitter is captured by the `/validate-operator-types` validating admission webhook, while mirrored-resource attribution is separately derived from kube-apiserver audit facts. The two paths answer different questions and must not be conflated. ## Contract -| Concern | CommitRequest submitter | Mirrored-resource author | +| Concern | SaveRequest submitter | Mirrored-resource author | |---|---|---| | Source | validating admission request `userInfo` | post-persist audit fact joined to watch event | -| Key | CommitRequest UID | source provider, GVR, UID, and resourceVersion | +| Key | SaveRequest UID | source provider, GVR, UID, and resourceVersion | | Timing | written before the object persists | waits up to `--author-attribution-grace` after the watch event | | Miss | request claims no actor | watch commit has the explicit unresolved author | The admission handler records the authenticated submitter in the Redis command-author store and always allows the request. It is deliberately best-effort: a Redis failure or an unavailable webhook must not -reject a user's command. The controller reads the record when the persisted `CommitRequest` is first +reject a user's command. The controller reads the record when the persisted `SaveRequest` is first reconciled. The result is **present-or-never**; waiting cannot create a record that admission did not write. ## Attach and final Git author @@ -63,6 +63,6 @@ An installation may therefore have either, both, or neither source of author inf Related live contracts: -- [architecture: CommitRequest Finalize](../architecture.md#commitrequest-finalize) -- [configuration: CommitRequest conditions](../configuration.md#when-there-is-nothing-to-commit) +- [architecture: SaveRequest Finalize](../architecture.md#saverequest-finalize) +- [configuration: SaveRequest conditions](../configuration.md#when-there-is-nothing-to-commit) - [status conditions guide](status-conditions-guide.md) diff --git a/docs/spec/commitrequest-design.md b/docs/spec/saverequest-design.md similarity index 96% rename from docs/spec/commitrequest-design.md rename to docs/spec/saverequest-design.md index 46896136..e073d648 100644 --- a/docs/spec/commitrequest-design.md +++ b/docs/spec/saverequest-design.md @@ -1,11 +1,11 @@ -# CommitRequest window finalization +# SaveRequest window finalization > **spec**: current behavior, reviewed at `c12f7970` on 2026-10-20. > The code depends on this document; change one, change the other. > Index: [`../INDEX.md`](../INDEX.md) -A `CommitRequest` is a one-shot “save now” command for one `GitTarget`. It does not mirror the -`CommitRequest` object. Instead, it can attach a message to a matching open commit window and asks the +A `SaveRequest` is a one-shot “save now” command for one `GitTarget`. It does not mirror the +`SaveRequest` object. Instead, it can attach a message to a matching open commit window and asks the worker to close that window after the requested collect delay. ## Request and window contract @@ -75,7 +75,7 @@ pushed commit is required; `Ready=True` also includes successful no-commit outco ## Authorship -The validating admission webhook captures a command submitter before the `CommitRequest` persists. The +The validating admission webhook captures a command submitter before the `SaveRequest` persists. The controller performs one best-effort, present-or-never lookup: | Admission result | `AuthorAttributed` | Request claim | @@ -91,14 +91,14 @@ that: - a resolved live attribution fact uses the authenticated actor; and - a live attribution miss uses `unknown (attribution unresolved) `. -See [CommitRequest admission authorship](commitrequest-admission-authorship.md) for capture provenance and +See [SaveRequest admission authorship](saverequest-admission-authorship.md) for capture provenance and [architecture: author identity](../architecture.md#author-and-committer-identity-in-git) for the three Git author states. ## Lifecycle and outcomes The controller stamps its conditions on first reconcile, immediately attaches the request, and polls the -worker. There is no audit wait for a `CommitRequest`; a delayed admission record cannot arrive after the +worker. There is no audit wait for a `SaveRequest`; a delayed admission record cannot arrive after the object is visible. | Outcome | Conditions | diff --git a/docs/spec/status-conditions-guide.md b/docs/spec/status-conditions-guide.md index 938df9c5..5d9d44f8 100644 --- a/docs/spec/status-conditions-guide.md +++ b/docs/spec/status-conditions-guide.md @@ -134,7 +134,7 @@ never reconciles one key twice at once), and nothing else re-enqueues the object `For()` of every controller using `reconcileStatus` filters status-only updates (`GenerationChangedPredicate`, or a variant of it that also admits a reconcile request). So the loss is recorded, and the reconcile comes back after `RequeueWriteLostInterval` (100ms) instead of on its -normal cadence, and recomputes from fresh data. `CommitRequest` does not use this helper: its +normal cadence, and recomputes from fresh data. `SaveRequest` does not use this helper: its `For()` has no such filter, and it retries its terminal status write in place. ### Reason vocabulary @@ -326,9 +326,9 @@ A `GitTarget` naming an applier claim gains `ApplierClaimReady`, a gate on its o claim (`ApplierClaimNotFound`) or one that is not `Ready` yet keeps it `Reconciling`. -### CommitRequest (one-shot) +### SaveRequest (one-shot) -CommitRequest runs once to a terminal outcome. Best-practice 1 above would suggest a `Succeeded` summary, +SaveRequest runs once to a terminal outcome. Best-practice 1 above would suggest a `Succeeded` summary, but it deliberately keeps `Ready` so every CRD in this project shares one summary type and the kstatus trio; the bounded "did the work actually happen" signal lives on the `Pushed` domain condition instead. diff --git a/docs/spec/where-validation-lives.md b/docs/spec/where-validation-lives.md index c003c44a..ee64ad1a 100644 --- a/docs/spec/where-validation-lives.md +++ b/docs/spec/where-validation-lives.md @@ -52,10 +52,10 @@ being refused at admission. [`/validate-operator-types`](../../internal/webhook/validate_operator_types_handler.go#L27) is **not a validation gate**. It captures the *submitter's identity* on a command kind -(`CommitRequest`) so a commit can be authored by a real Kubernetes user, and for a command it +(`SaveRequest`) so a commit can be authored by a real Kubernetes user, and for a command it **always allows**. Identity is the textbook case for admission: it exists in the `AdmissionRequest` and on no persisted field, so no reconciler could recover it later. See -[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). +[`saverequest-admission-authorship.md`](saverequest-admission-authorship.md). The same endpoint carries one rejection, and it is feedback, not a gate: a `WatchRule` that would bring a second source namespace to a `GitTarget` declaring `spec.serializeNamespace: false` is diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 7a492f82..76bbac9c 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -128,10 +128,10 @@ kinds: doc: >- Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) - - path: saveRequests + - path: remoteSaveRequests doc: >- - `Enabled` serves the `SaveRequest`s created on this cluster, which writes to it. See - [`SaveRequest`](#saverequest) + `Enabled` serves the `RemoteSaveRequest`s created on this cluster, which writes to it. See + [`RemoteSaveRequest`](#remotesaverequest) - kind: GitTarget title: "`GitTarget` (namespaced): which branch and folder" @@ -160,7 +160,7 @@ kinds: doc: How commits are phrased. See [commit messages](commit-messages.md) - path: commit.unattributedWindows doc: >- - How much a `CommitRequest` may claim of a window whose writes name no author, on a cluster + How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) - path: placement.byType @@ -296,8 +296,8 @@ kinds: the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` - - kind: CommitRequest - title: "`CommitRequest` (namespaced): save now" + - kind: SaveRequest + title: "`SaveRequest` (namespaced): save now" fields: - path: gitTargetRef doc: The target whose commit window the request attaches to @@ -327,14 +327,14 @@ kinds: pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it - - kind: SaveRequest - title: "`SaveRequest` (namespaced, on a source cluster): save the edits made here" + - kind: RemoteSaveRequest + title: "`RemoteSaveRequest` (namespaced, on a source cluster): save the edits made here" fields: - path: message default: each target's templates doc: Commit message, committed verbatim unless `requestTemplate` frames it - path: window.attachTimeout - doc: How long to wait for a window with the edits it saves. See [`SaveRequest`](#saverequest) + doc: How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) - path: window.idleTimeout default: no idle close doc: Close the attached window after this much silence diff --git a/internal/auditutil/identity_test.go b/internal/auditutil/identity_test.go index c0362eda..9d0d937f 100644 --- a/internal/auditutil/identity_test.go +++ b/internal/auditutil/identity_test.go @@ -15,11 +15,11 @@ import ( itypes "github.com/ConfigButler/gitops-reverser/internal/types" ) -func rawCommitRequestBody(t *testing.T, namespace, name, uid string) *runtime.Unknown { +func rawSaveRequestBody(t *testing.T, namespace, name, uid string) *runtime.Unknown { t.Helper() obj := &unstructured.Unstructured{} obj.SetAPIVersion("configbutler.ai/v1alpha3") - obj.SetKind("CommitRequest") + obj.SetKind("SaveRequest") if namespace != "" { obj.SetNamespace(namespace) } @@ -38,13 +38,13 @@ func TestIdentityFromAuditEvent_ObjectRefWins(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", Name: "save-1", UID: "uid-objref", }, } - ev.ResponseObject = rawCommitRequestBody(t, "team-a", "save-from-body", "uid-from-body") + ev.ResponseObject = rawSaveRequestBody(t, "team-a", "save-from-body", "uid-from-body") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -59,12 +59,12 @@ func TestIdentityFromAuditEvent_BackfillsNameFromResponseObject(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", Name: "", }, } - ev.ResponseObject = rawCommitRequestBody(t, "team-a", "save-generated-abcde", "uid-resp") + ev.ResponseObject = rawSaveRequestBody(t, "team-a", "save-generated-abcde", "uid-resp") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -79,10 +79,10 @@ func TestIdentityFromAuditEvent_BackfillsNamespaceFromBody(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", }, } - ev.ResponseObject = rawCommitRequestBody(t, "team-from-body", "save-x", "uid-x") + ev.ResponseObject = rawSaveRequestBody(t, "team-from-body", "save-x", "uid-x") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -96,13 +96,13 @@ func TestIdentityFromAuditEvent_FallsBackToRequestObject_NonDelete(t *testing.T) ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", }, } // Only RequestObject is populated; for non-delete the helper prefers // ResponseObject but must fall back to RequestObject when it is absent. - ev.RequestObject = rawCommitRequestBody(t, "team-a", "save-from-request", "uid-req") + ev.RequestObject = rawSaveRequestBody(t, "team-a", "save-from-request", "uid-req") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -116,12 +116,12 @@ func TestIdentityFromAuditEvent_DeletePrefersRequestObject(t *testing.T) { ev := auditv1.Event{ Verb: "delete", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", }, } - ev.RequestObject = rawCommitRequestBody(t, "team-a", "from-request", "uid-req") - ev.ResponseObject = rawCommitRequestBody(t, "team-a", "from-response", "uid-resp") + ev.RequestObject = rawSaveRequestBody(t, "team-a", "from-request", "uid-req") + ev.ResponseObject = rawSaveRequestBody(t, "team-a", "from-response", "uid-resp") got := IdentityFromAuditEvent(ev, itypes.OperationDelete) @@ -134,7 +134,7 @@ func TestIdentityFromAuditEvent_NoBodyKeepsObjectRef(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", Name: "save-1", }, @@ -161,7 +161,7 @@ func TestIdentityFromAuditEvent_IgnoresMalformedBody(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "commitrequests", + Resource: "saverequests", Namespace: "team-a", }, } diff --git a/internal/controller/constants.go b/internal/controller/constants.go index aaebcc87..37cb2b7f 100644 --- a/internal/controller/constants.go +++ b/internal/controller/constants.go @@ -86,19 +86,19 @@ const ( // still being established (the rule is blocked by validation), which must not be rendered as a // permanent failure. ConditionTypeSourceNamespaceAuthorized = "SourceNamespaceAuthorized" - // ConditionTypeAuthorAttributed indicates whether a CommitRequest's commit author + // ConditionTypeAuthorAttributed indicates whether a SaveRequest's commit author // was named from the submitter captured at admission. It is binary and immediately // settled (no Unknown, no timeout): True (AttributedFromAdmission) when the // validate-operator-types webhook recorded the submitter, False (CommitterFallback) when // capture ran but found no record, or False (AuthorCaptureDisabled) when capture is off. // False is not a failure and does not affect Ready: the // request claims no actor and can attach only to an unnamed watch window - // (docs/architecture.md#commitrequest-finalize). + // (docs/architecture.md#saverequest-finalize). ConditionTypeAuthorAttributed = "AuthorAttributed" - // ConditionTypePushed indicates whether a CommitRequest's commit reached the + // ConditionTypePushed indicates whether a SaveRequest's commit reached the // remote repository. ConditionTypePushed = "Pushed" - // ConditionTypeApplied reports, for a CommitRequest with catchUp: AfterPush, whether + // ConditionTypeApplied reports, for a SaveRequest with catchUp: AfterPush, whether // its GitTarget's applier applied a commit that holds the request's: Unknown while it applies or // retries, True once applied, False when there was nothing to apply. It never affects Ready. ConditionTypeApplied = "Applied" @@ -184,10 +184,10 @@ const ( // DefaultGitRefreshInterval is how often an IDLE branch's remote state is re-proved. A target // that is publishing renews its observation on every push and never waits for this. DefaultGitRefreshInterval = 10 * time.Minute - // DefaultCommitRequestTTL is how long a CommitRequest is kept once it finishes. Each save leaves + // DefaultSaveRequestTTL is how long a SaveRequest is kept once it finishes. Each save leaves // one object behind, so without an expiry they accumulate in etcd for as long as the namespace // lives. Two days leaves a weekend's worth of saves to inspect. - DefaultCommitRequestTTL = 48 * time.Hour + DefaultSaveRequestTTL = 48 * time.Hour // RequeueStreamSettleInterval is the requeue interval while a Ready GitTarget still // has streams pending replay completion. Stream status is computed during reconcile, so // this keeps status.streams fresh while watches converge. diff --git a/internal/controller/duration_fields_admission_test.go b/internal/controller/duration_fields_admission_test.go index e7ada17b..8144c995 100644 --- a/internal/controller/duration_fields_admission_test.go +++ b/internal/controller/duration_fields_admission_test.go @@ -186,18 +186,18 @@ var _ = Describe("Duration fields", func() { spec["gitTargetRef"] = map[string]any{"name": "any-target"} return &unstructured.Unstructured{Object: map[string]any{ "apiVersion": "configbutler.ai/v1alpha3", - "kind": "CommitRequest", + "kind": "SaveRequest", "metadata": map[string]any{"name": name, "namespace": "default"}, "spec": spec, }} } - It("defaults an omitted CommitRequest window and whenNothingToCommit, and keeps explicit zeros", func() { + It("defaults an omitted SaveRequest window and whenNothingToCommit, and keeps explicit zeros", func() { ctx := context.Background() - omitted := &configbutleraiv1alpha3.CommitRequest{ + omitted := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-omitted", Namespace: "default"}, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, }, } @@ -212,11 +212,11 @@ var _ = Describe("Duration fields", func() { // The pointers exist for exactly this: an explicit "0s" is a value, and defaulting must // not read it as an omission. - zero := &configbutleraiv1alpha3.CommitRequest{ + zero := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-zeros", Namespace: "default"}, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, - Window: &configbutleraiv1alpha3.CommitRequestWindow{ + Window: &configbutleraiv1alpha3.SaveRequestWindow{ AttachTimeout: &metav1.Duration{}, IdleTimeout: &metav1.Duration{}, MaxDuration: &metav1.Duration{}, @@ -230,7 +230,7 @@ var _ = Describe("Duration fields", func() { Expect(zero.Spec.Window.MaxDuration.Duration).To(BeZero()) }) - DescribeTable("refuse a CommitRequest window the rules forbid", + DescribeTable("refuse a SaveRequest window the rules forbid", func(name string, spec map[string]any, message string) { err := k8sClient.Create(context.Background(), newRequest(name, spec)) Expect(err).To(HaveOccurred()) @@ -265,12 +265,12 @@ var _ = Describe("Duration fields", func() { } }) - // CommitRequest carries the same hazard with immutability on top: its spec may not change + // SaveRequest carries the same hazard with immutability on top: its spec may not change // after creation, and a whole-object comparison compares a duration as a STRING. "1m" reads // back into Go and serializes as "1m0s", so before the rule compared durations instead of // spellings, every typed update — including one that only touches a label — was rejected as // an attempt to change an immutable spec. - It("lets a typed client update a CommitRequest created with a re-spelled duration", func() { + It("lets a typed client update a SaveRequest created with a re-spelled duration", func() { ctx := context.Background() // Created as UNSTRUCTURED, carrying the spelling a HUMAN writes. That is the whole @@ -290,7 +290,7 @@ var _ = Describe("Duration fields", func() { key := types.NamespacedName{Name: "window-respelled", Namespace: "default"} Eventually(func() error { - var typed configbutleraiv1alpha3.CommitRequest + var typed configbutleraiv1alpha3.SaveRequest if err := k8sClient.Get(ctx, key, &typed); err != nil { return err } @@ -303,15 +303,15 @@ var _ = Describe("Duration fields", func() { "the same duration spelled differently is not a spec change") }) - DescribeTable("still refuse a real change to an immutable CommitRequest spec", - func(name string, mutate func(*configbutleraiv1alpha3.CommitRequestSpec)) { + DescribeTable("still refuse a real change to an immutable SaveRequest spec", + func(name string, mutate func(*configbutleraiv1alpha3.SaveRequestSpec)) { ctx := context.Background() - request := &configbutleraiv1alpha3.CommitRequest{ + request := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, Message: "save", - Window: &configbutleraiv1alpha3.CommitRequestWindow{ + Window: &configbutleraiv1alpha3.SaveRequestWindow{ MaxDuration: &metav1.Duration{Duration: time.Minute}, }, }, @@ -321,7 +321,7 @@ var _ = Describe("Duration fields", func() { key := types.NamespacedName{Name: request.Name, Namespace: request.Namespace} Eventually(func() string { - var current configbutleraiv1alpha3.CommitRequest + var current configbutleraiv1alpha3.SaveRequest if err := k8sClient.Get(ctx, key, ¤t); err != nil { return err.Error() } @@ -333,16 +333,16 @@ var _ = Describe("Duration fields", func() { }, "10s", "200ms").Should(ContainSubstring("immutable"), "comparing by value must not become comparing by nothing") }, - Entry("maxDuration", "immutable-max", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { + Entry("maxDuration", "immutable-max", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { spec.Window.MaxDuration = &metav1.Duration{Duration: 30 * time.Second} }), - Entry("attach", "immutable-attach", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { + Entry("attach", "immutable-attach", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { spec.Window.Attach = configbutleraiv1alpha3.AttachNext }), - Entry("an idleTimeout added later", "immutable-idle", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { + Entry("an idleTimeout added later", "immutable-idle", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { spec.Window.IdleTimeout = &metav1.Duration{Duration: time.Second} }), - Entry("whenNothingToCommit", "immutable-empty", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { + Entry("whenNothingToCommit", "immutable-empty", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { spec.WhenNothingToCommit = configbutleraiv1alpha3.NothingToCommitCommitEmpty }), ) diff --git a/internal/controller/gitbranch_controller.go b/internal/controller/gitbranch_controller.go index fe86b80b..9b56c0b5 100644 --- a/internal/controller/gitbranch_controller.go +++ b/internal/controller/gitbranch_controller.go @@ -32,7 +32,7 @@ import ( ) // GitBranchFinalizer holds a GitBranch until its branch worker has given every pending write and -// CommitRequest an outcome. It never deletes the branch on the remote. +// SaveRequest an outcome. It never deletes the branch on the remote. const GitBranchFinalizer = "configbutler.ai/git-branch" const ( @@ -394,7 +394,7 @@ func (r *GitBranchReconciler) finalize(ctx context.Context, branch *configbutler // It is the one path for a GitBranch being deleted and for one that no grant admits any more, which // keeps its finalizer. The claim goes first, so no GitTarget reconcile can start this GitBranch's // worker again; then the worker is stopped, which closes its intake, pushes or refuses every pending -// write and settles every CommitRequest it holds (BranchWorker.Stop). Release, not a failed proof: +// write and settles every SaveRequest it holds (BranchWorker.Stop). Release, not a failed proof: // Observe keeps a holder's candidacy through a failed proof, and a revoked GitBranch must stop being // a candidate at all. Last, once the old worker has retired so pending work never moves on, the // GitBranches waiting for the branch are woken. Release handed it to none of them: each takes it only diff --git a/internal/controller/gitbranch_controller_test.go b/internal/controller/gitbranch_controller_test.go index 488a99e2..0125fffc 100644 --- a/internal/controller/gitbranch_controller_test.go +++ b/internal/controller/gitbranch_controller_test.go @@ -266,7 +266,7 @@ func TestGitBranch_AMissingOrUnreadyGitProvider(t *testing.T) { } // TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch: the finalizer stops the branch worker -// (which publishes or refuses its pending writes and settles its CommitRequests), releases the +// (which publishes or refuses its pending writes and settles its SaveRequests), releases the // claim, and goes. The next candidate then takes the branch. func TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch(t *testing.T) { f := newGitBranchFixture(t, diff --git a/internal/controller/gittarget_publication_test.go b/internal/controller/gittarget_publication_test.go index 2b7ab3fc..cb26a1e0 100644 --- a/internal/controller/gittarget_publication_test.go +++ b/internal/controller/gittarget_publication_test.go @@ -85,7 +85,7 @@ func TestPublicationReadiness_LeavesAMissingParentToTheParentGate(t *testing.T) // A save the worker holds says why it is held, with the same words the GitTarget uses, and the // status is written again when that reason changes, not only when the phase does. -func TestCommitRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { +func TestSaveRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { held := failingPublication(false).Message() reason, message := progressFor(git.PhaseWaitingForPush, held) assert.Equal(t, string(git.PhaseWaitingForPush), reason) @@ -95,11 +95,11 @@ func TestCommitRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { _, plain := progressFor(git.PhaseWaitingForPush, "") assert.Equal(t, waitingForPushMessage, plain, "a push that is merely pending says nothing more") - cr := &configbutleraiv1alpha3.CommitRequest{} - markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, "") + cr := &configbutleraiv1alpha3.SaveRequest{} + markSaveRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, "") assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, "")) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, held), "the outage is news to the save") - markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, held) + markSaveRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, held) assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, held)) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, ""), "and so is the recovery") } diff --git a/internal/controller/resource_condition_metrics.go b/internal/controller/resource_condition_metrics.go index 4959cd39..0c0dd7e9 100644 --- a/internal/controller/resource_condition_metrics.go +++ b/internal/controller/resource_condition_metrics.go @@ -19,10 +19,10 @@ import ( // bounds the series count and is the whole basis on which §7.1 of the metrics plan allows their // names as labels. // -// CommitRequest is deliberately absent. It is created once per save, so it is data-shaped rather +// SaveRequest is deliberately absent. It is created once per save, so it is data-shaped rather // than config-shaped: a gauge per object would churn series continuously — created, Ready, // finalized, deleted — and churn is the one cost Prometheus handles worst. Its state is answerable -// from git_commits_total{message_source="commit_request"} and from the object's own conditions, and +// from git_commits_total{message_source="save_request"} and from the object's own conditions, and // if a fleet view is ever wanted the right shape is a counter of terminal outcomes, because those // are events rather than state. Its reconciler does not use this status helper either, so the // exclusion holds at both ends. @@ -45,7 +45,7 @@ const ( // Which is why the kind does NOT come off statusObject, the accessor interface the status session // takes. That interface is about writing a status, and anything can be given the two methods it // asks for; hanging the kind on it would turn this list into an implication — implement the -// accessors over in api/v1alpha3 and you are on the metric — and CommitRequest would arrive here by +// accessors over in api/v1alpha3 and you are on the metric — and SaveRequest would arrive here by // a route nobody reviewing that file would see. func conditionMetricKind(obj client.Object) string { switch obj.(type) { diff --git a/internal/controller/resource_condition_metrics_test.go b/internal/controller/resource_condition_metrics_test.go index c39e454d..97260270 100644 --- a/internal/controller/resource_condition_metrics_test.go +++ b/internal/controller/resource_condition_metrics_test.go @@ -116,7 +116,7 @@ func TestPublishConditionMetrics_ForgetsAnObjectThatIsGoing(t *testing.T) { assert.False(t, ok, "an object with no finalizer left is going now and must stop publishing") } -// The kind switch is the surface. CommitRequest is data-shaped — one per save — so it is excluded +// The kind switch is the surface. SaveRequest is data-shaped — one per save — so it is excluded // structurally rather than by a rule someone has to remember. func TestConditionMetricKind_CoversTheFiveConfigKindsAndNothingElse(t *testing.T) { assert.Equal(t, conditionKindGitTarget, conditionMetricKind(&configbutleraiv1alpha3.GitTarget{})) @@ -124,7 +124,7 @@ func TestConditionMetricKind_CoversTheFiveConfigKindsAndNothingElse(t *testing.T assert.Equal(t, conditionKindClusterWatchRule, conditionMetricKind(&configbutleraiv1alpha3.ClusterWatchRule{})) assert.Equal(t, conditionKindGitProvider, conditionMetricKind(&configbutleraiv1alpha3.GitProvider{})) assert.Equal(t, conditionKindKubeProvider, conditionMetricKind(&configbutleraiv1alpha3.KubeProvider{})) - assert.Empty(t, conditionMetricKind(&configbutleraiv1alpha3.CommitRequest{})) + assert.Empty(t, conditionMetricKind(&configbutleraiv1alpha3.SaveRequest{})) } // A condition written with an empty reason publishes a visibly wrong placeholder rather than an diff --git a/internal/controller/commitrequest_apply.go b/internal/controller/saverequest_apply.go similarity index 77% rename from internal/controller/commitrequest_apply.go rename to internal/controller/saverequest_apply.go index 58d4c442..a2a6cfd8 100644 --- a/internal/controller/commitrequest_apply.go +++ b/internal/controller/saverequest_apply.go @@ -17,7 +17,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -// A CommitRequest with catchUp: AfterPush keeps being reconciled once it is Ready, to +// A SaveRequest with catchUp: AfterPush keeps being reconciled once it is Ready, to // report in its Applied condition how the apply it asked for went (git.SaveApply). Ready is the // save's alone and never changes here. @@ -32,33 +32,33 @@ const applyLostAfter = time.Minute // followsApply reports whether a finished request still reports its apply: it asked for one, its // save ended Ready, and Applied is not settled yet. -func followsApply(cr *configbutleraiv1alpha3.CommitRequest) bool { +func followsApply(cr *configbutleraiv1alpha3.SaveRequest) bool { return cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush && conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) && !applyReported(cr) } // applyReported reports whether Applied is settled: True, False, or Unknown because it was lost. -func applyReported(cr *configbutleraiv1alpha3.CommitRequest) bool { +func applyReported(cr *configbutleraiv1alpha3.SaveRequest) bool { c := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeApplied) return c != nil && (c.Status != metav1.ConditionUnknown || c.Reason == crReasonApplyLost) } // followApply reads where the request's apply stands, writes it to Applied, and comes back until it // is settled; then the request expires as any finished one does. -func (r *CommitRequestReconciler) followApply( +func (r *SaveRequestReconciler) followApply( ctx context.Context, - cr *configbutleraiv1alpha3.CommitRequest, + cr *configbutleraiv1alpha3.SaveRequest, ) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - attach := git.AttachCommitRequest{ + log := logf.FromContext(ctx).WithName("SaveRequestReconciler") + attach := git.AttachSaveRequest{ Namespace: cr.Namespace, Name: cr.Name, UID: string(cr.UID), GitTargetName: cr.Spec.GitTargetRef.Name, GitTargetNamespace: cr.Namespace, } - apply, known, err := r.Finalizer.LookupCommitRequestApply(ctx, attach) + apply, known, err := r.Finalizer.LookupSaveRequestApply(ctx, attach) if err != nil { - log.V(1).Info("Cannot read the CommitRequest's apply yet; will retry", + log.V(1).Info("Cannot read the SaveRequest's apply yet; will retry", "name", client.ObjectKeyFromObject(cr), "err", err.Error()) - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil } if !known { apply = unknownApply(cr) @@ -70,18 +70,18 @@ func (r *CommitRequestReconciler) followApply( // A conflict reads the request again; anything else retries the same way. return ctrl.Result{}, client.IgnoreNotFound(err) } - log.Info("CommitRequest apply reported", "name", client.ObjectKeyFromObject(cr), + log.Info("SaveRequest apply reported", "name", client.ObjectKeyFromObject(cr), "reason", apply.Reason, "appliedRevision", cr.Status.AppliedRevision, "message", apply.Message) } if applyReported(cr) { return r.expireFinished(ctx, cr) } - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil } // unknownApply is the apply of a request its worker knows nothing of: still being taken on, shortly // after Ready, and lost after applyLostAfter. -func unknownApply(cr *configbutleraiv1alpha3.CommitRequest) git.SaveApply { +func unknownApply(cr *configbutleraiv1alpha3.SaveRequest) git.SaveApply { if ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady); ready != nil && time.Since(ready.LastTransitionTime.Time) < applyLostAfter { return git.SaveApply{Reason: git.SaveApplyApplying, Message: "waiting for the branch worker to take the apply"} @@ -91,7 +91,7 @@ func unknownApply(cr *configbutleraiv1alpha3.CommitRequest) git.SaveApply { } // setAppliedCondition writes apply to Applied and status.appliedRevision. -func setAppliedCondition(cr *configbutleraiv1alpha3.CommitRequest, apply git.SaveApply) { +func setAppliedCondition(cr *configbutleraiv1alpha3.SaveRequest, apply git.SaveApply) { status := metav1.ConditionUnknown switch { case apply.Applied: @@ -100,5 +100,5 @@ func setAppliedCondition(cr *configbutleraiv1alpha3.CommitRequest, apply git.Sav case apply.Done: status = metav1.ConditionFalse } - setCommitRequestCondition(cr, ConditionTypeApplied, status, apply.Reason, apply.Message) + setSaveRequestCondition(cr, ConditionTypeApplied, status, apply.Reason, apply.Message) } diff --git a/internal/controller/commitrequest_apply_test.go b/internal/controller/saverequest_apply_test.go similarity index 59% rename from internal/controller/commitrequest_apply_test.go rename to internal/controller/saverequest_apply_test.go index b8adbd8c..348b9749 100644 --- a/internal/controller/commitrequest_apply_test.go +++ b/internal/controller/saverequest_apply_test.go @@ -16,26 +16,26 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -func catchingUp(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { +func catchingUp(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { cr.Spec.CatchUp = configv1alpha3.CatchUpAfterPush return cr } -// TestCommitRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with +// TestSaveRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with // catchUp: AfterPush is reconciled on, and Applied follows the worker's report until it // is settled. Ready stays the save's. -func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { - c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) +func TestSaveRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { + c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-1") + reconcileSaveRequest(t, r, "save-1") assert.Zero(t, f.applyLookups, "the save resolves first") f.apply, f.applyKnown = git.SaveApply{Reason: git.SaveApplyRetrying, Message: "the apply did not complete"}, true - res := reconcileCommitRequest(t, r, "save-1") + res := reconcileSaveRequest(t, r, "save-1") assert.Positive(t, res.RequeueAfter, "followed until it is settled") - got := fetchCommitRequest(t, c, "save-1") + got := fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyRetrying) requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) @@ -45,20 +45,20 @@ func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { Reason: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", } - reconcileCommitRequest(t, r, "save-1") - got = fetchCommitRequest(t, c, "save-1") + reconcileSaveRequest(t, r, "save-1") + got = fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionTrue, git.SaveApplyApplied) assert.Equal(t, "main@sha1:c0ffee", got.Status.AppliedRevision) lookups := f.applyLookups - reconcileCommitRequest(t, r, "save-1") + reconcileSaveRequest(t, r, "save-1") assert.Equal(t, lookups, f.applyLookups, "settled: not looked up again") } -// TestCommitRequestReconcile_ASettledApplyWithoutAnApply: a request that pushed nothing, or whose +// TestSaveRequestReconcile_ASettledApplyWithoutAnApply: a request that pushed nothing, or whose // GitTarget names no applier, is Applied=True: no apply was needed, and the cluster already holds what // it saved. One whose commit the branch lost is Applied=False. None touches Ready. -func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { +func TestSaveRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { for _, tc := range []struct { apply git.SaveApply want metav1.ConditionStatus @@ -68,14 +68,14 @@ func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { {git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyNoApplier}, metav1.ConditionTrue}, } { t.Run(tc.apply.Reason, func(t *testing.T) { - c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) + c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) f := committedFinalizer() f.apply, f.applyKnown = tc.apply, true - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-1") - reconcileCommitRequest(t, r, "save-1") - got := fetchCommitRequest(t, c, "save-1") + reconcileSaveRequest(t, r, "save-1") + reconcileSaveRequest(t, r, "save-1") + got := fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, tc.want, tc.apply.Reason) requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Empty(t, got.Status.AppliedRevision) @@ -83,36 +83,36 @@ func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { } } -// TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply +// TestSaveRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply // shortly after Ready is still taking it on; long after, it restarted, and Applied says it is lost. -func TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { - c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) +func TestSaveRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { + c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-1") + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileSaveRequest(t, r, "save-1") - reconcileCommitRequest(t, r, "save-1") - got := fetchCommitRequest(t, c, "save-1") + reconcileSaveRequest(t, r, "save-1") + got := fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyApplying) ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) ready.LastTransitionTime = metav1.NewTime(time.Now().Add(-2 * applyLostAfter)) require.NoError(t, c.Status().Update(context.Background(), &got)) - reconcileCommitRequest(t, r, "save-1") - got = fetchCommitRequest(t, c, "save-1") + reconcileSaveRequest(t, r, "save-1") + got = fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, crReasonApplyLost) assert.False(t, followsApply(&got), "lost is settled") } -// TestCommitRequestReconcile_NoCatchUpReportsNoApply: the default asks no apply, and has no Applied. -func TestCommitRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { - c := newCommitRequestClient(t, nil, newCommitRequest("save-1")) +// TestSaveRequestReconcile_NoCatchUpReportsNoApply: the default asks no apply, and has no Applied. +func TestSaveRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { + c := newSaveRequestClient(t, nil, newSaveRequest("save-1")) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-1") - reconcileCommitRequest(t, r, "save-1") - got := fetchCommitRequest(t, c, "save-1") + reconcileSaveRequest(t, r, "save-1") + reconcileSaveRequest(t, r, "save-1") + got := fetchSaveRequest(t, c, "save-1") assert.Nil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeApplied)) assert.Zero(t, f.applyLookups) } diff --git a/internal/controller/commitrequest_attribution_agreement_test.go b/internal/controller/saverequest_attribution_agreement_test.go similarity index 86% rename from internal/controller/commitrequest_attribution_agreement_test.go rename to internal/controller/saverequest_attribution_agreement_test.go index 80b44bf5..066c779f 100644 --- a/internal/controller/commitrequest_attribution_agreement_test.go +++ b/internal/controller/saverequest_attribution_agreement_test.go @@ -56,7 +56,7 @@ func windowOutcomeAttributionMissed(t *testing.T) git.AttributionOutcome { return outcome } -// TestCommitRequest_OutcomesAgree is the cross-subsystem half of the P0 regression test. +// TestSaveRequest_OutcomesAgree is the cross-subsystem half of the P0 regression test. // // The gap that let the blocker through was that every existing test set both sides of the // comparison to the same literal, so nothing ever checked that the two INDEPENDENT producers @@ -66,12 +66,12 @@ func windowOutcomeAttributionMissed(t *testing.T) git.AttributionOutcome { // // The default deployment is the first row: --admission-webhook defaults to false, so the // controller emits attributionNotAttempted, while --author-attribution=false leaves the window's -// outcome at the zero value. When those two were different strings, no CommitRequest could +// outcome at the zero value. When those two were different strings, no SaveRequest could // attach to any window and the user's commit message was silently dropped. -func TestCommitRequest_OutcomesAgree(t *testing.T) { +func TestSaveRequest_OutcomesAgree(t *testing.T) { tests := []struct { name string - requestSide commitRequestAttribution + requestSide saveRequestAttribution windowSide git.AttributionOutcome wantNamesActor bool }{ @@ -102,7 +102,7 @@ func TestCommitRequest_OutcomesAgree(t *testing.T) { requestOutcome := tc.requestSide.gitOutcome() assert.Equal(t, tc.wantNamesActor, requestOutcome.NamesActor()) assert.Equal(t, requestOutcome.NamesActor(), tc.windowSide.NamesActor(), - "the CommitRequest side produced %q and the window side produced %q; they must "+ + "the SaveRequest side produced %q and the window side produced %q; they must "+ "agree on whether an actor was named, or the request cannot attach and the "+ "user's commit message is silently dropped", requestOutcome, tc.windowSide) @@ -116,17 +116,17 @@ func TestAttributionOutcome_OffLeavesZeroOutcome(t *testing.T) { assert.Equal(t, git.AttributionNotAttempted, windowOutcomeAttributionOff(), "configured-author mode must leave the event's outcome at AttributionNotAttempted") assert.Equal(t, git.AttributionNotAttempted, attributionNotAttempted.gitOutcome(), - "a CommitRequest with command-author capture off must carry AttributionNotAttempted") + "a SaveRequest with command-author capture off must carry AttributionNotAttempted") assert.Equal(t, git.AttributionUnresolved, windowOutcomeAttributionMissed(t), "attribution that ran and found nothing must be AttributionUnresolved, not the zero value") } -// TestCommitRequestAttribution_GitOutcome keeps every controller outcome's projection explicit, +// TestSaveRequestAttribution_GitOutcome keeps every controller outcome's projection explicit, // including the defensive zero value. -func TestCommitRequestAttribution_GitOutcome(t *testing.T) { +func TestSaveRequestAttribution_GitOutcome(t *testing.T) { tests := []struct { name string - input commitRequestAttribution + input saveRequestAttribution want git.AttributionOutcome }{ {name: "unset is not attempted", input: attributionUnset, want: git.AttributionNotAttempted}, @@ -142,11 +142,11 @@ func TestCommitRequestAttribution_GitOutcome(t *testing.T) { } } -// TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled keeps an unset decision visible and safe. -func TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled(t *testing.T) { - var request configv1alpha3.CommitRequest +// TestSaveRequestAttribution_UnsetStatusIsCaptureDisabled keeps an unset decision visible and safe. +func TestSaveRequestAttribution_UnsetStatusIsCaptureDisabled(t *testing.T) { + var request configv1alpha3.SaveRequest - setCommitRequestAttributed(&request, attributionUnset) + setSaveRequestAttributed(&request, attributionUnset) requireCondition(t, request, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) } diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/saverequest_controller.go similarity index 62% rename from internal/controller/commitrequest_controller.go rename to internal/controller/saverequest_controller.go index 76352653..8ea1a6f3 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/saverequest_controller.go @@ -23,9 +23,9 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/queue" ) -// CommitRequestFinalizer is the EventRouter seam the reconciler drives, using the -// attach-then-poll protocol (docs/spec/commitrequest-design.md): -// ServiceCommitRequest registers the attach idempotently on the GitTarget's branch +// SaveRequestFinalizer is the EventRouter seam the reconciler drives, using the +// attach-then-poll protocol (docs/spec/saverequest-design.md): +// ServiceSaveRequest registers the attach idempotently on the GitTarget's branch // worker (attach it to the author's window with its message and timers) // and returns the request's current outcome — resolved=false means keep polling. // watch.EventRouter satisfies it without adaptation. @@ -33,21 +33,21 @@ import ( // There is no watermark barrier: the interactive case is covered by the human gap between the // edit and the save, UC2 by the request's attach wait. The worker anchors that wait at its own // registration and runs the window's timers, so the controller never holds the finalize itself. -type CommitRequestFinalizer interface { - ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - // WithdrawCommitRequest cancels a request the worker has not acted on and reports the worker's - // answer: resolved with git.ErrCommitRequestWithdrawn when it is cancelled, the real outcome +type SaveRequestFinalizer interface { + ServiceSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) + // WithdrawSaveRequest cancels a request the worker has not acted on and reports the worker's + // answer: resolved with git.ErrSaveRequestWithdrawn when it is cancelled, the real outcome // when the worker got there first, or resolved=false with the worker's phase while it decides. - WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - // LookupCommitRequestApply reports where the apply a request with catchUp: + WithdrawSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) + // LookupSaveRequestApply reports where the apply a request with catchUp: // AfterPush asked for stands, and false while the worker knows nothing of it. - LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) + LookupSaveRequestApply(ctx context.Context, attach git.AttachSaveRequest) (git.SaveApply, bool, error) } -// CommandAuthorLookup resolves the author of a CommitRequest from the submitter +// CommandAuthorLookup resolves the author of a SaveRequest from the submitter // captured at admission by the validate-operator-types webhook, keyed by the persisted // object's UID. *queue.CommandAuthorStore satisfies it without adaptation. The lookup -// is present-or-never (docs/spec/commitrequest-admission-authorship.md): a miss is +// is present-or-never (docs/spec/saverequest-admission-authorship.md): a miss is // immediate and final — the webhook is not configured (or a best-effort write missed) — // and the request claims no actor with no wait. type CommandAuthorLookup interface { @@ -61,30 +61,30 @@ const windowMismatchMessage = "the open commit window belongs to a different aut // resolveTimeoutMessage explains the fail-closed resolve bound: the worker never acted on the // request within the safety window, so it was withdrawn rather than polled forever. -const resolveTimeoutMessage = "the CommitRequest finalize did not resolve within the safety window" +const resolveTimeoutMessage = "the SaveRequest finalize did not resolve within the safety window" // noWorkerTimeoutMessage is resolveTimeoutMessage for a GitTarget that never started a branch // worker, the one cause the controller can name without asking anybody. -const noWorkerTimeoutMessage = "the CommitRequest's GitTarget did not start a branch worker within the safety window" +const noWorkerTimeoutMessage = "the SaveRequest's GitTarget did not start a branch worker within the safety window" const ( - // commitRequestPollInterval is the requeue cadence while polling the worker + // saveRequestPollInterval is the requeue cadence while polling the worker // for the attached request's outcome (attach-then-poll). - commitRequestPollInterval = 2 * time.Second + saveRequestPollInterval = 2 * time.Second - // commitRequestPushAllowance is what the safety bound allows on top of a request's own + // saveRequestPushAllowance is what the safety bound allows on top of a request's own // timers, for the push cooldown and its retries. Authorship is settled synchronously at first // sight, so no attribution wait belongs here. - commitRequestPushAllowance = 120 * time.Second + saveRequestPushAllowance = 120 * time.Second - // The request window defaults mirror the +kubebuilder:default markers on CommitRequestWindow. + // The request window defaults mirror the +kubebuilder:default markers on SaveRequestWindow. // The API server fills them in, so these apply only to a client that bypasses defaulting; // resolving to the same values keeps the two paths from disagreeing. defaultAttachTimeout = 2 * time.Second defaultMaxDuration = 2 * time.Second ) -// requestWindow is a CommitRequest's window, resolved to the values the worker applies. +// requestWindow is a SaveRequest's window, resolved to the values the worker applies. type requestWindow struct { attach configbutleraiv1alpha3.AttachPolicy attachTimeout time.Duration @@ -94,7 +94,7 @@ type requestWindow struct { // resolveRequestWindow reads spec.window, falling back field by field to the schema defaults. An // explicit "0s" stays zero, and an omitted idleTimeout stays nil (no idle close). -func resolveRequestWindow(spec configbutleraiv1alpha3.CommitRequestSpec) requestWindow { +func resolveRequestWindow(spec configbutleraiv1alpha3.SaveRequestSpec) requestWindow { resolved := requestWindow{ attach: configbutleraiv1alpha3.AttachCurrentOrNext, attachTimeout: defaultAttachTimeout, @@ -126,10 +126,10 @@ func resolveRequestWindow(spec configbutleraiv1alpha3.CommitRequestSpec) request // only the worker can say how that ends, however long a down remote or a recovery takes. Past the // bound, an unheld request is withdrawn, and fails only once the worker agrees. func (w requestWindow) resolveTimeout() time.Duration { - return w.attachTimeout + w.maxDuration + commitRequestPushAllowance + return w.attachTimeout + w.maxDuration + saveRequestPushAllowance } -// CommitRequestReconciler deliberately does NOT use reconcileStatus, which every other controller +// SaveRequestReconciler deliberately does NOT use reconcileStatus, which every other controller // here shares. The difference is not stylistic, so it is worth saying once: // // - Its For() carries no GenerationChangedPredicate, so a status write re-enqueues the object. @@ -146,9 +146,9 @@ func (w requestWindow) resolveTimeout() time.Duration { // one, but a re-run is no longer the only thing standing between a flushed window and a // NoOpenWindow that never happened.) // -// CommitRequestReconciler drives a CommitRequest through its state machine -// (docs/spec/commitrequest-design.md and -// docs/spec/commitrequest-admission-authorship.md): +// SaveRequestReconciler drives a SaveRequest through its state machine +// (docs/spec/saverequest-design.md and +// docs/spec/saverequest-admission-authorship.md): // // 1. ATTRIBUTE — a single synchronous read of the submitter captured at admission // (present-or-never). A hit names that submitter as the author @@ -162,7 +162,7 @@ func (w requestWindow) resolveTimeout() time.Duration { // registration, so there is no controller-side delay. While it polls, the controller // reports the phase the worker reports. No window before the attach deadline resolves // NoWindow; a window belonging to someone else stays open for its own author. -type CommitRequestReconciler struct { +type SaveRequestReconciler struct { client.Client Scheme *runtime.Scheme @@ -177,110 +177,110 @@ type CommitRequestReconciler struct { // AuthorLookup is nil (the validate-operator-types webhook is disabled), requests // claim no actor immediately, with AuthorAttributed=False. The attached window // determines the eventual Git author. - Finalizer CommitRequestFinalizer + Finalizer SaveRequestFinalizer AuthorLookup CommandAuthorLookup // TTL is how long a request is kept once it finishes. It is written onto the request as - // CommitRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. + // SaveRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. TTL time.Duration } -// CommitRequestDeleteAfterAnnotation carries the RFC 3339 time after which the controller deletes a -// finished CommitRequest. The controller writes it once, when the request finishes, and from then +// SaveRequestDeleteAfterAnnotation carries the RFC 3339 time after which the controller deletes a +// finished SaveRequest. The controller writes it once, when the request finishes, and from then // on the annotation alone decides: remove it to keep the request, or edit it to move the deletion. // A request that finished before this annotation existed carries none and is never deleted. -const CommitRequestDeleteAfterAnnotation = "configbutler.ai/delete-after" +const SaveRequestDeleteAfterAnnotation = "configbutler.ai/delete-after" -// +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests,verbs=get;list;watch;patch;delete -// +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=configbutler.ai,resources=saverequests,verbs=get;list;watch;patch;delete +// +kubebuilder:rbac:groups=configbutler.ai,resources=saverequests/status,verbs=get;update;patch -// Reconcile advances one CommitRequest through attribute → attach + poll → -// terminal status. With MaxConcurrentReconciles=1 concurrent CommitRequests are +// Reconcile advances one SaveRequest through attribute → attach + poll → +// terminal status. With MaxConcurrentReconciles=1 concurrent SaveRequests are // serialized by construction, and the worker keys attaches by request identity so // re-sends across poll requeues are idempotent. -func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") +func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := logf.FromContext(ctx).WithName("SaveRequestReconciler") - commitRequest, done, err := r.loadActionableCommitRequest(ctx, req) + saveRequest, done, err := r.loadActionableSaveRequest(ctx, req) if err != nil { return ctrl.Result{}, err } if done { - if commitRequest != nil { - if followsApply(commitRequest) { - return r.followApply(ctx, commitRequest) + if saveRequest != nil { + if followsApply(saveRequest) { + return r.followApply(ctx, saveRequest) } - return r.expireFinished(ctx, commitRequest) + return r.expireFinished(ctx, saveRequest) } return ctrl.Result{}, nil } if r.Finalizer == nil { - log.V(1).Info("CommitRequest finalize disabled: no Finalizer configured", + log.V(1).Info("SaveRequest finalize disabled: no Finalizer configured", "name", req.NamespacedName) return ctrl.Result{}, nil } - if handled, err := r.refusePrunedGitTargetRef(ctx, commitRequest); handled || err != nil { + if handled, err := r.refusePrunedGitTargetRef(ctx, saveRequest); handled || err != nil { return ctrl.Result{}, err } // 1. ATTRIBUTE: settle the request's actor synchronously (present-or-never). // A hit names the admission submitter; a miss claims no actor. Either way the // decision is final — there is no wait and no requeue for the author. - author, attribution := r.attributeAuthor(ctx, commitRequest) + author, attribution := r.attributeAuthor(ctx, saveRequest) // Refused only at first sight, before any attach was sent: a GitTarget's policy lowered later // does not fail a request the worker may already hold, which keeps what it was first sent. - claim, claimErr := r.unattributedClaim(ctx, commitRequest) - if claimErr != nil && findCondition(commitRequest.Status.Conditions, ConditionTypeReady) == nil { - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, claimErr, attribution, - resolvedCommitRequestTarget(commitRequest)) + claim, claimErr := r.unattributedClaim(ctx, saveRequest) + if claimErr != nil && findCondition(saveRequest.Status.Conditions, ConditionTypeReady) == nil { + r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, claimErr, attribution, + resolvedSaveRequestTarget(saveRequest)) return ctrl.Result{}, nil } - if err := r.stampFirstSightConditions(ctx, commitRequest, attribution); err != nil { + if err := r.stampFirstSightConditions(ctx, saveRequest, attribution); err != nil { return ctrl.Result{}, err } // The schema rejects an invalid literal message at admission, so a rejection here names an // object the schema could not have seen: one stored before the rule landed. It is terminal - // rather than retried, because CommitRequest.spec is immutable and no apply can repair it. - if err := git.ValidateLiteralCommitMessage(commitRequest.Spec.Message); err != nil { - // The fallback identity: this fails before ServiceCommitRequest, so nothing has + // rather than retried, because SaveRequest.spec is immutable and no apply can repair it. + if err := git.ValidateLiteralCommitMessage(saveRequest.Spec.Message); err != nil { + // The fallback identity: this fails before ServiceSaveRequest, so nothing has // confirmed the named GitTarget exists. - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, attribution, - commitRequestTarget{}) + r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, + saveRequestTarget{}) return ctrl.Result{}, nil } // 2. ATTACH + POLL: register the attach idempotently the instant we attribute // (no controller-side delay — the worker anchors the wait at its registration) // and poll the outcome. - window := resolveRequestWindow(commitRequest.Spec) - attach := git.AttachCommitRequest{ - Namespace: commitRequest.Namespace, - Name: commitRequest.Name, - UID: string(commitRequest.UID), + window := resolveRequestWindow(saveRequest.Spec) + attach := git.AttachSaveRequest{ + Namespace: saveRequest.Namespace, + Name: saveRequest.Name, + UID: string(saveRequest.UID), Author: author.Author, Attribution: attribution.gitOutcome(), - GitTargetName: commitRequest.Spec.GitTargetRef.Name, - GitTargetNamespace: commitRequest.Namespace, - Message: commitRequest.Spec.Message, + GitTargetName: saveRequest.Spec.GitTargetRef.Name, + GitTargetNamespace: saveRequest.Namespace, + Message: saveRequest.Spec.Message, Attach: window.attach, AttachTimeout: window.attachTimeout, IdleTimeout: window.idleTimeout, MaxDuration: window.maxDuration, - CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + CommitEmpty: saveRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: saveRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, Unattributed: claim, } - if time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout() { + if time.Since(saveRequest.CreationTimestamp.Time) >= window.resolveTimeout() { // Past the bound the controller only withdraws: an attach sent now would sit behind the // withdraw on the worker's FIFO and could register the request after it was cancelled. - return r.withdrawCommitRequest(ctx, log, req, commitRequest, attribution, attach) + return r.withdrawSaveRequest(ctx, log, req, saveRequest, attribution, attach) } - return r.attachCommitRequest(ctx, log, req, commitRequest, attribution, attach) + return r.attachSaveRequest(ctx, log, req, saveRequest, attribution, attach) } // errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author @@ -288,19 +288,19 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques var errTakeOverNotAllowed = errors.New("spec.window.author asks for more than the GitTarget allows") // unattributedClaim decides what the request may claim of a window on its GitTarget whose writes -// name no author (git.AttachCommitRequest.Unattributed). It claims what spec.window.author asks for, +// name no author (git.AttachSaveRequest.Unattributed). It claims what spec.window.author asks for, // and only while the GitTarget's KubeProvider has never received an audit fact: its // AuditFactsReceived condition is not True, so no write watched on that cluster can name its author. // On a cluster that has sent one, a write naming no author means its fact went missing, and nothing // is claimed. A request that asks for more than the GitTarget's spec.commit.unattributedWindows // allows is errTakeOverNotAllowed. A GitTarget or KubeProvider that cannot be read claims nothing, // and the request attaches by author alone. -func (r *CommitRequestReconciler) unattributedClaim( +func (r *SaveRequestReconciler) unattributedClaim( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, ) (git.UnattributedClaim, error) { ask := configbutleraiv1alpha3.WindowAuthorOwn - if window := commitRequest.Spec.Window; window != nil && window.Author != "" { + if window := saveRequest.Spec.Window; window != nil && window.Author != "" { ask = window.Author } if ask == configbutleraiv1alpha3.WindowAuthorOwn { @@ -308,7 +308,7 @@ func (r *CommitRequestReconciler) unattributedClaim( } var target configbutleraiv1alpha3.GitTarget if err := r.Get(ctx, client.ObjectKey{ - Namespace: commitRequest.Namespace, Name: commitRequest.Spec.GitTargetRef.Name, + Namespace: saveRequest.Namespace, Name: saveRequest.Spec.GitTargetRef.Name, }, &target); err != nil { return git.UnattributedClaimNone, nil //nolint:nilerr // a missing GitTarget is the attach path's to report } @@ -337,70 +337,70 @@ func (r *CommitRequestReconciler) unattributedClaim( return git.UnattributedClaimSavedBy, nil } -// attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then +// attachSaveRequest sends the attach and reports the outcome once the worker has one. Until then // it records the worker's phase and polls; a service error is a not-yet-serviceable worker, so it // polls on the same terms. -func (r *CommitRequestReconciler) attachCommitRequest( +func (r *SaveRequestReconciler) attachSaveRequest( ctx context.Context, log logr.Logger, req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, - attach git.AttachCommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + attach git.AttachSaveRequest, ) (ctrl.Result, error) { - result, resolved, serviceErr := r.Finalizer.ServiceCommitRequest(ctx, attach) + result, resolved, serviceErr := r.Finalizer.ServiceSaveRequest(ctx, attach) if serviceErr != nil || !resolved { if serviceErr != nil { - log.V(1).Info("CommitRequest attach not yet serviceable; will retry", + log.V(1).Info("SaveRequest attach not yet serviceable; will retry", "name", req.NamespacedName, "err", serviceErr.Error()) } - return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) + return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) } if result.Err != nil { - log.Error(result.Err, "CommitRequest finalize failed", - "gitTarget", commitRequest.Spec.GitTargetRef.Name, "name", req.NamespacedName) + log.Error(result.Err, "SaveRequest finalize failed", + "gitTarget", saveRequest.Spec.GitTargetRef.Name, "name", req.NamespacedName) } - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, - resolvedCommitRequestTarget(commitRequest)) + r.writeTerminalStatus(ctx, log, saveRequest, result, result.Err, attribution, + resolvedSaveRequestTarget(saveRequest)) return ctrl.Result{}, nil } -// withdrawCommitRequest handles a request past its safety window. A request the worker holds keeps +// withdrawSaveRequest handles a request past its safety window. A request the worker holds keeps // polling with no bound of its own: only the worker can say how it ends. Any other request is // withdrawn, and fails only on the worker's answer, so a request reported failed can never be // committed afterwards. A withdraw that cannot be serviced yet polls on the same terms. -func (r *CommitRequestReconciler) withdrawCommitRequest( +func (r *SaveRequestReconciler) withdrawSaveRequest( ctx context.Context, log logr.Logger, req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, - attach git.AttachCommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + attach git.AttachSaveRequest, ) (ctrl.Result, error) { - result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) + result, resolved, withdrawErr := r.Finalizer.WithdrawSaveRequest(ctx, attach) if withdrawErr != nil || !resolved { if withdrawErr != nil { - log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", + log.V(1).Info("SaveRequest withdraw not yet serviceable; will retry", "name", req.NamespacedName, "err", withdrawErr.Error()) } // Not answered yet, or the worker holds the request: keep polling. - return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) + return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) } - if !errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { + if !errors.Is(result.Err, git.ErrSaveRequestWithdrawn) { // The worker resolved it on its own before the withdraw: report that outcome, as the // attach path would have. - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, - resolvedCommitRequestTarget(commitRequest)) + r.writeTerminalStatus(ctx, log, saveRequest, result, result.Err, attribution, + resolvedSaveRequestTarget(saveRequest)) return ctrl.Result{}, nil } // Withdrawn. A worker's answer, or a GitTarget that was read, carries the branch, and only // then is the request's identity known. - target := commitRequestTarget{} + target := saveRequestTarget{} if result.Branch != "" { - target = resolvedCommitRequestTarget(commitRequest) + target = resolvedSaveRequestTarget(saveRequest) } message := resolveTimeoutMessage @@ -410,60 +410,60 @@ func (r *CommitRequestReconciler) withdrawCommitRequest( case result.Phase == git.PhaseWaitingForWorker: message = noWorkerTimeoutMessage } - log.Info("CommitRequest was not acted on within the safety window; withdrawn and failed closed", + log.Info("SaveRequest was not acted on within the safety window; withdrawn and failed closed", "name", req.NamespacedName, "reason", message) - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, errors.New(message), attribution, target) + r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, errors.New(message), attribution, target) return ctrl.Result{}, nil } -// pollCommitRequest reports where the worker says the request stands and polls again. -func (r *CommitRequestReconciler) pollCommitRequest( +// pollSaveRequest reports where the worker says the request stands and polls again. +func (r *SaveRequestReconciler) pollSaveRequest( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, - phase git.CommitRequestPhase, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + phase git.SaveRequestPhase, held string, ) (ctrl.Result, error) { - if err := r.recordWorkerPhase(ctx, commitRequest, attribution, phase, held); err != nil { + if err := r.recordWorkerPhase(ctx, saveRequest, attribution, phase, held); err != nil { return ctrl.Result{}, err } - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil } // stampFirstSightConditions stamps the still-running conditions the first time a request is seen, // so the object reports its progress (kstatus InProgress) and AuthorAttributed is settled // immediately. A disabled controller returns before this, so it never stamps. -func (r *CommitRequestReconciler) stampFirstSightConditions( +func (r *SaveRequestReconciler) stampFirstSightConditions( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, ) error { - if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) != nil { + if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) != nil { return nil } - markCommitRequestProgressing(commitRequest, attribution, "", "") - if err := r.Status().Update(ctx, commitRequest); err != nil { + markSaveRequestProgressing(saveRequest, attribution, "", "") + if err := r.Status().Update(ctx, saveRequest); err != nil { return err } - logf.FromContext(ctx).V(1).Info("Stamped CommitRequest in-progress conditions", - "name", client.ObjectKeyFromObject(commitRequest)) + logf.FromContext(ctx).V(1).Info("Stamped SaveRequest in-progress conditions", + "name", client.ObjectKeyFromObject(saveRequest)) return nil } -// refusePrunedGitTargetRef fails a CommitRequest whose spec names no GitTarget, which can only be +// refusePrunedGitTargetRef fails a SaveRequest whose spec names no GitTarget, which can only be // one thing: an object stored before spec.targetRef was renamed to spec.gitTargetRef, whose old // value the apiserver stopped serving the moment the new CRDs landed. Admission refuses an empty // name on every path, so nothing else can produce it. // // It is terminal rather than retried, and it has to be, because the object cannot be repaired: -// CommitRequest.spec is wholly immutable, so no apply can put the name back. Retrying would spend +// SaveRequest.spec is wholly immutable, so no apply can put the name back. Retrying would spend // the controller's attention on an object that will never resolve, and would report a transient // "get GitTarget" error for a permanent condition. The message names the only fix there is. -func (r *CommitRequestReconciler) refusePrunedGitTargetRef( +func (r *SaveRequestReconciler) refusePrunedGitTargetRef( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, ) (bool, error) { - if commitRequest.Spec.GitTargetRef.Name != "" { + if saveRequest.Spec.GitTargetRef.Name != "" { return false, nil } @@ -472,18 +472,18 @@ func (r *CommitRequestReconciler) refusePrunedGitTargetRef( // state that must not be missing from the counter. Note the status write below is returned for // requeue, so a failing write re-decides and increments again: see the instrument's doc // comment on what one increment means. - recordCommitRequestOutcome(ctx, crOutcomeFailed, commitRequestTarget{}) - failCommitRequest(commitRequest, crReasonGitTargetRefPruned, + recordSaveRequestOutcome(ctx, crOutcomeFailed, saveRequestTarget{}) + failSaveRequest(saveRequest, crReasonGitTargetRefPruned, "spec.gitTargetRef is empty: this request was created before spec.targetRef was renamed, "+ - "and its value was pruned by the upgrade. A CommitRequest spec is immutable, so this "+ + "and its value was pruned by the upgrade. A SaveRequest spec is immutable, so this "+ "one cannot be repaired — delete it and create a new one.") - if err := r.Status().Update(ctx, commitRequest); err != nil { + if err := r.Status().Update(ctx, saveRequest); err != nil { return true, err } - r.stampDeleteAfter(ctx, commitRequest) + r.stampDeleteAfter(ctx, saveRequest) logf.FromContext(ctx).Info( - "CommitRequest names no GitTarget: created before the gitTargetRef rename", - "name", client.ObjectKeyFromObject(commitRequest)) + "SaveRequest names no GitTarget: created before the gitTargetRef rename", + "name", client.ObjectKeyFromObject(saveRequest)) return true, nil } @@ -496,25 +496,25 @@ func (r *CommitRequestReconciler) refusePrunedGitTargetRef( // The lookup result is logged at Info: it is the counterpart to the admission handler's // "recorded command author" line, so a hit/miss pair makes the whole capture→read path // legible (the first thing to check when a request unexpectedly claims no actor). -func (r *CommitRequestReconciler) attributeAuthor( +func (r *SaveRequestReconciler) attributeAuthor( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (queue.CommandAuthor, commitRequestAttribution) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") + saveRequest *configbutleraiv1alpha3.SaveRequest, +) (queue.CommandAuthor, saveRequestAttribution) { + log := logf.FromContext(ctx).WithName("SaveRequestReconciler") if r.AuthorLookup == nil { log.Info("command-author lookup disabled (validate-operator-types webhook off); request claims no actor", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) + "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID) // Capture is off, so nothing was attempted — distinct from a capture that ran and // found no record, which is what attributionCommitter now means. return queue.CommandAuthor{}, attributionNotAttempted } - if author, ok := r.AuthorLookup.LookupCommandAuthor(ctx, commitRequest.UID); ok { + if author, ok := r.AuthorLookup.LookupCommandAuthor(ctx, saveRequest.UID); ok { log.Info("command author resolved from admission record", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID, "author", author.Author) + "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID, "author", author.Author) return author, attributionFromAdmission } log.Info("no admission command-author record found; request claims no actor", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) + "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID) return queue.CommandAuthor{}, attributionCommitter } @@ -522,23 +522,23 @@ func (r *CommitRequestReconciler) attributeAuthor( // change, so polling does not re-write status every interval. An empty phase — the worker has not // registered the request yet — keeps whatever is shown, so the controller never reports a phase // the worker has not confirmed. -func (r *CommitRequestReconciler) recordWorkerPhase( +func (r *SaveRequestReconciler) recordWorkerPhase( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, - phase git.CommitRequestPhase, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + phase git.SaveRequestPhase, held string, ) error { - if progressRecorded(commitRequest, phase, held) { + if progressRecorded(saveRequest, phase, held) { return nil } - markCommitRequestProgressing(commitRequest, attribution, phase, held) - return r.Status().Update(ctx, commitRequest) + markSaveRequestProgressing(saveRequest, attribution, phase, held) + return r.Status().Update(ctx, saveRequest) } // progressRecorded reports whether the request's status already says this phase, and why it is // held, so a poll writes only what changed. An empty phase says nothing new. -func progressRecorded(cr *configbutleraiv1alpha3.CommitRequest, phase git.CommitRequestPhase, held string) bool { +func progressRecorded(cr *configbutleraiv1alpha3.SaveRequest, phase git.SaveRequestPhase, held string) bool { current := findCondition(cr.Status.Conditions, ConditionTypeReconciling) if current == nil { return false @@ -550,35 +550,35 @@ func progressRecorded(cr *configbutleraiv1alpha3.CommitRequest, phase git.Commit return current.Reason == reason && current.Message == message } -// loadActionableCommitRequest fetches the CommitRequest and short-circuits +// loadActionableSaveRequest fetches the SaveRequest and short-circuits // everything that must not reach the finalize: a deleted object, a terminal // outcome, and — via an uncached re-read — a stale cache echo of our own // terminal write from a previous invocation (work past this point must happen -// at most once per CommitRequest). done=true means the finalize has nothing +// at most once per SaveRequest). done=true means the finalize has nothing // further to do; the request is returned with it when it is terminal, so its // expiry can be decided, and nil when it is gone. -func (r *CommitRequestReconciler) loadActionableCommitRequest( +func (r *SaveRequestReconciler) loadActionableSaveRequest( ctx context.Context, req ctrl.Request, -) (*configbutleraiv1alpha3.CommitRequest, bool, error) { - var commitRequest configbutleraiv1alpha3.CommitRequest - if err := r.Get(ctx, req.NamespacedName, &commitRequest); err != nil { +) (*configbutleraiv1alpha3.SaveRequest, bool, error) { + var saveRequest configbutleraiv1alpha3.SaveRequest + if err := r.Get(ctx, req.NamespacedName, &saveRequest); err != nil { return nil, true, client.IgnoreNotFound(err) } - if commitRequestIsTerminal(&commitRequest) { - return &commitRequest, true, nil + if saveRequestIsTerminal(&saveRequest) { + return &saveRequest, true, nil } if r.APIReader != nil { - if err := r.APIReader.Get(ctx, req.NamespacedName, &commitRequest); err != nil { + if err := r.APIReader.Get(ctx, req.NamespacedName, &saveRequest); err != nil { return nil, true, client.IgnoreNotFound(err) } - if commitRequestIsTerminal(&commitRequest) { - return &commitRequest, true, nil + if saveRequestIsTerminal(&saveRequest) { + return &saveRequest, true, nil } } - return &commitRequest, false, nil + return &saveRequest, false, nil } // expireFinished deletes a finished request once the time in its delete-after annotation has @@ -590,20 +590,20 @@ func (r *CommitRequestReconciler) loadActionableCommitRequest( // that races it fails the delete with a conflict, and the requeue reads the edit. The edit itself // needs no extra wiring: this controller has no generation filter, so a metadata change reconciles // the request again. -func (r *CommitRequestReconciler) expireFinished( +func (r *SaveRequestReconciler) expireFinished( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, ) (ctrl.Result, error) { - value, ok := commitRequest.Annotations[CommitRequestDeleteAfterAnnotation] - if !ok || !commitRequest.DeletionTimestamp.IsZero() { + value, ok := saveRequest.Annotations[SaveRequestDeleteAfterAnnotation] + if !ok || !saveRequest.DeletionTimestamp.IsZero() { return ctrl.Result{}, nil } - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") + log := logf.FromContext(ctx).WithName("SaveRequestReconciler") deleteAfter, parsed := parseDeleteAfter(value) if !parsed { // Kept rather than guessed at: an annotation a person mistyped is not consent to delete. - log.Info("CommitRequest delete-after annotation is not an RFC 3339 time; keeping the request", - "name", client.ObjectKeyFromObject(commitRequest), "value", value) + log.Info("SaveRequest delete-after annotation is not an RFC 3339 time; keeping the request", + "name", client.ObjectKeyFromObject(saveRequest), "value", value) return ctrl.Result{}, nil } @@ -611,13 +611,13 @@ func (r *CommitRequestReconciler) expireFinished( return ctrl.Result{RequeueAfter: remaining}, nil } - uid, resourceVersion := commitRequest.UID, commitRequest.ResourceVersion - err := r.Delete(ctx, commitRequest, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) + uid, resourceVersion := saveRequest.UID, saveRequest.ResourceVersion + err := r.Delete(ctx, saveRequest, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) if err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } - log.Info("Deleted finished CommitRequest past its delete-after time", - "name", client.ObjectKeyFromObject(commitRequest), "deleteAfter", value) + log.Info("Deleted finished SaveRequest past its delete-after time", + "name", client.ObjectKeyFromObject(saveRequest), "deleteAfter", value) return ctrl.Result{}, nil } @@ -638,74 +638,74 @@ func parseDeleteAfter(value string) (time.Time, bool) { // // A write that still fails is logged, not requeued: the request is then simply kept, the safe // direction, and a requeue would re-enter the finalize this controller must run at most once. -func (r *CommitRequestReconciler) stampDeleteAfter( +func (r *SaveRequestReconciler) stampDeleteAfter( ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, ) { if r.TTL <= 0 { return } - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") + log := logf.FromContext(ctx).WithName("SaveRequestReconciler") reader := r.APIReader if reader == nil { reader = r.Client } deleteAfter := time.Now().Add(r.TTL).UTC().Format(time.RFC3339) - current := commitRequest - for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { - if _, set := current.Annotations[CommitRequestDeleteAfterAnnotation]; set { + current := saveRequest + for attempt := 1; attempt <= saveRequestStatusUpdateAttempts; attempt++ { + if _, set := current.Annotations[SaveRequestDeleteAfterAnnotation]; set { return } patch := client.MergeFromWithOptions(current.DeepCopy(), client.MergeFromWithOptimisticLock{}) if current.Annotations == nil { current.Annotations = map[string]string{} } - current.Annotations[CommitRequestDeleteAfterAnnotation] = deleteAfter + current.Annotations[SaveRequestDeleteAfterAnnotation] = deleteAfter err := r.Patch(ctx, current, patch) if err == nil || apierrors.IsNotFound(err) { return } if !apierrors.IsConflict(err) { - log.Error(err, "Failed to stamp delete-after on a finished CommitRequest; it will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) + log.Error(err, "Failed to stamp delete-after on a finished SaveRequest; it will be kept", + "name", client.ObjectKeyFromObject(saveRequest)) return } - var fresh configbutleraiv1alpha3.CommitRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { + var fresh configbutleraiv1alpha3.SaveRequest + if getErr := reader.Get(ctx, client.ObjectKeyFromObject(saveRequest), &fresh); getErr != nil { if !apierrors.IsNotFound(getErr) { - log.Error(getErr, "Failed to re-read CommitRequest to stamp delete-after; it will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) + log.Error(getErr, "Failed to re-read SaveRequest to stamp delete-after; it will be kept", + "name", client.ObjectKeyFromObject(saveRequest)) } return } - if fresh.UID != commitRequest.UID { + if fresh.UID != saveRequest.UID { return } current = &fresh } log.Error(nil, "Gave up stamping delete-after after repeated conflicts; the request will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) + "name", client.ObjectKeyFromObject(saveRequest)) } -// commitRequestStatusUpdateAttempts bounds the terminal-status conflict retry. -const commitRequestStatusUpdateAttempts = 3 +// saveRequestStatusUpdateAttempts bounds the terminal-status conflict retry. +const saveRequestStatusUpdateAttempts = 3 -// writeTerminalStatus records the finalize outcome on the CommitRequest, +// writeTerminalStatus records the finalize outcome on the SaveRequest, // retrying on optimistic-concurrency conflicts. Like the audit-consumer path // it replaces, a permanently failing write is logged and given up on rather // than returned for requeue: re-running the reconcile would re-finalize an // already-flushed window and mis-report the outcome as NoOpenWindow. -func (r *CommitRequestReconciler) writeTerminalStatus( +func (r *SaveRequestReconciler) writeTerminalStatus( ctx context.Context, log logr.Logger, - commitRequest *configbutleraiv1alpha3.CommitRequest, + saveRequest *configbutleraiv1alpha3.SaveRequest, result git.FinalizeResult, finalizeErr error, - attribution commitRequestAttribution, - target commitRequestTarget, + attribution saveRequestAttribution, + target saveRequestTarget, ) { - expectedUID := commitRequest.UID + expectedUID := saveRequest.UID reader := r.APIReader if reader == nil { reader = r.Client @@ -714,10 +714,10 @@ func (r *CommitRequestReconciler) writeTerminalStatus( // Once, here, and deliberately not inside the loop below: result and finalizeErr are inputs // that do not change across attempts, so this is the point the outcome is settled. Recording // per attempt would over-count every request that hit a conflict. - recordCommitRequestOutcome(ctx, commitRequestOutcome(result, finalizeErr), target) + recordSaveRequestOutcome(ctx, saveRequestOutcome(result, finalizeErr), target) - current := commitRequest - for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { + current := saveRequest + for attempt := 1; attempt <= saveRequestStatusUpdateAttempts; attempt++ { applyFinalizeResultToStatus(current, result, finalizeErr, attribution) err := r.Status().Update(ctx, current) @@ -727,14 +727,14 @@ func (r *CommitRequestReconciler) writeTerminalStatus( if finalizeErr != nil { finalizeError = finalizeErr.Error() } - readyReason, readyMessage := commitRequestReadyReason(current) - log.Info("CommitRequest finalized", + readyReason, readyMessage := saveRequestReadyReason(current) + log.Info("SaveRequest finalized", "name", client.ObjectKeyFromObject(current), - "ready", commitRequestConditionStatus(current, ConditionTypeReady), + "ready", saveRequestConditionStatus(current, ConditionTypeReady), "reason", readyReason, "message", readyMessage, - "authorAttributed", commitRequestConditionStatus(current, ConditionTypeAuthorAttributed), - "pushed", commitRequestConditionStatus(current, ConditionTypePushed), + "authorAttributed", saveRequestConditionStatus(current, ConditionTypeAuthorAttributed), + "pushed", saveRequestConditionStatus(current, ConditionTypePushed), "branch", current.Status.Branch, "commit", current.Status.Commit, "outcome", result.Outcome, @@ -744,42 +744,42 @@ func (r *CommitRequestReconciler) writeTerminalStatus( return } if !apierrors.IsConflict(err) { - log.Error(err, "Failed to write CommitRequest status") + log.Error(err, "Failed to write SaveRequest status") return } - log.V(1).Info("Conflict writing CommitRequest status; retrying", "attempt", attempt) - var fresh configbutleraiv1alpha3.CommitRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { + log.V(1).Info("Conflict writing SaveRequest status; retrying", "attempt", attempt) + var fresh configbutleraiv1alpha3.SaveRequest + if getErr := reader.Get(ctx, client.ObjectKeyFromObject(saveRequest), &fresh); getErr != nil { if apierrors.IsNotFound(getErr) { - log.Info("CommitRequest deleted before status could be written; skipping") + log.Info("SaveRequest deleted before status could be written; skipping") return } - log.Error(getErr, "Failed to re-read CommitRequest for status update") + log.Error(getErr, "Failed to re-read SaveRequest for status update") return } // Never stamp the outcome onto a different incarnation or over a // terminal outcome another writer got in first. if fresh.UID != expectedUID { - log.Info("CommitRequest UID changed before status could be written; skipping", + log.Info("SaveRequest UID changed before status could be written; skipping", "expectedUID", expectedUID, "objectUID", fresh.UID) return } - if commitRequestIsTerminal(&fresh) { + if saveRequestIsTerminal(&fresh) { return } current = &fresh } - log.Error(nil, "Gave up writing CommitRequest status after repeated conflicts") + log.Error(nil, "Gave up writing SaveRequest status after repeated conflicts") } // SetupWithManager sets up the controller with the Manager. // MaxConcurrentReconciles is pinned to 1 on purpose: the single worker IS the -// multi-CommitRequest ordering design — concurrent CommitRequests for the same +// multi-SaveRequest ordering design — concurrent SaveRequests for the same // GitTarget are serialized exactly as a dedicated finalize-coordinator // goroutine would serialize them, without the extra moving parts (see -// docs/spec/commitrequest-design.md). +// docs/spec/saverequest-design.md). // // Restart recovery is best-effort by design: the // message is durable in spec.message, so on restart any non-terminal request is @@ -789,10 +789,10 @@ func (r *CommitRequestReconciler) writeTerminalStatus( // in-memory outcome is gone, the re-driven attach finds the work already mirrored, // and it resolves Rejected/AlreadyPresent. We do not build a durable record to // close that. -func (r *CommitRequestReconciler) SetupWithManager(mgr ctrl.Manager) error { +func (r *SaveRequestReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&configbutleraiv1alpha3.CommitRequest{}). + For(&configbutleraiv1alpha3.SaveRequest{}). WithOptions(controller.Options{MaxConcurrentReconciles: 1}). - Named("commitrequest"). + Named("saverequest"). Complete(r) } diff --git a/internal/controller/commitrequest_controller_test.go b/internal/controller/saverequest_controller_test.go similarity index 81% rename from internal/controller/commitrequest_controller_test.go rename to internal/controller/saverequest_controller_test.go index 2a65b0a6..18cde54b 100644 --- a/internal/controller/commitrequest_controller_test.go +++ b/internal/controller/saverequest_controller_test.go @@ -15,32 +15,32 @@ import ( configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -var _ = Describe("CommitRequest controller", func() { +var _ = Describe("SaveRequest controller", func() { const namespace = "default" // The suite registers the reconciler in attributed-author mode (a non-nil AuthorLookup // that never resolves) with a Finalizer that never resolves, so these specs cover // the in-progress stamp and the terminal short-circuit only; the full // attribute → attach → terminal flow and the configured-author path are covered by - // the unit tests in commitrequest_controller_unit_test.go. - It("stamps a freshly created CommitRequest with in-progress conditions", func() { - commitRequest := &configbutleraiv1alpha3.CommitRequest{ + // the unit tests in saverequest_controller_unit_test.go. + It("stamps a freshly created SaveRequest with in-progress conditions", func() { + saveRequest := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "save-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{ Name: "team-a-config", }, Message: "increase checkout API memory", }, } - Expect(k8sClient.Create(ctx, commitRequest)).To(Succeed()) - key := client.ObjectKeyFromObject(commitRequest) + Expect(k8sClient.Create(ctx, saveRequest)).To(Succeed()) + key := client.ObjectKeyFromObject(saveRequest) Eventually(func(g Gomega) { - var fetched configbutleraiv1alpha3.CommitRequest + var fetched configbutleraiv1alpha3.SaveRequest g.Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) reconciling := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReconciling) g.Expect(reconciling).NotTo(BeNil()) @@ -52,23 +52,23 @@ var _ = Describe("CommitRequest controller", func() { }) It("does not overwrite a terminal outcome that is already recorded", func() { - commitRequest := &configbutleraiv1alpha3.CommitRequest{ + saveRequest := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "save-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{ Name: "team-a-config", }, }, } - Expect(k8sClient.Create(ctx, commitRequest)).To(Succeed()) - key := client.ObjectKeyFromObject(commitRequest) + Expect(k8sClient.Create(ctx, saveRequest)).To(Succeed()) + key := client.ObjectKeyFromObject(saveRequest) // Wait for the initial in-progress stamp. Eventually(func(g Gomega) { - var fetched configbutleraiv1alpha3.CommitRequest + var fetched configbutleraiv1alpha3.SaveRequest g.Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) ready := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) @@ -76,7 +76,7 @@ var _ = Describe("CommitRequest controller", func() { }, 10*time.Second, 200*time.Millisecond).Should(Succeed()) // Simulate a finalize having recorded the terminal outcome (Ready=True). - var fetched configbutleraiv1alpha3.CommitRequest + var fetched configbutleraiv1alpha3.SaveRequest Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) apimeta.SetStatusCondition(&fetched.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: crReasonCommitted, Message: "committed", @@ -90,7 +90,7 @@ var _ = Describe("CommitRequest controller", func() { // The controller must leave the terminal outcome intact. Consistently(func(g Gomega) { - var checked configbutleraiv1alpha3.CommitRequest + var checked configbutleraiv1alpha3.SaveRequest g.Expect(k8sClient.Get(ctx, key, &checked)).To(Succeed()) ready := apimeta.FindStatusCondition(checked.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) @@ -104,16 +104,16 @@ var _ = Describe("CommitRequest controller", func() { // rejected only when the CRD is installed, and a rule that compiles but matches nothing simply // never rejects anything, so it is pinned against a real API server rather than only through the // controller's Go validator. -var _ = Describe("CommitRequest message schema", func() { +var _ = Describe("SaveRequest message schema", func() { const namespace = "default" - newRequest := func(message string) *configbutleraiv1alpha3.CommitRequest { - return &configbutleraiv1alpha3.CommitRequest{ + newRequest := func(message string) *configbutleraiv1alpha3.SaveRequest { + return &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "message-schema-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.CommitRequestSpec{ + Spec: configbutleraiv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: "team-a-config"}, Message: message, }, diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/saverequest_controller_unit_test.go similarity index 69% rename from internal/controller/commitrequest_controller_unit_test.go rename to internal/controller/saverequest_controller_unit_test.go index 48cce252..93d8e9c2 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/saverequest_controller_unit_test.go @@ -35,7 +35,7 @@ type fakeFinalizer struct { resolved bool err error - calls []git.AttachCommitRequest + calls []git.AttachSaveRequest // The withdraw reply. Left unset, the withdraw answers like the router does before it cancels // anything: with the worker's outcome, or its phase while unresolved. withdrawPending makes an @@ -44,7 +44,7 @@ type fakeFinalizer struct { withdrawResolved bool withdrawErr error withdrawPending bool - withdrawCalls []git.AttachCommitRequest + withdrawCalls []git.AttachSaveRequest // The apply a request with catchUp: AfterPush asked for, and the lookups made. apply git.SaveApply @@ -53,15 +53,15 @@ type fakeFinalizer struct { applyLookups int } -func (f *fakeFinalizer) LookupCommitRequestApply( - _ context.Context, _ git.AttachCommitRequest, +func (f *fakeFinalizer) LookupSaveRequestApply( + _ context.Context, _ git.AttachSaveRequest, ) (git.SaveApply, bool, error) { f.applyLookups++ return f.apply, f.applyKnown, f.applyErr } -func (f *fakeFinalizer) WithdrawCommitRequest( - _ context.Context, attach git.AttachCommitRequest, +func (f *fakeFinalizer) WithdrawSaveRequest( + _ context.Context, attach git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { f.withdrawCalls = append(f.withdrawCalls, attach) if f.withdrawPending || f.withdrawResolved || f.withdrawErr != nil || f.withdrawResult.Phase != "" { @@ -70,8 +70,8 @@ func (f *fakeFinalizer) WithdrawCommitRequest( return f.result, f.resolved, f.err } -func (f *fakeFinalizer) ServiceCommitRequest( - _ context.Context, attach git.AttachCommitRequest, +func (f *fakeFinalizer) ServiceSaveRequest( + _ context.Context, attach git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { f.calls = append(f.calls, attach) return f.result, f.resolved, f.err @@ -97,17 +97,17 @@ func attributedAlice() *fakeAuthorLookup { return &fakeAuthorLookup{author: queue.CommandAuthor{Author: "alice"}, found: true} } -// newCommitRequest builds a request created just now, inside its safety window, as the API server +// newSaveRequest builds a request created just now, inside its safety window, as the API server // would stamp it. pastTheBound ages one beyond it. -func newCommitRequest(name string) *configv1alpha3.CommitRequest { - return &configv1alpha3.CommitRequest{ +func newSaveRequest(name string) *configv1alpha3.SaveRequest { + return &configv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{ Name: name, Namespace: "default", UID: types.UID("uid-" + name), CreationTimestamp: metav1.Now(), }, - Spec: configv1alpha3.CommitRequestSpec{ + Spec: configv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: "team-a-config"}, Message: "save: " + name, }, @@ -115,13 +115,13 @@ func newCommitRequest(name string) *configv1alpha3.CommitRequest { } // pastTheBound ages a request beyond its safety window, where the controller only withdraws. -func pastTheBound(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { +func pastTheBound(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(cr.Spec).resolveTimeout())) return cr } -// withReadyCommitted stamps a terminal Committed CommitRequest (Ready=True). -func withReadyCommitted(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { +// withReadyCommitted stamps a terminal Committed SaveRequest (Ready=True). +func withReadyCommitted(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: crReasonCommitted, Message: "committed", @@ -131,7 +131,7 @@ func withReadyCommitted(cr *configv1alpha3.CommitRequest) *configv1alpha3.Commit // withInProgress stamps the still-running conditions a post-restart redelivery // would already carry (Ready=False, Reconciling=True), which is non-terminal. -func withInProgress(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { +func withInProgress(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionFalse, Reason: string(git.PhaseWaitingForWindow), Message: "in progress", @@ -143,21 +143,21 @@ func withInProgress(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequ return cr } -func newCommitRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...client.Object) client.WithWatch { +func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...client.Object) client.WithWatch { t.Helper() scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) builder := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(objects...). - WithStatusSubresource(&configv1alpha3.CommitRequest{}) + WithStatusSubresource(&configv1alpha3.SaveRequest{}) if fns != nil { builder = builder.WithInterceptorFuncs(*fns) } return builder.Build() } -func reconcileCommitRequest(t *testing.T, r *CommitRequestReconciler, name string) ctrl.Result { +func reconcileSaveRequest(t *testing.T, r *SaveRequestReconciler, name string) ctrl.Result { t.Helper() res, err := r.Reconcile(context.Background(), ctrl.Request{ NamespacedName: client.ObjectKey{Namespace: "default", Name: name}, @@ -166,9 +166,9 @@ func reconcileCommitRequest(t *testing.T, r *CommitRequestReconciler, name strin return res } -func fetchCommitRequest(t *testing.T, c client.Client, name string) configv1alpha3.CommitRequest { +func fetchSaveRequest(t *testing.T, c client.Client, name string) configv1alpha3.SaveRequest { t.Helper() - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest require.NoError(t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: name}, &cr)) return cr } @@ -177,7 +177,7 @@ func fetchCommitRequest(t *testing.T, c client.Client, name string) configv1alph // returns it for further assertions. func requireCondition( t *testing.T, - cr configv1alpha3.CommitRequest, + cr configv1alpha3.SaveRequest, condType string, status metav1.ConditionStatus, reason string, @@ -192,18 +192,18 @@ func requireCondition( return *c } -func TestCommitRequestReconcile_Committed(t *testing.T) { - cr := newCommitRequest("save-1") - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_Committed(t *testing.T) { + cr := newSaveRequest("save-1") + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-1") + reconcileSaveRequest(t, r, "save-1") - got := fetchCommitRequest(t, c, "save-1") + got := fetchSaveRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) @@ -223,18 +223,18 @@ func TestCommitRequestReconcile_Committed(t *testing.T) { assert.Equal(t, "uid-save-1", call.UID) } -func TestCommitRequestReconcile_NoOpenWindow(t *testing.T) { - cr := newCommitRequest("save-now") - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_NoOpenWindow(t *testing.T) { + cr := newSaveRequest("save-now") + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") - got := fetchCommitRequest(t, c, "save-now") + got := fetchSaveRequest(t, c, "save-now") // A benign no-commit is Ready=True (serviced) with the specific reason, Pushed=False. requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonNoWindow) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonNoWindow) @@ -244,7 +244,7 @@ func TestCommitRequestReconcile_NoOpenWindow(t *testing.T) { // whenNothingToCommit: CommitEmpty reaches the worker, and a recorded empty commit reports its // commit and Pushed=True while the Ready reason keeps the cause. -func TestCommitRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *testing.T) { +func TestSaveRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *testing.T) { for _, tc := range []struct { outcome git.FinalizeOutcome reason string @@ -253,20 +253,20 @@ func TestCommitRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t * {git.FinalizeAlreadyPresent, crReasonAlreadyPresent}, } { t.Run(tc.reason, func(t *testing.T) { - cr := newCommitRequest("save-empty") + cr := newSaveRequest("save-empty") cr.Spec.WhenNothingToCommit = configv1alpha3.NothingToCommitCommitEmpty - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: tc.outcome, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-empty") + reconcileSaveRequest(t, r, "save-empty") require.Len(t, f.calls, 1) assert.True(t, f.calls[0].CommitEmpty, "the choice must reach the worker") - got := fetchCommitRequest(t, c, "save-empty") + got := fetchSaveRequest(t, c, "save-empty") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, tc.reason) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeStalled, metav1.ConditionFalse, "") @@ -278,19 +278,19 @@ func TestCommitRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t * } // The author-bound refusal: an open window belonging to someone else is left -// open and the CommitRequest reports the WindowMismatch reason (no commit). -func TestCommitRequestReconcile_WindowMismatchIsExplained(t *testing.T) { - cr := newCommitRequest("save-foreign") - c := newCommitRequestClient(t, nil, cr) +// open and the SaveRequest reports the WindowMismatch reason (no commit). +func TestSaveRequestReconcile_WindowMismatchIsExplained(t *testing.T) { + cr := newSaveRequest("save-foreign") + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-foreign") + reconcileSaveRequest(t, r, "save-foreign") - got := fetchCommitRequest(t, c, "save-foreign") + got := fetchSaveRequest(t, c, "save-foreign") ready := requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonWindowMismatch) assert.Equal(t, windowMismatchMessage, ready.Message) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonWindowMismatch) @@ -299,65 +299,65 @@ func TestCommitRequestReconcile_WindowMismatchIsExplained(t *testing.T) { // A matching window that finalized with no diff (loop prevention) reports the // AlreadyPresent reason — never left hanging. -func TestCommitRequestReconcile_AlreadyPresentRejected(t *testing.T) { - cr := newCommitRequest("save-noop") - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_AlreadyPresentRejected(t *testing.T) { + cr := newSaveRequest("save-noop") + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-noop") + reconcileSaveRequest(t, r, "save-noop") - got := fetchCommitRequest(t, c, "save-noop") + got := fetchSaveRequest(t, c, "save-noop") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonAlreadyPresent) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonAlreadyPresent) assert.Empty(t, got.Status.Commit) } // A resolved outcome that carries an error becomes a Failed (Stalled=True) request. -func TestCommitRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { - cr := newCommitRequest("save-fail") - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { + cr := newSaveRequest("save-fail") + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Err: errors.New("commit failed: unreachable remote")}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-fail") + reconcileSaveRequest(t, r, "save-fail") - got := fetchCommitRequest(t, c, "save-fail") + got := fetchSaveRequest(t, c, "save-fail") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonFinalizeFailed) stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Contains(t, stalled.Message, "unreachable remote") requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, "") } -// TestCommitRequestReconcile_LookupMissClaimsNoActor verifies that a lookup miss claims no actor -// immediately — present-or-never, no wait. A freshly created CommitRequest attaches at once with a +// TestSaveRequestReconcile_LookupMissClaimsNoActor verifies that a lookup miss claims no actor +// immediately — present-or-never, no wait. A freshly created SaveRequest attaches at once with a // blank author and records AuthorAttributed=False (CommitterFallback). The matching window determines // the Git author. -func TestCommitRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { - cr := newCommitRequest("save-fresh") +func TestSaveRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { + cr := newSaveRequest("save-fresh") cr.CreationTimestamp = metav1.Now() // fresh: the old path would have requeued here - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Branch: "main"}, resolved: true, } lookup := &fakeAuthorLookup{found: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} - res := reconcileCommitRequest(t, r, "save-fresh") + res := reconcileSaveRequest(t, r, "save-fresh") assert.Zero(t, res.RequeueAfter, "a miss is final; the request must not poll for an author") assert.Equal(t, 1, lookup.calls, "the author is looked up exactly once, synchronously") require.Len(t, f.calls, 1, "the attach is sent immediately on a miss") assert.Empty(t, f.calls[0].Author, "a miss attaches with a blank, unnamed actor") - got := fetchCommitRequest(t, c, "save-fresh") + got := fetchSaveRequest(t, c, "save-fresh") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback) @@ -367,15 +367,15 @@ func TestCommitRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { // not zeros. The API server normally fills the block in, so nil arrives here only from a client // that bypassed defaulting — and a zero there would be the old defect: the request expires before // the write it exists to publish can open a window. -func TestCommitRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { - cr := newCommitRequest("save-default") +func TestSaveRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { + cr := newSaveRequest("save-default") cr.CreationTimestamp = metav1.Now() require.Nil(t, cr.Spec.Window, "the fixture must leave the block unset") - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-default") + reconcileSaveRequest(t, r, "save-default") require.Len(t, f.calls, 1) assert.Equal(t, configv1alpha3.AttachCurrentOrNext, f.calls[0].Attach) @@ -386,19 +386,19 @@ func TestCommitRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { // The pointer types exist so that an explicit zero survives defaulting, and each field keeps its // own meaning: a zero attachTimeout is not a zero maxDuration. -func TestCommitRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing.T) { - cr := newCommitRequest("save-now") +func TestSaveRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing.T) { + cr := newSaveRequest("save-now") cr.CreationTimestamp = metav1.Now() - cr.Spec.Window = &configv1alpha3.CommitRequestWindow{ + cr.Spec.Window = &configv1alpha3.SaveRequestWindow{ AttachTimeout: &metav1.Duration{}, IdleTimeout: &metav1.Duration{}, MaxDuration: &metav1.Duration{Duration: 30 * time.Second}, } - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") require.Len(t, f.calls, 1) assert.Zero(t, f.calls[0].AttachTimeout, "an explicit \"0s\" must not be re-read as an omitted field") @@ -410,78 +410,78 @@ func TestCommitRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing // The window is the worker's job: the controller does not hold the finalize itself. While the // worker has not resolved the attach, the controller polls and reports the phase the WORKER // reports, never one it inferred from having sent the attach. -func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) { - cr := newCommitRequest("save-linger") +func TestSaveRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) { + cr := newSaveRequest("save-linger") cr.CreationTimestamp = metav1.Now() - cr.Spec.Window = &configv1alpha3.CommitRequestWindow{MaxDuration: &metav1.Duration{Duration: 30 * time.Second}} - c := newCommitRequestClient(t, nil, cr) + cr.Spec.Window = &configv1alpha3.SaveRequestWindow{MaxDuration: &metav1.Duration{Duration: 30 * time.Second}} + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileCommitRequest(t, r, "save-linger") + res := reconcileSaveRequest(t, r, "save-linger") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, "an unresolved attach must be polled, not held by a controller-side delay") require.Len(t, f.calls, 1, "the attach is sent the instant the author is known") assert.Equal(t, 30*time.Second, f.calls[0].MaxDuration, "the window is passed to the worker, not consumed here") - got := fetchCommitRequest(t, c, "save-linger") + got := fetchSaveRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, commitRequestIsTerminal(&got), "an unresolved request is not terminal") + assert.False(t, saveRequestIsTerminal(&got), "an unresolved request is not terminal") - for _, phase := range []git.CommitRequestPhase{ + for _, phase := range []git.SaveRequestPhase{ git.PhaseWaitingForWindow, git.PhaseCollectingWindow, git.PhaseWaitingForPush, } { f.result = git.FinalizeResult{Phase: phase} - reconcileCommitRequest(t, r, "save-linger") - got = fetchCommitRequest(t, c, "save-linger") + reconcileSaveRequest(t, r, "save-linger") + got = fetchSaveRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, string(phase)) requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, string(phase)) } // A phase the worker has not (yet) reported keeps what is shown rather than guessing. f.result = git.FinalizeResult{} - reconcileCommitRequest(t, r, "save-linger") - got = fetchCommitRequest(t, c, "save-linger") + reconcileSaveRequest(t, r, "save-linger") + got = fetchSaveRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, string(git.PhaseWaitingForPush)) } // A transient service error (e.g. the GitTarget momentarily unreadable) keeps the // request polling within the safety window rather than failing it outright. -func TestCommitRequestReconcile_ServiceErrorPolls(t *testing.T) { - cr := newCommitRequest("save-transient") +func TestSaveRequestReconcile_ServiceErrorPolls(t *testing.T) { + cr := newSaveRequest("save-transient") cr.CreationTimestamp = metav1.Now() - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{err: errors.New("get GitTarget: not found")} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileCommitRequest(t, r, "save-transient") + res := reconcileSaveRequest(t, r, "save-transient") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) - got := fetchCommitRequest(t, c, "save-transient") + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) + got := fetchSaveRequest(t, c, "save-transient") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, commitRequestIsTerminal(&got)) + assert.False(t, saveRequestIsTerminal(&got)) } // Past the safety window a request the worker never acted on is withdrawn, and fails closed once // the worker confirms the withdraw. -func TestCommitRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { - cr := pastTheBound(newCommitRequest("save-stuck")) - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { + cr := pastTheBound(newSaveRequest("save-stuck")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWindow}, - withdrawResult: git.FinalizeResult{Branch: "main", Err: git.ErrCommitRequestWithdrawn}, + withdrawResult: git.FinalizeResult{Branch: "main", Err: git.ErrSaveRequestWithdrawn}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-stuck") + reconcileSaveRequest(t, r, "save-stuck") require.Len(t, f.withdrawCalls, 1, "the request is withdrawn before it is failed") assert.Empty(t, f.calls, "past the bound no attach is sent: it would queue behind the withdraw") - got := fetchCommitRequest(t, c, "save-stuck") + got := fetchSaveRequest(t, c, "save-stuck") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonFinalizeFailed) stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Equal(t, resolveTimeoutMessage, stalled.Message) @@ -489,27 +489,27 @@ func TestCommitRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { // A request the worker holds is never failed on the controller's clock: a down remote or a parent // branch recovery can keep it in WaitingForPush far past the bound, and the worker still commits it. -func TestCommitRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { - for _, phase := range []git.CommitRequestPhase{git.PhaseCollectingWindow, git.PhaseWaitingForPush} { +func TestSaveRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { + for _, phase := range []git.SaveRequestPhase{git.PhaseCollectingWindow, git.PhaseWaitingForPush} { t.Run(string(phase), func(t *testing.T) { - cr := pastTheBound(newCommitRequest("save-held")) - c := newCommitRequestClient(t, nil, cr) + cr := pastTheBound(newSaveRequest("save-held")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{result: git.FinalizeResult{Phase: phase}} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileCommitRequest(t, r, "save-held") + res := reconcileSaveRequest(t, r, "save-held") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) assert.Empty(t, f.calls, "past the bound no attach is sent") assert.Len(t, f.withdrawCalls, 1, "the withdraw finds the request held and changes nothing") - got := fetchCommitRequest(t, c, "save-held") - assert.False(t, commitRequestIsTerminal(&got)) + got := fetchSaveRequest(t, c, "save-held") + assert.False(t, saveRequestIsTerminal(&got)) // The push lands much later: the request resolves Committed, not FinalizeFailed. f.result = git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"} f.resolved = true - reconcileCommitRequest(t, r, "save-held") - got = fetchCommitRequest(t, c, "save-held") + reconcileSaveRequest(t, r, "save-held") + got = fetchSaveRequest(t, c, "save-held") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) }) } @@ -517,115 +517,115 @@ func TestCommitRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { // Until the worker answers the withdraw the request keeps polling; and when the worker turns out to // hold it after all (it attached between the poll and the withdraw), nothing fails. -func TestCommitRequestReconcile_AnUnansweredWithdrawKeepsPolling(t *testing.T) { +func TestSaveRequestReconcile_AnUnansweredWithdrawKeepsPolling(t *testing.T) { for name, reply := range map[string]git.FinalizeResult{ "not handled yet": {}, "the worker now holds it": {Phase: git.PhaseCollectingWindow}, } { t.Run(name, func(t *testing.T) { - cr := pastTheBound(newCommitRequest("save-race")) - c := newCommitRequestClient(t, nil, cr) + cr := pastTheBound(newSaveRequest("save-race")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWindow}, withdrawResult: reply, withdrawPending: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileCommitRequest(t, r, "save-race") + res := reconcileSaveRequest(t, r, "save-race") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) - got := fetchCommitRequest(t, c, "save-race") - assert.False(t, commitRequestIsTerminal(&got)) + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) + got := fetchSaveRequest(t, c, "save-race") + assert.False(t, saveRequestIsTerminal(&got)) }) } } // A worker that resolved the request on its own just before the withdraw wins: its real outcome is // what the request reports. -func TestCommitRequestReconcile_AWithdrawThatLostTheRaceReportsTheRealOutcome(t *testing.T) { - cr := pastTheBound(newCommitRequest("save-late")) - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_AWithdrawThatLostTheRaceReportsTheRealOutcome(t *testing.T) { + cr := pastTheBound(newSaveRequest("save-late")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ withdrawResult: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-late") + reconcileSaveRequest(t, r, "save-late") - got := fetchCommitRequest(t, c, "save-late") + got := fetchSaveRequest(t, c, "save-late") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Equal(t, "abc", got.Status.Commit) } // A GitTarget that never started a branch worker is named as the cause. -func TestCommitRequestReconcile_NoWorkerIsNamedWhenItFailsClosed(t *testing.T) { - cr := pastTheBound(newCommitRequest("save-no-worker")) - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_NoWorkerIsNamedWhenItFailsClosed(t *testing.T) { + cr := pastTheBound(newSaveRequest("save-no-worker")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWorker}, withdrawResult: git.FinalizeResult{ - Branch: "main", Phase: git.PhaseWaitingForWorker, Err: git.ErrCommitRequestWithdrawn, + Branch: "main", Phase: git.PhaseWaitingForWorker, Err: git.ErrSaveRequestWithdrawn, }, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-no-worker") + reconcileSaveRequest(t, r, "save-no-worker") - got := fetchCommitRequest(t, c, "save-no-worker") + got := fetchSaveRequest(t, c, "save-no-worker") stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Equal(t, noWorkerTimeoutMessage, stalled.Message) } -func TestCommitRequestReconcile_TerminalShortCircuits(t *testing.T) { - cr := withReadyCommitted(newCommitRequest("save-done")) - c := newCommitRequestClient(t, nil, cr) +func TestSaveRequestReconcile_TerminalShortCircuits(t *testing.T) { + cr := withReadyCommitted(newSaveRequest("save-done")) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{} lookup := attributedAlice() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} - reconcileCommitRequest(t, r, "save-done") + reconcileSaveRequest(t, r, "save-done") assert.Zero(t, lookup.calls) - assert.Empty(t, f.calls, "a terminal CommitRequest must never re-attach") + assert.Empty(t, f.calls, "a terminal SaveRequest must never re-attach") } // A reconcile triggered by a stale cache echo (the cached object still says // in-progress while the live object is already terminal) must not re-run the // finalize: the uncached APIReader read is the guard. -func TestCommitRequestReconcile_StaleCacheEchoDoesNotRefinalize(t *testing.T) { - stale := withInProgress(newCommitRequest("save-echo")) - cached := newCommitRequestClient(t, nil, stale) - terminal := withReadyCommitted(newCommitRequest("save-echo")) - live := newCommitRequestClient(t, nil, terminal) +func TestSaveRequestReconcile_StaleCacheEchoDoesNotRefinalize(t *testing.T) { + stale := withInProgress(newSaveRequest("save-echo")) + cached := newSaveRequestClient(t, nil, stale) + terminal := withReadyCommitted(newSaveRequest("save-echo")) + live := newSaveRequestClient(t, nil, terminal) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: cached, APIReader: live, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: cached, APIReader: live, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-echo") + reconcileSaveRequest(t, r, "save-echo") assert.Empty(t, f.calls) } -// Webhook-disabled mode (no AuthorLookup) never waits: a freshly created CommitRequest +// Webhook-disabled mode (no AuthorLookup) never waits: a freshly created SaveRequest // attaches immediately with a blank author and records AuthorAttributed=False // (AuthorCaptureDisabled). The matching window determines the Git author. -func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T) { - cr := newCommitRequest("save-configured-author") +func TestSaveRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T) { + cr := newSaveRequest("save-configured-author") cr.CreationTimestamp = metav1.Now() // fresh: a waiting path would requeue instead of commit - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil - res := reconcileCommitRequest(t, r, "save-configured-author") + res := reconcileSaveRequest(t, r, "save-configured-author") assert.Zero(t, res.RequeueAfter, "webhook-disabled mode must not requeue waiting for an author") - got := fetchCommitRequest(t, c, "save-configured-author") + got := fetchSaveRequest(t, c, "save-configured-author") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) @@ -637,55 +637,55 @@ func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing // Webhook-disabled mode settles AuthorAttributed=False (AuthorCaptureDisabled) immediately // and never parks the request in a "waiting for the author" state: even while the // attach is still being polled, the request reports progress. -func TestCommitRequestReconcile_ConfiguredAuthorAttributedImmediately(t *testing.T) { - cr := newCommitRequest("save-committer-poll") +func TestSaveRequestReconcile_ConfiguredAuthorAttributedImmediately(t *testing.T) { + cr := newSaveRequest("save-committer-poll") cr.CreationTimestamp = metav1.Now() - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil - res := reconcileCommitRequest(t, r, "save-committer-poll") + res := reconcileSaveRequest(t, r, "save-committer-poll") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "an unresolved attach is polled") + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, "an unresolved attach is polled") require.Len(t, f.calls, 1) assert.Empty(t, f.calls[0].Author) - got := fetchCommitRequest(t, c, "save-committer-poll") + got := fetchSaveRequest(t, c, "save-committer-poll") requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, commitRequestIsTerminal(&got), "in progress, not terminal") + assert.False(t, saveRequestIsTerminal(&got), "in progress, not terminal") } // With no Finalizer wired the controller is inert: it neither attaches, stamps any // condition, nor drives the request to a terminal outcome. (Production always wires // the Finalizer; this is the disabled guard at the top of Reconcile.) -func TestCommitRequestReconcile_NilFinalizerIsInert(t *testing.T) { - cr := newCommitRequest("save-no-finalizer") - c := newCommitRequestClient(t, nil, cr) - r := &CommitRequestReconciler{Client: c, APIReader: c} // Finalizer nil, AuthorLookup nil +func TestSaveRequestReconcile_NilFinalizerIsInert(t *testing.T) { + cr := newSaveRequest("save-no-finalizer") + c := newSaveRequestClient(t, nil, cr) + r := &SaveRequestReconciler{Client: c, APIReader: c} // Finalizer nil, AuthorLookup nil - res := reconcileCommitRequest(t, r, "save-no-finalizer") + res := reconcileSaveRequest(t, r, "save-no-finalizer") assert.Zero(t, res.RequeueAfter) - got := fetchCommitRequest(t, c, "save-no-finalizer") + got := fetchSaveRequest(t, c, "save-no-finalizer") assert.Empty(t, got.Status.Conditions, "a disabled controller stamps nothing") - assert.False(t, commitRequestIsTerminal(&got)) + assert.False(t, saveRequestIsTerminal(&got)) } -func TestCommitRequestReconcile_ObjectDeletedIsBenign(t *testing.T) { - c := newCommitRequestClient(t, nil) +func TestSaveRequestReconcile_ObjectDeletedIsBenign(t *testing.T) { + c := newSaveRequestClient(t, nil) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "gone") + reconcileSaveRequest(t, r, "gone") assert.Empty(t, f.calls) } -func TestCommitRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { +func TestSaveRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { // Already in-progress: this models a post-restart redelivery, so the only // status write is the terminal one — the write the conflict hits. - cr := withInProgress(newCommitRequest("save-retry")) + cr := withInProgress(newSaveRequest("save-retry")) conflicts := 1 fns := interceptor.Funcs{ @@ -699,32 +699,32 @@ func TestCommitRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { if conflicts > 0 { conflicts-- return apierrors.NewConflict( - schema.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}, + schema.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}, obj.GetName(), errors.New("simulated")) } return c.SubResource(subResource).Update(ctx, obj, opts...) }, } - c := newCommitRequestClient(t, &fns, cr) + c := newSaveRequestClient(t, &fns, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "ddd111", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-retry") + reconcileSaveRequest(t, r, "save-retry") - got := fetchCommitRequest(t, c, "save-retry") + got := fetchSaveRequest(t, c, "save-retry") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Equal(t, "ddd111", got.Status.Commit) require.Len(t, f.calls, 1, "the conflict retry must re-write status, not re-attach") } -// --- pure helpers (formerly internal/queue/commit_request_test.go) --- +// --- pure helpers (formerly internal/queue/save_request_test.go) --- func TestApplyFinalizeResultToStatus(t *testing.T) { t.Run("committed", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus( &cr, git.FinalizeResult{ @@ -745,7 +745,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("no window in grace is a benign ready (no admission author)", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, nil, attributionCommitter) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonNoWindow) @@ -755,7 +755,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("window mismatch surfaces the reason", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, nil, attributionFromAdmission) ready := requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonWindowMismatch) @@ -763,7 +763,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("already present is a benign ready", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, nil, attributionFromAdmission) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonAlreadyPresent) @@ -771,7 +771,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("finalize error stalls", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc"}, errors.New("boom"), attributionFromAdmission) @@ -781,7 +781,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("unknown outcome stalls", func(t *testing.T) { - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{}, nil, attributionFromAdmission) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionFalse, crReasonUnexpectedOutcome) stalled := requireCondition(t, cr, ConditionTypeStalled, metav1.ConditionTrue, crReasonUnexpectedOutcome) @@ -789,30 +789,30 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) } -func TestCommitRequestIsTerminal(t *testing.T) { - assert.False(t, commitRequestIsTerminal(newCommitRequest("empty"))) - assert.False(t, commitRequestIsTerminal(withInProgress(newCommitRequest("in-progress")))) - assert.True(t, commitRequestIsTerminal(withReadyCommitted(newCommitRequest("committed")))) +func TestSaveRequestIsTerminal(t *testing.T) { + assert.False(t, saveRequestIsTerminal(newSaveRequest("empty"))) + assert.False(t, saveRequestIsTerminal(withInProgress(newSaveRequest("in-progress")))) + assert.True(t, saveRequestIsTerminal(withReadyCommitted(newSaveRequest("committed")))) - stalled := newCommitRequest("failed") + stalled := newSaveRequest("failed") apimeta.SetStatusCondition(&stalled.Status.Conditions, metav1.Condition{ Type: ConditionTypeStalled, Status: metav1.ConditionTrue, Reason: crReasonFinalizeFailed, Message: "boom", }) - assert.True(t, commitRequestIsTerminal(stalled)) + assert.True(t, saveRequestIsTerminal(stalled)) } -// A CommitRequest stored before spec.targetRef became spec.gitTargetRef serves an EMPTY name after +// A SaveRequest stored before spec.targetRef became spec.gitTargetRef serves an EMPTY name after // the upgrade prunes it, and its spec is immutable, so no apply can put the name back. The // reconciler must say that once and stop, rather than retry a "get GitTarget" that names the empty // string and can never succeed. -func TestCommitRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { - request := &configv1alpha3.CommitRequest{ +func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { + request := &configv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{ Name: "stored-before-the-rename", Namespace: "team-a", UID: "cr-uid", }, - Spec: configv1alpha3.CommitRequestSpec{ + Spec: configv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: ""}, }, } @@ -826,7 +826,7 @@ func TestCommitRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { Build() finalizer := &fakeFinalizer{} - reconciler := &CommitRequestReconciler{Client: k8sClient, Scheme: scheme, Finalizer: finalizer} + reconciler := &SaveRequestReconciler{Client: k8sClient, Scheme: scheme, Finalizer: finalizer} result, err := reconciler.Reconcile(context.Background(), ctrl.Request{ NamespacedName: types.NamespacedName{Name: request.Name, Namespace: request.Namespace}, @@ -836,10 +836,10 @@ func TestCommitRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { assert.Zero(t, result.RequeueAfter, "it must not be retried: no apply can ever fix it") assert.Empty(t, finalizer.calls, "the router must never be asked for GitTarget \"\"") - var got configv1alpha3.CommitRequest + var got configv1alpha3.SaveRequest require.NoError(t, k8sClient.Get(context.Background(), types.NamespacedName{Name: request.Name, Namespace: request.Namespace}, &got)) - assert.True(t, commitRequestIsTerminal(&got), "the request must reach a terminal condition") + assert.True(t, saveRequestIsTerminal(&got), "the request must reach a terminal condition") stalled := findCondition(got.Status.Conditions, ConditionTypeStalled) require.NotNil(t, stalled) assert.Equal(t, metav1.ConditionTrue, stalled.Status) @@ -848,7 +848,7 @@ func TestCommitRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { "the message must name the only repair there is, because the spec is immutable") } -func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { +func TestSaveRequestReconciler_LiteralValidation(t *testing.T) { for _, tc := range []struct { name, message string valid bool @@ -860,18 +860,18 @@ func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { {"control", "bad\tmessage", false}, } { t.Run(tc.name, func(t *testing.T) { - cr := newCommitRequest(tc.name) + cr := newSaveRequest(tc.name) cr.Spec.Message = tc.message - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) finalizer := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, Finalizer: finalizer} - reconcileCommitRequest(t, r, cr.Name) + r := &SaveRequestReconciler{Client: c, Finalizer: finalizer} + reconcileSaveRequest(t, r, cr.Name) if tc.valid { require.Len(t, finalizer.calls, 1) assert.Equal(t, tc.message, finalizer.calls[0].Message) } else { require.Empty(t, finalizer.calls) - var got configv1alpha3.CommitRequest + var got configv1alpha3.SaveRequest require.NoError(t, c.Get(context.Background(), client.ObjectKeyFromObject(cr), &got)) assert.True(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled)) } @@ -882,7 +882,7 @@ func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { // A request claims a window whose writes name no author only when it asked, its GitTarget allows // what it asked, and the GitTarget's KubeProvider has never received an audit fact. A request that // asks for more than its GitTarget allows fails at once, before anything is attached. -func TestCommitRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { +func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { target := func(policy configv1alpha3.UnattributedWindowPolicy) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, @@ -971,17 +971,17 @@ func TestCommitRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUna } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - cr := newCommitRequest("save") - cr.Spec.Window = &configv1alpha3.CommitRequestWindow{Author: tc.ask} - c := newCommitRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) + cr := newSaveRequest("save") + cr.Spec.Window = &configv1alpha3.SaveRequestWindow{Author: tc.ask} + c := newSaveRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileCommitRequest(t, r, "save") + reconcileSaveRequest(t, r, "save") if tc.refused { assert.Empty(t, f.calls, "a refused request is never attached") - var got configv1alpha3.CommitRequest + var got configv1alpha3.SaveRequest require.NoError( t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got), diff --git a/internal/controller/commitrequest_finalize.go b/internal/controller/saverequest_finalize.go similarity index 66% rename from internal/controller/commitrequest_finalize.go rename to internal/controller/saverequest_finalize.go index 9c3e1db4..1fa6d4bc 100644 --- a/internal/controller/commitrequest_finalize.go +++ b/internal/controller/saverequest_finalize.go @@ -16,7 +16,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/telemetry" ) -// CommitRequest condition reasons (CamelCase tokens surfaced on status.conditions). +// SaveRequest condition reasons (CamelCase tokens surfaced on status.conditions). const ( crReasonCommitted = "Committed" crReasonNoWindow = "NoWindow" @@ -31,7 +31,7 @@ const ( crReasonTakeOverNotAllowed = "TakeOverNotAllowed" ) -// The bounded `outcome` values on gitopsreverser_commit_requests_total. +// The bounded `outcome` values on gitopsreverser_save_requests_total. // // They are derived from git.FinalizeOutcome rather than from the condition reasons above: the // FinalizeResult is what DECIDES, and the reasons include non-terminal and attribution-only values @@ -45,13 +45,13 @@ const ( crOutcomeFailed = "failed" ) -// commitRequestOutcome maps one finalize result to its counter value. +// saveRequestOutcome maps one finalize result to its counter value. // // A finalize error outranks the outcome because FinalizeResult.Outcome is only set when Err is nil, // and an unrecognized outcome counts as failed for the same reason applyFinalizeResultToStatus // fails it: an empty or unknown outcome with no error is a bug, and a bug that increments nothing // is a bug nobody sees. -func commitRequestOutcome(result git.FinalizeResult, finalizeErr error) string { +func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { if finalizeErr != nil { return crOutcomeFailed } @@ -69,47 +69,47 @@ func commitRequestOutcome(result git.FinalizeResult, finalizeErr error) string { } } -// commitRequestTarget is the GitTarget identity one terminal outcome is counted under. +// saveRequestTarget is the GitTarget identity one terminal outcome is counted under. // // The zero value is the documented FALLBACK for a request whose target could not be resolved, and // it publishes empty labels rather than the requested name. That is deliberate: spec.gitTargetRef // is whatever the client wrote, so labelling an unresolved reference would let anyone with create -// rights on commitrequests mint an unbounded number of series by naming targets that do not exist. +// rights on saverequests mint an unbounded number of series by naming targets that do not exist. // The requested reference is still on the object's conditions and in the logs, which is where an // unbounded string belongs. -type commitRequestTarget struct { +type saveRequestTarget struct { namespace string name string } -// resolvedCommitRequestTarget names the GitTarget a request was serviced against. +// resolvedSaveRequestTarget names the GitTarget a request was serviced against. // // It is only correct to call once the workflow has RESOLVED that target, which -// EventRouter.ServiceCommitRequest does by getting the GitTarget and returning an error when it is +// EventRouter.ServiceSaveRequest does by getting the GitTarget and returning an error when it is // absent. So a nil service error is the proof that this identity exists, and no extra API lookup is // needed to bound the label — the plan's rule that metric recording must not block terminal status // on another failing call. // -// The namespace is the REQUEST's, not a field: a CommitRequest names a same-namespace GitTarget, -// which is the same assumption ServiceCommitRequest makes when it builds the object key. -func resolvedCommitRequestTarget(cr *configv1alpha3.CommitRequest) commitRequestTarget { - return commitRequestTarget{namespace: cr.Namespace, name: cr.Spec.GitTargetRef.Name} +// The namespace is the REQUEST's, not a field: a SaveRequest names a same-namespace GitTarget, +// which is the same assumption ServiceSaveRequest makes when it builds the object key. +func resolvedSaveRequestTarget(cr *configv1alpha3.SaveRequest) saveRequestTarget { + return saveRequestTarget{namespace: cr.Namespace, name: cr.Spec.GitTargetRef.Name} } -// recordCommitRequestOutcome increments the terminal-outcome counter once. +// recordSaveRequestOutcome increments the terminal-outcome counter once. // // It is called at the point the outcome is SETTLED, never from inside applyFinalizeResultToStatus: -// that runs inside writeTerminalStatus's conflict-retry loop, up to commitRequestStatusUpdateAttempts +// that runs inside writeTerminalStatus's conflict-retry loop, up to saveRequestStatusUpdateAttempts // times for one request, so recording there would over-report exactly the contended case. The // counter measures what the pipeline decided; whether the terminal status then reached the API // server is a separate failure, already logged. // // See the instrument's doc comment for what this does and does not promise across reconciles. -func recordCommitRequestOutcome(ctx context.Context, outcome string, target commitRequestTarget) { - if telemetry.CommitRequestsTotal == nil { +func recordSaveRequestOutcome(ctx context.Context, outcome string, target saveRequestTarget) { + if telemetry.SaveRequestsTotal == nil { return } - telemetry.CommitRequestsTotal.Add(ctx, 1, metric.WithAttributes( + telemetry.SaveRequestsTotal.Add(ctx, 1, metric.WithAttributes( attribute.String("outcome", outcome), attribute.String("gittarget_namespace", target.namespace), attribute.String("gittarget_name", target.name), @@ -126,18 +126,18 @@ const noWindowMessage = "no matching commit window was there to attach to before const alreadyPresentMessage = "the change already matches the remote, so no commit was made" // notStalledMessage is the boilerplate message for a False Stalled condition. -const notStalledMessage = "the CommitRequest is not stalled" +const notStalledMessage = "the SaveRequest is not stalled" -// commitRequestAttribution is the settled author decision threaded from the +// saveRequestAttribution is the settled author decision threaded from the // attribute step into the terminal status write so the AuthorAttributed condition can // record whether the commit author was named. It is binary and always settled at // first sight (present-or-never, §2) — there is no pending or timed-out state. -type commitRequestAttribution int +type saveRequestAttribution int const ( // attributionUnset is the defensive zero value. It must never report a resolved // actor when a caller neglected to initialize the attribution decision. - attributionUnset commitRequestAttribution = iota + attributionUnset saveRequestAttribution = iota // attributionFromAdmission means the submitter was captured at admission by the // validate-operator-types webhook and named as the commit author. attributionFromAdmission @@ -153,7 +153,7 @@ const ( // gitOutcome projects the controller's attribution state onto the git layer's outcome, which // is what the worker matches windows on. Keeping the projection here — rather than passing the // controller's own enum down — means the git package never learns about admission records. -func (a commitRequestAttribution) gitOutcome() git.AttributionOutcome { +func (a saveRequestAttribution) gitOutcome() git.AttributionOutcome { switch a { case attributionUnset: return git.AttributionNotAttempted @@ -168,10 +168,10 @@ func (a commitRequestAttribution) gitOutcome() git.AttributionOutcome { } } -// setCommitRequestCondition upserts a condition keyed by type, stamping it with +// setSaveRequestCondition upserts a condition keyed by type, stamping it with // the request's generation so kstatus can tell a current status from a stale one. -func setCommitRequestCondition( - cr *configv1alpha3.CommitRequest, +func setSaveRequestCondition( + cr *configv1alpha3.SaveRequest, conditionType string, status metav1.ConditionStatus, reason, message string, @@ -194,7 +194,7 @@ const ( // progressFor maps a worker-reported phase to its progress reason and message. held is why a // request waiting for its push has not reached the remote yet: the branch worker's publication // report, in the words its GitTarget uses. -func progressFor(phase git.CommitRequestPhase, held string) (string, string) { +func progressFor(phase git.SaveRequestPhase, held string) (string, string) { switch phase { case git.PhaseWaitingForWorker: return string(phase), waitingForWorkerMessage @@ -212,54 +212,54 @@ func progressFor(phase git.CommitRequestPhase, held string) (string, string) { } } -// setCommitRequestProgress stamps the four progress conditions (Ready=False, +// setSaveRequestProgress stamps the four progress conditions (Ready=False, // Reconciling=True, Stalled=False, Pushed=Unknown) with one unifying reason, so a // single wait reads consistently across them. -func setCommitRequestProgress(cr *configv1alpha3.CommitRequest, reason, message string) { - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) - setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionTrue, reason, message) - setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionUnknown, reason, pushPendingMessage) +func setSaveRequestProgress(cr *configv1alpha3.SaveRequest, reason, message string) { + setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) + setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionTrue, reason, message) + setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionUnknown, reason, pushPendingMessage) } -// markCommitRequestProgressing records the still-running state for a worker-reported phase +// markSaveRequestProgressing records the still-running state for a worker-reported phase // (Reconciling=True). An empty phase is the generic Progressing: sent, not yet confirmed. -func markCommitRequestProgressing( - cr *configv1alpha3.CommitRequest, - attribution commitRequestAttribution, - phase git.CommitRequestPhase, +func markSaveRequestProgressing( + cr *configv1alpha3.SaveRequest, + attribution saveRequestAttribution, + phase git.SaveRequestPhase, held string, ) { cr.Status.ObservedGeneration = cr.Generation - setCommitRequestAttributed(cr, attribution) + setSaveRequestAttributed(cr, attribution) reason, message := progressFor(phase, held) - setCommitRequestProgress(cr, reason, message) + setSaveRequestProgress(cr, reason, message) } -// setCommitRequestAttributed records the settled, binary author decision on the +// setSaveRequestAttributed records the settled, binary author decision on the // AuthorAttributed condition. A False reason distinguishes an absent record // (CommitterFallback) from capture disabled (AuthorCaptureDisabled). Neither is a failure or // affects Ready; the request claims no actor. -func setCommitRequestAttributed(cr *configv1alpha3.CommitRequest, attribution commitRequestAttribution) { +func setSaveRequestAttributed(cr *configv1alpha3.SaveRequest, attribution saveRequestAttribution) { switch attribution { case attributionUnset: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled, "command-author capture state was unset; the request can attach only to a window with no named actor") case attributionFromAdmission: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, + setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission, "the submitter was captured at admission and named as the commit author") case attributionCommitter: // Capture RAN and found nothing. Distinct from attributionNotAttempted below, whose // message this used to carry — reporting "the webhook is not configured" for a // webhook that is configured and simply missed sends operators to the wrong place. - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback, "no admission author record was found for this request; it can attach only to a "+ "window with no named actor") case attributionNotAttempted: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled, "command-author capture is disabled (the validate-operator-types webhook is not "+ "configured); the request can attach only to a window with no named actor") @@ -267,25 +267,25 @@ func setCommitRequestAttributed(cr *configv1alpha3.CommitRequest, attribution co } // applyFinalizeResultToStatus maps a FinalizeResult (or a finalize error) and the -// settled attribution onto a CommitRequest's terminal conditions. Ready is the +// settled attribution onto a SaveRequest's terminal conditions. Ready is the // summary; Reconciling/Stalled carry kstatus; Attributed records the author // decision; Pushed records whether a commit reached the remote. func applyFinalizeResultToStatus( - cr *configv1alpha3.CommitRequest, + cr *configv1alpha3.SaveRequest, result git.FinalizeResult, finalizeErr error, - attribution commitRequestAttribution, + attribution saveRequestAttribution, ) { cr.Status.ObservedGeneration = cr.Generation cr.Status.Branch = result.Branch - setCommitRequestAttributed(cr, attribution) + setSaveRequestAttributed(cr, attribution) if finalizeErr != nil { reason := crReasonFinalizeFailed if errors.Is(finalizeErr, errTakeOverNotAllowed) { reason = crReasonTakeOverNotAllowed } - failCommitRequest(cr, reason, finalizeErr.Error()) + failSaveRequest(cr, reason, finalizeErr.Error()) return } @@ -293,25 +293,25 @@ func applyFinalizeResultToStatus( case git.FinalizeCommitted: cr.Status.Commit = result.Commit const committedMsg = "the open commit window was closed, committed, and pushed" - setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, crReasonCommitted, committedMsg) - setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, crReasonCommitted, notStalledMessage) - setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, + setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, crReasonCommitted, committedMsg) + setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, crReasonCommitted, notStalledMessage) + setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, "the commit was pushed to the remote repository") - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted, committedMsg) + setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted, committedMsg) case git.FinalizeNoOpenWindow: // Benign: the attach timeout ran out with nothing pending to save. resolveNothingToCommit(cr, result, crReasonNoWindow, noWindowMessage) case git.FinalizeWindowMismatch: // The author-bound refusal: deliberately not a failure — the foreign // window stays open for its own author — but the reason is surfaced. - rejectCommitRequest(cr, crReasonWindowMismatch, windowMismatchMessage) + rejectSaveRequest(cr, crReasonWindowMismatch, windowMismatchMessage) case git.FinalizeAlreadyPresent: // The change already matches the remote, so the commit was dropped. resolveNothingToCommit(cr, result, crReasonAlreadyPresent, alreadyPresentMessage) default: // An empty or unknown outcome with no error is a bug, not a benign // rejection; fail it so it is not silently hidden. - failCommitRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) + failSaveRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) } } @@ -322,58 +322,58 @@ const recordedEmptyMessage = "; the message was recorded in an empty commit" // resolveNothingToCommit records an outcome with nothing to commit. Without a commit it is a benign // rejection. With one, whenNothingToCommit: CommitEmpty recorded the message: the reason keeps the // cause, and status.commit and Pushed=True say the empty commit reached the remote. -func resolveNothingToCommit(cr *configv1alpha3.CommitRequest, result git.FinalizeResult, reason, message string) { +func resolveNothingToCommit(cr *configv1alpha3.SaveRequest, result git.FinalizeResult, reason, message string) { if result.Commit == "" { - rejectCommitRequest(cr, reason, message) + rejectSaveRequest(cr, reason, message) return } cr.Status.Commit = result.Commit message += recordedEmptyMessage - setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, + setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, "the empty commit was pushed to the remote repository") - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) + setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) } -// rejectCommitRequest records a benign terminal outcome that produced no commit: +// rejectSaveRequest records a benign terminal outcome that produced no commit: // Ready=True (the request was serviced correctly, with the specific reason), // Pushed=False, and Stalled=False — kstatus Current, not Failed. -func rejectCommitRequest(cr *configv1alpha3.CommitRequest, reason, message string) { - setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, message) - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) +func rejectSaveRequest(cr *configv1alpha3.SaveRequest, reason, message string) { + setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, message) + setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) } -// failCommitRequest records a hard terminal failure: Ready=False, Pushed=False, and +// failSaveRequest records a hard terminal failure: Ready=False, Pushed=False, and // Stalled=True — kstatus Failed, a human-fixable block. -func failCommitRequest(cr *configv1alpha3.CommitRequest, reason, message string) { - setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, "no commit was pushed") - setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionTrue, reason, message) - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) +func failSaveRequest(cr *configv1alpha3.SaveRequest, reason, message string) { + setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, "no commit was pushed") + setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionTrue, reason, message) + setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) } -// commitRequestIsTerminal reports whether the request has reached a terminal +// saveRequestIsTerminal reports whether the request has reached a terminal // outcome: Ready=True (committed or benignly rejected) or Stalled=True (failed). -func commitRequestIsTerminal(cr *configv1alpha3.CommitRequest) bool { +func saveRequestIsTerminal(cr *configv1alpha3.SaveRequest) bool { return conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) || conditionIsTrue(cr.Status.Conditions, ConditionTypeStalled) } -// commitRequestConditionStatus returns the status of one condition as a loggable +// saveRequestConditionStatus returns the status of one condition as a loggable // string, or "" when the condition is absent. -func commitRequestConditionStatus(cr *configv1alpha3.CommitRequest, conditionType string) string { +func saveRequestConditionStatus(cr *configv1alpha3.SaveRequest, conditionType string) string { if c := apimeta.FindStatusCondition(cr.Status.Conditions, conditionType); c != nil { return string(c.Status) } return "" } -// commitRequestReadyReason returns the Ready condition's reason and message for +// saveRequestReadyReason returns the Ready condition's reason and message for // logging (both empty when the condition is absent). -func commitRequestReadyReason(cr *configv1alpha3.CommitRequest) (string, string) { +func saveRequestReadyReason(cr *configv1alpha3.SaveRequest) (string, string) { if c := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady); c != nil { return c.Reason, c.Message } diff --git a/internal/controller/commitrequest_kstatus_test.go b/internal/controller/saverequest_kstatus_test.go similarity index 92% rename from internal/controller/commitrequest_kstatus_test.go rename to internal/controller/saverequest_kstatus_test.go index c84221dc..8ab71e84 100644 --- a/internal/controller/commitrequest_kstatus_test.go +++ b/internal/controller/saverequest_kstatus_test.go @@ -13,10 +13,10 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -// TestCommitRequestKstatusContract pins the CommitRequest condition set to the +// TestSaveRequestKstatusContract pins the SaveRequest condition set to the // kstatus the GitOps tooling derives from it: Reconciling=True → InProgress, // Stalled=True → Failed, otherwise Current (with observedGeneration current). -func TestCommitRequestKstatusContract(t *testing.T) { +func TestSaveRequestKstatusContract(t *testing.T) { tests := []struct { name string conds []map[string]interface{} @@ -96,7 +96,7 @@ func TestCommitRequestKstatusContract(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - result, err := kstatus.Compute(commitRequestStatusObject(tt.conds)) + result, err := kstatus.Compute(saveRequestStatusObject(tt.conds)) require.NoError(t, err) assert.Equal(t, tt.wantStatus, result.Status) if tt.wantMsg != "" { @@ -106,10 +106,10 @@ func TestCommitRequestKstatusContract(t *testing.T) { } } -func commitRequestStatusObject(conditions []map[string]interface{}) *unstructured.Unstructured { +func saveRequestStatusObject(conditions []map[string]interface{}) *unstructured.Unstructured { return &unstructured.Unstructured{Object: map[string]interface{}{ "apiVersion": "configbutler.ai/v1alpha3", - "kind": "CommitRequest", + "kind": "SaveRequest", "metadata": map[string]interface{}{ "name": "save-a", "namespace": "default", diff --git a/internal/controller/commitrequest_outcome_metric_test.go b/internal/controller/saverequest_outcome_metric_test.go similarity index 74% rename from internal/controller/commitrequest_outcome_metric_test.go rename to internal/controller/saverequest_outcome_metric_test.go index 7e8b7de7..2b68623d 100644 --- a/internal/controller/commitrequest_outcome_metric_test.go +++ b/internal/controller/saverequest_outcome_metric_test.go @@ -29,12 +29,12 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/watch" ) -const commitRequestsMetric = "gitopsreverser_commit_requests_total" +const saveRequestsMetric = "gitopsreverser_save_requests_total" // outcomeCount reads one bounded outcome's counter value, and reports whether the series exists. func outcomeCount(t *testing.T, reader *sdkmetric.ManualReader, outcome string) (int64, bool) { t.Helper() - return telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{"outcome": outcome}) + return telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{"outcome": outcome}) } // outcomeCountForTarget reads one outcome's counter for one GitTarget identity. The empty pair is @@ -45,7 +45,7 @@ func outcomeCountForTarget( outcome, namespace, name string, ) (int64, bool) { t.Helper() - return telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{ + return telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{ "outcome": outcome, "gittarget_namespace": namespace, "gittarget_name": name, @@ -54,20 +54,20 @@ func outcomeCountForTarget( // The reason the label was added: two targets' saves must be tellable apart, so an operator can // alert on one tenant's target rather than on the fleet. -func TestCommitRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { +func TestSaveRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) for _, target := range []string{"team-a-config", "team-b-config"} { - cr := newCommitRequest("save-" + target) + cr := newSaveRequest("save-" + target) cr.Spec.GitTargetRef = meta.LocalObjectReference{Name: target} - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-"+target) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileSaveRequest(t, r, "save-"+target) } for _, target := range []string{"team-a-config", "team-b-config"} { @@ -79,19 +79,19 @@ func TestCommitRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { // Cardinality is bounded by configuration rather than by saves: repeated saves against one target // reuse one series. That is the property that makes the label affordable. -func TestCommitRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { +func TestSaveRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) for i := range 3 { name := fmt.Sprintf("save-repeat-%d", i) - c := newCommitRequestClient(t, nil, newCommitRequest(name)) + c := newSaveRequestClient(t, nil, newSaveRequest(name)) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, name) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileSaveRequest(t, r, name) } value, ok := outcomeCountForTarget(t, reader, crOutcomeCommitted, "default", "team-a-config") @@ -101,33 +101,33 @@ func TestCommitRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { // The bound on the label. spec.gitTargetRef is whatever the client wrote, so a request naming a // target that does not exist must NOT mint a series under that name: anyone with create rights on -// commitrequests could otherwise grow the metric without limit. The requested reference stays on +// saverequests could otherwise grow the metric without limit. The requested reference stays on // the conditions and in the logs, which is where an unbounded string belongs. -func TestCommitRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { +func TestSaveRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - cr := newCommitRequest("save-bogus") + cr := newSaveRequest("save-bogus") cr.Spec.GitTargetRef = meta.LocalObjectReference{Name: "no-such-target"} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(cr.Spec).resolveTimeout())) - c := newCommitRequestClient(t, nil, cr) - // A service error is what EventRouter.ServiceCommitRequest returns when the GitTarget Get fails, + c := newSaveRequestClient(t, nil, cr) + // A service error is what EventRouter.ServiceSaveRequest returns when the GitTarget Get fails, // and a missing GitTarget has no worker to hold the request, so the withdraw resolves at once. f := &fakeFinalizer{ err: errors.New("get GitTarget default/no-such-target: not found"), - withdrawResult: git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, + withdrawResult: git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-bogus") + reconcileSaveRequest(t, r, "save-bogus") value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "", "") require.True(t, ok, "an unresolved target must still be counted, under the fallback identity") assert.Equal(t, int64(1), value) - _, minted := telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{ + _, minted := telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{ "gittarget_name": "no-such-target", }) assert.False(t, minted, "a name that resolved to nothing must never become a label value") @@ -136,7 +136,7 @@ func TestCommitRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { // The mapping is derived from FinalizeResult rather than from the condition reasons, so pin it // directly. The default arm matters most: an unmapped outcome must count as failed, because a bug // that increments nothing is a bug nobody sees. -func TestCommitRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { +func TestSaveRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { tests := []struct { name string result git.FinalizeResult @@ -159,14 +159,14 @@ func TestCommitRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - assert.Equal(t, tc.want, commitRequestOutcome(tc.result, tc.err)) + assert.Equal(t, tc.want, saveRequestOutcome(tc.result, tc.err)) }) } } // Every terminal outcome must reach the counter through a real reconcile, not only through the // mapping function. A value with no raise site is a label nobody can alert on. -func TestCommitRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) { +func TestSaveRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) { tests := []struct { name string result git.FinalizeResult @@ -199,11 +199,11 @@ func TestCommitRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing. reader, err := telemetry.InitTestExporter() require.NoError(t, err) - c := newCommitRequestClient(t, nil, newCommitRequest("save-outcome")) + c := newSaveRequestClient(t, nil, newSaveRequest("save-outcome")) f := &fakeFinalizer{result: tc.result, err: tc.err, resolved: true} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-outcome") + reconcileSaveRequest(t, r, "save-outcome") value, ok := outcomeCount(t, reader, tc.want) require.True(t, ok, "outcome %q published no series", tc.want) @@ -215,7 +215,7 @@ func TestCommitRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing. // The trap the recording site exists to avoid. applyFinalizeResultToStatus is called once per // attempt inside writeTerminalStatus's conflict-retry loop, so recording there would over-report // exactly the contended request. Two conflicts, one increment. -func TestCommitRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { +func TestSaveRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -231,20 +231,20 @@ func TestCommitRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { if conflicts > 0 { conflicts-- return apierrors.NewConflict( - schema.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}, + schema.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}, obj.GetName(), errors.New("simulated")) } return c.SubResource(subResource).Update(ctx, obj, opts...) }, } - c := newCommitRequestClient(t, &fns, withInProgress(newCommitRequest("save-conflict"))) + c := newSaveRequestClient(t, &fns, withInProgress(newSaveRequest("save-conflict"))) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-conflict") + reconcileSaveRequest(t, r, "save-conflict") value, ok := outcomeCount(t, reader, crOutcomeCommitted) require.True(t, ok) @@ -254,8 +254,8 @@ func TestCommitRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { // The documented limit, asserted as behavior rather than left to be discovered. A status write that // fails permanently is not retried into a terminal state, so the next reconcile re-decides and // increments again. This is why the instrument promises one increment per terminal DECISION -// ATTEMPT and not one per CommitRequest; deduplicating would need a durable marker on the object. -func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing.T) { +// ATTEMPT and not one per SaveRequest; deduplicating would need a durable marker on the object. +func TestSaveRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -270,15 +270,15 @@ func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testi return apierrors.NewInternalError(errors.New("status write is down")) }, } - c := newCommitRequestClient(t, &fns, withInProgress(newCommitRequest("save-lost"))) + c := newSaveRequestClient(t, &fns, withInProgress(newSaveRequest("save-lost"))) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-lost") - reconcileCommitRequest(t, r, "save-lost") + reconcileSaveRequest(t, r, "save-lost") + reconcileSaveRequest(t, r, "save-lost") value, ok := outcomeCount(t, reader, crOutcomeCommitted) require.True(t, ok) @@ -289,24 +289,24 @@ func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testi // refusePrunedGitTargetRef reaches a terminal state without passing through writeTerminalStatus. // It is rare and migration-only, which is exactly the kind of terminal state that must not be // missing from the counter. -func TestCommitRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { +func TestSaveRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - pruned := &configv1alpha3.CommitRequest{ + pruned := &configv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "save-pruned", Namespace: "default", UID: types.UID("uid-save-pruned")}, - Spec: configv1alpha3.CommitRequestSpec{ + Spec: configv1alpha3.SaveRequestSpec{ GitTargetRef: meta.LocalObjectReference{Name: ""}, Message: "save: pruned", }, } - c := newCommitRequestClient(t, nil, pruned) + c := newSaveRequestClient(t, nil, pruned) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-pruned") + reconcileSaveRequest(t, r, "save-pruned") - got := fetchCommitRequest(t, c, "save-pruned") + got := fetchSaveRequest(t, c, "save-pruned") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonGitTargetRefPruned) value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "", "") @@ -315,41 +315,41 @@ func TestCommitRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { require.Empty(t, f.calls, "a request naming no GitTarget must never reach the finalizer") } -// A recreated CommitRequest is a new incarnation of the same save, and each one is its own +// A recreated SaveRequest is a new incarnation of the same save, and each one is its own // decision. Counting per incarnation is what makes a rate over the counter meaningful. -func TestCommitRequestMetric_RecreatedRequestCountsPerIncarnation(t *testing.T) { +func TestSaveRequestMetric_RecreatedRequestCountsPerIncarnation(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - c := newCommitRequestClient(t, nil, newCommitRequest("save-again")) + c := newSaveRequestClient(t, nil, newSaveRequest("save-again")) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileCommitRequest(t, r, "save-again") + reconcileSaveRequest(t, r, "save-again") // Delete and recreate under a fresh UID, as a second save from the same editor would. - first := fetchCommitRequest(t, c, "save-again") + first := fetchSaveRequest(t, c, "save-again") require.NoError(t, c.Delete(context.Background(), &first)) - recreated := newCommitRequest("save-again") + recreated := newSaveRequest("save-again") recreated.UID = types.UID("uid-save-again-2") require.NoError(t, c.Create(context.Background(), recreated)) - reconcileCommitRequest(t, r, "save-again") + reconcileSaveRequest(t, r, "save-again") value, ok := outcomeCount(t, reader, crOutcomeWindowMismatch) require.True(t, ok) assert.Equal(t, int64(2), value) } -// TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels runs the real EventRouter against a +// TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels runs the real EventRouter against a // GitTarget that exists but whose branch worker never starts. The request waits in // WaitingForWorker, then fails closed at the safety bound, and that failure must be counted under // its GitTarget: the target DID resolve, so the empty fallback pair would hide a real tenant's // failed save from every per-target dashboard and alert. -func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { +func TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -360,28 +360,28 @@ func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } - fresh := newCommitRequest("save-fresh") + fresh := newSaveRequest("save-fresh") fresh.Spec.GitTargetRef = meta.LocalObjectReference{Name: target.Name} fresh.CreationTimestamp = metav1.Now() - stale := newCommitRequest("save-stale") + stale := newSaveRequest("save-stale") stale.Spec.GitTargetRef = meta.LocalObjectReference{Name: target.Name} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. stale.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(stale.Spec).resolveTimeout())) - c := newCommitRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) + c := newSaveRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, itypes.SensitiveResourcePolicy{}) router := watch.NewEventRouter(workers, nil, c, logr.Discard()) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} // Inside the safety window: pending, and the status says why. - res := reconcileCommitRequest(t, r, fresh.Name) + res := reconcileSaveRequest(t, r, fresh.Name) assert.Positive(t, res.RequeueAfter, "a missing worker is polled, not resolved") reconciling := apimeta.FindStatusCondition( - fetchCommitRequest(t, c, fresh.Name).Status.Conditions, ConditionTypeReconciling) + fetchSaveRequest(t, c, fresh.Name).Status.Conditions, ConditionTypeReconciling) require.NotNil(t, reconciling) assert.Equal(t, string(git.PhaseWaitingForWorker), reconciling.Reason) // Past it: failed, under the GitTarget that resolved. - reconcileCommitRequest(t, r, stale.Name) + reconcileSaveRequest(t, r, stale.Name) value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "default", target.Name) require.True(t, ok, "the failure is counted under the GitTarget it resolved to") diff --git a/internal/controller/commitrequest_ttl_test.go b/internal/controller/saverequest_ttl_test.go similarity index 51% rename from internal/controller/commitrequest_ttl_test.go rename to internal/controller/saverequest_ttl_test.go index 5f84ba02..b06d139b 100644 --- a/internal/controller/commitrequest_ttl_test.go +++ b/internal/controller/saverequest_ttl_test.go @@ -19,15 +19,15 @@ import ( ) // finishedWithDeleteAfter is a Committed request carrying the given delete-after value. -func finishedWithDeleteAfter(name, deleteAfter string) *configv1alpha3.CommitRequest { - cr := withReadyCommitted(newCommitRequest(name)) - cr.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: deleteAfter} +func finishedWithDeleteAfter(name, deleteAfter string) *configv1alpha3.SaveRequest { + cr := withReadyCommitted(newSaveRequest(name)) + cr.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: deleteAfter} return cr } -func commitRequestExists(t *testing.T, c client.Client, name string) bool { +func saveRequestExists(t *testing.T, c client.Client, name string) bool { t.Helper() - var cr configv1alpha3.CommitRequest + var cr configv1alpha3.SaveRequest err := c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: name}, &cr) if apierrors.IsNotFound(err) { return false @@ -45,146 +45,146 @@ func committedFinalizer() *fakeFinalizer { // Finishing is what writes the deletion time, so it is on the object for anyone to read or change // from the moment the outcome is. -func TestCommitRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { - c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) - r := &CommitRequestReconciler{ +func TestSaveRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { + c := newSaveRequestClient(t, nil, newSaveRequest("save-now")) + r := &SaveRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: 48 * time.Hour, } - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") - got := fetchCommitRequest(t, c, "save-now") - value, ok := got.Annotations[CommitRequestDeleteAfterAnnotation] + got := fetchSaveRequest(t, c, "save-now") + value, ok := got.Annotations[SaveRequestDeleteAfterAnnotation] require.True(t, ok, "a finished request carries its deletion time") deleteAfter, err := time.Parse(time.RFC3339, value) require.NoError(t, err) assert.WithinDuration(t, time.Now().Add(48*time.Hour), deleteAfter, time.Minute) } -func TestCommitRequestTTL_ZeroStampsNothing(t *testing.T) { - c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) - r := &CommitRequestReconciler{ +func TestSaveRequestTTL_ZeroStampsNothing(t *testing.T) { + c := newSaveRequestClient(t, nil, newSaveRequest("save-now")) + r := &SaveRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), } - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") - assert.NotContains(t, fetchCommitRequest(t, c, "save-now").Annotations, CommitRequestDeleteAfterAnnotation) + assert.NotContains(t, fetchSaveRequest(t, c, "save-now").Annotations, SaveRequestDeleteAfterAnnotation) } // A request that finished before the annotation existed, or one someone removed it from, is kept: // only the transition to terminal stamps, never a later look at a terminal request. -func TestCommitRequestTTL_AFinishedRequestWithoutTheAnnotationIsKept(t *testing.T) { - c := newCommitRequestClient(t, nil, withReadyCommitted(newCommitRequest("save-before"))) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}, TTL: time.Nanosecond} +func TestSaveRequestTTL_AFinishedRequestWithoutTheAnnotationIsKept(t *testing.T) { + c := newSaveRequestClient(t, nil, withReadyCommitted(newSaveRequest("save-before"))) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}, TTL: time.Nanosecond} - res := reconcileCommitRequest(t, r, "save-before") + res := reconcileSaveRequest(t, r, "save-before") assert.Zero(t, res.RequeueAfter) - got := fetchCommitRequest(t, c, "save-before") - assert.NotContains(t, got.Annotations, CommitRequestDeleteAfterAnnotation, "it is not stamped after the fact") + got := fetchSaveRequest(t, c, "save-before") + assert.NotContains(t, got.Annotations, SaveRequestDeleteAfterAnnotation, "it is not stamped after the fact") } -func TestCommitRequestTTL_DeletesOnceTheTimeHasPassed(t *testing.T) { +func TestSaveRequestTTL_DeletesOnceTheTimeHasPassed(t *testing.T) { past := time.Now().Add(-time.Minute).UTC().Format(time.RFC3339) - c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-old", past)) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} + c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-old", past)) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileCommitRequest(t, r, "save-old") + res := reconcileSaveRequest(t, r, "save-old") assert.Zero(t, res.RequeueAfter) - assert.False(t, commitRequestExists(t, c, "save-old"), + assert.False(t, saveRequestExists(t, c, "save-old"), "the annotation alone decides; a TTL of zero now does not reach back to it") } // The request comes back when its time is up rather than on a poll, so a namespace full of recent // saves costs one requeue each and nothing in between. -func TestCommitRequestTTL_ComesBackWhenTheTimeIsUp(t *testing.T) { +func TestSaveRequestTTL_ComesBackWhenTheTimeIsUp(t *testing.T) { future := time.Now().Add(47 * time.Hour).UTC().Format(time.RFC3339) - c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-recent", future)) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} + c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-recent", future)) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileCommitRequest(t, r, "save-recent") + res := reconcileSaveRequest(t, r, "save-recent") - assert.True(t, commitRequestExists(t, c, "save-recent")) + assert.True(t, saveRequestExists(t, c, "save-recent")) assert.InDelta(t, (47 * time.Hour).Seconds(), res.RequeueAfter.Seconds(), 60) } -func TestCommitRequestTTL_AnUnreadableTimeKeepsTheRequest(t *testing.T) { - c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-typo", "tomorrow")) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} +func TestSaveRequestTTL_AnUnreadableTimeKeepsTheRequest(t *testing.T) { + c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-typo", "tomorrow")) + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileCommitRequest(t, r, "save-typo") + res := reconcileSaveRequest(t, r, "save-typo") assert.Zero(t, res.RequeueAfter) - assert.True(t, commitRequestExists(t, c, "save-typo")) + assert.True(t, saveRequestExists(t, c, "save-typo")) } // A request still waiting for its outcome is neither stamped nor deleted; it is polled as before. -func TestCommitRequestTTL_LeavesARequestInProgressAlone(t *testing.T) { - cr := withInProgress(newCommitRequest("save-pending")) +func TestSaveRequestTTL_LeavesARequestInProgressAlone(t *testing.T) { + cr := withInProgress(newSaveRequest("save-pending")) cr.CreationTimestamp = metav1.Now() // inside the resolve window, so it polls rather than fails closed - c := newCommitRequestClient(t, nil, cr) + c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{ + r := &SaveRequestReconciler{ Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice(), TTL: time.Nanosecond, } - res := reconcileCommitRequest(t, r, "save-pending") + res := reconcileSaveRequest(t, r, "save-pending") - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "it keeps polling for the outcome") - assert.NotContains(t, fetchCommitRequest(t, c, "save-pending").Annotations, CommitRequestDeleteAfterAnnotation) + assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, "it keeps polling for the outcome") + assert.NotContains(t, fetchSaveRequest(t, c, "save-pending").Annotations, SaveRequestDeleteAfterAnnotation) } // A person who extends delete-after in the moment between the terminal status write and the stamp // must keep the extension. The stamp is computed from the object the status write returned, so an // unconditional patch would overwrite the edit and delete the request early. -func TestCommitRequestTTL_StampDoesNotOverwriteAConcurrentExtension(t *testing.T) { +func TestSaveRequestTTL_StampDoesNotOverwriteAConcurrentExtension(t *testing.T) { const extended = "2099-01-01T00:00:00Z" - c := newCommitRequestClient(t, &interceptor.Funcs{ + c := newSaveRequestClient(t, &interceptor.Funcs{ SubResourceUpdate: func( ctx context.Context, cl client.Client, sub string, obj client.Object, opts ...client.SubResourceUpdateOption, ) error { if err := cl.SubResource(sub).Update(ctx, obj, opts...); err != nil { return err } - if !commitRequestIsTerminal(obj.(*configv1alpha3.CommitRequest)) { + if !saveRequestIsTerminal(obj.(*configv1alpha3.SaveRequest)) { return nil } // The edit lands right after the terminal status is published, before the stamp. - var live configv1alpha3.CommitRequest + var live configv1alpha3.SaveRequest require.NoError(t, cl.Get(ctx, client.ObjectKeyFromObject(obj), &live)) - live.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: extended} + live.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: extended} require.NoError(t, cl.Update(ctx, &live)) return nil }, - }, newCommitRequest("save-now")) - r := &CommitRequestReconciler{ + }, newSaveRequest("save-now")) + r := &SaveRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: time.Hour, } - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") assert.Equal(t, extended, - fetchCommitRequest(t, c, "save-now").Annotations[CommitRequestDeleteAfterAnnotation], + fetchSaveRequest(t, c, "save-now").Annotations[SaveRequestDeleteAfterAnnotation], "the extension made after the request finished survives the stamp") } // A submitter may choose its own deletion time up front; finishing does not replace it. -func TestCommitRequestTTL_AValueSetAtCreationIsKept(t *testing.T) { +func TestSaveRequestTTL_AValueSetAtCreationIsKept(t *testing.T) { const chosen = "2099-01-01T00:00:00Z" - cr := newCommitRequest("save-now") - cr.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: chosen} - c := newCommitRequestClient(t, nil, cr) - r := &CommitRequestReconciler{ + cr := newSaveRequest("save-now") + cr.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: chosen} + c := newSaveRequestClient(t, nil, cr) + r := &SaveRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: time.Hour, } - reconcileCommitRequest(t, r, "save-now") + reconcileSaveRequest(t, r, "save-now") assert.Equal(t, chosen, - fetchCommitRequest(t, c, "save-now").Annotations[CommitRequestDeleteAfterAnnotation]) + fetchSaveRequest(t, c, "save-now").Annotations[SaveRequestDeleteAfterAnnotation]) } diff --git a/internal/controller/status.go b/internal/controller/status.go index 32f5f678..fb4426f2 100644 --- a/internal/controller/status.go +++ b/internal/controller/status.go @@ -104,7 +104,7 @@ func (s *reconcileStatus) writeLost() bool { return s.writeLostToRace } // commit() drops the write, and the GenerationChangedPredicate on their For() filters the // status-only update that won, so nothing re-enqueues the object — it keeps the older answer for // its full steady interval. Fixing that at one call site would have left the same bug in the other -// four. CommitRequestReconciler carries neither half; see its Reconcile for why. +// four. SaveRequestReconciler carries neither half; see its Reconcile for why. func (s *reconcileStatus) requeueAfter(cadence time.Duration) time.Duration { if s.writeLostToRace { return RequeueWriteLostInterval @@ -123,7 +123,7 @@ func (s *reconcileStatus) requeueAfter(cadence time.Duration) time.Duration { // handing it one object's conditions and another object's metadata. // // It carries no kind name, on purpose. conditionMetricKind resolves that from its own list, -// because CommitRequest is excluded from the condition metric and an interface anything could +// because SaveRequest is excluded from the condition metric and an interface anything could // implement is not a place to keep an exclusion. See resource_condition_metrics.go. type statusObject interface { client.Object diff --git a/internal/controller/suite_test.go b/internal/controller/suite_test.go index 5e740d58..ac0717f4 100644 --- a/internal/controller/suite_test.go +++ b/internal/controller/suite_test.go @@ -186,7 +186,7 @@ var _ = BeforeSuite(func() { // committer immediately, and a Finalizer that never resolves keeps requests in the // Progressing state. Neither ever completes, so these specs cover only the // initial in-progress stamp and the terminal short-circuit. - err = (&CommitRequestReconciler{ + err = (&SaveRequestReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), Finalizer: &fakeFinalizer{resolved: false}, diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index d956e233..80f97894 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -251,9 +251,9 @@ type applierState struct { // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, // so its attempt yields to an edit (convergeHolds). lag bool - // waitingSaves is the CommitRequests waiting for this applier's next apply that succeeds, each + // waitingSaves is the SaveRequests waiting for this applier's next apply that succeeds, each // with the commit it pushed (save_apply.go); the attempt sent next carries them. - waitingSaves map[commitRequestID]string + waitingSaves map[saveRequestID]string // appliedRevision is the revision the last operation that succeeded applied, as the applier // reported it, recovered on restart from the applier claim's status.operation. A converge of the commit // it names asks nothing. @@ -326,9 +326,9 @@ type inflightAttempt struct { refusals map[refusalKey]unsentRefusal // lag is whether it carries only scheduleConverges' converge. lag bool - // saves is the CommitRequests whose apply it carries, each with the commit it pushed + // saves is the SaveRequests whose apply it carries, each with the commit it pushed // (save_apply.go). - saves map[commitRequestID]string + saves map[saveRequestID]string // phase is the last report of its operation, empty before the first. phase ApplyOutcome // previous is the operation recorded before this attempt's, with the attempt it belonged to, @@ -357,7 +357,7 @@ func (s *applierState) hasReason(refusals map[refusalKey]unsentRefusal) bool { const barrierRecheckInterval = time.Second // writeBusy reports whether the loop is in a state in which no flush can be made, and so no request -// sent: a registered CommitRequest, whose save defines its own commit boundary (a save is announced +// sent: a registered SaveRequest, whose save defines its own commit boundary (a save is announced // before its writes arrive, and an apply landing between the two would revert its first edits before // they reach Git), a parent recovery, or a checkout a failed write left dirty. In the last two the // worker cannot say what is published. @@ -367,7 +367,7 @@ func (l *branchWorkerEventLoop) writeBusy() bool { // closeWindowForPushHint closes the open commit window early when a Git host has said the branch may // have moved, so its writes are published, and replayed onto the move, now rather than when the -// window's timers allow. A window a CommitRequest holds is left to its save. +// window's timers allow. A window a SaveRequest holds is left to its save. func (l *branchWorkerEventLoop) closeWindowForPushHint() { hinted := l.pushHinted l.pushHinted = false diff --git a/internal/git/applier_claim_test.go b/internal/git/applier_claim_test.go index b4a986ee..8d657f79 100644 --- a/internal/git/applier_claim_test.go +++ b/internal/git/applier_claim_test.go @@ -150,7 +150,7 @@ func applierClaimLoop(t *testing.T, targets ...client.Object) (*branchWorkerEven // holdApplies puts the loop in a state in which no flush can be made, so no apply is sent: a save is // registered, and it defines its own commit boundary (writeBusy). releaseApplies ends it. func holdApplies(l *branchWorkerEventLoop) { - l.pendingCRs = map[commitRequestID]*pendingCommitRequest{{Namespace: "team-a", Name: "save"}: {}} + l.pendingCRs = map[saveRequestID]*pendingSaveRequest{{Namespace: "team-a", Name: "save"}: {}} } func releaseApplies(l *branchWorkerEventLoop) { l.pendingCRs = nil } diff --git a/internal/git/apply_attempt_test.go b/internal/git/apply_attempt_test.go index 4a6059ac..7a6d4318 100644 --- a/internal/git/apply_attempt_test.go +++ b/internal/git/apply_attempt_test.go @@ -349,12 +349,12 @@ func TestPushNotice_TheTipAlreadySeenDoesNotCutAWindow(t *testing.T) { } } -// TestWriteBusy_ASaveAnnouncedBeforeItsWrites: a registered CommitRequest makes the branch busy, so +// TestWriteBusy_ASaveAnnouncedBeforeItsWrites: a registered SaveRequest makes the branch busy, so // a catch-up cannot land between a save's announcement and its first writes. func TestWriteBusy_ASaveAnnouncedBeforeItsWrites(t *testing.T) { loop, _ := applierClaimLoop(t, applierTestTarget("alpha", true)) assert.False(t, loop.writeBusy()) - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{{}: {}} + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{{}: {}} assert.True(t, loop.writeBusy()) } diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index edc3973c..07771d4e 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -198,20 +198,20 @@ func TestCatchUpAfter_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { assert.Equal(t, "bbb", requests[0].Commit) } -// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a CommitRequest with catchUp: AfterPush +// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a SaveRequest with catchUp: AfterPush // leaves a converge pending for its GitTarget's applier once its commit is pushed, without waiting // for catchUpAfter; one that pushed no commit asks nothing. It does not yield to an edit: it was // asked for. func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} save := func(name string, commit string) { - id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} - loop.pendingCRs[id] = &pendingCommitRequest{ + id := saveRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingSaveRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) } save("nothing-pushed", "") @@ -237,16 +237,16 @@ func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true), applierTestTarget("loner", false)) convergedAt(t, loop, requester) - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} - save := func(name, target, commit string) commitRequestID { - id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} - loop.pendingCRs[id] = &pendingCommitRequest{ + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + save := func(name, target, commit string) saveRequestID { + id := saveRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingSaveRequest{ id: id, gitTargetName: target, gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) return id } - lookup := func(id commitRequestID) SaveApply { + lookup := func(id saveRequestID) SaveApply { apply, known := loop.w.LookupSaveApply(id.Namespace, id.Name, id.UID) require.True(t, known, "the worker knows %s", id.Name) return apply @@ -289,15 +289,15 @@ func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { func TestSaveApply_AnApplierAlreadyAtTheTipSettlesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} - id := commitRequestID{Namespace: "team-a", Name: "late", UID: "uid-late"} - loop.pendingCRs[id] = &pendingCommitRequest{ + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + id := saveRequestID{Namespace: "team-a", Name: "late", UID: "uid-late"} + loop.pendingCRs[id] = &pendingSaveRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "aaa"}) + loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "aaa"}) loop.serviceAppliers() assert.Empty(t, requester.taken()) @@ -332,12 +332,12 @@ func TestCatchUpAfter_TurningItOffCancelsAPendingConverge(t *testing.T) { assert.Empty(t, requester.taken(), "off: the timed converge is gone") assert.False(t, state.converge) - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} - id := commitRequestID{Namespace: "team-a", Name: "asks", UID: "uid-asks"} - loop.pendingCRs[id] = &pendingCommitRequest{ + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + id := saveRequestID{Namespace: "team-a", Name: "asks", UID: "uid-asks"} + loop.pendingCRs[id] = &pendingSaveRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "bbb"}) + loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "bbb"}) loop.serviceAppliers() asked := requester.taken() require.Len(t, asked, 1, "a save that asks is no timed converge") @@ -403,12 +403,12 @@ func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { startup[0].Report(ApplyResult{ID: startup[0].ID, Outcome: ApplySucceeded, AppliedRevision: base}) loop.serviceAppliers() - loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} - id := commitRequestID{Namespace: "team-a", Name: "save", UID: "uid-save"} - loop.pendingCRs[id] = &pendingCommitRequest{ + loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + id := saveRequestID{Namespace: "team-a", Name: "save", UID: "uid-save"} + loop.pendingCRs[id] = &pendingSaveRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: saved}) + loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: saved}) tip, by := tc.tip(base, saved, later, rewritten) loop.w.inbox.observe(RemoteObservation{Commit: tip, By: by}) loop.serviceAppliers() @@ -435,6 +435,6 @@ func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { func TestApplierState_ASaveIsNeverOnlyALag(t *testing.T) { state := &applierState{lag: true} assert.True(t, state.onlyLag(nil)) - state.waitSave(commitRequestID{Namespace: "team-a", Name: "save", UID: "uid"}, "ccc") + state.waitSave(saveRequestID{Namespace: "team-a", Name: "save", UID: "uid"}, "ccc") assert.False(t, state.onlyLag(nil)) } diff --git a/internal/git/attribution_outcome_test.go b/internal/git/attribution_outcome_test.go index 53117d3e..d4390bf7 100644 --- a/internal/git/attribution_outcome_test.go +++ b/internal/git/attribution_outcome_test.go @@ -157,30 +157,30 @@ func TestOpenWindow_ResolvedWindowsStillSplitByAuthor(t *testing.T) { // THE REGRESSION THIS DESIGN EXISTS TO AVOID. // -// A CommitRequest that could not be attributed carries an empty author, and so does a window +// A SaveRequest that could not be attributed carries an empty author, and so does a window // whose attribution ran and named nobody — the sentinel is a git-signature rendering only and // never reaches the window's Author. The two therefore still match on the author, and the // outcomes must not be allowed to push them apart: the request's outcome comes from command // authorship and the window's from mirrored-resource attribution, which are configured // independently, so they routinely differ. See the full cross-subsystem matrix in // TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems. -func TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *testing.T) { +func TestPendingSaveRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *testing.T) { window := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) require.Empty(t, window.Author, "an unresolved window carries no author; the sentinel is applied at the write path") - request := &pendingCommitRequest{ - author: "", // an unnamed CommitRequest claims no actor + request := &pendingSaveRequest{ + author: "", // an unnamed SaveRequest claims no actor attribution: AttributionUnresolved, gitTargetName: "target", gitTargetNamespace: "default", } assert.True(t, request.matchesWindow(window), - "an unnamed CommitRequest must attach to the unresolved window it was meant for") + "an unnamed SaveRequest must attach to the unresolved window it was meant for") // The default deployment: command-author capture is off, so the request says "not // attempted" while the window says "unresolved". Neither names an actor, so they match. - notAttempted := &pendingCommitRequest{ + notAttempted := &pendingSaveRequest{ author: "", attribution: AttributionNotAttempted, gitTargetName: "target", @@ -191,10 +191,10 @@ func TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *tes "requiring the two subsystems' outcomes to be equal drops the user's commit message") } -func TestPendingCommitRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { +func TestPendingSaveRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { resolvedWindow := newOpenWindow(attributedEvent("jane@acme.com", AttributionResolved), nil) - unresolvedRequest := &pendingCommitRequest{ + unresolvedRequest := &pendingSaveRequest{ author: "", attribution: AttributionUnresolved, gitTargetName: "target", @@ -203,7 +203,7 @@ func TestPendingCommitRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { assert.False(t, unresolvedRequest.matchesWindow(resolvedWindow), "an unattributed request must not claim a named actor's window") - resolvedRequest := &pendingCommitRequest{ + resolvedRequest := &pendingSaveRequest{ author: "jane@acme.com", attribution: AttributionResolved, gitTargetName: "target", @@ -211,7 +211,7 @@ func TestPendingCommitRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { } assert.True(t, resolvedRequest.matchesWindow(resolvedWindow)) - otherActor := &pendingCommitRequest{ + otherActor := &pendingSaveRequest{ author: "bob@acme.com", attribution: AttributionResolved, gitTargetName: "target", @@ -221,9 +221,9 @@ func TestPendingCommitRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { "a resolved request still matches on the actor") } -func TestPendingCommitRequest_StillScopedToOneGitTarget(t *testing.T) { +func TestPendingSaveRequest_StillScopedToOneGitTarget(t *testing.T) { window := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) - request := &pendingCommitRequest{ + request := &pendingSaveRequest{ author: "", attribution: AttributionUnresolved, gitTargetName: "other-target", @@ -260,11 +260,11 @@ func TestRenderEventCommitMessage_UnresolvedUsernameStaysEmptyInTemplates(t *tes // A request that names its submitter claims a window whose writes name no author only when it was // allowed to, and never a window whose writes name an actor, nor one on another GitTarget. -func TestPendingCommitRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { +func TestPendingSaveRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { unattributed := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) named := newOpenWindow(attributedEvent("bob@acme.com", AttributionResolved), nil) - request := func(claim UnattributedClaim, target string) *pendingCommitRequest { - return &pendingCommitRequest{ + request := func(claim UnattributedClaim, target string) *pendingSaveRequest { + return &pendingSaveRequest{ author: "jane@acme.com", attribution: AttributionResolved, gitTargetName: target, @@ -291,12 +291,12 @@ func TestPendingCommitRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testi } func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { - id := &commitRequestID{Namespace: "team-a", Name: "save-1"} + id := &saveRequestID{Namespace: "team-a", Name: "save-1"} assert.Equal(t, - "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes\n", + "save: edit\n\nAuthor-Source: SaveRequest team-a/save-1; no audit record names who made these changes\n", withTakeOverNote("save: edit\n", id), "the message's trailing newline is kept") assert.Equal(t, - "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes", + "save: edit\n\nAuthor-Source: SaveRequest team-a/save-1; no audit record names who made these changes", withTakeOverNote("save: edit", id)) } diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index f3aa556b..98e53268 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -213,7 +213,7 @@ type BranchWorker struct { // to read it — and still answer "accepted". stoppingState bool // exitedState is set once the event loop has returned, for whatever reason, and the worker - // has settled every CommitRequest it knew. From then on nothing can act on a request, so a + // has settled every SaveRequest it knew. From then on nothing can act on a request, so a // withdraw is answered at once instead of being queued for a loop that is gone. Guarded by // pendingResyncsMu, like stoppingState. exitedState bool @@ -364,27 +364,27 @@ type BranchWorker struct { // Guarded by pendingResyncsMu. See queuedResyncsFor. queuedResyncs map[string]int - // crOutcomes holds resolved CommitRequest outcomes for the controller to poll - // via LookupCommitRequestOutcome. The event loop is the only writer (on its + // crOutcomes holds resolved SaveRequest outcomes for the controller to poll + // via LookupSaveRequestOutcome. The event loop is the only writer (on its // goroutine), the controller the only reader (on a reconcile goroutine), so the // mutex guards the cross-goroutine handoff. Entries are GC'd by age. The - // accessors live in commit_request_attach_loop.go alongside the attach logic. + // accessors live in save_request_attach_loop.go alongside the attach logic. crOutcomesMu sync.Mutex - crOutcomes map[commitRequestID]commitRequestOutcomeEntry - // crPhases holds where each unresolved CommitRequest stands, for the controller to report + crOutcomes map[saveRequestID]saveRequestOutcomeEntry + // crPhases holds where each unresolved SaveRequest stands, for the controller to report // while it waits. Guarded by crOutcomesMu; the event loop is the only writer. - crPhases map[commitRequestID]CommitRequestPhase - // crApplies holds where the apply each CommitRequest asked for stands (save_apply.go). Guarded by + crPhases map[saveRequestID]SaveRequestPhase + // crApplies holds where the apply each SaveRequest asked for stands (save_apply.go). Guarded by // crOutcomesMu; the event loop is the only writer. - crApplies map[commitRequestID]saveApplyEntry + crApplies map[saveRequestID]saveApplyEntry // crQueuedAttaches counts, per request, the attaches on the FIFO that the loop has not handled // yet. An outcome is not collected while any are queued: the outcome is what turns a late attach // into a no-op, so collecting it first would let that attach register a withdrawn request // again, however long the outcome had been kept. Guarded by crOutcomesMu. - crQueuedAttaches map[commitRequestID]int - // crOwners is the manager's table of which worker holds each request. See commitRequestOwners. + crQueuedAttaches map[saveRequestID]int + // crOwners is the manager's table of which worker holds each request. See saveRequestOwners. // Set by the WorkerManager before Start; nil for a worker built on its own. - crOwners *commitRequestOwners + crOwners *saveRequestOwners // clock is the time source of the retry schedule; nil is time.Now. See now(). clock func() time.Time @@ -561,7 +561,7 @@ func (w *BranchWorker) Start(parentCtx context.Context) error { go func() { defer w.wg.Done() w.processEvents() - w.settleCommitRequestsAfterExit() + w.settleSaveRequestsAfterExit() }() return nil @@ -596,10 +596,10 @@ func (w *BranchWorker) Enqueue(event Event) bool { return w.enqueueRequest(&WriteRequest{Events: []Event{event}}) } -// EnqueueWithdraw asks the loop to cancel a CommitRequest it has not acted on. It never blocks: a +// EnqueueWithdraw asks the loop to cancel a SaveRequest it has not acted on. It never blocks: a // full queue or a stopping worker drops it, and the controller sends it again on its next poll. A // worker whose loop has exited answers at once, because nothing is left that could act on it. -func (w *BranchWorker) EnqueueWithdraw(req *AttachCommitRequest) { +func (w *BranchWorker) EnqueueWithdraw(req *AttachSaveRequest) { if req == nil { return } @@ -620,17 +620,17 @@ func (w *BranchWorker) EnqueueWithdraw(req *AttachCommitRequest) { case w.eventQueue <- WorkItem{Withdraw: req}: default: w.releaseInflightItem() - w.Log.V(1).Info("Queue full, CommitRequest withdraw dropped (the controller re-sends)", + w.Log.V(1).Info("Queue full, SaveRequest withdraw dropped (the controller re-sends)", "request", req.Namespace+"/"+req.Name) } } -// EnqueueAttach adds a CommitRequest attach to this worker's queue. Riding the +// EnqueueAttach adds a SaveRequest attach to this worker's queue. Riding the // same queue as resource events is what makes it process in audit order, after // every earlier write. The attach is fire-and-forget: the controller polls the -// outcome via LookupCommitRequestOutcome and re-sends idempotently, so a queue- +// outcome via LookupSaveRequestOutcome and re-sends idempotently, so a queue- // full drop is recovered by the next poll rather than a synchronous reply. -func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { +func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { if req == nil { return } @@ -648,14 +648,14 @@ func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.V(1).Info("Worker is stopping, CommitRequest attach refused (the controller re-sends)") + w.Log.V(1).Info("Worker is stopping, SaveRequest attach refused (the controller re-sends)") return } if !w.admitPayload(charge) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.V(1).Info("CommitRequest attach refused while the branch has paused intake "+ + w.Log.V(1).Info("SaveRequest attach refused while the branch has paused intake "+ "(the controller re-sends)", "request", req.Namespace+"/"+req.Name) return } @@ -666,7 +666,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { w.intake.release(charge) w.pendingResyncsMu.Unlock() w.releaseInflightItem() - w.Log.Info("CommitRequest attach refused: another branch worker holds the request", + w.Log.Info("SaveRequest attach refused: another branch worker holds the request", "request", req.Namespace+"/"+req.Name) return } @@ -675,7 +675,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { select { case w.eventQueue <- WorkItem{Attach: req, charge: charge}: w.pendingResyncsMu.Unlock() - w.Log.Info("CommitRequest attach enqueued", + w.Log.Info("SaveRequest attach enqueued", "request", req.Namespace+"/"+req.Name, "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName, @@ -690,7 +690,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.Error(nil, "Event queue full, CommitRequest attach refused (controller will re-send)") + w.Log.Error(nil, "Event queue full, SaveRequest attach refused (controller will re-send)") } } @@ -1049,15 +1049,15 @@ type branchWorkerEventLoop struct { // See apply_ownership.go. ownershipLogged map[string]struct{} - // pendingCRs holds CommitRequests registered (via AttachCommitRequest) but not + // pendingCRs holds SaveRequests registered (via AttachSaveRequest) but not // yet resolved, keyed by identity. A request is parked here until a same-author // window opens (then it attaches to it) or its attach deadline passes. // Loop-goroutine only. - pendingCRs map[commitRequestID]*pendingCommitRequest - // savedCatchUps is the CommitRequests, by GitTarget, whose pushed commit asked their applier to + pendingCRs map[saveRequestID]*pendingSaveRequest + // savedCatchUps is the SaveRequests, by GitTarget, whose pushed commit asked their applier to // catch up at once (catchUp: AfterPush), with that commit, until the appliers are next listed // (takeReasons, save_apply.go). - savedCatchUps map[itypes.ResourceReference]map[commitRequestID]string + savedCatchUps map[itypes.ResourceReference]map[saveRequestID]string // crSeq numbers registrations, so competing requests attach first come, first served. crSeq uint64 // refusalTimer fires at the earliest deadline in refusalPending. See refusal_reapply.go. @@ -1097,7 +1097,7 @@ type branchWorkerEventLoop struct { flushMark uint64 flushEdits map[itypes.ResourceReference]map[string]time.Time - // attachTimer fires at the earliest attach deadline of a waiting CommitRequest, so a request + // attachTimer fires at the earliest attach deadline of a waiting SaveRequest, so a request // that no window reaches still resolves on time. An attached request needs no timer of its own: // its window's timers close it. attachTimer *time.Timer @@ -1165,7 +1165,7 @@ func (l *branchWorkerEventLoop) run() { case <-attachC: l.attachTimer = nil // The work (attach waiting requests, finalize due ones) is done by - // serviceCommitRequests below. + // serviceSaveRequests below. case <-refusalC: l.refusalTimer = nil l.flushPendingReapply() @@ -1195,7 +1195,7 @@ func (l *branchWorkerEventLoop) handleQueueItem(item WorkItem) { // // - Release the live events held for a GitTarget whose render fidelity is now established, or drop // them on a divergence or past the hold's bounds (serviceHeldWrites). -// - Bind any waiting CommitRequest to an open window, resolve any whose attach deadline has +// - Bind any waiting SaveRequest to an open window, resolve any whose attach deadline has // passed, and re-arm the deadline timer. // - Close the open window early if a Git host reported a push (closeWindowForPushHint). // - Drive the pending writes: commit what the wake decided, schedule its push (advance). @@ -1206,7 +1206,7 @@ func (l *branchWorkerEventLoop) handleQueueItem(item WorkItem) { // - Publish the result, moving the handled item's charge to what the loop holds. func (l *branchWorkerEventLoop) endWake(released int64) { l.serviceHeldWrites() - l.serviceCommitRequests() + l.serviceSaveRequests() l.closeWindowForPushHint() l.advance() l.serviceAppliers() @@ -1259,9 +1259,9 @@ func (l *branchWorkerEventLoop) totalRetainedBytes() int64 { func (l *branchWorkerEventLoop) dispatch(item WorkItem) int64 { switch { case item.Attach != nil: - l.handleAttachCommitRequest(item.Attach) + l.handleAttachSaveRequest(item.Attach) case item.Withdraw != nil: - l.handleWithdrawCommitRequest(item.Withdraw) + l.handleWithdrawSaveRequest(item.Withdraw) case item.ApplyBarrier != nil: l.handleApplyBarrier(item.ApplyBarrier) case item.Resync != nil: @@ -1309,7 +1309,7 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { // diagnosable: the common cause is an incoming event whose author is empty // (an unattributed update, e.g. a /status change that carries no audit fact) // arriving against a non-empty window author, which forces a finalize and - // can split a CommitRequest collect window. windowAuthor vs eventAuthor makes + // can split a SaveRequest collect window. windowAuthor vs eventAuthor makes // that visible at a glance. l.w.Log.Info("Window identity change forces finalize before appending event", "windowAuthor", l.openWindow.Author, @@ -1345,13 +1345,13 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { l.windowBytes += eventRetainedCharge(&event) if opened { - // A waiting CommitRequest gets the new window BEFORE anything can close it: before the + // A waiting SaveRequest gets the new window BEFORE anything can close it: before the // target's timers are applied, and before a buffer-limit flush. With idleTimeout or // maxDuration at 0s, or a write that trips the cap, the window would otherwise close in // this very step, and a waiting request would miss the write it was waiting for — or, // waiting on another author, never see the window that refuses it. l.noteForeignWindow() - l.attachWaitingCommitRequests() + l.attachWaitingSaveRequests() if l.openWindow == nil { return // the attached request's maxDuration: 0s closed it already } @@ -1430,24 +1430,24 @@ func (l *branchWorkerEventLoop) handleShutdown() { // worker exits, even if a push was just sent. The push commits what was just decided. l.pushPending() } - l.failUnpushedCommitRequests() + l.failUnpushedSaveRequests() l.drainUnhandledQueueItems() } -// failUnpushedCommitRequests settles every CommitRequest still riding a write the exiting loop +// failUnpushedSaveRequests settles every SaveRequest still riding a write the exiting loop // could not push. // // A request that reached a local commit is resolved by the push and by nothing else, so a // shutdown whose final push failed would leave it in flight with no loop left to settle it. The // controller would poll until its own safety window expired and then fail closed on a timeout, // which says nothing about what happened. Say what happened instead. -func (l *branchWorkerEventLoop) failUnpushedCommitRequests() { +func (l *branchWorkerEventLoop) failUnpushedSaveRequests() { for i := range l.pendingWrites { - id := l.pendingWrites[i].CommitRequest + id := l.pendingWrites[i].SaveRequest if id == nil { continue } - l.resolveCommitRequest(*id, FinalizeResult{ + l.resolveSaveRequest(*id, FinalizeResult{ Err: errors.New("worker stopped before the commit reached the remote"), }) } @@ -1458,7 +1458,7 @@ func (l *branchWorkerEventLoop) drainUnhandledQueueItems() { l.w.drainQueue() } // drainQueue clears items no loop will handle. Each was counted into inflightItems at enqueue and // decremented only after handling, so without this the final depth would stay non-zero forever: -// the loop has stopped, so nothing republishes a corrected value. A buffered CommitRequest attach +// the loop has stopped, so nothing republishes a corrected value. A buffered SaveRequest attach // is dropped, and the request given back unless the worker knows it otherwise; the controller // re-sends on its next poll, to whichever worker holds it then. func (w *BranchWorker) drainQueue() { @@ -1495,10 +1495,10 @@ func (l *branchWorkerEventLoop) resetCommitTimer(delay time.Duration) { } // finalizeOpenWindowWithReason closes the live event window into one decided write. The attached -// CommitRequest message overrides the live template. +// SaveRequest message overrides the live template. // // The closed window is a decided write (decide): a remote that cannot be reached leaves it pending -// for the publication retry, and its CommitRequest held. Only a failure of the write itself +// for the publication retry, and its SaveRequest held. Only a failure of the write itself // drops it, and resolves that request Failed (settleFailed). func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinalizeReason) { if l.openWindow == nil { @@ -1573,11 +1573,11 @@ func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinali "pendingWrites", len(l.pendingWrites)) } -// carryRequestOnWrite carries the attached CommitRequest onto the write so its result follows the +// carryRequestOnWrite carries the attached SaveRequest onto the write so its result follows the // data: it is resolved Committed once this write is pushed (§6.5). The request's // whenNothingToCommit rides along, so a window that changed nothing can still record its message. -func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, pendingCR *commitRequestID) { - pendingWrite.CommitRequest = pendingCR +func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, pendingCR *saveRequestID) { + pendingWrite.SaveRequest = pendingCR if pendingCR == nil { return } @@ -1596,8 +1596,8 @@ func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, // target's scan stays fresh, but it commits nothing, and the push would report that as // AlreadyPresent, so only the request fails. See write_gate.go. func (l *branchWorkerEventLoop) gateBuiltWindow( - pendingCR *commitRequestID, target ResolvedTargetMetadata, reason windowFinalizeReason, -) (*commitRequestID, bool) { + pendingCR *saveRequestID, target ResolvedTargetMetadata, reason windowFinalizeReason, +) (*saveRequestID, bool) { if err := l.w.targetWriteRefusal(target.Name, target.Namespace, false); err != nil { l.w.Log.V(1).Info("Discarding built window: render fidelity closed while it was finalized", "reason", string(reason), "gitTarget", target.Namespace+"/"+target.Name) @@ -1605,19 +1605,19 @@ func (l *branchWorkerEventLoop) gateBuiltWindow( return nil, false } if pendingCR != nil && target.Suspend { - l.resolveCommitRequest(*pendingCR, FinalizeResult{Err: errTargetSuspended}) + l.resolveSaveRequest(*pendingCR, FinalizeResult{Err: errTargetSuspended}) return nil, true } return pendingCR, true } // dropOpenWindow discards a window whose finalize failed before it was decided, resolving any -// attached CommitRequest as Failed so the controller does not poll forever. -func (l *branchWorkerEventLoop) dropOpenWindow(pendingCR *commitRequestID, cause error) { +// attached SaveRequest as Failed so the controller does not poll forever. +func (l *branchWorkerEventLoop) dropOpenWindow(pendingCR *saveRequestID, cause error) { l.openWindow = nil l.windowBytes = 0 if pendingCR != nil { - l.resolveCommitRequest(*pendingCR, FinalizeResult{Branch: l.w.Branch, Err: cause}) + l.resolveSaveRequest(*pendingCR, FinalizeResult{Branch: l.w.Branch, Err: cause}) } } @@ -1684,15 +1684,15 @@ func (l *branchWorkerEventLoop) pushPending() { if err != nil { // Leave pendingWrites in place; do NOT advance lastPushAt — the // design specifies lastPushAt only advances on a successful push. A - // CommitRequest riding a retained write stays unresolved while we retry. + // SaveRequest riding a retained write stays unresolved while we retry. l.publicationFailed(err, "Push failed; pending writes retained for retry") l.settleReplayRefusals(refused) return } - // The writes are now on the remote: resolve every CommitRequest riding one with + // The writes are now on the remote: resolve every SaveRequest riding one with // the pushed commit's own SHA (§6.5) — "Committed" means "on the remote". - l.resolvePushedCommitRequests() + l.resolvePushedSaveRequests() l.pendingWrites = nil l.pendingWritesBytes = 0 @@ -1710,7 +1710,7 @@ func (l *branchWorkerEventLoop) publicationFailed(err error, message string) { l.w.Log.Error(err, message, "pendingWrites", len(l.pendingWrites), "retryAt", l.retry.due) } -// resolvePushedCommitRequests settles every CommitRequest carried by a just-pushed write, now +// resolvePushedSaveRequests settles every SaveRequest carried by a just-pushed write, now // that the remote has spoken. // // Two outcomes, decided by whether the write ended up with a commit of its own. A real SHA is @@ -1719,27 +1719,27 @@ func (l *branchWorkerEventLoop) publicationFailed(err error, message string) { // push confirmed the remote agreed, which is the difference between this and resolving at // finalize time. If the remote had moved, the replay would have turned this write into a real // commit before we got here, and it would resolve Committed instead. -func (l *branchWorkerEventLoop) resolvePushedCommitRequests() { +func (l *branchWorkerEventLoop) resolvePushedSaveRequests() { for i := range l.pendingWrites { pw := l.pendingWrites[i] - if pw.CommitRequest == nil { + if pw.SaveRequest == nil { continue } if pw.CommitSHA.IsZero() { - l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{Outcome: FinalizeAlreadyPresent}) + l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{Outcome: FinalizeAlreadyPresent}) continue } if pw.emptyCommitted { // Nothing changed, and the request asked to record its message anyway. The outcome // keeps the cause, and the commit proves the message is in Git. - l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{ + l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{ Outcome: pw.nothingToCommitOutcome(), Commit: pw.CommitSHA.String(), Branch: l.w.Branch, }) continue } - l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{ + l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{ Outcome: FinalizeCommitted, Commit: pw.CommitSHA.String(), Branch: l.w.Branch, @@ -2858,8 +2858,8 @@ func (w *BranchWorker) recordCommitFailure(reason string) { // Commit window timer sources, reported as the commit_windows_total `timer_source` label. const ( - windowTimerSourceTarget = "target" - windowTimerSourceCommitRequest = "commit_request" + windowTimerSourceTarget = "target" + windowTimerSourceSaveRequest = "save_request" ) // metricLabel is the commit_windows_total `close_reason` value for this reason. The log field keeps @@ -2906,7 +2906,7 @@ func (w *BranchWorker) recordWindowClosed(window *openWindow, reason windowFinal if telemetry.GitCommitWindowsTotal != nil { source := windowTimerSourceTarget if window.pendingCR != nil { - source = windowTimerSourceCommitRequest + source = windowTimerSourceSaveRequest } telemetry.GitCommitWindowsTotal.Add(ctx, 1, metric.WithAttributes(append(target, attribute.String("close_reason", reason.metricLabel()), diff --git a/internal/git/branch_worker_empty_repo_test.go b/internal/git/branch_worker_empty_repo_test.go index 3795235d..53e5092e 100644 --- a/internal/git/branch_worker_empty_repo_test.go +++ b/internal/git/branch_worker_empty_repo_test.go @@ -107,7 +107,7 @@ func branchOnDisk(t *testing.T, repoDir, branch, content string) plumbing.Hash { // standbyRequest waits for a window and closes it at once: the write it attaches to is committed // and, in a cooldown, retained. -func standbyRequest() *AttachCommitRequest { +func standbyRequest() *AttachSaveRequest { req := attachReq("alice", time.Hour) req.GitTargetName = newBranchTarget req.MaxDuration = 0 diff --git a/internal/git/branch_worker_metrics_test.go b/internal/git/branch_worker_metrics_test.go index 02313b25..876f5491 100644 --- a/internal/git/branch_worker_metrics_test.go +++ b/internal/git/branch_worker_metrics_test.go @@ -242,7 +242,7 @@ func TestHandleShutdown_DrainsBufferedItemsToZeroDepth(t *testing.T) { assert.Equal(t, int64(0), w.queueDepth(), "a drained, exiting worker must read depth 0") } -// A CommitRequest attach still buffered at shutdown is fire-and-forget: it is +// A SaveRequest attach still buffered at shutdown is fire-and-forget: it is // simply drained from the inflight count (the controller re-sends on its next // poll), so the exiting worker's depth gauge settles to 0. func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { @@ -250,7 +250,7 @@ func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { require.NoError(t, err) w := newMetricsTestWorker() - w.EnqueueAttach(&AttachCommitRequest{Namespace: "default", Name: "save", Author: "alice"}) + w.EnqueueAttach(&AttachSaveRequest{Namespace: "default", Name: "save", Author: "alice"}) require.Equal(t, int64(1), w.inflightItems.Load()) loop := newBranchWorkerEventLoop(w, time.Second) @@ -260,7 +260,7 @@ func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { "a buffered attach must be drained from the inflight count on shutdown") } -// The message_source label is what separates a commit a person named through a CommitRequest from +// The message_source label is what separates a commit a person named through a SaveRequest from // automatic mirroring, so it is asserted on the COLLECTED samples rather than only on // messageSource(): a label that never reaches the exporter is not observable. func TestRecordPendingWritesMetrics_LabelsCommitsByMessageSource(t *testing.T) { @@ -287,9 +287,9 @@ func TestRecordPendingWritesMetrics_LabelsCommitsByMessageSource(t *testing.T) { }, 3) for source, want := range map[string]int64{ - messageSourceLive: 1, - messageSourceCommitRequest: 1, - messageSourceReconcile: 1, + messageSourceLive: 1, + messageSourceSaveRequest: 1, + messageSourceReconcile: 1, } { labels := workerLabels() labels["message_source"] = source @@ -333,7 +333,7 @@ func TestRecordPendingWritesMetrics_MessageSourceSplitsOneAuthorKind(t *testing. require.True(t, ok) assert.Equal(t, int64(1), live) - labels["message_source"] = messageSourceCommitRequest + labels["message_source"] = messageSourceSaveRequest request, ok := telemetry.CollectInt64Sum(reader, commitsTotalMetric, labels) require.True(t, ok) assert.Equal(t, int64(1), request) diff --git a/internal/git/branch_worker_queue_depth_test.go b/internal/git/branch_worker_queue_depth_test.go index f76f03aa..20da5b06 100644 --- a/internal/git/branch_worker_queue_depth_test.go +++ b/internal/git/branch_worker_queue_depth_test.go @@ -24,7 +24,7 @@ import ( // queueDepth() sums the in-flight count and the retained-work flag, so an item that opened a commit // window is covered by the first until the second is published. Releasing the count first leaves a // window where neither covers it, and the loop does real work inside that window -// (serviceCommitRequests, advance, serviceAppliers) before the flag is published. A scrape +// (serviceSaveRequests, advance, serviceAppliers) before the flag is published. A scrape // landing there reports a drained queue for a worker holding an uncommitted event, which can satisfy // a drain gate early. The loop releases the count only after the wake that handled the item. func TestHandledItem_PublishesRetainedWorkBeforeReleasingInflight(t *testing.T) { diff --git a/internal/git/commit.go b/internal/git/commit.go index d2b82b7e..165fc47c 100644 --- a/internal/git/commit.go +++ b/internal/git/commit.go @@ -55,7 +55,7 @@ func renderLiveCommitMessage(pendingWrite PendingWrite, config CommitConfig) (st return renderCommitTemplate("live", config.Message.LiveTemplate, pendingWrite.liveMessageData()) } -// renderRequestCommitMessage frames an attached CommitRequest's message with the target's +// renderRequestCommitMessage frames an attached SaveRequest's message with the target's // requestTemplate. The literal rides in as .RequestMessage rather than being parsed, so nothing a // requester wrote is ever executed. func renderRequestCommitMessage(pendingWrite PendingWrite, config CommitConfig) (string, error) { @@ -265,7 +265,7 @@ func validateRequestTemplate(config CommitConfig, events []Event) error { // requester's reason reaches the commit whatever the window happened to contain. if !strings.Contains(rendered, probe) { return errors.New("requestTemplate must render {{.RequestMessage}}: as written it would " + - "drop the CommitRequest's message from the commit. Omit requestTemplate to commit that " + + "drop the SaveRequest's message from the commit. Omit requestTemplate to commit that " + "message verbatim") } return nil diff --git a/internal/git/commit_executor.go b/internal/git/commit_executor.go index df76d030..b279fe1d 100644 --- a/internal/git/commit_executor.go +++ b/internal/git/commit_executor.go @@ -81,7 +81,7 @@ func (w *BranchWorker) runPendingWrites( refusals := map[int]error{} // Index over the slice so the per-write commit hash is written back onto the - // caller's PendingWrite (§6.5): a CommitRequest riding a write resolves to this + // caller's PendingWrite (§6.5): a SaveRequest riding a write resolves to this // SHA on push, and a rebase-replay (which re-runs this loop on the retained // writes) refreshes it to the post-rebase hash. for i := range pendingWrites { @@ -164,11 +164,11 @@ func restoreWorktree(repo *gogit.Repository, start plumbing.Hash) error { // retained reports whether the loop keeps this write once it is committed: a write that changes // nothing in Git does not exist (docs/design/applier-echoes.md, F2), so a write whose plan left no // action, a window or a resync alike, is neither retained nor pushed. A save is the exception: it -// keeps its objects, and the push decides its outcome (resolvePushedCommitRequests). The checkout +// keeps its objects, and the push decides its outcome (resolvePushedSaveRequests). The checkout // counts the same writes (checkoutApplied), so the two can never disagree. func (p PendingWrite) retained() bool { switch { - case p.Kind == PendingWriteRequestRecord || p.CommitRequest != nil: + case p.Kind == PendingWriteRequestRecord || p.SaveRequest != nil: return true case p.Kind == PendingWriteResync: return p.resyncActions == nil || !p.resyncActions.empty() @@ -179,7 +179,7 @@ func (p PendingWrite) retained() bool { // keepsEveryObject reports whether a write keeps the objects whose plan changed nothing: only a // save does (docs/design/applier-echoes.md, decision 9). -func (p PendingWrite) keepsEveryObject() bool { return p.CommitRequest != nil || p.AllowEmpty } +func (p PendingWrite) keepsEveryObject() bool { return p.SaveRequest != nil || p.AllowEmpty } func (p PendingWrite) path() string { if targetPath := p.Target().Path; targetPath != "" { @@ -203,12 +203,12 @@ const ( messageResolutionUnsupported messageResolution = iota // messageResolutionPreRendered is a resync, which renders the target's reconcile template // itself and hands the result over as the write's message. That is generated text, not a - // request's override, so the CommitRequest literal contract does not bind it: an operator's + // request's override, so the SaveRequest literal contract does not bind it: an operator's // reconcile template may legitimately be long or contain a tab. messageResolutionPreRendered - // messageResolutionRequest is a message a CommitRequest supplied, used verbatim. + // messageResolutionRequest is a message a SaveRequest supplied, used verbatim. messageResolutionRequest - // messageResolutionRequestTemplate is a CommitRequest's message framed by the target's + // messageResolutionRequestTemplate is a SaveRequest's message framed by the target's // requestTemplate. The request's own bytes are never parsed — they ride in as .RequestMessage — // so the formatting policy is the operator's while the message stays the requester's. messageResolutionRequestTemplate @@ -226,7 +226,7 @@ func (p PendingWrite) resolveMessage() messageResolution { switch { case p.Kind == PendingWriteResync: return messageResolutionPreRendered - // Framing is bound to the GROUPED kind, not merely to "carries a message". A CommitRequest + // Framing is bound to the GROUPED kind, not merely to "carries a message". A SaveRequest // attaches its message to an open live window, which finalizes as PendingWriteCommit, so that // is the only write requestTemplate describes. case p.Kind == PendingWriteCommit && @@ -246,11 +246,11 @@ func (p PendingWrite) resolveMessage() messageResolution { func (r messageResolution) label() string { switch r { case messageResolutionRequest: - return messageSourceCommitRequest + return messageSourceSaveRequest case messageResolutionRequestTemplate: - return messageSourceCommitRequestFramed + return messageSourceSaveRequestFramed case messageResolutionRequestFallback: - return messageSourceCommitRequestFallback + return messageSourceSaveRequestFallback case messageResolutionPreRendered: return messageSourceReconcile case messageResolutionLiveTemplate: @@ -294,7 +294,7 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes // Validated on THIS arm too, not just the verbatim one. The literal reaches Git either // way — framed on success, and by itself on the fallback below — so framing must not // become a route that smuggles a message past the check the verbatim path enforces. - // The controller validates earlier (commitrequest_controller.go), which makes this + // The controller validates earlier (saverequest_controller.go), which makes this // belt-and-braces rather than reachable today; it is exactly the kind of invariant that // stops being true once a second producer of PendingWrite appears. if err = ValidateLiteralCommitMessage(p.CommitMessage); err == nil { @@ -315,7 +315,7 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes } switch { case p.TakenOver: - message = withTakeOverNote(message, p.CommitRequest) + message = withTakeOverNote(message, p.SaveRequest) case p.SavedBy.Username != "": message = withSavedByNote(message, p.SavedBy) } @@ -323,17 +323,17 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes return message, commitOptionsFor(p, p.CommitConfig, p.Signer, time.Now()), resolution, nil } -// withTakeOverNote ends the message of a commit whose CommitRequest took over a window with no author +// withTakeOverNote ends the message of a commit whose SaveRequest took over a window with no author // named (PendingWrite.TakenOver) with a trailer that says the author is the request's submitter. -func withTakeOverNote(message string, request *commitRequestID) string { - name := "a CommitRequest" +func withTakeOverNote(message string, request *saveRequestID) string { + name := "a SaveRequest" if request != nil { - name = "CommitRequest " + request.Namespace + "/" + request.Name + name = "SaveRequest " + request.Namespace + "/" + request.Name } return withTrailer(message, "Author-Source: "+name+"; no audit record names who made these changes") } -// withSavedByNote ends the message of a commit whose CommitRequest gave a window with no author named +// withSavedByNote ends the message of a commit whose SaveRequest gave a window with no author named // its message (PendingWrite.SavedBy) with a trailer naming the request's submitter, in the shape of a // Co-authored-by trailer. func withSavedByNote(message string, savedBy UserInfo) string { @@ -346,7 +346,7 @@ func withTrailer(message, trailer string) string { return body + "\n\n" + trailer + message[len(body):] } -// framedRequestMessage renders the target's requestTemplate around an attached CommitRequest's +// framedRequestMessage renders the target's requestTemplate around an attached SaveRequest's // message, falling back to that message verbatim if the render fails. // // The fallback deviates from liveTemplate, where a render failure fails the write, and the @@ -357,7 +357,7 @@ func withTrailer(message, trailer string) string { // harmless. // // It is never silent. The Error log names the target, and the returned resolution moves the commit -// to message_source="commit_request_fallback" — because a fallback is a SUCCESSFUL commit, so no +// to message_source="save_request_fallback" — because a fallback is a SUCCESSFUL commit, so no // condition moves, nothing is refused, and a rate is the only way an operator sees a template that // has quietly stopped applying. // @@ -365,7 +365,7 @@ func withTrailer(message, trailer string) string { func (p PendingWrite) framedRequestMessage() (string, messageResolution) { framed, err := renderRequestCommitMessage(p, p.CommitConfig) if err != nil { - log.Log.Error(err, "requestTemplate failed to render; committing the CommitRequest message verbatim", + log.Log.Error(err, "requestTemplate failed to render; committing the SaveRequest message verbatim", "gitTarget", p.Target().Namespace+"/"+p.Target().Name) return p.CommitMessage, messageResolutionRequestFallback } @@ -385,7 +385,7 @@ func (w *BranchWorker) executePendingWrite( case PendingWriteRequestRecord: return w.executeRequestRecord(ctx, worktree, *pendingWrite) case PendingWriteResync: - // Resync writes never carry a CommitRequest, so their commit hash is unused; + // Resync writes never carry a SaveRequest, so their commit hash is unused; // report ZeroHash to keep the per-write SHA bookkeeping uniform. created, err := w.executeResyncPendingWrite(ctx, repo, worktree, pendingWrite) return created, plumbing.ZeroHash, messageResolutionPreRendered, err @@ -442,7 +442,7 @@ func (w *BranchWorker) executeEventWrite( if err != nil { return 0, plumbing.ZeroHash, messageResolutionUnsupported, err } - // A window whose writes changed nothing still commits when its CommitRequest asked to record + // A window whose writes changed nothing still commits when its SaveRequest asked to record // its message: whenNothingToCommit: CommitEmpty. commitOptions.AllowEmptyCommits = recordEmpty @@ -546,7 +546,7 @@ func (w *BranchWorker) planPendingWriteEvents( return anyChanges, changedBy, nil } -// executeRequestRecord commits a CommitRequest's message with an untouched tree: the record a +// executeRequestRecord commits a SaveRequest's message with an untouched tree: the record a // request that ran out of time to attach leaves when it asked for whenNothingToCommit: CommitEmpty. func (w *BranchWorker) executeRequestRecord( ctx context.Context, @@ -563,7 +563,7 @@ func (w *BranchWorker) executeRequestRecord( return 0, plumbing.ZeroHash, messageResolutionUnsupported, fmt.Errorf("failed to create request record commit: %w", err) } - log.FromContext(ctx).Info("Empty commit created to record a CommitRequest that found nothing to commit", + log.FromContext(ctx).Info("Empty commit created to record a SaveRequest that found nothing to commit", "gitTarget", pendingWrite.GitTargetNamespace+"/"+pendingWrite.GitTargetName, "branch", w.Branch, "commit", hash.String()) return 1, hash, source, nil diff --git a/internal/git/commit_executor_test.go b/internal/git/commit_executor_test.go index 927544de..315a207d 100644 --- a/internal/git/commit_executor_test.go +++ b/internal/git/commit_executor_test.go @@ -205,7 +205,7 @@ func TestExecutor_AppliesEncryptionFromPendingWrite_NotFromWorker(t *testing.T) } // A resync renders the target's reconcile template and hands the result over as the pending -// write's message. That generated text is not a CommitRequest literal override, so the request +// write's message. That generated text is not a SaveRequest literal override, so the request // contract's length and control-character limits must not reject an operator's own template. func TestCommitMetadata_ResyncRenderedMessageIsNotHeldToTheLiteralRequestContract(t *testing.T) { longMessage := "chore: reconcile " + strings.Repeat("x", 1200) @@ -247,13 +247,13 @@ func TestMessageSource_MatchesTheMessageActuallyRendered(t *testing.T) { want: messageSourceLive, }, { - name: "an attached request message is commit_request", + name: "an attached request message is save_request", write: PendingWrite{ Kind: PendingWriteCommit, Events: []Event{makeEvent("api")}, CommitMessage: "fix(api): correct the port", }, - want: messageSourceCommitRequest, + want: messageSourceSaveRequest, message: "fix(api): correct the port", }, { diff --git a/internal/git/commit_window_metrics_test.go b/internal/git/commit_window_metrics_test.go index e58ce24c..b3be11e1 100644 --- a/internal/git/commit_window_metrics_test.go +++ b/internal/git/commit_window_metrics_test.go @@ -112,7 +112,7 @@ func TestCommitWindowMetrics_AnAttachKeepsTheOpeningTimeAndCountsTheRequestsTime assert.Equal(t, opened, loop.openWindow.openedAt, "the attach restarts the timers, not the opening time") forceDue(loop) - assert.Equal(t, int64(1), windowsClosed(t, reader, "max_duration", windowTimerSourceCommitRequest)) + assert.Equal(t, int64(1), windowsClosed(t, reader, "max_duration", windowTimerSourceSaveRequest)) assert.Zero(t, windowsClosed(t, reader, "max_duration", windowTimerSourceTarget)) requireOneObservationPerWindow(t, reader, 1) sum, _ := telemetry.CollectHistogramSum(reader, commitWindowDurationMetric, diff --git a/internal/git/commit_window_timers_test.go b/internal/git/commit_window_timers_test.go index 38e27f5e..95eba69e 100644 --- a/internal/git/commit_window_timers_test.go +++ b/internal/git/commit_window_timers_test.go @@ -13,7 +13,7 @@ import ( ) // These tests pin the clock rules of docs/design/commit-timing-surface.md: a window opened by a -// write takes the GitTarget's timers, and a CommitRequest that attaches replaces them, shorter or +// write takes the GitTarget's timers, and a SaveRequest that attaches replaces them, shorter or // longer. Each one drives the loop directly, so no test waits on a real timer longer than a few // milliseconds. @@ -25,7 +25,7 @@ func writeTo(loop *branchWorkerEventLoop, name string) { }}) } -func namedAttachReq(name string, d time.Duration) *AttachCommitRequest { +func namedAttachReq(name string, d time.Duration) *AttachSaveRequest { req := attachReq("alice", d) req.Name = name req.UID = "uid-" + name @@ -229,7 +229,7 @@ func TestAttachSelection_TheWorkerReportsEachPhase(t *testing.T) { createPlainGitTarget(t, worker, "team-a", "team-a") loop := newBranchWorkerEventLoop(worker, time.Hour) defer loop.stopTimers() - phase := func() CommitRequestPhase { return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName) } + phase := func() SaveRequestPhase { return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName) } assert.Empty(t, phase(), "nothing is reported before the worker registers the request") loop.lastPushAt = time.Now() // keep the push behind its cooldown, so WaitingForPush is observable @@ -266,7 +266,7 @@ func TestWindowTimers_AWaitingRequestWithMaxDurationZeroCommitsExactlyTheNextWri req.Message = "save: exactly the next change" serviceAttach(loop, req) require.Nil(t, loop.openWindow, "the request waits; it opens nothing") - assert.Equal(t, PhaseWaitingForWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) writeTo(loop, "the-change") loop.endWake(0) diff --git a/internal/git/decided_write_test.go b/internal/git/decided_write_test.go index 806c74ac..85281321 100644 --- a/internal/git/decided_write_test.go +++ b/internal/git/decided_write_test.go @@ -76,7 +76,7 @@ func TestDecidedWrite_SurvivesAFailedRebuildAndLandsThroughTheRetry(t *testing.T _, resolved := outcome(t, f.worker) require.False(t, resolved, "the save is held, not failed: its write can still land") - assert.Equal(t, PhaseWaitingForPush, f.worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForPush, f.worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) require.Len(t, loop.pendingWrites, 2, "the window is decided and kept pending") assert.False(t, loop.pendingWrites[1].committedOnce) require.True(t, loop.retry.pending(), "the failure armed the publication retry") @@ -328,7 +328,7 @@ func TestDecidedWrite_SameObjectEditsReplayAsDecidedOntoARemoteHoldingTheLast(t assert.Equal(t, string(second), fileAt(t, replayed[1], gitPath), "the second applies it again") assert.Equal(t, "set value to 2", replayed[1].Message) - outcome, ok := f.worker.LookupCommitRequestOutcome(save.Namespace, save.Name, save.UID) + outcome, ok := f.worker.LookupSaveRequestOutcome(save.Namespace, save.Name, save.UID) require.True(t, ok) assert.Equal(t, FinalizeCommitted, outcome.Outcome, "the save resolves on its own write's commit") assert.Equal(t, replayed[1].Hash.String(), outcome.Commit) diff --git a/internal/git/dirty_worktree_recovery_test.go b/internal/git/dirty_worktree_recovery_test.go index 4bd07f75..923ef984 100644 --- a/internal/git/dirty_worktree_recovery_test.go +++ b/internal/git/dirty_worktree_recovery_test.go @@ -120,7 +120,7 @@ func TestEveryLoopCommitPathRecoversADirtyWorktree(t *testing.T) { }, }, { - name: "empty commit recording a CommitRequest", + name: "empty commit recording a SaveRequest", commit: func(t *testing.T, loop *branchWorkerEventLoop) { t.Helper() serviceAttach(loop, commitEmptyReq("alice", "save: nothing changed")) diff --git a/internal/git/intake_test.go b/internal/git/intake_test.go index 7a7854fa..b84ccd89 100644 --- a/internal/git/intake_test.go +++ b/internal/git/intake_test.go @@ -78,7 +78,7 @@ func TestIntake_EveryKindOfHeldWorkCounts(t *testing.T) { req := attachReq("alice", time.Hour) req.GitTargetName = ledgerTargetName req.Message = "save once the window opens" - loop.handleAttachCommitRequest(req) + loop.handleAttachSaveRequest(req) require.Contains(t, loop.pendingCRs, req.id()) assert.GreaterOrEqual(t, loop.heldBytes()-held, int64(pendingWriteOverheadBytes), "a registered save is charged for what it keeps") @@ -155,7 +155,7 @@ func TestIntake_AWithdrawalProgressesWhilePaused(t *testing.T) { f, loop, _ := outageLoop(t, "intake-withdraw-while-paused") req := attachReq("alice", time.Hour) req.GitTargetName = ledgerTargetName - loop.handleAttachCommitRequest(req) + loop.handleAttachSaveRequest(req) f.worker.branchBufferMaxBytes = 1 loop.publishLoopState(0) require.NotNil(t, f.worker.IntakePaused()) @@ -163,9 +163,9 @@ func TestIntake_AWithdrawalProgressesWhilePaused(t *testing.T) { f.worker.EnqueueWithdraw(req) require.Len(t, f.worker.eventQueue, 1, "a withdrawal is not refused by the pause") loop.handleQueueItem(<-f.worker.eventQueue) - outcome, ok := f.worker.LookupCommitRequestOutcome(req.Namespace, req.Name, req.UID) + outcome, ok := f.worker.LookupSaveRequestOutcome(req.Namespace, req.Name, req.UID) require.True(t, ok) - assert.ErrorIs(t, outcome.Err, ErrCommitRequestWithdrawn) + assert.ErrorIs(t, outcome.Err, ErrSaveRequestWithdrawn) } // bigConfigMap is a ConfigMap whose data is about size bytes. diff --git a/internal/git/network_bound_test.go b/internal/git/network_bound_test.go index 797a711c..1549edcc 100644 --- a/internal/git/network_bound_test.go +++ b/internal/git/network_bound_test.go @@ -298,8 +298,8 @@ func TestPushCycle_ALostReplyThatLandedIsPublishedNotReplayed(t *testing.T) { f := newLedgerFixture(t, "lost-reply", true) f.createLedgerTarget("apps", nil) - record, err := f.worker.buildRequestRecordWrite(f.worker.ctx, &pendingCommitRequest{ - id: commitRequestID{Namespace: "default", Name: "save", UID: "save-uid"}, + record, err := f.worker.buildRequestRecordWrite(f.worker.ctx, &pendingSaveRequest{ + id: saveRequestID{Namespace: "default", Name: "save", UID: "save-uid"}, author: "alice", gitTargetName: ledgerTargetName, gitTargetNamespace: "default", diff --git a/internal/git/open_window.go b/internal/git/open_window.go index a1fe5299..087f3e42 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -16,8 +16,8 @@ type openWindow struct { // author agree, so an unresolved event never coalesces with a resolved one that happens // to carry an empty username. Exact enum equality is right HERE — canAppend compares two // events from the same watch pipeline, so the outcomes are directly comparable. It is not - // right when comparing against a CommitRequest, which a different subsystem attributes; - // see pendingCommitRequest.matchesWindow. + // right when comparing against a SaveRequest, which a different subsystem attributes; + // see pendingSaveRequest.matchesWindow. Attribution AttributionOutcome // GitTarget is the target this window is bound to. Each finalized commit // covers exactly one target so per-target encryption and bootstrap @@ -39,28 +39,28 @@ type openWindow struct { // window collapse onto the same path key. writer eventContentWriter - // pendingMessage is the CommitRequest message attached to this window (§6.4). + // pendingMessage is the SaveRequest message attached to this window (§6.4). // Once set, whichever path finalizes the window uses it instead of the // generated grouped-commit message, so an early cut-off still carries intent. pendingMessage string - // pendingCR identifies the CommitRequest attached to this window; at most one. On + // pendingCR identifies the SaveRequest attached to this window; at most one. On // finalize its outcome is resolved (Committed once the carrying write pushes). - pendingCR *commitRequestID - // claim and claimedBy say what the attached CommitRequest claimed of the window, whose writes - // name no author, and who submitted it (pendingCommitRequest.claims): its message (SavedBy), or + pendingCR *saveRequestID + // claim and claimedBy say what the attached SaveRequest claimed of the window, whose writes + // name no author, and who submitted it (pendingSaveRequest.claims): its message (SavedBy), or // the window, which the submitter then authors (TakeOver). Empty when it claimed nothing. claim UnattributedClaim claimedBy UserInfo // timers decide when the window closes. They are the GitTarget's, snapshotted when the window - // opened, until a CommitRequest attaches and replaces them with its own. + // opened, until a SaveRequest attaches and replaces them with its own. timers windowTimers // openedAt is when the write that opened the window arrived. Unlike the timers and lastWriteAt it - // never moves, not even when a CommitRequest attaches, so it is what the window's collection time + // never moves, not even when a SaveRequest attaches, so it is what the window's collection time // is measured from. openedAt time.Time // lastWriteAt is when the idle interval last restarted: the latest collected write, or the - // attach of a CommitRequest. + // attach of a SaveRequest. lastWriteAt time.Time } @@ -68,7 +68,7 @@ type openWindow struct { type windowTimers struct { // idle is the silence that closes the window. Ignored when noIdle is set. idle time.Duration - // noIdle disables the idle close: a CommitRequest that set no idleTimeout collects until + // noIdle disables the idle close: a SaveRequest that set no idleTimeout collects until // maxAt alone. noIdle bool // maxAt closes the window whatever keeps arriving: the window's opening (or the attach) plus @@ -149,9 +149,9 @@ func windowPathKey(e Event, writer eventContentWriter) string { // window contributes the LAST routed event's resourceVersion, which is the state the commit // actually writes. // -// requestMessage is the attached CommitRequest's message, or empty when no request attached. It is +// requestMessage is the attached SaveRequest's message, or empty when no request attached. It is // carried through unaltered — never parsed, never trimmed — because the request's literal bytes -// reaching the commit is the audit property the whole CommitRequest contract rests on. +// reaching the commit is the audit property the whole SaveRequest contract rests on. func buildLiveCommitMessageData( author, gitTarget, requestMessage string, events []Event, diff --git a/internal/git/parent_recovery_test.go b/internal/git/parent_recovery_test.go index 0abc2e83..100d0102 100644 --- a/internal/git/parent_recovery_test.go +++ b/internal/git/parent_recovery_test.go @@ -325,12 +325,12 @@ func TestParentRecovery_AParentChangeMakesTheProbeDue(t *testing.T) { assert.False(t, f.worker.awaitingParentProbe(), "a new parent is looked for at once") } -// A CommitRequest that rides a write held back by a missing parent stays held through the whole +// A SaveRequest that rides a write held back by a missing parent stays held through the whole // recovery, however long it takes, and resolves Committed when the work lands. This runs the real // event loop on its own goroutine; the clock stands in for a recovery far longer than the bound the // controller used to fail requests on (attachTimeout + maxDuration + 120s). -func TestParentRecovery_AHeldCommitRequestIsCommittedAfterALongRecovery(t *testing.T) { - f := newRealServerNewBranch(t, "recovery-held-commit-request", func(repoDir string) { +func TestParentRecovery_AHeldSaveRequestIsCommittedAfterALongRecovery(t *testing.T) { + f := newRealServerNewBranch(t, "recovery-held-save-request", func(repoDir string) { simulateClientCommitOnDisk(t, repoDir, "main", "README.md", "main\n") gitIn(t, repoDir, "branch", "release", "main") }) @@ -350,8 +350,8 @@ func TestParentRecovery_AHeldCommitRequestIsCommittedAfterALongRecovery(t *testi require.True(t, f.worker.Enqueue(configMapTargetEvent("cm1", "alice", newBranchTarget))) req := standbyRequest() f.worker.EnqueueAttach(req) - phase := func() CommitRequestPhase { - return f.worker.LookupCommitRequestPhase(req.Namespace, req.Name, req.UID) + phase := func() SaveRequestPhase { + return f.worker.LookupSaveRequestPhase(req.Namespace, req.Name, req.UID) } require.Eventually(t, func() bool { open, _ := f.worker.ParentRecovery() diff --git a/internal/git/pending_writes.go b/internal/git/pending_writes.go index 71cebe44..c404b42e 100644 --- a/internal/git/pending_writes.go +++ b/internal/git/pending_writes.go @@ -273,22 +273,22 @@ const ( // Commit message sources, reported as the commits_total `message_source` label. Each names // where the message came from, so they read as one set alongside each other. const ( - // messageSourceCommitRequest is a message a CommitRequest supplied and that was used - // verbatim. It counts commits that USED such a message, not CommitRequests: a request + // messageSourceSaveRequest is a message a SaveRequest supplied and that was used + // verbatim. It counts commits that USED such a message, not SaveRequests: a request // omitting spec.message takes the target's liveTemplate and counts as live. - messageSourceCommitRequest = "commit_request" - // messageSourceCommitRequestFramed is a CommitRequest message rendered through the target's - // requestTemplate. Split from commit_request rather than folded into it, even though the text + messageSourceSaveRequest = "save_request" + // messageSourceSaveRequestFramed is a SaveRequest message rendered through the target's + // requestTemplate. Split from save_request rather than folded into it, even though the text // originated with the request either way: the difference is not HOW the text was produced (the // distinction reconcile deliberately collapses, below) but WHETHER the operator's configured // framing applied at all, which is a question an operator actually asks. - messageSourceCommitRequestFramed = "commit_request_framed" - // messageSourceCommitRequestFallback is a requestTemplate that failed to render, so the + messageSourceSaveRequestFramed = "save_request_framed" + // messageSourceSaveRequestFallback is a requestTemplate that failed to render, so the // request's message was committed verbatim instead. It is the ONLY signal for that: the commit // succeeds, nothing is refused, and no condition moves, so without a counter a template that // has quietly stopped applying looks exactly like one that was never configured. Alert on its // rate; see docs/interpreting-metrics.md. - messageSourceCommitRequestFallback = "commit_request_fallback" + messageSourceSaveRequestFallback = "save_request_fallback" // messageSourceLive is a live window rendered through the target's liveTemplate. messageSourceLive = "live" // messageSourceReconcile is a resync rendered through reconcileTemplate. diff --git a/internal/git/pending_writes_loop.go b/internal/git/pending_writes_loop.go index 68cdb6c5..f5a3ee39 100644 --- a/internal/git/pending_writes_loop.go +++ b/internal/git/pending_writes_loop.go @@ -88,9 +88,9 @@ func (l *branchWorkerEventLoop) decide(pendingWrite PendingWrite) { pendingWrite.decidedAt = l.w.now() l.pendingWrites = append(l.pendingWrites, pendingWrite) l.pendingWritesBytes += pendingWrite.charge - if id := pendingWrite.CommitRequest; id != nil { - l.w.setCommitRequestPhase(*id, PhaseWaitingForPush) - // Bound to its write from here on: see pendingCommitRequest.attached. + if id := pendingWrite.SaveRequest; id != nil { + l.w.setSaveRequestPhase(*id, PhaseWaitingForPush) + // Bound to its write from here on: see pendingSaveRequest.attached. if pcr := l.pendingCRs[*id]; pcr != nil { pcr.attached = true } @@ -430,7 +430,7 @@ func (l *branchWorkerEventLoop) settleDropped(pendingWrite PendingWrite, err err l.w.Log.Error(err, "Resync commit failed after its caller was answered; dropping it", "resources", len(req.Desired)) case pendingWrite.Kind == PendingWriteRequestRecord: - l.w.Log.Error(err, "The empty commit recording a CommitRequest failed", + l.w.Log.Error(err, "The empty commit recording a SaveRequest failed", "gitTarget", pendingWrite.GitTargetNamespace+"/"+pendingWrite.GitTargetName) l.failDecidedRequest(pendingWrite, fmt.Errorf("record the message in an empty commit: %w", err)) default: @@ -476,7 +476,7 @@ func (l *branchWorkerEventLoop) replyToDeferredResyncs(err error) { // failDecidedRequest fails the request a dropped write carried. func (l *branchWorkerEventLoop) failDecidedRequest(pendingWrite PendingWrite, cause error) { - if pendingWrite.CommitRequest != nil { - l.resolveCommitRequest(*pendingWrite.CommitRequest, FinalizeResult{Branch: l.w.Branch, Err: cause}) + if pendingWrite.SaveRequest != nil { + l.resolveSaveRequest(*pendingWrite.SaveRequest, FinalizeResult{Branch: l.w.Branch, Err: cause}) } } diff --git a/internal/git/render_fidelity_gate.go b/internal/git/render_fidelity_gate.go index fe1e56eb..cb46f2eb 100644 --- a/internal/git/render_fidelity_gate.go +++ b/internal/git/render_fidelity_gate.go @@ -146,7 +146,7 @@ func (g *RenderFidelityGate) Reconcile( // An EMPTY plan restarts every scope vacuously, and a vacuous full restart is the absence // of a measurement rather than a fresh one. Without the length guard a target whose rules // stopped selecting anything would clear a write divergence it never repaired, and the - // writes that do not come from a watch — a CommitRequest — would be + // writes that do not come from a watch — a SaveRequest — would be // admitted again on the strength of nothing. if len(scopes) > 0 && fresh == len(scopes) { state.writeDivergence = nil diff --git a/internal/git/render_fidelity_gate_test.go b/internal/git/render_fidelity_gate_test.go index 3c47ba6d..f1517946 100644 --- a/internal/git/render_fidelity_gate_test.go +++ b/internal/git/render_fidelity_gate_test.go @@ -255,7 +255,7 @@ func TestRenderFidelityGate_DroppedScopeStopsCounting(t *testing.T) { // A plan that selects nothing restarts every scope only vacuously, which is the absence of a // measurement rather than a fresh one. Clearing on it would readmit the writes that do not come -// from a watch — a CommitRequest — on the strength of nothing. +// from a watch — a SaveRequest — on the strength of nothing. func TestRenderFidelityGate_AnEmptyPlanDoesNotClearAWriteDivergence(t *testing.T) { gate := NewRenderFidelityGate() target := types.NewResourceReference("apps", "default") @@ -405,7 +405,7 @@ func TestRenderFidelityGate_InvalidateAdvancesEveryScopeRevision(t *testing.T) { // TestRenderFidelityGate_InvalidateHoldsATargetWithNoScopes is the vacuous case, and it is the one // an unregistered target got wrong: with no scopes to report, an empty plan reduces to True, so a // target whose rules select nothing would have been writable against a repository nothing has -// looked at. A CommitRequest does not come from a watch. +// looked at. A SaveRequest does not come from a watch. func TestRenderFidelityGate_InvalidateHoldsATargetWithNoScopes(t *testing.T) { gate := NewRenderFidelityGate() target := types.NewResourceReference("apps", "default") diff --git a/internal/git/replay_failure_test.go b/internal/git/replay_failure_test.go index b11d2d26..2d833c3f 100644 --- a/internal/git/replay_failure_test.go +++ b/internal/git/replay_failure_test.go @@ -12,7 +12,7 @@ package git // A push in that state is silently wrong rather than noisily wrong. The local branch is exactly // the remote tip, so validatePushState reports "already up to date" and PushAtomic returns nil // before it ever compares the cycle's root hash. The worker then counts the writes as published, -// clears them, and resolves any CommitRequest riding one as Committed, naming a SHA that is not on +// clears them, and resolves any SaveRequest riding one as Committed, naming a SHA that is not on // the remote and never will be. // // worktreeDirty is false throughout: the reset cleared it, and the worktree really is clean. It is @@ -114,9 +114,9 @@ func TestReplayFailure_DoesNotLetThePushSettleWorkThatWasReset(t *testing.T) { "the write survived the failed replay and reached the remote on the retry") } -// TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded is the same hazard seen from the +// TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded is the same hazard seen from the // caller's side, which is where it does real damage: a user is told their save landed, with a SHA. -func TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded(t *testing.T) { +func TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded(t *testing.T) { f := newLedgerFixture(t, "replay-failure-cr", true) f.createLedgerTarget("team-a", nil) f.publish("prime") @@ -134,7 +134,7 @@ func TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded(t *testing.T) req.Message = "save: this must not be reported as committed" serviceAttach(loop, req) require.Len(t, loop.pendingWrites, 1) - require.NotNil(t, loop.pendingWrites[0].CommitRequest) + require.NotNil(t, loop.pendingWrites[0].SaveRequest) restoreAPI := failGitTargetReads(t, f.worker, errors.New("etcdserver: request timed out")) require.Error(t, f.worker.refreshRemoteAndRebuildPendingWrites( @@ -142,7 +142,7 @@ func TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded(t *testing.T) loop.pushPending() - _, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) + _, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) require.False(t, resolved, "the request must stay in flight: nothing of its commit is on the remote") @@ -151,7 +151,7 @@ func TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded(t *testing.T) restoreAPI() loop.pushPending() - res, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) + res, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) require.True(t, resolved) assert.Equal(t, FinalizeCommitted, res.Outcome) assert.Contains(t, remoteFileNames(t, f.repoDir), "saved-by-hand") diff --git a/internal/git/replay_refusal_test.go b/internal/git/replay_refusal_test.go index 7a175e80..a917a0bf 100644 --- a/internal/git/replay_refusal_test.go +++ b/internal/git/replay_refusal_test.go @@ -59,7 +59,7 @@ func retainTwoWrites(t *testing.T, loop *branchWorkerEventLoop) { req.Message = "save: lands in a folder that is refused before it is pushed" serviceAttach(loop, req) require.Len(t, loop.pendingWrites, 1) - require.NotNil(t, loop.pendingWrites[0].CommitRequest) + require.NotNil(t, loop.pendingWrites[0].SaveRequest) loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("kept-behind-it", "bob", replayRefusalTargetB)}, @@ -91,7 +91,7 @@ func assertOnlyTheRefusedWriteWasDropped( require.Len(t, *refusals, 1, "the refusal is reported on the target's status, once") assert.Equal(t, itypes.NewResourceReference(ledgerTargetName, "default"), (*refusals)[0].target) - res, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) + res, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) require.True(t, resolved, "the refused write's save is answered rather than held") require.Error(t, res.Err, "and it fails: its write will never land") assert.Contains(t, res.Err.Error(), "kustomization.yaml", "naming what refused it") @@ -140,7 +140,7 @@ func TestReplayRefusal_AnotherFailureStillHoldsEveryWrite(t *testing.T) { require.Len(t, loop.pendingWrites, 2, "an aborted replay settles nothing") assert.Empty(t, *refusals, "and reports no refusal it did not finish deciding") - _, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) + _, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) assert.False(t, resolved, "the save stays held") assert.True(t, loop.retry.pending(), "the retry is scheduled") diff --git a/internal/git/request_template_test.go b/internal/git/request_template_test.go index 41d4cfd6..28219d6d 100644 --- a/internal/git/request_template_test.go +++ b/internal/git/request_template_test.go @@ -57,7 +57,7 @@ func TestFramedRequestMessage_ComposesTheReasonWithWhatWasSaved(t *testing.T) { // The contract PR 1 of this series must not break: the request's bytes reach the commit unaltered. // A target with no requestTemplate commits exactly what was supplied — no trimming, no framing, no -// reformatting — which is what makes a CommitRequest auditable against the commit it produced. +// reformatting — which is what makes a SaveRequest auditable against the commit it produced. func TestCommitMetadata_WithoutARequestTemplateTheLiteralIsCommittedByteForByte(t *testing.T) { literal := " fix(api): correct the service port \n\n Route traffic to the API container. " write := requestWrite(literal, ResolveCommitConfig(nil)) @@ -67,7 +67,7 @@ func TestCommitMetadata_WithoutARequestTemplateTheLiteralIsCommittedByteForByte( assert.Equal(t, literal, message, "an unframed request message must not be altered at all") assert.Equal(t, messageResolutionRequest, resolution) - assert.Equal(t, messageSourceCommitRequest, resolution.label()) + assert.Equal(t, messageSourceSaveRequest, resolution.label()) } // A requestTemplate that fails at render time must not cost the window. The literal already passed @@ -88,7 +88,7 @@ func TestFramedRequestMessage_RenderFailureFallsBackToTheLiteralAndSaysSo(t *tes assert.Equal(t, "fix(api): correct the service port", message) assert.Equal(t, messageResolutionRequestFallback, resolution) - assert.Equal(t, messageSourceCommitRequestFallback, resolution.label()) + assert.Equal(t, messageSourceSaveRequestFallback, resolution.label()) } // The framed and fallback arms must stay distinguishable in commits_total. Folding them together @@ -96,15 +96,15 @@ func TestFramedRequestMessage_RenderFailureFallsBackToTheLiteralAndSaysSo(t *tes func TestMessageSource_SeparatesFramedFromFallbackAndPlainRequest(t *testing.T) { framedConfig := requestConfig(requestBodyTemplate) - assert.Equal(t, messageSourceCommitRequest, + assert.Equal(t, messageSourceSaveRequest, requestWrite("save", ResolveCommitConfig(nil)).messageSource()) - assert.Equal(t, messageSourceCommitRequestFramed, + assert.Equal(t, messageSourceSaveRequestFramed, requestWrite("save", framedConfig).messageSource()) // Stamped at commit time, which is the only place the fallback is known. stamped := requestWrite("save", framedConfig) stamped.committedMessageSource = messageResolutionRequestFallback - assert.Equal(t, messageSourceCommitRequestFallback, stamped.messageSource()) + assert.Equal(t, messageSourceSaveRequestFallback, stamped.messageSource()) } // Precedence: a configured requestTemplate outranks the verbatim arm, and a window with no request @@ -244,7 +244,7 @@ func TestNewRequestTemplateProbe_IsFreshEachTime(t *testing.T) { assert.NotEmpty(t, first) } -// requestTemplate describes the commit a CommitRequest's message produces, and a CommitRequest +// requestTemplate describes the commit a SaveRequest's message produces, and a SaveRequest // attaches to a live window, which finalizes as PendingWriteCommit. A resync arrives pre-rendered // and is never framed, whatever requestTemplate says. func TestResolveMessage_AResyncIsNeverFramed(t *testing.T) { diff --git a/internal/git/resync_push_test.go b/internal/git/resync_push_test.go index 03e24b21..9942c6d4 100644 --- a/internal/git/resync_push_test.go +++ b/internal/git/resync_push_test.go @@ -308,7 +308,7 @@ func TestEnqueueResync_ALabelExitFencesTheSelectedSnapshot(t *testing.T) { } // TestHandleResyncRequest_ClosedWindowIsPushedEvenWhenNoOpResync pins the -// stranded-write fix in the CommitRequest window contract: a +// stranded-write fix in the SaveRequest window contract: a // resync that closes a live commit window but commits nothing of its own must // still schedule the window's commit for push. Before the fix maybeSchedulePush // ran only when the resync itself committed, so a window the resync closed was @@ -438,7 +438,7 @@ func TestEnqueueResync_AReleasedMarkerRunsTheRequestCoalescedIntoIt(t *testing.T } // TestEnqueueResync_DoesNotCoalescePastQueuedAttach pins the same fence for a -// CommitRequest attach, which carries no resource identity of its own but decides +// SaveRequest attach, which carries no resource identity of its own but decides // which commit window later work joins. func TestEnqueueResync_DoesNotCoalescePastQueuedAttach(t *testing.T) { w := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 4)} @@ -448,7 +448,7 @@ func TestEnqueueResync_DoesNotCoalescePastQueuedAttach(t *testing.T) { GitTargetNamespace: "ns", GitTargetName: "target", ResourceVersion: "100", Result: first, })) - w.EnqueueAttach(&AttachCommitRequest{ + w.EnqueueAttach(&AttachSaveRequest{ Namespace: "ns", Name: "cr", GitTargetNamespace: "ns", GitTargetName: "target", }) require.True(t, w.EnqueueResync(&ResyncRequest{ diff --git a/internal/git/retry.go b/internal/git/retry.go index 13e1df5b..c88269bc 100644 --- a/internal/git/retry.go +++ b/internal/git/retry.go @@ -10,7 +10,7 @@ import "time" // The cooldown in maybeSchedulePush spaces SUCCESSFUL pushes; it says nothing about failed ones, // because lastPushAt advances only on success. Without a schedule of its own a failure had two // outcomes, both wrong: a branch that then went quiet kept its work indefinitely (and any -// CommitRequest riding it stayed WaitingForPush, since the controller never fails a request the +// SaveRequest riding it stayed WaitingForPush, since the controller never fails a request the // worker holds), and a branch that kept receiving commits spent one failed attempt on each of them. // // So a failed attempt schedules the next one: due one backoff later, 10s doubling to 5m, on a timer diff --git a/internal/git/retry_test.go b/internal/git/retry_test.go index 18ed1438..93cd2356 100644 --- a/internal/git/retry_test.go +++ b/internal/git/retry_test.go @@ -5,7 +5,7 @@ package git // A failed publication schedules its own retry: a branch that goes quiet after the failure still // publishes, a branch that keeps receiving commits does not spend a push on each of them, and a // failure caused by a missing parent stays with parent recovery's schedule. These pin all three, -// and the CommitRequest that rides the retained write. +// and the SaveRequest that rides the retained write. import ( "context" @@ -198,15 +198,15 @@ func TestPublicationRetry_AFailureAfterTheParentReturnsWaitsForRecovery(t *testi assert.Equal(t, int32(2), attempts.Load(), "recovery's next deadline does") } -// The user-facing half, on the real event loop: a CommitRequest riding a write whose push failed is +// The user-facing half, on the real event loop: a SaveRequest riding a write whose push failed is // held (the controller never fails a held request), so before this it stayed WaitingForPush until // some unrelated write came along. It now resolves Committed on its own. -func TestPublicationRetry_AHeldCommitRequestIsCommittedWithoutAnotherWrite(t *testing.T) { +func TestPublicationRetry_AHeldSaveRequestIsCommittedWithoutAnotherWrite(t *testing.T) { original := retryInitialBackoff retryInitialBackoff = 50 * time.Millisecond t.Cleanup(func() { retryInitialBackoff = original }) - f := newRealServerNewBranch(t, "retry-held-commit-request", seedMain(t)) + f := newRealServerNewBranch(t, "retry-held-save-request", seedMain(t)) attempts := failNextPushes(t, 1) require.NoError(t, f.worker.Start(context.Background())) t.Cleanup(f.worker.Stop) diff --git a/internal/git/save_apply.go b/internal/git/save_apply.go index e9149957..d9a47a7b 100644 --- a/internal/git/save_apply.go +++ b/internal/git/save_apply.go @@ -13,7 +13,7 @@ import ( itypes "github.com/ConfigButler/gitops-reverser/internal/types" ) -// A CommitRequest with catchUp: AfterPush asks its GitTarget's applier to apply the tip as soon as its +// A SaveRequest with catchUp: AfterPush asks its GitTarget's applier to apply the tip as soon as its // commit is pushed, and the controller reports how that went in its Applied condition. The save's own // outcome is decided by the push alone; this is reported beside it and never changes it. // @@ -35,7 +35,7 @@ const ( SaveApplyCommitDropped = "CommitDropped" ) -// SaveApply is how the apply a CommitRequest asked for stands. +// SaveApply is how the apply a SaveRequest asked for stands. type SaveApply struct { // Done is whether it is settled: it changes no more. Done bool @@ -55,56 +55,56 @@ type saveApplyEntry struct { } // recordSaveApply stores where a save's apply stands, for the controller to read. The event loop is -// the only writer; the entries are collected with the CommitRequest outcomes. -func (w *BranchWorker) recordSaveApply(id commitRequestID, apply SaveApply) { +// the only writer; the entries are collected with the SaveRequest outcomes. +func (w *BranchWorker) recordSaveApply(id saveRequestID, apply SaveApply) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if w.crApplies == nil { - w.crApplies = map[commitRequestID]saveApplyEntry{} + w.crApplies = map[saveRequestID]saveApplyEntry{} } w.crApplies[id] = saveApplyEntry{apply: apply, updatedAt: time.Now()} } -// LookupSaveApply reports where the apply a CommitRequest asked for stands, and false while the +// LookupSaveApply reports where the apply a SaveRequest asked for stands, and false while the // worker knows nothing of it. func (w *BranchWorker) LookupSaveApply(namespace, name, uid string) (SaveApply, bool) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - entry, ok := w.crApplies[commitRequestID{Namespace: namespace, Name: name, UID: uid}] + entry, ok := w.crApplies[saveRequestID{Namespace: namespace, Name: name, UID: uid}] return entry.apply, ok } // collectSaveApplies drops the entries older than the outcome TTL. The caller holds crOutcomesMu. func (w *BranchWorker) collectSaveApplies(now time.Time) { for id, entry := range w.crApplies { - if now.Sub(entry.updatedAt) > commitRequestOutcomeTTL { + if now.Sub(entry.updatedAt) > saveRequestOutcomeTTL { delete(w.crApplies, id) } } } -// catchUpAfterSave is called as a CommitRequest that asked to catch up resolves. With a pushed +// catchUpAfterSave is called as a SaveRequest that asked to catch up resolves. With a pushed // commit, its GitTarget's applier gets a converge when the appliers are next listed, at the end of // this wake (takeSavedCatchUps); with none, there is nothing to apply. -func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, commit string) { +func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingSaveRequest, commit string) { if commit == "" { l.w.recordSaveApply(pcr.id, SaveApply{Done: true, Applied: true, Reason: SaveApplyNothingToApply, Message: "the request pushed no commit: what it saved was in Git already, and the cluster holds it"}) return } if l.savedCatchUps == nil { - l.savedCatchUps = map[itypes.ResourceReference]map[commitRequestID]string{} + l.savedCatchUps = map[itypes.ResourceReference]map[saveRequestID]string{} } ref := itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace) if l.savedCatchUps[ref] == nil { - l.savedCatchUps[ref] = map[commitRequestID]string{} + l.savedCatchUps[ref] = map[saveRequestID]string{} } l.savedCatchUps[ref][pcr.id] = commit l.w.recordSaveApply(pcr.id, SaveApply{Reason: SaveApplyApplying, Message: "commit " + commit + " is pushed; its applier is asked to apply it"}) } -// takeSavedCatchUps leaves a converge pending for every applier a pushed CommitRequest asked to catch +// takeSavedCatchUps leaves a converge pending for every applier a pushed SaveRequest asked to catch // up, and records the request as waiting on it. It was asked for, so it is no lag converge: it does // not yield to an edit (convergeHolds). A request whose GitTarget names no applier is settled. func (l *branchWorkerEventLoop) takeSavedCatchUps() { @@ -122,7 +122,7 @@ func (l *branchWorkerEventLoop) takeSavedCatchUps() { for id, commit := range saves { state.waitSave(id, commit) } - l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", + l.w.Log.Info("A SaveRequest asked its applier to catch up with the pushed commit", "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", ref.String(), "requests", len(saves)) } l.savedCatchUps = nil @@ -140,11 +140,11 @@ func (l *branchWorkerEventLoop) applierNaming(ref itypes.ResourceReference) *app return nil } -// waitSave records a CommitRequest, which pushed commit, as waiting for the applier's next apply that +// waitSave records a SaveRequest, which pushed commit, as waiting for the applier's next apply that // succeeds. -func (s *applierState) waitSave(id commitRequestID, commit string) { +func (s *applierState) waitSave(id saveRequestID, commit string) { if s.waitingSaves == nil { - s.waitingSaves = map[commitRequestID]string{} + s.waitingSaves = map[saveRequestID]string{} } s.waitingSaves[id] = commit } @@ -157,7 +157,7 @@ func (s *applierState) waitSave(id commitRequestID, commit string) { // - one whose commit the branch holds and revision does not waits for the converge left pending; // - one the checkout cannot tell about, because it lacks a commit, waits as well, and the remote is // looked at: revision is a commit pushed elsewhere, and its fetch brings the apply that settles it. -func (l *branchWorkerEventLoop) settleSaves(state *applierState, saves map[commitRequestID]string, revision string) { +func (l *branchWorkerEventLoop) settleSaves(state *applierState, saves map[saveRequestID]string, revision string) { if len(saves) == 0 { return } @@ -235,7 +235,7 @@ func commitHolds(repo *gogit.Repository, descendant, commit string) (bool, bool) } // retrySaves records the saves an attempt that did not succeed gave back as waiting again, with why. -func (l *branchWorkerEventLoop) retrySaves(saves map[commitRequestID]string, why string) { +func (l *branchWorkerEventLoop) retrySaves(saves map[saveRequestID]string, why string) { for id := range saves { l.w.recordSaveApply(id, SaveApply{Reason: SaveApplyRetrying, Message: "the apply did not complete, and is asked again: " + why}) @@ -248,7 +248,7 @@ func (l *branchWorkerEventLoop) dropSaves(state *applierState) { saves := maps.Clone(state.waitingSaves) if state.inflight != nil { if saves == nil { - saves = map[commitRequestID]string{} + saves = map[saveRequestID]string{} } maps.Copy(saves, state.inflight.saves) } diff --git a/internal/git/commit_request_attach.go b/internal/git/save_request_attach.go similarity index 85% rename from internal/git/commit_request_attach.go rename to internal/git/save_request_attach.go index 15c8352f..d6e0381a 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/save_request_attach.go @@ -31,7 +31,7 @@ var ErrResyncDiscarded = errors.New("resync snapshot discarded: an apply made it // it stays pending in its place and is applied with the writes around it once the remote answers. var ErrResyncDeferred = errors.New("resync deferred until the remote can be reached") -// FinalizeOutcome is the terminal result of resolving a CommitRequest. +// FinalizeOutcome is the terminal result of resolving a SaveRequest. type FinalizeOutcome string const ( @@ -45,7 +45,7 @@ const ( // window this request could attach to. It is a refusal a human can see: the author's own edits // went into somebody else's commit, under a generated message rather than theirs. // - // Raised at expiry from pendingCommitRequest.sawForeignWindow, never at attach time: under + // Raised at expiry from pendingSaveRequest.sawForeignWindow, never at attach time: under // eager attach a request WAITS for a matching window rather than being refused on sight, so // "a foreign window is open right now" is not yet an outcome. FinalizeWindowMismatch FinalizeOutcome = "WindowMismatch" @@ -55,38 +55,38 @@ const ( FinalizeAlreadyPresent FinalizeOutcome = "AlreadyPresent" ) -// CommitRequestPhase is where an unresolved CommitRequest stands on the worker. It is what the +// SaveRequestPhase is where an unresolved SaveRequest stands on the worker. It is what the // controller reports while it waits, and it comes from the worker because sending an attach does // not prove the request was registered, attached or committed. -type CommitRequestPhase string +type SaveRequestPhase string const ( // PhaseWaitingForWorker is a request whose GitTarget has no branch worker yet, at startup or // before the target's first reconcile. It is reported by the router, since there is no worker to // report it, and the request is sent again on the controller's next poll. - PhaseWaitingForWorker CommitRequestPhase = "WaitingForWorker" + PhaseWaitingForWorker SaveRequestPhase = "WaitingForWorker" // PhaseWaitingForWindow is a registered request waiting for a window to attach to. - PhaseWaitingForWindow CommitRequestPhase = "WaitingForWindow" + PhaseWaitingForWindow SaveRequestPhase = "WaitingForWindow" // PhaseCollectingWindow is an attached request collecting writes until the window's timers close it. - PhaseCollectingWindow CommitRequestPhase = "CollectingWindow" + PhaseCollectingWindow SaveRequestPhase = "CollectingWindow" // PhaseWaitingForPush is a request whose write is decided and not yet confirmed by the remote. - PhaseWaitingForPush CommitRequestPhase = "WaitingForPush" + PhaseWaitingForPush SaveRequestPhase = "WaitingForPush" ) // Held reports whether the worker has acted on a request in this phase: it is attached to a window // or committed. From there only the worker can say how it ends, so the controller waits for it // however long that takes, instead of failing the request on a clock of its own. -func (p CommitRequestPhase) Held() bool { +func (p SaveRequestPhase) Held() bool { return p == PhaseCollectingWindow || p == PhaseWaitingForPush } -// ErrCommitRequestWithdrawn is the outcome of a request the controller withdrew before the worker +// ErrSaveRequestWithdrawn is the outcome of a request the controller withdrew before the worker // acted on it. The worker records it, so a late re-send of the attach can never register the // request again, and the controller's failure and the worker's answer are the same answer. -var ErrCommitRequestWithdrawn = errors.New("the CommitRequest was withdrawn before the worker acted on it") +var ErrSaveRequestWithdrawn = errors.New("the SaveRequest was withdrawn before the worker acted on it") -// FinalizeResult carries the resolved outcome of a CommitRequest back to the -// controller, polled via LookupCommitRequestOutcome. +// FinalizeResult carries the resolved outcome of a SaveRequest back to the +// controller, polled via LookupSaveRequestOutcome. type FinalizeResult struct { // Outcome is set when Err is nil. Outcome FinalizeOutcome @@ -98,19 +98,19 @@ type FinalizeResult struct { Err error // Phase is where an UNRESOLVED request stands on the worker; empty once resolved, and empty // before the worker has registered it. - Phase CommitRequestPhase + Phase SaveRequestPhase // Held is why a request the worker holds has not reached the remote yet: the worker's // publication report while it cannot publish, empty otherwise. See PublicationStatus. Held string } -// AttachCommitRequest is the "attach this CommitRequest to the author's window, with its +// AttachSaveRequest is the "attach this SaveRequest to the author's window, with its // message and its timers" work item. It rides the same per-worker FIFO // event queue as resource events, so by audit-stream ordering it is processed // after every earlier write for that worker. Re-sends are idempotent: the worker // keys pending requests by identity and keeps everything the first delivery set. -type AttachCommitRequest struct { - // Namespace, Name, UID identify the CommitRequest. UID may be empty (a +type AttachSaveRequest struct { + // Namespace, Name, UID identify the SaveRequest. UID may be empty (a // Metadata-level audit policy can omit it); identity then keys on // namespace/name only. Namespace string @@ -122,7 +122,7 @@ type AttachCommitRequest struct { // matches is attached; this binds "the open window" to "the requesting // author's open window". Author string - // Attribution is the outcome of attributing THIS CommitRequest, from the command-authorship + // Attribution is the outcome of attributing THIS SaveRequest, from the command-authorship // path. It is matched alongside Author rather than inferred from it, because an empty Author // alone cannot say whether an actor was sought: it is both "attribution is off" and // "attribution ran and named nobody". Only the NamesActor half is compared against the @@ -157,7 +157,7 @@ type AttachCommitRequest struct { Unattributed UnattributedClaim } -// UnattributedClaim is what a CommitRequest may claim of a commit window whose writes name no author. +// UnattributedClaim is what a SaveRequest may claim of a commit window whose writes name no author. type UnattributedClaim string const ( @@ -171,24 +171,24 @@ const ( UnattributedClaimTakeOver UnattributedClaim = "TakeOver" ) -// commitRequestID is the worker-local key for a CommitRequest: its namespaced -// name plus UID when available. Two CommitRequests with the same name but +// saveRequestID is the worker-local key for a SaveRequest: its namespaced +// name plus UID when available. Two SaveRequests with the same name but // different UIDs (a delete-and-recreate) are distinct. -type commitRequestID struct { +type saveRequestID struct { Namespace string Name string UID string } -func (a AttachCommitRequest) id() commitRequestID { - return commitRequestID{Namespace: a.Namespace, Name: a.Name, UID: a.UID} +func (a AttachSaveRequest) id() saveRequestID { + return saveRequestID{Namespace: a.Namespace, Name: a.Name, UID: a.UID} } -// pendingCommitRequest is a CommitRequest registered with the worker and not yet +// pendingSaveRequest is a SaveRequest registered with the worker and not yet // resolved: waiting for a same-author window to attach to, attached and collecting until the // window's timers close it, or committed locally and awaiting the push that settles it. -type pendingCommitRequest struct { - id commitRequestID +type pendingSaveRequest struct { + id saveRequestID author string attribution AttributionOutcome gitTargetName string @@ -210,7 +210,7 @@ type pendingCommitRequest struct { // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. catchUpNow bool // unattributed is what the request may claim of a window whose writes name no author - // (AttachCommitRequest.Unattributed). + // (AttachSaveRequest.Unattributed). unattributed UnattributedClaim // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays @@ -240,9 +240,9 @@ type pendingCommitRequest struct { // to save" and "someone else held the window the whole time" are different events with the same // shape, and only the second one silently substitutes a generated commit message for the sentence // the request's author typed. FinalizeWindowMismatch has always been declared, surfaced by the -// controller as the WindowMismatch reason, and counted by commit_requests_total; from the eager +// controller as the WindowMismatch reason, and counted by save_requests_total; from the eager // attach onwards nothing produced it, so every one of those refusals reported as NoOpenWindow. -func (p *pendingCommitRequest) expiryOutcome() FinalizeOutcome { +func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { if p.sawForeignWindow { return FinalizeWindowMismatch } @@ -270,7 +270,7 @@ func (p *pendingCommitRequest) expiryOutcome() FinalizeOutcome { // outcome fields being right, so any path that leaves an outcome unset while the author IS set // would let one author's request finalize another's window. Comparing both costs nothing and // keeps "bob never attaches to alice's window" true regardless of what the outcomes say. -func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { +func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { if p == nil || w == nil { return false } @@ -282,14 +282,14 @@ func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { // claims reports whether the request matches a window on its GitTarget whose writes name no author // only because it may claim such a window: it names its submitter, and asked for more than its own -// windows (AttachCommitRequest.Unattributed). -func (p *pendingCommitRequest) claims(w *openWindow) bool { +// windows (AttachSaveRequest.Unattributed). +func (p *pendingSaveRequest) claims(w *openWindow) bool { return p.unattributed != UnattributedClaimNone && p.attribution.NamesActor() && !w.Attribution.NamesActor() } -// commitRequestOutcomeEntry is a resolved outcome retained for the controller to +// saveRequestOutcomeEntry is a resolved outcome retained for the controller to // poll, GC'd by age. -type commitRequestOutcomeEntry struct { +type saveRequestOutcomeEntry struct { result FinalizeResult resolvedAt time.Time } diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/save_request_attach_loop.go similarity index 74% rename from internal/git/commit_request_attach_loop.go rename to internal/git/save_request_attach_loop.go index 4f7c7d7d..fdba8eaa 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/save_request_attach_loop.go @@ -11,14 +11,14 @@ import ( "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// This file holds the worker-loop side of CommitRequest attach. All methods run on the single +// This file holds the worker-loop side of SaveRequest attach. All methods run on the single // event loop goroutine, so pendingCRs needs no locking; only the resolved-outcome and phase tables // (BranchWorker.crOutcomes, crPhases) cross goroutines and are mutex-guarded. // -// The lifecycle of one CommitRequest, worker-side. A window is only ever opened by a write; a +// The lifecycle of one SaveRequest, worker-side. A window is only ever opened by a write; a // request attaches to one and never opens one. // -// register (handleAttachCommitRequest) → attach: Next closes the author's open window first +// register (handleAttachSaveRequest) → attach: Next closes the author's open window first // → [matching window open, CurrentOrNext] Attached // → otherwise WaitingForWindow // a same-author window opens before the attach deadline → Attached (first come, first served) @@ -27,35 +27,35 @@ import ( // attach deadline passes while WaitingForWindow → resolved NoOpenWindow, or WindowMismatch when // a window it could not attach to was open meanwhile; never a later window -// commitRequestOutcomeTTL bounds how long a resolved CommitRequest outcome is +// saveRequestOutcomeTTL bounds how long a resolved SaveRequest outcome is // retained for the controller to poll before it is GC'd. It comfortably exceeds // the controller's poll cadence. It is not what keeps a withdrawn request withdrawn: // an outcome is never collected while an attach for it is still queued (crQueuedAttaches), // and the controller sends no attach once it has started withdrawing. -const commitRequestOutcomeTTL = 15 * time.Minute +const saveRequestOutcomeTTL = 15 * time.Minute // ErrBranchWorkerStopped is the outcome of a request whose branch worker stopped before it could // finish it. Once the loop has exited nothing can publish the request, so this answer is final. -var ErrBranchWorkerStopped = errors.New("the GitTarget's branch worker stopped before it finished the CommitRequest") +var ErrBranchWorkerStopped = errors.New("the GitTarget's branch worker stopped before it finished the SaveRequest") -// recordCommitRequestOutcome stores a resolved outcome and GCs stale entries. The +// recordSaveRequestOutcome stores a resolved outcome and GCs stale entries. The // event loop is the only caller, but it takes the mutex because the controller -// reads concurrently via LookupCommitRequestOutcome. -func (w *BranchWorker) recordCommitRequestOutcome(id commitRequestID, result FinalizeResult) { +// reads concurrently via LookupSaveRequestOutcome. +func (w *BranchWorker) recordSaveRequestOutcome(id saveRequestID, result FinalizeResult) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - w.recordCommitRequestOutcomeLocked(id, result) + w.recordSaveRequestOutcomeLocked(id, result) } -// recordCommitRequestOutcomeLocked is recordCommitRequestOutcome for a caller holding crOutcomesMu. -func (w *BranchWorker) recordCommitRequestOutcomeLocked(id commitRequestID, result FinalizeResult) { +// recordSaveRequestOutcomeLocked is recordSaveRequestOutcome for a caller holding crOutcomesMu. +func (w *BranchWorker) recordSaveRequestOutcomeLocked(id saveRequestID, result FinalizeResult) { if w.crOutcomes == nil { - w.crOutcomes = map[commitRequestID]commitRequestOutcomeEntry{} + w.crOutcomes = map[saveRequestID]saveRequestOutcomeEntry{} } now := time.Now() - w.crOutcomes[id] = commitRequestOutcomeEntry{result: result, resolvedAt: now} + w.crOutcomes[id] = saveRequestOutcomeEntry{result: result, resolvedAt: now} for k, entry := range w.crOutcomes { - if now.Sub(entry.resolvedAt) > commitRequestOutcomeTTL && w.crQueuedAttaches[k] == 0 { + if now.Sub(entry.resolvedAt) > saveRequestOutcomeTTL && w.crQueuedAttaches[k] == 0 { delete(w.crOutcomes, k) w.crOwners.release(k, w) } @@ -65,11 +65,11 @@ func (w *BranchWorker) recordCommitRequestOutcomeLocked(id commitRequestID, resu // countQueuedAttach records an attach entering the FIFO. The caller holds pendingResyncsMu, so the // count is raised before the loop can receive the item. -func (w *BranchWorker) countQueuedAttach(id commitRequestID) { +func (w *BranchWorker) countQueuedAttach(id saveRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if w.crQueuedAttaches == nil { - w.crQueuedAttaches = map[commitRequestID]int{} + w.crQueuedAttaches = map[saveRequestID]int{} } w.crQueuedAttaches[id]++ } @@ -77,7 +77,7 @@ func (w *BranchWorker) countQueuedAttach(id commitRequestID) { // uncountQueuedAttach records an attach leaving the FIFO: handled by the loop, dropped on a full // queue, or drained by an exiting worker. A request the worker then knows nothing else about is // given back, so the controller's next poll finds the worker its GitTarget names. -func (w *BranchWorker) uncountQueuedAttach(id commitRequestID) { +func (w *BranchWorker) uncountQueuedAttach(id saveRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if n := w.crQueuedAttaches[id]; n > 1 { @@ -92,7 +92,7 @@ func (w *BranchWorker) uncountQueuedAttach(id commitRequestID) { } } -// settleCommitRequestsAfterExit runs once the event loop has returned: after a shutdown, after a +// settleSaveRequestsAfterExit runs once the event loop has returned: after a shutdown, after a // retirement, or when the loop never started because its GitProvider could not be read. Either way // nothing will act on a request again, and every request the worker knew must get an answer // rather than wait for one that cannot come. @@ -103,7 +103,7 @@ func (w *BranchWorker) uncountQueuedAttach(id commitRequestID) { // controller is free to send it to the worker the GitTarget names now, or to withdraw it. // // From then on a withdraw is answered at once (answerWithdrawAfterExit). -func (w *BranchWorker) settleCommitRequestsAfterExit() { +func (w *BranchWorker) settleSaveRequestsAfterExit() { w.pendingResyncsMu.Lock() w.stoppingState = true w.pendingResyncsMu.Unlock() @@ -114,7 +114,7 @@ func (w *BranchWorker) settleCommitRequestsAfterExit() { w.crOutcomesMu.Lock() for id, phase := range w.crPhases { if phase.Held() { - w.recordCommitRequestOutcomeLocked(id, FinalizeResult{Branch: w.Branch, Err: ErrBranchWorkerStopped}) + w.recordSaveRequestOutcomeLocked(id, FinalizeResult{Branch: w.Branch, Err: ErrBranchWorkerStopped}) } else { w.crOwners.release(id, w) } @@ -129,30 +129,30 @@ func (w *BranchWorker) settleCommitRequestsAfterExit() { // answerWithdrawAfterExit resolves a withdraw sent to a worker whose loop has exited. Nothing can // act on the request any more, so it is withdrawn for good, unless it already has an outcome. -func (w *BranchWorker) answerWithdrawAfterExit(id commitRequestID) { +func (w *BranchWorker) answerWithdrawAfterExit(id saveRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if _, resolved := w.crOutcomes[id]; resolved { return } - w.recordCommitRequestOutcomeLocked(id, FinalizeResult{ + w.recordSaveRequestOutcomeLocked(id, FinalizeResult{ Branch: w.Branch, - Err: errors.Join(ErrCommitRequestWithdrawn, ErrBranchWorkerStopped), + Err: errors.Join(ErrSaveRequestWithdrawn, ErrBranchWorkerStopped), }) } -// LookupCommitRequestOutcome returns a resolved CommitRequest outcome, or ok=false +// LookupSaveRequestOutcome returns a resolved SaveRequest outcome, or ok=false // when the request is still in flight (or already GC'd). The controller polls this -// after sending its AttachCommitRequest. -func (w *BranchWorker) LookupCommitRequestOutcome(namespace, name, uid string) (FinalizeResult, bool) { +// after sending its AttachSaveRequest. +func (w *BranchWorker) LookupSaveRequestOutcome(namespace, name, uid string) (FinalizeResult, bool) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - entry, ok := w.crOutcomes[commitRequestID{Namespace: namespace, Name: name, UID: uid}] + entry, ok := w.crOutcomes[saveRequestID{Namespace: namespace, Name: name, UID: uid}] return entry.result, ok } -// setCommitRequestPhase records where an unresolved request stands; an empty phase forgets it. -func (w *BranchWorker) setCommitRequestPhase(id commitRequestID, phase CommitRequestPhase) { +// setSaveRequestPhase records where an unresolved request stands; an empty phase forgets it. +func (w *BranchWorker) setSaveRequestPhase(id saveRequestID, phase SaveRequestPhase) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if phase == "" { @@ -160,40 +160,40 @@ func (w *BranchWorker) setCommitRequestPhase(id commitRequestID, phase CommitReq return } if w.crPhases == nil { - w.crPhases = map[commitRequestID]CommitRequestPhase{} + w.crPhases = map[saveRequestID]SaveRequestPhase{} } w.crPhases[id] = phase } -// LookupCommitRequestPhase returns where an unresolved request stands on this worker, or "" when +// LookupSaveRequestPhase returns where an unresolved request stands on this worker, or "" when // the worker has not registered it (yet) or it is resolved. -func (w *BranchWorker) LookupCommitRequestPhase(namespace, name, uid string) CommitRequestPhase { +func (w *BranchWorker) LookupSaveRequestPhase(namespace, name, uid string) SaveRequestPhase { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - return w.crPhases[commitRequestID{Namespace: namespace, Name: name, UID: uid}] + return w.crPhases[saveRequestID{Namespace: namespace, Name: name, UID: uid}] } -// hasCommitRequestOutcome reports whether a request is already resolved, so a late +// hasSaveRequestOutcome reports whether a request is already resolved, so a late // idempotent re-send of its attach is a no-op. -func (w *BranchWorker) hasCommitRequestOutcome(id commitRequestID) bool { +func (w *BranchWorker) hasSaveRequestOutcome(id saveRequestID) bool { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() _, ok := w.crOutcomes[id] return ok } -// handleAttachCommitRequest registers a CommitRequest with the worker. Everything is stamped once: +// handleAttachSaveRequest registers a SaveRequest with the worker. Everything is stamped once: // an idempotent re-send restarts nothing and closes nothing. -func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitRequest) { +func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) { id := req.id() // Uncounted only once the request is registered or resolved, so the worker never looks as if it // had forgotten a request it is in the middle of taking. defer l.w.uncountQueuedAttach(id) - if l.w.hasCommitRequestOutcome(id) { + if l.w.hasSaveRequestOutcome(id) { return // already resolved: a late idempotent re-send. } if l.pendingCRs == nil { - l.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + l.pendingCRs = map[saveRequestID]*pendingSaveRequest{} } if _, exists := l.pendingCRs[id]; exists { return // idempotent re-send: keep the first registration. @@ -202,7 +202,7 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque // delayed ingestion pipeline this lets attachTimeout cover only the inter-stream spread // instead of the absolute latency. l.crSeq++ - pcr := &pendingCommitRequest{ + pcr := &pendingSaveRequest{ id: id, author: req.Author, attribution: req.Attribution, @@ -219,8 +219,8 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque unattributed: req.Unattributed, } l.pendingCRs[id] = pcr - l.w.setCommitRequestPhase(id, PhaseWaitingForWindow) - l.w.Log.Info("CommitRequest registered with worker", + l.w.setSaveRequestPhase(id, PhaseWaitingForWindow) + l.w.Log.Info("SaveRequest registered with worker", "request", id.Namespace+"/"+id.Name, "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName, @@ -252,31 +252,31 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque } } -// serviceCommitRequests runs after every loop wake: note foreign windows, resolve waiting requests +// serviceSaveRequests runs after every loop wake: note foreign windows, resolve waiting requests // whose attach deadline passed, attach the next waiting request to an open window, and re-arm the // attach timer. // // Expiry runs BEFORE attach. When a request's deadline and a matching write are ready on the same // wake, the request has already run out: it resolves, and the write keeps its own window. -func (l *branchWorkerEventLoop) serviceCommitRequests() { +func (l *branchWorkerEventLoop) serviceSaveRequests() { if len(l.pendingCRs) == 0 { l.stopAttachTimer() return } l.noteForeignWindow() - l.expireWaitingCommitRequests() - l.attachWaitingCommitRequests() + l.expireWaitingSaveRequests() + l.attachWaitingSaveRequests() l.rearmAttachTimer() } // waiting reports whether a request is still looking for a window it may attach to at now. -func (p *pendingCommitRequest) waiting(now time.Time) bool { +func (p *pendingSaveRequest) waiting(now time.Time) bool { return !p.attached && p.attachDeadline.After(now) } // noteForeignWindow records, on every waiting request, that a window it cannot attach to is open. // -// It runs BEFORE attachWaitingCommitRequests and independently of it, because that function +// It runs BEFORE attachWaitingSaveRequests and independently of it, because that function // returns early when the window is already taken — so the scan inside it cannot be the place // this is observed. It also runs on every pass rather than at expiry: the foreign window is // finalized on its own timer, usually before this request's wait runs out. @@ -299,7 +299,7 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { continue } pcr.sawForeignWindow = true - l.w.Log.Info("CommitRequest is waiting on a window that belongs to someone else", + l.w.Log.Info("SaveRequest is waiting on a window that belongs to someone else", "request", pcr.id.Namespace+"/"+pcr.id.Name, "requestAuthor", pcr.author, "requestTarget", pcr.gitTargetNamespace+"/"+pcr.gitTargetName, @@ -308,15 +308,15 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { } } -// attachWaitingCommitRequests attaches the first-registered waiting request that matches the open +// attachWaitingSaveRequests attaches the first-registered waiting request that matches the open // window, when the window carries none yet. A window carries at most one request; a second waits // for the next window, and a request whose deadline has passed takes none. -func (l *branchWorkerEventLoop) attachWaitingCommitRequests() { +func (l *branchWorkerEventLoop) attachWaitingSaveRequests() { if l.openWindow == nil || l.openWindow.pendingCR != nil { return } now := time.Now() - var first *pendingCommitRequest + var first *pendingSaveRequest for _, pcr := range l.pendingCRs { if !pcr.waiting(now) || !pcr.matchesWindow(l.openWindow) { continue @@ -334,7 +334,7 @@ func (l *branchWorkerEventLoop) attachWaitingCommitRequests() { // with the request's own: the request's maxDuration runs from this attach, and so does its first // idle interval. A zero maxDuration closes the window right here. The message is the window's to // keep from now on, so it moves into the window's charge. -func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { +func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { now := time.Now() l.openWindow.pendingMessage = pcr.message id := pcr.id @@ -349,12 +349,12 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { l.openWindow.lastWriteAt = now if pcr.claims(l.openWindow) { l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} - l.w.Log.Info("CommitRequest claimed a window whose writes name no author", + l.w.Log.Info("SaveRequest claimed a window whose writes name no author", "request", id.Namespace+"/"+id.Name, "author", pcr.author, "claim", string(pcr.unattributed), "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) } - l.w.setCommitRequestPhase(id, PhaseCollectingWindow) - l.w.Log.Info("CommitRequest attached to open window", + l.w.setSaveRequestPhase(id, PhaseCollectingWindow) + l.w.Log.Info("SaveRequest attached to open window", "request", id.Namespace+"/"+id.Name, "author", pcr.author, "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName, @@ -362,52 +362,52 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { l.closeOrArmWindow() } -// expireWaitingCommitRequests resolves every request whose attach deadline passed while it was +// expireWaitingSaveRequests resolves every request whose attach deadline passed while it was // still waiting. An attached request is not here: its window's timers close it, and whichever path // closes the window resolves it. -func (l *branchWorkerEventLoop) expireWaitingCommitRequests() { +func (l *branchWorkerEventLoop) expireWaitingSaveRequests() { now := time.Now() for id, pcr := range l.pendingCRs { if pcr.attached || pcr.attachDeadline.After(now) { continue } // Which of the two refusals this is depends on whether anything was open that this - // request could not have: see pendingCommitRequest.expiryOutcome. + // request could not have: see pendingSaveRequest.expiryOutcome. outcome := pcr.expiryOutcome() if outcome == FinalizeNoOpenWindow { // NoWindow says the save saw no writes. On a target that refuses them that is not the // cause, and with CommitEmpty the record would say so in Git: see write_gate.go. if err := l.w.requestWriteRefusal(l.w.ctx, pcr); err != nil { - l.resolveCommitRequest(id, FinalizeResult{Err: err}) + l.resolveSaveRequest(id, FinalizeResult{Err: err}) continue } } if outcome == FinalizeNoOpenWindow && pcr.commitEmpty { - recorded, err := l.recordCommitRequest(pcr) + recorded, err := l.recordSaveRequest(pcr) if err != nil { // The request asked for its message to be recorded and it was not: a failure, not // a benign NoWindow that would report Ready=True for a save that left no trace. - l.resolveCommitRequest(id, FinalizeResult{Err: err}) + l.resolveSaveRequest(id, FinalizeResult{Err: err}) continue } if recorded { continue // it now rides the record write, and the push resolves it } } - l.resolveCommitRequest(id, FinalizeResult{Outcome: outcome}) + l.resolveSaveRequest(id, FinalizeResult{Outcome: outcome}) } } -// recordCommitRequest decides an empty commit recording a request's message, for a request that ran +// recordSaveRequest decides an empty commit recording a request's message, for a request that ran // out of time to attach and asked for whenNothingToCommit: CommitEmpty. Every reason no record can // be made is returned, including a write gate (an empty commit is a write too), so the request fails // instead of reporting a save it did not make. Once decided, the request rides the record write // like an attached request rides its window: a remote that cannot be reached holds it, and only a // failure of the commit itself fails it. -func (l *branchWorkerEventLoop) recordCommitRequest(pcr *pendingCommitRequest) (bool, error) { +func (l *branchWorkerEventLoop) recordSaveRequest(pcr *pendingSaveRequest) (bool, error) { pendingWrite, err := l.w.buildRequestRecordWrite(l.w.ctx, pcr) if err != nil { - l.w.Log.Error(err, "Cannot build the empty commit recording a CommitRequest", + l.w.Log.Error(err, "Cannot build the empty commit recording a SaveRequest", "request", pcr.id.Namespace+"/"+pcr.id.Name) return false, fmt.Errorf("record the message in an empty commit: %w", err) } @@ -422,7 +422,7 @@ func (l *branchWorkerEventLoop) recordCommitRequest(pcr *pendingCommitRequest) ( // gets none, and the refusal is returned (see write_gate.go). func (w *BranchWorker) buildRequestRecordWrite( ctx context.Context, - pcr *pendingCommitRequest, + pcr *pendingSaveRequest, ) (*PendingWrite, error) { metadata, err := w.resolveTargetMetadata(ctx, pcr.gitTargetName, pcr.gitTargetNamespace) if err != nil { @@ -453,41 +453,41 @@ func (w *BranchWorker) buildRequestRecordWrite( }, RequestAuthor: UserInfo{Username: pcr.author}, RequestAttribution: pcr.attribution, - CommitRequest: &id, + SaveRequest: &id, }, nil } -// handleWithdrawCommitRequest cancels a request the worker has not acted on: one that is only +// handleWithdrawSaveRequest cancels a request the worker has not acted on: one that is only // waiting for a window, or that was never registered at all. A request attached to a window or // already committed is held, and withdrawing it is a no-op; the controller reads the held phase and // keeps waiting. Either way the worker's answer and the controller's agree. -func (l *branchWorkerEventLoop) handleWithdrawCommitRequest(req *AttachCommitRequest) { +func (l *branchWorkerEventLoop) handleWithdrawSaveRequest(req *AttachSaveRequest) { id := req.id() - if l.w.hasCommitRequestOutcome(id) { + if l.w.hasSaveRequestOutcome(id) { return // already resolved, or already withdrawn. } if pcr, ok := l.pendingCRs[id]; ok && pcr.attached { - l.w.Log.Info("CommitRequest withdraw ignored: the worker already holds it", + l.w.Log.Info("SaveRequest withdraw ignored: the worker already holds it", "request", id.Namespace+"/"+id.Name) return } - l.resolveCommitRequest(id, FinalizeResult{Err: ErrCommitRequestWithdrawn}) + l.resolveSaveRequest(id, FinalizeResult{Err: ErrSaveRequestWithdrawn}) l.rearmAttachTimer() } -// resolveCommitRequest records a request's terminal outcome for the controller to +// resolveSaveRequest records a request's terminal outcome for the controller to // poll and forgets it from the pending set. -func (l *branchWorkerEventLoop) resolveCommitRequest(id commitRequestID, result FinalizeResult) { +func (l *branchWorkerEventLoop) resolveSaveRequest(id saveRequestID, result FinalizeResult) { if result.Branch == "" { result.Branch = l.w.Branch } - l.w.recordCommitRequestOutcome(id, result) + l.w.recordSaveRequestOutcome(id, result) if pcr := l.pendingCRs[id]; pcr != nil && pcr.catchUpNow { l.catchUpAfterSave(pcr, result.Commit) } delete(l.pendingCRs, id) - l.w.setCommitRequestPhase(id, "") - l.w.Log.Info("CommitRequest resolved", + l.w.setSaveRequestPhase(id, "") + l.w.Log.Info("SaveRequest resolved", "request", id.Namespace+"/"+id.Name, "outcome", string(result.Outcome), "commit", result.Commit, diff --git a/internal/git/commit_request_attach_test.go b/internal/git/save_request_attach_test.go similarity index 96% rename from internal/git/commit_request_attach_test.go rename to internal/git/save_request_attach_test.go index 75f09341..12214818 100644 --- a/internal/git/commit_request_attach_test.go +++ b/internal/git/save_request_attach_test.go @@ -23,7 +23,7 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// crName is the single CommitRequest name these tests register; each test uses one. +// crName is the single SaveRequest name these tests register; each test uses one. // crTarget is the GitTarget the worker serves (see createPlainGitTarget). const ( crName = "save" @@ -32,15 +32,15 @@ const ( // serviceAttach drives one attach + servicing pass on the loop, the way run() // does after dequeuing an attach work item. -func serviceAttach(loop *branchWorkerEventLoop, req *AttachCommitRequest) { - loop.handleAttachCommitRequest(req) +func serviceAttach(loop *branchWorkerEventLoop, req *AttachSaveRequest) { + loop.handleAttachSaveRequest(req) loop.endWake(0) } // attachReq builds a CurrentOrNext request that waits up to d for a window and, once attached, // collects for d with no idle close: the defaults' shape, at any length. -func attachReq(author string, d time.Duration) *AttachCommitRequest { - return &AttachCommitRequest{ +func attachReq(author string, d time.Duration) *AttachSaveRequest { + return &AttachSaveRequest{ Namespace: "default", Name: crName, UID: "uid-" + crName, @@ -57,7 +57,7 @@ func attachReq(author string, d time.Duration) *AttachCommitRequest { // waiting request's attach deadline passes, and an attached request's window reaches its // maxDuration and closes. func forceDue(loop *branchWorkerEventLoop) { - id := commitRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} + id := saveRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} past := time.Now().Add(-time.Millisecond) if pcr := loop.pendingCRs[id]; pcr != nil && !pcr.attached { pcr.attachDeadline = past @@ -71,7 +71,7 @@ func forceDue(loop *branchWorkerEventLoop) { func outcome(t *testing.T, w *BranchWorker) (FinalizeResult, bool) { t.Helper() - return w.LookupCommitRequestOutcome("default", crName, "uid-"+crName) + return w.LookupSaveRequestOutcome("default", crName, "uid-"+crName) } // TestEnqueueAttach_Success verifies an attach lands on the worker's event queue. @@ -246,7 +246,7 @@ func TestAttach_TheAttachStartsTheRequestsMaxDuration(t *testing.T) { const maxDuration = 60 * time.Second serviceAttach(loop, attachReq("alice", maxDuration)) - id := commitRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} + id := saveRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} // Almost all of the wait spent: the writes arrive with a second of it left. loop.pendingCRs[id].attachDeadline = time.Now().Add(time.Second) @@ -374,7 +374,7 @@ func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) require.NoError(t, err) assert.Equal(t, "jane@acme.com", commit.Author.Name, "the submitter authors the commit") - assert.Equal(t, "save: from the editor\n\nAuthor-Source: CommitRequest default/save; "+ + assert.Equal(t, "save: from the editor\n\nAuthor-Source: SaveRequest default/save; "+ "no audit record names who made these changes", commit.Message) } @@ -529,7 +529,7 @@ func TestAttach_ClaimedSameAuthorWindowIsNotAMismatch(t *testing.T) { second.UID = "uid-save-second" serviceAttach(loop, second) - pending := loop.pendingCRs[commitRequestID{Namespace: second.Namespace, Name: second.Name, UID: second.UID}] + pending := loop.pendingCRs[saveRequestID{Namespace: second.Namespace, Name: second.Name, UID: second.UID}] require.NotNil(t, pending) assert.False(t, pending.sawForeignWindow, "a window this author could have claimed is contention, never a mismatch") @@ -579,7 +579,7 @@ func TestAttach_IdempotentReSendKeepsFirstDeadline(t *testing.T) { defer loop.stopTimers() serviceAttach(loop, attachReq("alice", 60*time.Second)) - id := commitRequestID{Namespace: "default", Name: "save", UID: "uid-save"} + id := saveRequestID{Namespace: "default", Name: "save", UID: "uid-save"} require.Contains(t, loop.pendingCRs, id) firstDeadline := loop.pendingCRs[id].attachDeadline @@ -631,7 +631,7 @@ func TestAttach_AnUnreachableRemoteHoldsTheRequest(t *testing.T) { _, resolved := outcome(t, worker) assert.False(t, resolved, "a request whose write can still land is not failed") - assert.Equal(t, PhaseWaitingForPush, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForPush, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) assert.Nil(t, loop.openWindow, "the window closed on its own timers") require.Len(t, loop.pendingWrites, 1, "the decided window stays pending") assert.False(t, loop.pendingWrites[0].committedOnce, "nothing could commit it yet") @@ -679,7 +679,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) loop := newBranchWorkerEventLoop(worker, time.Hour) defer loop.stopTimers() - // First, commit the ConfigMap (no CommitRequest) and push it, so it is already + // First, commit the ConfigMap (no SaveRequest) and push it, so it is already // present in Git. loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("present", "alice", "team-a")}, @@ -691,7 +691,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) // Hold the push back, so the next assertion is about the finalize and nothing else. loop.lastPushAt = time.Now() - // A second window re-asserts the SAME object: no diff. Attach a CommitRequest and + // A second window re-asserts the SAME object: no diff. Attach a SaveRequest and // finalize it (maxDuration 0s). loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("present", "alice", "team-a")}, @@ -704,7 +704,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) "a no-diff finalize must NOT resolve before the remote has been consulted") require.Len(t, loop.pendingWrites, 1, "the no-diff write is retained so it reaches the push") require.True(t, loop.pendingWrites[0].CommitSHA.IsZero(), "no diff means no commit of its own") - require.NotNil(t, loop.pendingWrites[0].CommitRequest, "the request rides the retained write") + require.NotNil(t, loop.pendingWrites[0].SaveRequest, "the request rides the retained write") // The push reads the advertisement, finds the remote already holds this state, and that is // what settles the request. @@ -855,14 +855,14 @@ func TestAttributionOutcome_ZeroValueIsNotAttempted(t *testing.T) { } // TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems is the permanent regression test for -// the merge blocker: in the DEFAULT deployment no CommitRequest could attach to any window, so +// the merge blocker: in the DEFAULT deployment no SaveRequest could attach to any window, so // the user's commit message was silently dropped and the change landed under the generated // message with no error anywhere. // // The window's outcome and the request's outcome come from two subsystems configured by two // different flags (--author-attribution vs --admission-webhook, cmd/main.go:311-316), so this // walks every pair either side can actually produce and names the deployment that produces it. -// TestCommitRequest_OutcomesAgree in internal/controller pins that these are the real +// TestSaveRequest_OutcomesAgree in internal/controller pins that these are the real // values the two producers emit; here we pin what the matching rule does with them. func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { const otherAuthor = "bob" @@ -932,7 +932,7 @@ func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - p := &pendingCommitRequest{ + p := &pendingSaveRequest{ author: tc.reqAuth, attribution: tc.request, gitTargetName: crTarget, @@ -953,7 +953,7 @@ func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { // rework: no attribution pairing may let a request finalize another target's window. func TestMatchesWindow_GitTargetAlwaysScopes(t *testing.T) { for _, o := range []AttributionOutcome{AttributionNotAttempted, AttributionUnresolved, AttributionResolved} { - p := &pendingCommitRequest{attribution: o, gitTargetName: crTarget, gitTargetNamespace: "default"} + p := &pendingSaveRequest{attribution: o, gitTargetName: crTarget, gitTargetNamespace: "default"} assert.False(t, p.matchesWindow(&openWindow{ Attribution: o, GitTarget: "team-b", GitTargetNamespace: "default", }), "outcome %q must not match across GitTarget names", o) @@ -963,7 +963,7 @@ func TestMatchesWindow_GitTargetAlwaysScopes(t *testing.T) { } } -// The re-attach contract. A CommitRequest is settled by the push and by nothing else, so between +// The re-attach contract. A SaveRequest is settled by the push and by nothing else, so between // its window's finalize and that push it is neither pending nor resolved. The controller re-sends // its attach every couple of seconds throughout that gap, and these four tests pin what the worker // must do with those re-sends. @@ -988,11 +988,11 @@ func TestAttach_ReSentAttachDuringPushCooldownDoesNotResolve(t *testing.T) { serviceAttach(loop, attachReq("alice", 0)) require.Len(t, loop.pendingWrites, 1, "the window committed locally and is retained") - require.NotNil(t, loop.pendingWrites[0].CommitRequest, "the request rides that write") + require.NotNil(t, loop.pendingWrites[0].SaveRequest, "the request rides that write") _, resolved := outcome(t, worker) require.False(t, resolved, "nothing has reached the remote yet") - // The controller re-sends, twice, exactly as it does every commitRequestPollInterval while it + // The controller re-sends, twice, exactly as it does every saveRequestPollInterval while it // has no outcome to read. for range 2 { serviceAttach(loop, attachReq("alice", 0)) diff --git a/internal/git/commit_request_empty_test.go b/internal/git/save_request_empty_test.go similarity index 99% rename from internal/git/commit_request_empty_test.go rename to internal/git/save_request_empty_test.go index 9e78f3e0..f3142f55 100644 --- a/internal/git/commit_request_empty_test.go +++ b/internal/git/save_request_empty_test.go @@ -21,7 +21,7 @@ import ( // request that ends with nothing to commit records its message in an empty commit, the outcome // keeps the cause, and a window that belonged to someone else never falls back to one. -func commitEmptyReq(author, message string) *AttachCommitRequest { +func commitEmptyReq(author, message string) *AttachSaveRequest { req := attachReq(author, time.Hour) req.Message = message req.CommitEmpty = true @@ -244,7 +244,7 @@ func TestCommitEmpty_AFailedPushTreatsEveryCommitShapeAlike(t *testing.T) { _, resolved := outcome(t, worker) require.False(t, resolved, "the remote has neither accepted nor refused it") assert.Equal(t, PhaseWaitingForPush, - worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) require.Len(t, loop.pendingWrites, 1, "and the re-send made no second write") } diff --git a/internal/git/commit_request_owners.go b/internal/git/save_request_owners.go similarity index 80% rename from internal/git/commit_request_owners.go rename to internal/git/save_request_owners.go index 132b2131..25736d54 100644 --- a/internal/git/commit_request_owners.go +++ b/internal/git/save_request_owners.go @@ -4,7 +4,7 @@ package git import "sync" -// commitRequestOwners records which branch worker accepted each CommitRequest's attach, so the +// saveRequestOwners records which branch worker accepted each SaveRequest's attach, so the // controller's questions about a request reach the worker that can still act on it. // // The GitTarget cannot answer that. A worker serves every GitTarget on its (provider, branch), so it @@ -20,14 +20,14 @@ import "sync" // // It is shared by the manager and every worker it creates. The lock is a leaf: callers may hold // a worker's pendingResyncsMu or crOutcomesMu when they take it, never the other way round. -type commitRequestOwners struct { +type saveRequestOwners struct { mu sync.Mutex - owners map[commitRequestID]*BranchWorker + owners map[saveRequestID]*BranchWorker } // owner returns the worker holding the request, if any. A nil table (a worker built outside a // manager) knows no owners. -func (o *commitRequestOwners) owner(id commitRequestID) (*BranchWorker, bool) { +func (o *saveRequestOwners) owner(id saveRequestID) (*BranchWorker, bool) { if o == nil { return nil, false } @@ -39,7 +39,7 @@ func (o *commitRequestOwners) owner(id commitRequestID) (*BranchWorker, bool) { // claim makes w the request's owner, unless another worker already is. It reports whether w owns // the request afterwards. A nil table lets every worker claim. -func (o *commitRequestOwners) claim(id commitRequestID, w *BranchWorker) bool { +func (o *saveRequestOwners) claim(id saveRequestID, w *BranchWorker) bool { if o == nil { return true } @@ -49,14 +49,14 @@ func (o *commitRequestOwners) claim(id commitRequestID, w *BranchWorker) bool { return current == w } if o.owners == nil { - o.owners = map[commitRequestID]*BranchWorker{} + o.owners = map[saveRequestID]*BranchWorker{} } o.owners[id] = w return true } // release forgets the request, if w is still its owner. -func (o *commitRequestOwners) release(id commitRequestID, w *BranchWorker) { +func (o *saveRequestOwners) release(id saveRequestID, w *BranchWorker) { if o == nil { return } diff --git a/internal/git/commit_request_withdraw_test.go b/internal/git/save_request_withdraw_test.go similarity index 83% rename from internal/git/commit_request_withdraw_test.go rename to internal/git/save_request_withdraw_test.go index f2ffa423..95f6c1da 100644 --- a/internal/git/commit_request_withdraw_test.go +++ b/internal/git/save_request_withdraw_test.go @@ -2,7 +2,7 @@ package git -// The controller withdraws a CommitRequest before failing it closed. These pin the worker's half of +// The controller withdraws a SaveRequest before failing it closed. These pin the worker's half of // that agreement: a request the worker has not acted on is cancelled for good, and a request it // holds (attached, or committed) is not, so a request reported failed is never committed later. @@ -28,14 +28,14 @@ func TestWithdraw_AWaitingRequestIsCancelledForGood(t *testing.T) { req := attachReq("alice", time.Hour) serviceAttach(loop, req) - require.Equal(t, PhaseWaitingForWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + require.Equal(t, PhaseWaitingForWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) loop.handleQueueItem(WorkItem{Withdraw: req}) res, ok := outcome(t, worker) require.True(t, ok, "a withdrawn request resolves") - require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) - assert.Empty(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + assert.Empty(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) // A late re-send of the attach, and a window it would have attached to, change nothing. serviceAttach(loop, req) @@ -46,7 +46,7 @@ func TestWithdraw_AWaitingRequestIsCancelledForGood(t *testing.T) { assert.Nil(t, loop.openWindow.pendingCR, "a withdrawn request never attaches") assert.Empty(t, loop.pendingCRs) res, _ = outcome(t, worker) - require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) } // A withdraw that arrives before the worker ever saw the attach leaves a tombstone, so the attach @@ -62,7 +62,7 @@ func TestWithdraw_AnUnregisteredRequestCannotRegisterLater(t *testing.T) { res, ok := outcome(t, worker) require.True(t, ok) - require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) assert.Empty(t, loop.pendingCRs) } @@ -79,13 +79,13 @@ func TestWithdraw_AnAttachedRequestIsHeldAndStillCommits(t *testing.T) { }}) req := attachReq("alice", time.Hour) serviceAttach(loop, req) - require.Equal(t, PhaseCollectingWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + require.Equal(t, PhaseCollectingWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) loop.handleQueueItem(WorkItem{Withdraw: req}) _, resolved := outcome(t, worker) require.False(t, resolved, "a held request is not withdrawn") - assert.True(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName).Held()) + assert.True(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName).Held()) forceDue(loop) loop.pushPending() @@ -136,8 +136,8 @@ func TestWithdraw_AnOutcomeIsKeptWhileAnAttachForItIsQueued(t *testing.T) { worker.EnqueueAttach(req) // a late attach, still on the FIFO // The outcome runs past its TTL while the attach waits, and a GC pass runs. - ageCommitRequestOutcome(worker, req.id(), 2*commitRequestOutcomeTTL) - worker.recordCommitRequestOutcome(commitRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) + ageSaveRequestOutcome(worker, req.id(), 2*saveRequestOutcomeTTL) + worker.recordSaveRequestOutcome(saveRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) _, kept := outcome(t, worker) require.True(t, kept, "an outcome with an attach still queued is not collected") @@ -145,11 +145,11 @@ func TestWithdraw_AnOutcomeIsKeptWhileAnAttachForItIsQueued(t *testing.T) { loop.endWake(0) assert.Empty(t, loop.pendingCRs, "the late attach does not register the withdrawn request") res, _ := outcome(t, worker) - require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) // With nothing queued for it any more, the outcome is collected as usual. - ageCommitRequestOutcome(worker, req.id(), 2*commitRequestOutcomeTTL) - worker.recordCommitRequestOutcome(commitRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) + ageSaveRequestOutcome(worker, req.id(), 2*saveRequestOutcomeTTL) + worker.recordSaveRequestOutcome(saveRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) _, kept = outcome(t, worker) assert.False(t, kept) } @@ -162,7 +162,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { require.NoError(t, configv1alpha3.AddToScheme(scheme)) worker := NewBranchWorker(fake.NewClientBuilder().WithScheme(scheme).Build(), logr.Discard(), "no-such-provider", "default", "main", RepoIdentity{URL: "file:///nowhere"}, nil, BranchWorkerLimits{}) - owners := &commitRequestOwners{} + owners := &saveRequestOwners{} worker.crOwners = owners req := attachReq("alice", time.Hour) @@ -181,7 +181,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { worker.EnqueueWithdraw(req) res, resolved := outcome(t, worker) require.True(t, resolved, "a withdraw is answered at once") - require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) require.ErrorIs(t, res.Err, ErrBranchWorkerStopped) assert.Zero(t, worker.inflightItems.Load()) } @@ -192,7 +192,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { worker, _, _ := setupCommitPushSplitWorker(t) createPlainGitTarget(t, worker, "team-a", "team-a") - owners := &commitRequestOwners{} + owners := &saveRequestOwners{} worker.crOwners = owners require.NoError(t, worker.Start(context.Background())) @@ -200,7 +200,7 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { req := attachReq("alice", time.Hour) worker.EnqueueAttach(req) require.Eventually(t, func() bool { - return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName).Held() + return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName).Held() }, 5*time.Second, 10*time.Millisecond) worker.Stop() @@ -210,7 +210,7 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { assert.Same(t, worker, owner) res, resolved := outcome(t, worker) require.True(t, resolved, "its shutdown settled the request, by its last push or by failing it") - require.NotErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + require.NotErrorIs(t, res.Err, ErrSaveRequestWithdrawn) worker.EnqueueWithdraw(req) after, _ := outcome(t, worker) @@ -222,29 +222,29 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { func TestWithdraw_AStoppedWorkerGivesBackWhatItNeverActedOn(t *testing.T) { worker, _, _ := setupCommitPushSplitWorker(t) createPlainGitTarget(t, worker, "team-a", "team-a") - owners := &commitRequestOwners{} + owners := &saveRequestOwners{} worker.crOwners = owners require.NoError(t, worker.Start(context.Background())) req := attachReq("alice", time.Hour) worker.EnqueueAttach(req) require.Eventually(t, func() bool { - return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName) == PhaseWaitingForWindow + return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName) == PhaseWaitingForWindow }, 5*time.Second, 10*time.Millisecond) worker.Stop() _, owned := owners.owner(req.id()) assert.False(t, owned) - assert.Empty(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) + assert.Empty(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) _, resolved := outcome(t, worker) assert.False(t, resolved, "giving it back is not an outcome") } -func TestCommitRequestOwners_OneOwnerAtATime(t *testing.T) { +func TestSaveRequestOwners_OneOwnerAtATime(t *testing.T) { a, b := &BranchWorker{}, &BranchWorker{} - id := commitRequestID{Namespace: "default", Name: crName} - owners := &commitRequestOwners{} + id := saveRequestID{Namespace: "default", Name: crName} + owners := &saveRequestOwners{} require.True(t, owners.claim(id, a)) assert.True(t, owners.claim(id, a), "claiming again is idempotent") @@ -256,7 +256,7 @@ func TestCommitRequestOwners_OneOwnerAtATime(t *testing.T) { owners.release(id, a) assert.True(t, owners.claim(id, b)) - var none *commitRequestOwners + var none *saveRequestOwners assert.True(t, none.claim(id, a), "a worker outside a manager owns what it is sent") _, ok = none.owner(id) assert.False(t, ok) @@ -264,7 +264,7 @@ func TestCommitRequestOwners_OneOwnerAtATime(t *testing.T) { } func TestEnqueueAttach_AnotherWorkersRequestIsRefused(t *testing.T) { - owners := &commitRequestOwners{} + owners := &saveRequestOwners{} holder := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} other := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} req := attachReq("alice", 0) @@ -279,7 +279,7 @@ func TestEnqueueAttach_AnotherWorkersRequestIsRefused(t *testing.T) { // A dropped attach gives the request back unless the worker knows it otherwise. func TestEnqueueAttach_AQueueFullDropGivesTheRequestBack(t *testing.T) { - owners := &commitRequestOwners{} + owners := &saveRequestOwners{} w := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} w.eventQueue <- WorkItem{} // saturate @@ -291,7 +291,7 @@ func TestEnqueueAttach_AQueueFullDropGivesTheRequestBack(t *testing.T) { assert.Empty(t, w.crQueuedAttaches) } -func ageCommitRequestOutcome(w *BranchWorker, id commitRequestID, by time.Duration) { +func ageSaveRequestOutcome(w *BranchWorker, id saveRequestID, by time.Duration) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() entry := w.crOutcomes[id] @@ -299,10 +299,10 @@ func ageCommitRequestOutcome(w *BranchWorker, id commitRequestID, by time.Durati w.crOutcomes[id] = entry } -func TestCommitRequestPhase_Held(t *testing.T) { +func TestSaveRequestPhase_Held(t *testing.T) { assert.True(t, PhaseCollectingWindow.Held()) assert.True(t, PhaseWaitingForPush.Held()) assert.False(t, PhaseWaitingForWindow.Held()) assert.False(t, PhaseWaitingForWorker.Held()) - assert.False(t, CommitRequestPhase("").Held()) + assert.False(t, SaveRequestPhase("").Held()) } diff --git a/internal/git/types.go b/internal/git/types.go index 0c2edde0..9469e3ec 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -37,7 +37,7 @@ const ( // // Deliberately the EMPTY string so it is also the ZERO VALUE: the many paths that never assign // Attribution (reconcile, resync, bootstrap) all mean exactly this. Any other value makes the - // zero value a silent fourth state, which is the bug that stopped every CommitRequest + // zero value a silent fourth state, which is the bug that stopped every SaveRequest // attaching. TestAttributionZeroValueIsNotAttempted pins it. AttributionNotAttempted AttributionOutcome = "" // AttributionResolved means an audit fact named the actor. @@ -272,7 +272,7 @@ const ( // content-derived mark-and-sweep against the worktree (upsert every desired // resource, drop every watched managed document the snapshot did not contain). PendingWriteResync PendingWriteKind = "resync" - // PendingWriteRequestRecord carries no content. It records a CommitRequest's message + // PendingWriteRequestRecord carries no content. It records a SaveRequest's message // in an empty commit when the request ran out of time to attach, and asked for that with // whenNothingToCommit: CommitEmpty. It is not a window: only writes open windows. PendingWriteRequestRecord PendingWriteKind = "request_record" @@ -376,7 +376,7 @@ type PendingWrite struct { Committed *bool // AllowEmpty makes a grouped window commit even when its events changed nothing, because the - // CommitRequest attached to it asked for whenNothingToCommit: CommitEmpty. A suspended target + // SaveRequest attached to it asked for whenNothingToCommit: CommitEmpty. A suspended target // still writes nothing. AllowEmpty bool // RequestAuthor and RequestAttribution author a PendingWriteRequestRecord, which has no @@ -384,11 +384,11 @@ type PendingWrite struct { // window's author is. RequestAuthor UserInfo RequestAttribution AttributionOutcome - // TakenOver says the CommitRequest attached to this grouped write took over a window whose + // TakenOver says the SaveRequest attached to this grouped write took over a window whose // writes named no author: RequestAuthor and RequestAttribution, the submitter's, author the // commit, and its message ends with an Author-Source trailer (withTakeOverNote). TakenOver bool - // SavedBy is the submitter of the CommitRequest that gave this grouped write its message without + // SavedBy is the submitter of the SaveRequest that gave this grouped write its message without // claiming its writes, which named no author: the commit stays unattributed, and its message ends // with a Saved-By trailer naming them (withSavedByNote). Zero when no request did. SavedBy UserInfo @@ -398,10 +398,10 @@ type PendingWrite struct { // message". Re-stamped on a rebase-replay, where the same write may now change files. emptyCommitted bool - // CommitRequest, when set, is the CommitRequest attached to this write: it is + // SaveRequest, when set, is the SaveRequest attached to this write: it is // resolved Committed (with CommitSHA) once this write is pushed. It rides the write through the // push cooldown and the conflict rebase-replay, so the result follows the data. - CommitRequest *commitRequestID + SaveRequest *saveRequestID // committedMessageSource is the message source this write actually committed under, stamped by // executePendingWrites the way CommitSHA is and for the same reason: publishCommitsForPush runs // after the push, and recomputing the source from the write cannot know that a requestTemplate @@ -411,7 +411,7 @@ type PendingWrite struct { // actionsDecided is set once the write was planned on the tree it was first committed on. A // write carries the actions that plan decided, never a picture of the cluster to plan again // (docs/design/applier-echoes.md, F2): from then on Events holds only the objects whose plan - // changed that tree, except in a save (CommitRequest), which keeps every object it captured, + // changed that tree, except in a save (SaveRequest), which keeps every object it captured, // and a resync carries resyncActions instead of its snapshot. A replay applies those actions to // the newer tip, and plans nothing else again. Loop-goroutine only. actionsDecided bool @@ -454,18 +454,18 @@ type PendingWrite struct { type WorkItem struct { // Request is a resource-write request. Request *WriteRequest - // Attach is a CommitRequest attach: bind a message to the author's window and + // Attach is a SaveRequest attach: bind a message to the author's window and // let its timers close that window. - Attach *AttachCommitRequest + Attach *AttachSaveRequest // Resync is the FIFO position of a streaming-snapshot resync request (M8): a synchronous // request/reply that materialises a GitTarget's complete desired set. Resync *resyncMarker // Refresh asks the worker to re-prove where its branch is on the remote. It is the only // work item that never writes anything: see RefreshRequest. Refresh *RefreshRequest - // Withdraw cancels a CommitRequest the worker has not acted on yet. It rides the same FIFO as + // Withdraw cancels a SaveRequest the worker has not acted on yet. It rides the same FIFO as // the attach, so it is always handled after every attach sent before it. - Withdraw *AttachCommitRequest + Withdraw *AttachSaveRequest // ApplyBarrier is one answer to the barrier an apply window took (apply_barrier.go). A stream's // answer rides the FIFO behind every event that stream routed before it. ApplyBarrier *ApplyBarrier @@ -753,7 +753,7 @@ type Event struct { UserInfo UserInfo // Attribution is the authority for author rendering, the author_kind metric, and - // CommitRequest window matching. None may infer the outcome from UserInfo: an empty username + // SaveRequest window matching. None may infer the outcome from UserInfo: an empty username // cannot distinguish "attribution is off" from "it ran and found nothing". attachAuthor is // the only assignment outside tests. Attribution AttributionOutcome @@ -832,7 +832,7 @@ type CommitterConfig struct { type CommitMessageConfig struct { LiveTemplate string ReconcileTemplate string - // RequestTemplate frames a CommitRequest's message. Empty — the default — commits that + // RequestTemplate frames a SaveRequest's message. Empty — the default — commits that // message verbatim, which is the behaviour every target had before the field existed. RequestTemplate string } @@ -992,7 +992,7 @@ type LiveCommitMessageData struct { // Resources is the per-resource list, deduplicated by file path so the // final state is what's being committed. Resources []ResourceRef - // RequestMessage is the message an attached CommitRequest supplied, unaltered. It is empty for + // RequestMessage is the message an attached SaveRequest supplied, unaltered. It is empty for // every window no request attached to, which is the ordinary case and why liveTemplate may // reference it freely. // diff --git a/internal/git/worker_manager.go b/internal/git/worker_manager.go index 16d49f00..a0630cc8 100644 --- a/internal/git/worker_manager.go +++ b/internal/git/worker_manager.go @@ -104,11 +104,11 @@ type WorkerManager struct { // the scrape goroutine. Take it BEFORE mu, never the other way round. lifecycleMu sync.Mutex workers map[BranchKey]*BranchWorker - // commitRequests records which worker holds each CommitRequest. It outlives the map above on + // saveRequests records which worker holds each SaveRequest. It outlives the map above on // purpose: a retired worker leaves the map before it stops, and keeps answering for the - // requests it held. See commitRequestOwners. - commitRequests *commitRequestOwners - ctx context.Context + // requests it held. See saveRequestOwners. + saveRequests *saveRequestOwners + ctx context.Context // mapper is the GVK->GVR resolver injected into every worker so store scans build a // resource-identity inventory. It is set once at startup (SetMapper) before any // worker is created; a nil mapper keeps workers structure-only. It is the LOCAL cluster's @@ -223,7 +223,7 @@ func NewWorkerManager( limits: limits.withDefaults(), sensitiveResources: sensitiveResources, workers: make(map[BranchKey]*BranchWorker), - commitRequests: &commitRequestOwners{}, + saveRequests: &saveRequestOwners{}, remotes: make(map[BranchKey]RemoteObservation), replacements: make(map[BranchKey]struct{}), renderFidelityGate: NewRenderFidelityGate(), @@ -538,7 +538,7 @@ func (m *WorkerManager) startWorkerLocked(key BranchKey, req WorkerRequest, gitB worker.replacementPending = func() bool { return m.ReplacementPending(key) } worker.applierAdmitted = m.applierAdmitted worker.renderFidelityGate = m.renderFidelityGate - worker.crOwners = m.commitRequests + worker.crOwners = m.saveRequests if err := worker.Start(m.ctx); err != nil { return fmt.Errorf("failed to start worker for %s: %w", key.String(), err) @@ -685,7 +685,7 @@ func (m *WorkerManager) removeWorkersLocked(keys []BranchKey, reason string) { // RetireGitBranch stops the worker writing for gitBranch, if one is live, and returns once it has // stopped. Stopping closes the worker's intake, pushes or refuses every pending write and settles -// every CommitRequest it holds, so a GitBranch being deleted calls this before its finalizer goes. +// every SaveRequest it holds, so a GitBranch being deleted calls this before its finalizer goes. // What the manager kept about the branch goes with it: the next holder starts from the remote. func (m *WorkerManager) RetireGitBranch(gitBranch k8stypes.NamespacedName) { m.lifecycleMu.Lock() @@ -736,12 +736,12 @@ func (m *WorkerManager) WorkerForTarget(target *configv1alpha3.GitTarget) (*Bran return m.WorkerForGitBranch(target.GitBranchKey()) } -// CommitRequestOwner returns the worker that holds a CommitRequest: the one that accepted its +// SaveRequestOwner returns the worker that holds a SaveRequest: the one that accepted its // attach and has not given it back. It is found whatever has happened to the request's GitTarget // since, including a worker retired and still finishing its last push, so a caller that finds no // owner knows that no worker can still commit the request. -func (m *WorkerManager) CommitRequestOwner(namespace, name, uid string) (*BranchWorker, bool) { - return m.commitRequests.owner(commitRequestID{Namespace: namespace, Name: name, UID: uid}) +func (m *WorkerManager) SaveRequestOwner(namespace, name, uid string) (*BranchWorker, bool) { + return m.saveRequests.owner(saveRequestID{Namespace: namespace, Name: name, UID: uid}) } // afterOrphanSelection runs between choosing the workers to retire and retiring them. It is nil in diff --git a/internal/git/worker_repo_identity_test.go b/internal/git/worker_repo_identity_test.go index 11ab2fde..29180177 100644 --- a/internal/git/worker_repo_identity_test.go +++ b/internal/git/worker_repo_identity_test.go @@ -261,7 +261,7 @@ func TestEnqueue_RefusesOnceTheWorkerIsStopping(t *testing.T) { "and a resync must be answered rather than left waiting on a reply that never comes") // The two fire-and-forget paths report nothing, so they must at least leave no trace: an // increment here would be work the depth gauge counts and nothing will ever finish. - worker.EnqueueAttach(&AttachCommitRequest{Namespace: "ns", Name: "cr"}) + worker.EnqueueAttach(&AttachSaveRequest{Namespace: "ns", Name: "cr"}) worker.EnqueueRefresh(&RefreshRequest{}) assert.Equal(t, inflight, worker.inflightItems.Load(), diff --git a/internal/git/write_gate.go b/internal/git/write_gate.go index f849c7f6..5baea32f 100644 --- a/internal/git/write_gate.go +++ b/internal/git/write_gate.go @@ -9,12 +9,12 @@ import ( // The write gates are the two reasons a GitTarget may not be written at all: spec.suspend, and a // render-fidelity gate that has not established render-vs-live for it. They are rules about the -// TARGET, and every path that decides a CommitRequest's outcome asks targetWriteRefusal before it +// TARGET, and every path that decides a SaveRequest's outcome asks targetWriteRefusal before it // reports one, so a save on a target that refuses writes fails with the reason instead of resolving // Ready=True with nothing written. That holds whichever path ends the request and whatever its // whenNothingToCommit says: // -// - a request no window reached (expireWaitingCommitRequests, before the record it may make); +// - a request no window reached (expireWaitingSaveRequests, before the record it may make); // - the record itself (buildRequestRecordWrite); // - a window a request is attached to (finalizeOpenWindowWithReason before it builds the write, // and gateBuiltWindow after, right before the commit). A render-fidelity refusal seen at @@ -59,7 +59,7 @@ func (w *BranchWorker) targetWriteRefusal(name, namespace string, suspended bool // spec.suspend from the live GitTarget. Only evidence refuses: a target that cannot be read is // left to the path that writes, which fails on it, and a request that writes nothing (Resolve) // keeps its answer. -func (w *BranchWorker) requestWriteRefusal(ctx context.Context, pcr *pendingCommitRequest) error { +func (w *BranchWorker) requestWriteRefusal(ctx context.Context, pcr *pendingSaveRequest) error { suspended := false if target, err := w.getGitTarget(ctx, pcr.gitTargetName, pcr.gitTargetNamespace); err == nil { suspended = target.Spec.Suspend diff --git a/internal/git/write_gate_test.go b/internal/git/write_gate_test.go index 49f01db2..3f33d4e8 100644 --- a/internal/git/write_gate_test.go +++ b/internal/git/write_gate_test.go @@ -21,7 +21,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/types" ) -// These tests pin the write gates every CommitRequest outcome passes (write_gate.go): a request +// These tests pin the write gates every SaveRequest outcome passes (write_gate.go): a request // that ends on a target that refuses writes fails with the reason, whatever path it ends on and // whatever whenNothingToCommit says. They drive the transitions — a gate that closes after the // attach, after the local commit, on a sibling target, across a failed push — and not only the @@ -46,7 +46,7 @@ func closeRenderGate(worker *BranchWorker) { } // requestFor is a request named name, for target, by alice. -func requestFor(name, target string, commitEmpty bool) *AttachCommitRequest { +func requestFor(name, target string, commitEmpty bool) *AttachSaveRequest { req := attachReq("alice", time.Hour) req.Name, req.UID = name, "uid-"+name req.GitTargetName = target @@ -127,17 +127,17 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { endings := []struct { name string // run closes the gate at the point this ending puts it, and ends the request. - run func(t *testing.T, loop *branchWorkerEventLoop, closeGate func(), req *AttachCommitRequest) + run func(t *testing.T, loop *branchWorkerEventLoop, closeGate func(), req *AttachSaveRequest) }{ {"no window before the attach timeout", func(_ *testing.T, loop *branchWorkerEventLoop, closeGate func(), - req *AttachCommitRequest) { + req *AttachSaveRequest) { closeGate() serviceAttach(loop, req) expire(loop) loop.pushPending() }}, {"attached, the gate closing before the window closes", func(t *testing.T, loop *branchWorkerEventLoop, - closeGate func(), req *AttachCommitRequest) { + closeGate func(), req *AttachSaveRequest) { serviceAttach(loop, req) writeTo(loop, "new") require.NotNil(t, loop.openWindow, "the request is attached to an open window") @@ -146,7 +146,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { loop.pushPending() }}, {"attached to a window that changed nothing", func(t *testing.T, loop *branchWorkerEventLoop, - closeGate func(), req *AttachCommitRequest) { + closeGate func(), req *AttachSaveRequest) { writeTo(loop, "existing") serviceAttach(loop, req) require.NotNil(t, loop.openWindow) @@ -170,7 +170,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { ending.run(t, loop, func() { gate.close(t, worker) }, req) - res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.True(t, ok, "the request is resolved") require.ErrorIs(t, res.Err, gate.want, "a refused save is a failure, never Ready=True") assert.Empty(t, res.Commit) @@ -183,7 +183,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { } // forceDueNamed closes req's attached window at its maxDuration, as forceDue does for crName. -func forceDueNamed(loop *branchWorkerEventLoop, req *AttachCommitRequest) { +func forceDueNamed(loop *branchWorkerEventLoop, req *AttachSaveRequest) { id := req.id() if loop.openWindow != nil && loop.openWindow.pendingCR != nil && *loop.openWindow.pendingCR == id { loop.openWindow.timers.maxAt = time.Now().Add(-time.Millisecond) @@ -207,7 +207,7 @@ func TestWriteGate_ASaveWhoseWritesTheRenderGateDroppedSaysSo(t *testing.T) { expire(loop) loop.pushPending() - res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.ErrorIs(t, res.Err, errRenderFidelityClosed) assert.Equal(t, before, remoteHead(t, serverRepo), "no empty commit claims the save saw nothing") @@ -227,15 +227,15 @@ func TestWriteGate_AGateClosingAfterTheLocalCommitDoesNotRetractIt(t *testing.T) } commits := []struct { name string - commit func(loop *branchWorkerEventLoop, req *AttachCommitRequest) + commit func(loop *branchWorkerEventLoop, req *AttachSaveRequest) outcome FinalizeOutcome }{ - {"a window that changed files", func(loop *branchWorkerEventLoop, req *AttachCommitRequest) { + {"a window that changed files", func(loop *branchWorkerEventLoop, req *AttachSaveRequest) { serviceAttach(loop, req) writeTo(loop, "new") forceDueNamed(loop, req) }, FinalizeCommitted}, - {"the record of a request no window reached", func(loop *branchWorkerEventLoop, req *AttachCommitRequest) { + {"the record of a request no window reached", func(loop *branchWorkerEventLoop, req *AttachSaveRequest) { serviceAttach(loop, req) expire(loop) }, FinalizeNoOpenWindow}, @@ -255,12 +255,12 @@ func TestWriteGate_AGateClosingAfterTheLocalCommitDoesNotRetractIt(t *testing.T) gate.close(t, worker) loop.pushPending() - _, resolved := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + _, resolved := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.False(t, resolved, "the gate closing does not resolve a request still waiting for its push") restore() loop.pushPending() - res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.NoError(t, res.Err) assert.Equal(t, c.outcome, res.Outcome) @@ -299,11 +299,11 @@ func TestWriteGate_SiblingTargetsAreGatedApart(t *testing.T) { expire(loop) loop.pushPending() - res, ok := worker.LookupCommitRequestOutcome("default", refused.Name, refused.UID) + res, ok := worker.LookupSaveRequestOutcome("default", refused.Name, refused.UID) require.True(t, ok) require.ErrorIs(t, res.Err, gate.want) - res, ok = worker.LookupCommitRequestOutcome("default", served.Name, served.UID) + res, ok = worker.LookupSaveRequestOutcome("default", served.Name, served.UID) require.True(t, ok) require.NoError(t, res.Err, "a sibling's gate is not this target's") assert.Equal(t, FinalizeNoOpenWindow, res.Outcome) @@ -324,7 +324,7 @@ func TestWriteGate_AWindowMismatchIsStillAMismatch(t *testing.T) { writeTo(loop, "alices-edit") expire(loop) - res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.NoError(t, res.Err) assert.Equal(t, FinalizeWindowMismatch, res.Outcome) @@ -376,7 +376,7 @@ func TestWriteGate_ARenderGateClosingDuringFinalizeStopsTheWholeWindow(t *testin assert.Empty(t, loop.pendingWrites, "nothing is retained") assert.Equal(t, before, remoteHead(t, serverRepo), "nothing reached the remote") if withRequest { - res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.ErrorIs(t, res.Err, errRenderFidelityClosed) } diff --git a/internal/telemetry/exporter.go b/internal/telemetry/exporter.go index 7bde4072..7a362919 100644 --- a/internal/telemetry/exporter.go +++ b/internal/telemetry/exporter.go @@ -44,7 +44,7 @@ var ( // GitTarget: a resync can apply every document and then abort on a precondition, writing // nothing at all. GitDocumentsTotal metric.Int64Counter - // CommitRequestsTotal counts CommitRequests reaching a terminal outcome, labelled by + // SaveRequestsTotal counts SaveRequests reaching a terminal outcome, labelled by // {outcome, gittarget_namespace, gittarget_name}. outcome is committed / no_window / // window_mismatch / already_present / failed. // @@ -58,13 +58,13 @@ var ( // // An UNRESOLVED target publishes the empty pair rather than the requested name. spec.gitTargetRef // is whatever the client wrote, so labelling it would let anyone with create rights on - // commitrequests mint unbounded series by naming targets that do not exist; the requested + // saverequests mint unbounded series by naming targets that do not exist; the requested // reference stays on the object's conditions and in the logs. // // It is the fleet view resource_condition deliberately refuses to give this kind. A - // CommitRequest is created once per save, so a gauge per object would churn a series + // SaveRequest is created once per save, so a gauge per object would churn a series // continuously; a counter over terminal outcomes is bounded by the enum instead. The pointer - // that used to stand in for it, git_commits_total{message_source="commit_request"}, counts + // that used to stand in for it, git_commits_total{message_source="save_request"}, counts // only the requests that succeeded AND were used, so the whole failure population had no // series at all — and this is the one object a human waits on synchronously. A request that // does not land still produces a green, pushed commit carrying a GENERATED message instead of @@ -75,19 +75,19 @@ var ( // // COUNTING SEMANTICS, because this counter cannot promise what it looks like it promises. It // counts one increment per TERMINAL DECISION ATTEMPT, recorded outside the status-write retry - // loop. That is not once per CommitRequest: a status write that fails permanently is requeued + // loop. That is not once per SaveRequest: a status write that fails permanently is requeued // and re-decides, and restart recovery re-reconciles any request whose terminal status never - // persisted (documented at CommitRequestReconciler.SetupWithManager as a knowingly-accepted + // persisted (documented at SaveRequestReconciler.SetupWithManager as a knowingly-accepted // gap). Deduplicating across invocations needs a durable marker on the object, which is an API // change rather than an instrument. Read rates and ratios from it, not exact request counts. - CommitRequestsTotal metric.Int64Counter + SaveRequestsTotal metric.Int64Counter // GitCommitsTotal counts commit batches that REACHED THE REMOTE, labelled by the recording // BranchWorker's identity {gitbranch_namespace, gitbranch_name, provider_namespace, // provider_name} (gitbranch_* and provider_* below) plus author_kind and // message_source. Live, snapshot and resync paths all feed this one counter; message_source - // (live / reconcile / commit_request / commit_request_framed / commit_request_fallback) is what + // (live / reconcile / save_request / save_request_framed / save_request_fallback) is what // tells them apart. It counts commits that USED a - // request-supplied message, not CommitRequests: a request omitting spec.message renders through + // request-supplied message, not SaveRequests: a request omitting spec.message renders through // liveTemplate and counts as live. // // It is recorded on a successful push, not on local commit creation. Recorded at creation it @@ -186,7 +186,7 @@ var ( // // This is the head-of-line signal, and it is a proven failure rather than a theoretical one: a // slow attribution resolution blocks the events queued BEHIND it on the same single-threaded - // stream, which broke a CommitRequest e2e spec and was only ever visible by correlating two + // stream, which broke a SaveRequest e2e spec and was only ever visible by correlating two // log lines by hand. The attribution wait histogram times each resolution in isolation and // cannot see the delay one imposes on its neighbours; occupancy can, because a stream that is // busy is a stream nothing else is being read from. @@ -257,7 +257,7 @@ var ( // (idle_timeout, max_duration, attach_next, identity_change, buffer_limit, resync_before_apply, // shutdown, foreign_commit for a Git host's push notice, apply_due for the // flush before an apply); timer_source says whose timers were in effect: the GitTarget's, - // or a CommitRequest's that attached and replaced them. Every window is counted exactly once, at + // or a SaveRequest's that attached and replaced them. Every window is counted exactly once, at // the start of its finalize, so a window that then produces no diff or fails to commit is counted // too. A push replay rebuilds commits and closes no window, and a CommitEmpty record has none. // @@ -580,7 +580,7 @@ func registerCounters() error { counters := []cSpec{ {"gitopsreverser_git_documents_total", &GitDocumentsTotal}, {"gitopsreverser_git_commits_total", &GitCommitsTotal}, - {"gitopsreverser_commit_requests_total", &CommitRequestsTotal}, + {"gitopsreverser_save_requests_total", &SaveRequestsTotal}, {"gitopsreverser_git_commit_failures_total", &GitCommitFailuresTotal}, {"gitopsreverser_git_writes_dropped_total", &GitWritesDroppedTotal}, {"gitopsreverser_git_commit_windows_total", &GitCommitWindowsTotal}, diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index f6b2187a..d8057cd2 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -73,8 +73,8 @@ func NewEventRouter( } } -// ServiceCommitRequest is the controller's attach-then-poll seam (§6.4.3): it -// resolves the worker that holds the request, or else the GitTarget's branch worker, registers the CommitRequest attach +// ServiceSaveRequest is the controller's attach-then-poll seam (§6.4.3): it +// resolves the worker that holds the request, or else the GitTarget's branch worker, registers the SaveRequest attach // idempotently on that worker's FIFO event queue (attach it to the author's window // with its message and timers), and returns the request's current // outcome. resolved=false means the worker has not finished — the controller @@ -88,16 +88,16 @@ func NewEventRouter( // It is reported as pending and NOT as an error, because the GitTarget did resolve: the controller // reads a service error as "the target never resolved" and would count the request's eventual // failure without its GitTarget labels. A GitTarget that cannot be read is that error. -func (r *EventRouter) ServiceCommitRequest( +func (r *EventRouter) ServiceSaveRequest( ctx context.Context, - attach git.AttachCommitRequest, + attach git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { - worker, branch, err := r.commitRequestWorker(ctx, attach) + worker, branch, err := r.saveRequestWorker(ctx, attach) if err != nil { return git.FinalizeResult{}, false, err } if worker == nil { - r.Log.V(1).Info("ServiceCommitRequest: no branch worker for the GitTarget yet; will retry", + r.Log.V(1).Info("ServiceSaveRequest: no branch worker for the GitTarget yet; will retry", "gitTarget", attach.GitTargetNamespace+"/"+attach.GitTargetName) return git.FinalizeResult{Branch: branch, Phase: git.PhaseWaitingForWorker}, false, nil } @@ -105,17 +105,17 @@ func (r *EventRouter) ServiceCommitRequest( // Idempotent register (the worker keys by request identity and keeps the first // finalize deadline), then poll the outcome. worker.EnqueueAttach(&attach) - result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID) + result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID) if !resolved { // Report where the worker says the request stands, so the controller never has to infer // a phase from having sent the attach. - result.Phase = worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) + result.Phase = worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) result.Held = heldBecause(worker, result.Phase) } return result, resolved, nil } -// WithdrawCommitRequest is the controller's fail-closed seam: it cancels a request the worker has +// WithdrawSaveRequest is the controller's fail-closed seam: it cancels a request the worker has // not acted on, and reports the worker's answer, so the controller never fails a request the worker // could still commit. // @@ -126,14 +126,14 @@ func (r *EventRouter) ServiceCommitRequest( // - Nothing holds it, and the GitTarget is gone or names no worker: nothing can commit the // request, so it resolves at once as withdrawn. The controller sends no attach after it starts // withdrawing, so a worker that starts later never sees it. -func (r *EventRouter) WithdrawCommitRequest( +func (r *EventRouter) WithdrawSaveRequest( ctx context.Context, - attach git.AttachCommitRequest, + attach git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { - worker, branch, err := r.commitRequestWorker(ctx, attach) + worker, branch, err := r.saveRequestWorker(ctx, attach) if errors.Is(err, errGitTargetGone) { // No worker holds the request, and its GitTarget can never start one. - return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil + return git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, true, nil } if err != nil { return git.FinalizeResult{}, false, err @@ -142,32 +142,32 @@ func (r *EventRouter) WithdrawCommitRequest( return git.FinalizeResult{ Branch: branch, Phase: git.PhaseWaitingForWorker, - Err: git.ErrCommitRequestWithdrawn, + Err: git.ErrSaveRequestWithdrawn, }, true, nil } - if result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { + if result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { return result, true, nil } - phase := worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) + phase := worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) if phase.Held() { return git.FinalizeResult{Branch: branch, Phase: phase, Held: heldBecause(worker, phase)}, false, nil } worker.EnqueueWithdraw(&attach) // A worker whose loop has exited answers the withdraw at once. - if result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { + if result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { return result, true, nil } return git.FinalizeResult{Branch: branch, Phase: phase}, false, nil } -// LookupCommitRequestApply reports where the apply a CommitRequest asked for stands, from the +// LookupSaveRequestApply reports where the apply a SaveRequest asked for stands, from the // branch worker of its GitTarget, and false while that worker knows nothing of it or none runs. -func (r *EventRouter) LookupCommitRequestApply( +func (r *EventRouter) LookupSaveRequestApply( ctx context.Context, - attach git.AttachCommitRequest, + attach git.AttachSaveRequest, ) (git.SaveApply, bool, error) { - worker, _, err := r.commitRequestWorker(ctx, attach) + worker, _, err := r.saveRequestWorker(ctx, attach) if errors.Is(err, errGitTargetGone) { return git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyApplierGone, Message: "the GitTarget is gone, so its applier is asked for nothing"}, true, nil @@ -181,24 +181,24 @@ func (r *EventRouter) LookupCommitRequestApply( // heldBecause is why a request waiting for its push has not reached the remote: the worker's // publication report while it cannot publish, empty otherwise. -func heldBecause(worker *git.BranchWorker, phase git.CommitRequestPhase) string { +func heldBecause(worker *git.BranchWorker, phase git.SaveRequestPhase) string { if phase != git.PhaseWaitingForPush { return "" } return worker.Publication().Message() } -// commitRequestWorker finds the worker to ask about a CommitRequest, and the branch it serves. +// saveRequestWorker finds the worker to ask about a SaveRequest, and the branch it serves. // // The worker that holds the request comes first, whatever its GitTarget says now: a shared worker // outlives a deleted target, and a retired one finishes its last push after the manager stopped // listing it. Only a request no worker holds falls back to the GitTarget's current worker, which is // nil while that worker has not started. A GitTarget that is gone is errGitTargetGone. -func (r *EventRouter) commitRequestWorker( +func (r *EventRouter) saveRequestWorker( ctx context.Context, - attach git.AttachCommitRequest, + attach git.AttachSaveRequest, ) (*git.BranchWorker, string, error) { - if owner, ok := r.WorkerManager.CommitRequestOwner(attach.Namespace, attach.Name, attach.UID); ok { + if owner, ok := r.WorkerManager.SaveRequestOwner(attach.Namespace, attach.Name, attach.UID); ok { return owner, owner.Branch, nil } diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 03ad9df5..79559442 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -32,8 +32,8 @@ func eventRouterScheme(t *testing.T) *runtime.Scheme { return scheme } -func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachCommitRequest { - return git.AttachCommitRequest{ +func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachSaveRequest { + return git.AttachSaveRequest{ Namespace: gitTargetNamespace, Name: "save", UID: "uid-save", @@ -43,7 +43,7 @@ func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachCommitReques } } -func TestServiceCommitRequest_GitTargetNotFound(t *testing.T) { +func TestServiceSaveRequest_GitTargetNotFound(t *testing.T) { scheme := eventRouterScheme(t) client := fake.NewClientBuilder().WithScheme(scheme).Build() workerManager := git.NewWorkerManager( @@ -55,7 +55,7 @@ func TestServiceCommitRequest_GitTargetNotFound(t *testing.T) { router := NewEventRouter(workerManager, nil, client, logr.Discard()) - _, resolved, err := router.ServiceCommitRequest(context.Background(), saveAttach("missing", "team-a")) + _, resolved, err := router.ServiceSaveRequest(context.Background(), saveAttach("missing", "team-a")) require.Error(t, err) assert.False(t, resolved) require.ErrorIs(t, err, errGitTargetGone) @@ -63,7 +63,7 @@ func TestServiceCommitRequest_GitTargetNotFound(t *testing.T) { // A save whose GitTarget is gone needs no apply: Applied=True with ApplierGone. Only a commit the // branch lost is ever Applied=False. -func TestLookupCommitRequestApply_GitTargetGoneIsApplied(t *testing.T) { +func TestLookupSaveRequestApply_GitTargetGoneIsApplied(t *testing.T) { scheme := eventRouterScheme(t) client := fake.NewClientBuilder().WithScheme(scheme).Build() workerManager := git.NewWorkerManager( @@ -74,7 +74,7 @@ func TestLookupCommitRequestApply_GitTargetGoneIsApplied(t *testing.T) { ) router := NewEventRouter(workerManager, nil, client, logr.Discard()) - apply, known, err := router.LookupCommitRequestApply(context.Background(), saveAttach("missing", "team-a")) + apply, known, err := router.LookupSaveRequestApply(context.Background(), saveAttach("missing", "team-a")) require.NoError(t, err) require.True(t, known) assert.True(t, apply.Done) @@ -314,11 +314,11 @@ func TestGitPathRefusalReason(t *testing.T) { } } -// TestServiceCommitRequest_NoWorkerKeepsTheRequestPending pins that a missing worker is not an -// outcome. At startup a CommitRequest can reach the controller before its GitTarget's worker exists; +// TestServiceSaveRequest_NoWorkerKeepsTheRequestPending pins that a missing worker is not an +// outcome. At startup a SaveRequest can reach the controller before its GitTarget's worker exists; // resolving it then would end the save before anything could collect its writes or record its // message, so it stays pending and the controller polls within its safety window. -func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { +func TestServiceSaveRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { scheme := eventRouterScheme(t) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, @@ -336,7 +336,7 @@ func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { router := NewEventRouter(workerManager, nil, client, logr.Discard()) - result, resolved, err := router.ServiceCommitRequest(context.Background(), saveAttach("team-a-config", "team-a")) + result, resolved, err := router.ServiceSaveRequest(context.Background(), saveAttach("team-a-config", "team-a")) require.NoError(t, err, "the GitTarget resolved; only its worker is missing") assert.False(t, resolved, "a worker that does not exist yet is a retry, not an answer") assert.Empty(t, result.Outcome) @@ -344,7 +344,7 @@ func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { assert.Equal(t, "main", result.Branch) } -func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { +func TestServiceSaveRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -387,13 +387,13 @@ func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) // No events routed and attachTimeout 0s, so the worker has no window: the attach // is enqueued, processed by the worker loop, and resolves NoOpenWindow. The - // controller polls via repeated (idempotent) ServiceCommitRequest calls. + // controller polls via repeated (idempotent) ServiceSaveRequest calls. attach := saveAttach("team-a-config", "team-a") var result git.FinalizeResult require.Eventually(t, func() bool { var resolved bool var err error - result, resolved, err = router.ServiceCommitRequest(context.Background(), attach) + result, resolved, err = router.ServiceSaveRequest(context.Background(), attach) require.NoError(t, err) return resolved }, 5*time.Second, 50*time.Millisecond, "the worker must resolve the attached request") @@ -401,10 +401,10 @@ func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) assert.Equal(t, "main", result.Branch) } -// TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker: a deleted GitTarget, or one whose worker +// TestWithdrawSaveRequest_NothingHoldsItWithoutAWorker: a deleted GitTarget, or one whose worker // never started, has nothing that could commit the request, so the withdraw resolves at once and // says why when it can. -func TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker(t *testing.T) { +func TestWithdrawSaveRequest_NothingHoldsItWithoutAWorker(t *testing.T) { scheme := eventRouterScheme(t) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, @@ -417,24 +417,24 @@ func TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker(t *testing.T) { client, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) router := NewEventRouter(workerManager, nil, client, logr.Discard()) - result, resolved, err := router.WithdrawCommitRequest(context.Background(), saveAttach("team-a-config", "team-a")) + result, resolved, err := router.WithdrawSaveRequest(context.Background(), saveAttach("team-a-config", "team-a")) require.NoError(t, err) require.True(t, resolved) - require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) assert.Equal(t, git.PhaseWaitingForWorker, result.Phase) assert.Equal(t, "main", result.Branch) - result, resolved, err = router.WithdrawCommitRequest(context.Background(), saveAttach("deleted", "team-b")) + result, resolved, err = router.WithdrawSaveRequest(context.Background(), saveAttach("deleted", "team-b")) require.NoError(t, err) require.True(t, resolved) - require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) assert.Empty(t, result.Branch, "a deleted GitTarget has no identity to report") } -// TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest: the withdraw rides the FIFO behind the +// TestWithdrawSaveRequest_ALiveWorkerCancelsAWaitingRequest: the withdraw rides the FIFO behind the // attach, cancels the request while it only waits for a window, and an attach sent afterwards // cannot bring it back. -func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { +func TestWithdrawSaveRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -473,29 +473,29 @@ func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { attach := saveAttach("team-a-config", "team-a") attach.AttachTimeout = time.Hour // it would wait an hour for a window - _, resolved, err := router.ServiceCommitRequest(context.Background(), attach) + _, resolved, err := router.ServiceSaveRequest(context.Background(), attach) require.NoError(t, err) require.False(t, resolved) var result git.FinalizeResult require.Eventually(t, func() bool { var err error - result, resolved, err = router.WithdrawCommitRequest(context.Background(), attach) + result, resolved, err = router.WithdrawSaveRequest(context.Background(), attach) require.NoError(t, err) return resolved }, 5*time.Second, 50*time.Millisecond, "the worker must answer the withdraw") - require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) - result, resolved, err = router.ServiceCommitRequest(context.Background(), attach) + result, resolved, err = router.ServiceSaveRequest(context.Background(), attach) require.NoError(t, err) require.True(t, resolved, "a re-sent attach finds the withdrawn outcome") - require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) } -// TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers: a worker is shared by every +// TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers: a worker is shared by every // GitTarget on its branch, so it outlives a deleted one. The withdraw must reach that worker, and // must never report success while the request is still registered there. -func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) { +func TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -535,10 +535,10 @@ func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T router := NewEventRouter(workerManager, nil, client, logr.Discard()) attach := saveAttach("team-a-config", "team-a") attach.AttachTimeout = time.Hour - _, _, err := router.ServiceCommitRequest(context.Background(), attach) + _, _, err := router.ServiceSaveRequest(context.Background(), attach) require.NoError(t, err) require.Eventually(t, func() bool { - return worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) == git.PhaseWaitingForWindow + return worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) == git.PhaseWaitingForWindow }, 5*time.Second, 10*time.Millisecond) require.NoError(t, client.Delete(context.Background(), gitTarget)) @@ -546,15 +546,15 @@ func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T var result git.FinalizeResult require.Eventually(t, func() bool { var resolved bool - result, resolved, err = router.WithdrawCommitRequest(context.Background(), attach) + result, resolved, err = router.WithdrawSaveRequest(context.Background(), attach) require.NoError(t, err) if resolved { - require.Empty(t, worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID), + require.Empty(t, worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID), "the withdraw reported success while the worker still held the registration") } return resolved }, 5*time.Second, 50*time.Millisecond) - require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) assert.Equal(t, "main", result.Branch, "the worker answered, not the missing GitTarget") } diff --git a/internal/watch/save_requests.go b/internal/watch/remote_save_requests.go similarity index 68% rename from internal/watch/save_requests.go rename to internal/watch/remote_save_requests.go index da78bba7..73f04ba8 100644 --- a/internal/watch/save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -29,10 +29,10 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/types" ) -// A SaveRequest is a save made on a source cluster, next to the edits it saves +// A RemoteSaveRequest is a save made on a source cluster, next to the edits it saves // (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose -// spec.saveRequests is "Enabled", through that provider's identity, and serves each one the way the -// CommitRequest controller serves a CommitRequest: it attaches the request to the commit window of +// spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way the +// SaveRequest controller serves a SaveRequest: it attaches the request to the commit window of // every GitTarget that mirrors the request's namespace from that cluster, polls the branch workers // for the outcomes, and writes them back to the request's status on the source cluster. // @@ -43,35 +43,35 @@ import ( // missing; it is refused rather than attached, so it cannot claim the window that should stay // unattributed. -// +kubebuilder:rbac:groups=source.configbutler.ai,resources=saverequests,verbs=get;list;watch;delete -// +kubebuilder:rbac:groups=source.configbutler.ai,resources=saverequests/status,verbs=get;update +// +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests,verbs=get;list;watch;delete +// +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests/status,verbs=get;update -// saveRequestGVR() is the served resource of a SaveRequest. -func saveRequestGVR() schema.GroupVersionResource { - return sourcev1alpha1.GroupVersion.WithResource("saverequests") +// remoteSaveRequestGVR() is the served resource of a RemoteSaveRequest. +func remoteSaveRequestGVR() schema.GroupVersionResource { + return sourcev1alpha1.GroupVersion.WithResource("remotesaverequests") } const ( - // saveRequestProviderResync is how often the server looks for KubeProviders to serve. - saveRequestProviderResync = 10 * time.Second - // saveRequestReconnect is the wait before a watch of a cluster's SaveRequests opens again. - saveRequestReconnect = 10 * time.Second - // saveRequestRelist ends a watch this often, so finished requests past their TTL are deleted. - saveRequestRelist = 5 * time.Minute - // saveRequestPoll is how often a request's outcome is asked for. - saveRequestPoll = time.Second - // saveRequestPushAllowance is what a request may take beyond its own waits: the push, its + // remoteSaveRequestProviderResync is how often the server looks for KubeProviders to serve. + remoteSaveRequestProviderResync = 10 * time.Second + // remoteSaveRequestReconnect is the wait before a watch of a cluster's RemoteSaveRequests opens again. + remoteSaveRequestReconnect = 10 * time.Second + // remoteSaveRequestRelist ends a watch this often, so finished requests past their TTL are deleted. + remoteSaveRequestRelist = 5 * time.Minute + // remoteSaveRequestPoll is how often a request's outcome is asked for. + remoteSaveRequestPoll = time.Second + // remoteSaveRequestPushAllowance is what a request may take beyond its own waits: the push, its // cooldown and a retry. Past it, a request no worker holds is withdrawn. - saveRequestPushAllowance = 2 * time.Minute - // saveRequestApplyLost is how long a save's apply may stay unknown to its worker before it is + remoteSaveRequestPushAllowance = 2 * time.Minute + // remoteSaveRequestApplyLost is how long a save's apply may stay unknown to its worker before it is // reported lost: GitOps Reverser restarted, or the record aged out. - saveRequestApplyLost = time.Minute + remoteSaveRequestApplyLost = time.Minute // defaultSaveAttachTimeout and defaultSaveMaxDuration are the window when a request sets none. defaultSaveAttachTimeout = 5 * time.Second defaultSaveMaxDuration = 2 * time.Second ) -// SaveRequest reasons, on the Ready condition and each target. +// RemoteSaveRequest reasons, on the Ready condition and each target. const ( saveReasonCommitted = "Committed" saveReasonNoWindow = "NoWindow" @@ -85,17 +85,17 @@ const ( saveReasonApplied = "Applied" ) -// SaveFinalizer is what serves a save once it is attached: the CommitRequest service of the event +// SaveFinalizer is what serves a save once it is attached: the SaveRequest service of the event // router, which owns the branch workers. type SaveFinalizer interface { - ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) + ServiceSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) + WithdrawSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) + LookupSaveRequestApply(ctx context.Context, attach git.AttachSaveRequest) (git.SaveApply, bool, error) } -// SaveRequestServer serves the SaveRequests of every KubeProvider that enables them. It runs on the +// RemoteSaveRequestServer serves the RemoteSaveRequests of every KubeProvider that enables them. It runs on the // leader only, as every status writer does. -type SaveRequestServer struct { +type RemoteSaveRequestServer struct { // Manager is the watch manager: the source clusters' clients, their attribution, and which // GitTargets mirror which namespace. Manager *Manager @@ -111,11 +111,11 @@ type SaveRequestServer struct { dynamicClient func(ctx context.Context, provider k8stypes.NamespacedName) (dynamic.Interface, error) mu sync.Mutex - watchers map[k8stypes.NamespacedName]*saveRequestWatcher + watchers map[k8stypes.NamespacedName]*remoteSaveRequestWatcher } -// saveRequestWatcher is the watch of one KubeProvider's SaveRequests. -type saveRequestWatcher struct { +// remoteSaveRequestWatcher is the watch of one KubeProvider's RemoteSaveRequests. +type remoteSaveRequestWatcher struct { provider k8stypes.NamespacedName route string cancel context.CancelFunc @@ -124,8 +124,8 @@ type saveRequestWatcher struct { } // Start runs the server until ctx ends. -func (s *SaveRequestServer) Start(ctx context.Context) error { - ticker := time.NewTicker(saveRequestProviderResync) +func (s *RemoteSaveRequestServer) Start(ctx context.Context) error { + ticker := time.NewTicker(remoteSaveRequestProviderResync) defer ticker.Stop() for { s.syncProviders(ctx) @@ -138,24 +138,24 @@ func (s *SaveRequestServer) Start(ctx context.Context) error { } } -// syncProviders runs a watcher for every KubeProvider that enables SaveRequests, and stops the rest. -func (s *SaveRequestServer) syncProviders(ctx context.Context) { +// syncProviders runs a watcher for every KubeProvider that enables RemoteSaveRequests, and stops the rest. +func (s *RemoteSaveRequestServer) syncProviders(ctx context.Context) { var providers configv1alpha3.KubeProviderList if err := s.Client.List(ctx, &providers); err != nil { - s.Log.V(1).Info("Cannot list KubeProviders for SaveRequests; will retry", "err", err.Error()) + s.Log.V(1).Info("Cannot list KubeProviders for RemoteSaveRequests; will retry", "err", err.Error()) return } want := map[k8stypes.NamespacedName]string{} for i := range providers.Items { provider := &providers.Items[i] - if provider.DeletionTimestamp == nil && provider.Spec.SaveRequests == configv1alpha3.SaveRequestsEnabled { + if provider.DeletionTimestamp == nil && provider.Spec.RemoteSaveRequests == configv1alpha3.RemoteSaveRequestsEnabled { want[provider.Key()] = provider.AuditRoute() } } s.mu.Lock() defer s.mu.Unlock() if s.watchers == nil { - s.watchers = map[k8stypes.NamespacedName]*saveRequestWatcher{} + s.watchers = map[k8stypes.NamespacedName]*remoteSaveRequestWatcher{} } for key, watcher := range s.watchers { if route, ok := want[key]; !ok || route != watcher.route { @@ -168,13 +168,13 @@ func (s *SaveRequestServer) syncProviders(ctx context.Context) { continue } watchCtx, cancel := context.WithCancel(ctx) - watcher := &saveRequestWatcher{provider: key, route: route, cancel: cancel} + watcher := &remoteSaveRequestWatcher{provider: key, route: route, cancel: cancel} s.watchers[key] = watcher go s.run(watchCtx, watcher) } } -func (s *SaveRequestServer) stopAll() { +func (s *RemoteSaveRequestServer) stopAll() { s.mu.Lock() defer s.mu.Unlock() for key, watcher := range s.watchers { @@ -183,47 +183,47 @@ func (s *SaveRequestServer) stopAll() { } } -// run watches one KubeProvider's SaveRequests until its context ends, reconnecting after a failure. -func (s *SaveRequestServer) run(ctx context.Context, watcher *saveRequestWatcher) { +// run watches one KubeProvider's RemoteSaveRequests until its context ends, reconnecting after a failure. +func (s *RemoteSaveRequestServer) run(ctx context.Context, watcher *remoteSaveRequestWatcher) { log := s.Log.WithValues("kubeProvider", watcher.provider.String()) - log.Info("Serving the SaveRequests of a KubeProvider") + log.Info("Serving the RemoteSaveRequests of a KubeProvider") if s.Manager != nil && s.Manager.FactStreams != nil { release := s.Manager.FactStreams.Acquire( - queue.FactStreamKeyFor(watcher.route, saveRequestGVR().GroupResource()), + queue.FactStreamKeyFor(watcher.route, remoteSaveRequestGVR().GroupResource()), ) defer release() } for ctx.Err() == nil { if err := s.session(ctx, log, watcher); err != nil && ctx.Err() == nil { - log.Info("The watch of SaveRequests ended; reconnecting", "err", err.Error()) + log.Info("The watch of RemoteSaveRequests ended; reconnecting", "err", err.Error()) } select { case <-ctx.Done(): - case <-time.After(saveRequestReconnect): + case <-time.After(remoteSaveRequestReconnect): } } } -// session lists the cluster's SaveRequests and watches them, until the watch ends. -func (s *SaveRequestServer) session(ctx context.Context, log logr.Logger, watcher *saveRequestWatcher) error { +// session lists the cluster's RemoteSaveRequests and watches them, until the watch ends. +func (s *RemoteSaveRequestServer) session(ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher) error { dc, err := s.clusterClient(ctx, watcher.provider) if err != nil { return err } - resource := dc.Resource(saveRequestGVR()) + resource := dc.Resource(remoteSaveRequestGVR()) list, err := resource.List(ctx, metav1.ListOptions{}) if err != nil { - return fmt.Errorf("list SaveRequests: %w", err) + return fmt.Errorf("list RemoteSaveRequests: %w", err) } for i := range list.Items { s.consider(ctx, log, watcher, resource, &list.Items[i]) } - timeout := int64(saveRequestRelist / time.Second) + timeout := int64(remoteSaveRequestRelist / time.Second) w, err := resource.Watch(ctx, metav1.ListOptions{ ResourceVersion: list.GetResourceVersion(), AllowWatchBookmarks: true, TimeoutSeconds: &timeout, }) if err != nil { - return fmt.Errorf("watch SaveRequests: %w", err) + return fmt.Errorf("watch RemoteSaveRequests: %w", err) } defer w.Stop() for { @@ -240,14 +240,14 @@ func (s *SaveRequestServer) session(ctx context.Context, log logr.Logger, watche s.consider(ctx, log, watcher, resource, u) } case k8swatch.Error: - return fmt.Errorf("watch SaveRequests: %w", apierrors.FromObject(ev.Object)) + return fmt.Errorf("watch RemoteSaveRequests: %w", apierrors.FromObject(ev.Object)) case k8swatch.Deleted, k8swatch.Bookmark: } } } } -func (s *SaveRequestServer) clusterClient( +func (s *RemoteSaveRequestServer) clusterClient( ctx context.Context, provider k8stypes.NamespacedName, ) (dynamic.Interface, error) { @@ -259,21 +259,21 @@ func (s *SaveRequestServer) clusterClient( // consider serves a request that has not finished and is not served yet, and deletes one that // finished longer than the TTL ago. -func (s *SaveRequestServer) consider( - ctx context.Context, log logr.Logger, watcher *saveRequestWatcher, resource dynamic.NamespaceableResourceInterface, +func (s *RemoteSaveRequestServer) consider( + ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher, resource dynamic.NamespaceableResourceInterface, u *unstructured.Unstructured, ) { - var request sourcev1alpha1.SaveRequest + var request sourcev1alpha1.RemoteSaveRequest if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, &request); err != nil { - log.Info("Cannot read a SaveRequest", "saveRequest", u.GetNamespace()+"/"+u.GetName(), "err", err.Error()) + log.Info("Cannot read a RemoteSaveRequest", "remoteSaveRequest", u.GetNamespace()+"/"+u.GetName(), "err", err.Error()) return } - if finishedAt, finished := saveRequestFinished(&request); finished { + if finishedAt, finished := remoteSaveRequestFinished(&request); finished { if s.TTL > 0 && time.Since(finishedAt) > s.TTL && request.DeletionTimestamp == nil { if err := resource.Namespace(request.Namespace).Delete(ctx, request.Name, metav1.DeleteOptions{ Preconditions: &metav1.Preconditions{UID: &request.UID}, }); err != nil && !apierrors.IsNotFound(err) { - log.V(1).Info("Cannot delete a finished SaveRequest", "saveRequest", request.Namespace+"/"+request.Name, + log.V(1).Info("Cannot delete a finished RemoteSaveRequest", "remoteSaveRequest", request.Namespace+"/"+request.Name, "err", err.Error()) } } @@ -291,9 +291,9 @@ func (s *SaveRequestServer) consider( }() } -// saveRequestFinished reports whether a request has an outcome that changes no more, and since when: +// remoteSaveRequestFinished reports whether a request has an outcome that changes no more, and since when: // Ready decided, and, for one that asked to catch up and committed, Applied decided too. -func saveRequestFinished(request *sourcev1alpha1.SaveRequest) (time.Time, bool) { +func remoteSaveRequestFinished(request *sourcev1alpha1.RemoteSaveRequest) (time.Time, bool) { ready := apimeta.FindStatusCondition(request.Status.Conditions, "Ready") if ready == nil || ready.Status == metav1.ConditionUnknown { return time.Time{}, false @@ -311,15 +311,15 @@ func saveRequestFinished(request *sourcev1alpha1.SaveRequest) (time.Time, bool) // serve takes one request from attribution to its outcome, and to its apply when it asked to catch // up, writing each step to its status. -func (s *SaveRequestServer) serve( - ctx context.Context, log logr.Logger, watcher *saveRequestWatcher, resource dynamic.ResourceInterface, - request *sourcev1alpha1.SaveRequest, +func (s *RemoteSaveRequestServer) serve( + ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher, resource dynamic.ResourceInterface, + request *sourcev1alpha1.RemoteSaveRequest, ) { - log = log.WithValues("saveRequest", request.Namespace+"/"+request.Name) + log = log.WithValues("remoteSaveRequest", request.Namespace+"/"+request.Name) status := request.Status.DeepCopy() write := func() { - if err := writeSaveRequestStatus(ctx, resource, request, status); err != nil { - log.Info("Cannot write the SaveRequest's status", "err", err.Error()) + if err := writeRemoteSaveRequestStatus(ctx, resource, request, status); err != nil { + log.Info("Cannot write the RemoteSaveRequest's status", "err", err.Error()) } } if err := git.ValidateLiteralCommitMessage(request.Spec.Message); err != nil { @@ -337,13 +337,13 @@ func (s *SaveRequestServer) serve( "no audit event of this cluster names who created this request, though the cluster sends audit "+ "events: its event went missing, so it saves nothing. Save again") write() - log.Info("Refused a SaveRequest whose author no audit event names, on a cluster that sends them") + log.Info("Refused a RemoteSaveRequest whose author no audit event names, on a cluster that sends them") return } targets, err := s.mirroringTargets(ctx, watcher.provider, request.Namespace) if err != nil { - log.Info("Cannot find the GitTargets that mirror a SaveRequest's namespace; will retry", "err", err.Error()) + log.Info("Cannot find the GitTargets that mirror a RemoteSaveRequest's namespace; will retry", "err", err.Error()) return } if len(targets) == 0 { @@ -352,11 +352,11 @@ func (s *SaveRequestServer) serve( write() return } - status.Targets = make([]sourcev1alpha1.SaveRequestTarget, len(targets)) - attaches := make([]git.AttachCommitRequest, len(targets)) + status.Targets = make([]sourcev1alpha1.RemoteSaveRequestTarget, len(targets)) + attaches := make([]git.AttachSaveRequest, len(targets)) for i, target := range targets { - status.Targets[i] = sourcev1alpha1.SaveRequestTarget{GitTarget: target.String()} - attaches[i] = saveRequestAttach(request, target, author.Username, attribution) + status.Targets[i] = sourcev1alpha1.RemoteSaveRequestTarget{GitTarget: target.String()} + attaches[i] = remoteSaveRequestAttach(request, target, author.Username, attribution) } setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, "waiting for the edits to reach a commit window") @@ -364,7 +364,7 @@ func (s *SaveRequestServer) serve( s.finalize(ctx, request, status, attaches) write() - log.Info("SaveRequest saved", "reason", apimeta.FindStatusCondition(status.Conditions, "Ready").Reason, + log.Info("RemoteSaveRequest saved", "reason", apimeta.FindStatusCondition(status.Conditions, "Ready").Reason, "author", status.Author) if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && readyReason(status) == saveReasonCommitted { @@ -374,15 +374,15 @@ func (s *SaveRequestServer) serve( // resolveSaveAuthor finds the request's creator from the source cluster's audit facts, as a watched // edit's author is found. -func (s *SaveRequestServer) resolveSaveAuthor( - ctx context.Context, route string, request *sourcev1alpha1.SaveRequest, +func (s *RemoteSaveRequestServer) resolveSaveAuthor( + ctx context.Context, route string, request *sourcev1alpha1.RemoteSaveRequest, ) (git.UserInfo, git.AttributionOutcome) { if s.Manager == nil || s.Manager.AuthorResolver == nil { return git.UserInfo{}, git.AttributionNotAttempted } return s.Manager.AuthorResolver.ResolveAuthor(ctx, AuthorQuery{ AuditRoute: route, - GVR: saveRequestGVR(), + GVR: remoteSaveRequestGVR(), UID: request.UID, ResourceVersion: request.ResourceVersion, Namespace: request.Namespace, @@ -394,7 +394,7 @@ func (s *SaveRequestServer) resolveSaveAuthor( // sentAuditFacts reports whether the KubeProvider has ever received an audit fact: its // AuditFactsReceived condition is True. A provider that cannot be read says no. -func (s *SaveRequestServer) sentAuditFacts(ctx context.Context, provider k8stypes.NamespacedName) bool { +func (s *RemoteSaveRequestServer) sentAuditFacts(ctx context.Context, provider k8stypes.NamespacedName) bool { var kp configv1alpha3.KubeProvider if err := s.Client.Get(ctx, provider, &kp); err != nil { return false @@ -404,7 +404,7 @@ func (s *SaveRequestServer) sentAuditFacts(ctx context.Context, provider k8stype // mirroringTargets are the GitTargets that mirror namespace from the KubeProvider: they name it, and // one of their running streams watches that namespace, or every namespace. -func (s *SaveRequestServer) mirroringTargets( +func (s *RemoteSaveRequestServer) mirroringTargets( ctx context.Context, provider k8stypes.NamespacedName, namespace string, ) ([]types.ResourceReference, error) { var targets configv1alpha3.GitTargetList @@ -438,13 +438,13 @@ func (m *Manager) mirrorsNamespace(gitDest types.ResourceReference, namespace st return false } -// saveRequestAttach is the attach of a request to one GitTarget's window. Its identity carries the GitTarget, +// remoteSaveRequestAttach is the attach of a request to one GitTarget's window. Its identity carries the GitTarget, // so one request saving several GitTargets on one branch worker is several requests there. -func saveRequestAttach( - request *sourcev1alpha1.SaveRequest, target types.ResourceReference, author string, +func remoteSaveRequestAttach( + request *sourcev1alpha1.RemoteSaveRequest, target types.ResourceReference, author string, attribution git.AttributionOutcome, -) git.AttachCommitRequest { - attach := git.AttachCommitRequest{ +) git.AttachSaveRequest { + attach := git.AttachSaveRequest{ Namespace: request.Namespace, Name: request.Name, UID: string(request.UID) + "@" + target.String(), @@ -475,20 +475,20 @@ func saveRequestAttach( // finalize polls every GitTarget's outcome until each one resolved, withdrawing those still unheld past // the request's bound, and records them on status. -func (s *SaveRequestServer) finalize( - ctx context.Context, request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, - attaches []git.AttachCommitRequest, +func (s *RemoteSaveRequestServer) finalize( + ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, + attaches []git.AttachSaveRequest, ) { - bound := time.Now().Add(attaches[0].AttachTimeout + attaches[0].MaxDuration + saveRequestPushAllowance) + bound := time.Now().Add(attaches[0].AttachTimeout + attaches[0].MaxDuration + remoteSaveRequestPushAllowance) resolved := make([]bool, len(attaches)) for remaining := len(attaches); remaining > 0; { for i := range attaches { if resolved[i] { continue } - ask := s.Finalizer.ServiceCommitRequest + ask := s.Finalizer.ServiceSaveRequest if time.Now().After(bound) { - ask = s.Finalizer.WithdrawCommitRequest + ask = s.Finalizer.WithdrawSaveRequest } result, done, err := ask(ctx, attaches[i]) if err != nil || !done { @@ -504,18 +504,18 @@ func (s *SaveRequestServer) finalize( select { case <-ctx.Done(): return - case <-time.After(saveRequestPoll): + case <-time.After(remoteSaveRequestPoll): } } setSaveReady(request, status) } // recordSaveOutcome records one GitTarget's outcome on its target status. -func recordSaveOutcome(target *sourcev1alpha1.SaveRequestTarget, result git.FinalizeResult) { +func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestTarget, result git.FinalizeResult) { target.Branch = result.Branch if result.Err != nil { target.Reason, target.Message = saveReasonFailed, result.Err.Error() - if errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { + if errors.Is(result.Err, git.ErrSaveRequestWithdrawn) { target.Message = "the save was not served in time and was withdrawn" } return @@ -540,7 +540,7 @@ func recordSaveOutcome(target *sourcev1alpha1.SaveRequestTarget, result git.Fina // setSaveReady sets Ready from the targets' outcomes: a failure in any fails the request, a commit in // any is Committed, and otherwise the first target's reason says why nothing was committed. -func setSaveReady(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus) { +func setSaveReady(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { for _, target := range status.Targets { if target.Reason == saveReasonFailed { message := target.GitTarget + ": " + target.Message @@ -560,20 +560,20 @@ func setSaveReady(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.Sa } // followApplies polls the apply each committed save asked for until every one is settled, and records -// them; a save its worker knows nothing of for saveRequestApplyLost is lost. -func (s *SaveRequestServer) followApplies( - ctx context.Context, request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, - attaches []git.AttachCommitRequest, write func(), +// them; a save its worker knows nothing of for remoteSaveRequestApplyLost is lost. +func (s *RemoteSaveRequestServer) followApplies( + ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, + attaches []git.AttachSaveRequest, write func(), ) { setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, "waiting for the appliers to apply the commit") write() started := time.Now() - for s.pollApplies(ctx, status, attaches, time.Since(started) > saveRequestApplyLost) { + for s.pollApplies(ctx, status, attaches, time.Since(started) > remoteSaveRequestApplyLost) { select { case <-ctx.Done(): return - case <-time.After(saveRequestPoll): + case <-time.After(remoteSaveRequestPoll): } } setSaveApplied(request, status) @@ -582,8 +582,8 @@ func (s *SaveRequestServer) followApplies( // pollApplies asks once for the apply of every committed target not settled yet, records it, and // reports whether one is still pending. Once lost, a save its worker knows nothing of is lost. -func (s *SaveRequestServer) pollApplies( - ctx context.Context, status *sourcev1alpha1.SaveRequestStatus, attaches []git.AttachCommitRequest, lost bool, +func (s *RemoteSaveRequestServer) pollApplies( + ctx context.Context, status *sourcev1alpha1.RemoteSaveRequestStatus, attaches []git.AttachSaveRequest, lost bool, ) bool { pending := false for i := range attaches { @@ -591,7 +591,7 @@ func (s *SaveRequestServer) pollApplies( if target.Reason != saveReasonCommitted || saveApplySettled(target.Applied) { continue } - apply, known, err := s.Finalizer.LookupCommitRequestApply(ctx, attaches[i]) + apply, known, err := s.Finalizer.LookupSaveRequestApply(ctx, attaches[i]) switch { case err == nil && known: target.Applied, target.AppliedRevision = apply.Reason, apply.AppliedRevision @@ -617,7 +617,7 @@ func saveApplySettled(reason string) bool { // setSaveApplied sets Applied from the committed targets' applies: False only for a commit a branch // lost, Unknown when one was lost track of, and True otherwise. -func setSaveApplied(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus) { +func setSaveApplied(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { reason, condition := saveReasonApplied, metav1.ConditionTrue message := "every applier applied a revision that holds the commit, or needed no apply" for _, target := range status.Targets { @@ -634,7 +634,7 @@ func setSaveApplied(request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1. setSaveCondition(request, status, "Applied", condition, reason, message) } -func readyReason(status *sourcev1alpha1.SaveRequestStatus) string { +func readyReason(status *sourcev1alpha1.RemoteSaveRequestStatus) string { if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil { return ready.Reason } @@ -642,7 +642,7 @@ func readyReason(status *sourcev1alpha1.SaveRequestStatus) string { } func setSaveCondition( - request *sourcev1alpha1.SaveRequest, status *sourcev1alpha1.SaveRequestStatus, conditionType string, + request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, conditionType string, condition metav1.ConditionStatus, reason, message string, ) { status.ObservedGeneration = request.Generation @@ -652,10 +652,10 @@ func setSaveCondition( }) } -// writeSaveRequestStatus writes status onto the request's latest version, retrying a conflict. -func writeSaveRequestStatus( - ctx context.Context, resource dynamic.ResourceInterface, request *sourcev1alpha1.SaveRequest, - status *sourcev1alpha1.SaveRequestStatus, +// writeRemoteSaveRequestStatus writes status onto the request's latest version, retrying a conflict. +func writeRemoteSaveRequestStatus( + ctx context.Context, resource dynamic.ResourceInterface, request *sourcev1alpha1.RemoteSaveRequest, + status *sourcev1alpha1.RemoteSaveRequestStatus, ) error { fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(status) if err != nil { @@ -668,7 +668,7 @@ func writeSaveRequestStatus( return err } if current.GetUID() != request.UID { - return fmt.Errorf("SaveRequest %s/%s was replaced", request.Namespace, request.Name) + return fmt.Errorf("RemoteSaveRequest %s/%s was replaced", request.Namespace, request.Name) } if err := unstructured.SetNestedMap(current.Object, fields, "status"); err != nil { return err diff --git a/internal/watch/save_requests_test.go b/internal/watch/remote_save_requests_test.go similarity index 76% rename from internal/watch/save_requests_test.go rename to internal/watch/remote_save_requests_test.go index a6f3e68f..3518c3d7 100644 --- a/internal/watch/save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -34,11 +34,11 @@ type saveFinalizer struct { mu sync.Mutex result git.FinalizeResult apply git.SaveApply - attaches []git.AttachCommitRequest + attaches []git.AttachSaveRequest } -func (f *saveFinalizer) ServiceCommitRequest( - _ context.Context, attach git.AttachCommitRequest, +func (f *saveFinalizer) ServiceSaveRequest( + _ context.Context, attach git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { f.mu.Lock() defer f.mu.Unlock() @@ -46,14 +46,14 @@ func (f *saveFinalizer) ServiceCommitRequest( return f.result, true, nil } -func (f *saveFinalizer) WithdrawCommitRequest( - _ context.Context, _ git.AttachCommitRequest, +func (f *saveFinalizer) WithdrawSaveRequest( + _ context.Context, _ git.AttachSaveRequest, ) (git.FinalizeResult, bool, error) { - return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil + return git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, true, nil } -func (f *saveFinalizer) LookupCommitRequestApply( - _ context.Context, _ git.AttachCommitRequest, +func (f *saveFinalizer) LookupSaveRequestApply( + _ context.Context, _ git.AttachSaveRequest, ) (git.SaveApply, bool, error) { return f.apply, true, nil } @@ -71,11 +71,11 @@ func (a fixedAuthor) ResolveAuthor(context.Context, AuthorQuery) (git.UserInfo, var saveProvider = k8stypes.NamespacedName{Namespace: "providers", Name: "workspace"} // saveServer is a server for the KubeProvider providers/workspace, whose GitTarget team-a/app mirrors -// the source namespace "shop" when mirrored is set, with a SaveRequest shop/save on the source cluster. +// the source namespace "shop" when mirrored is set, with a RemoteSaveRequest shop/save on the source cluster. func saveServer( - t *testing.T, request *sourcev1alpha1.SaveRequest, mirrored bool, auditFacts metav1.ConditionStatus, + t *testing.T, request *sourcev1alpha1.RemoteSaveRequest, mirrored bool, auditFacts metav1.ConditionStatus, author AuthorResolver, finalizer *saveFinalizer, -) (*SaveRequestServer, dynamic.ResourceInterface) { +) (*RemoteSaveRequestServer, dynamic.ResourceInterface) { t.Helper() scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) @@ -103,20 +103,20 @@ func saveServer( require.NoError(t, err) obj := &unstructured.Unstructured{Object: fields} obj.SetAPIVersion(sourcev1alpha1.GroupVersion.String()) - obj.SetKind("SaveRequest") + obj.SetKind("RemoteSaveRequest") dc := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), - map[schema.GroupVersionResource]string{saveRequestGVR(): "SaveRequestList"}, obj) - server := &SaveRequestServer{Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard()} - return server, dc.Resource(saveRequestGVR()).Namespace("shop") + map[schema.GroupVersionResource]string{remoteSaveRequestGVR(): "RemoteSaveRequestList"}, obj) + server := &RemoteSaveRequestServer{Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard()} + return server, dc.Resource(remoteSaveRequestGVR()).Namespace("shop") } -func newSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1.SaveRequest { - return &sourcev1alpha1.SaveRequest{ +func newRemoteSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1.RemoteSaveRequest { + return &sourcev1alpha1.RemoteSaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: "shop", UID: "uid-save", Generation: 1}, - Spec: sourcev1alpha1.SaveRequestSpec{ + Spec: sourcev1alpha1.RemoteSaveRequestSpec{ Message: "save: from the editor", CatchUp: catchUp, - Window: &sourcev1alpha1.SaveRequestWindow{ + Window: &sourcev1alpha1.RemoteSaveRequestWindow{ AttachTimeout: &metav1.Duration{Duration: 3 * time.Second}, MaxDuration: &metav1.Duration{Duration: time.Second}, }, @@ -126,25 +126,25 @@ func newSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1.SaveRe // serveOnce serves the request and returns its status as written to the source cluster. func serveOnce( - t *testing.T, server *SaveRequestServer, resource dynamic.ResourceInterface, request *sourcev1alpha1.SaveRequest, -) sourcev1alpha1.SaveRequestStatus { + t *testing.T, server *RemoteSaveRequestServer, resource dynamic.ResourceInterface, request *sourcev1alpha1.RemoteSaveRequest, +) sourcev1alpha1.RemoteSaveRequestStatus { t.Helper() - watcher := &saveRequestWatcher{provider: saveProvider, route: "providers.workspace"} + watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} server.serve(context.Background(), logr.Discard(), watcher, resource, request) got, err := resource.Get(context.Background(), request.Name, metav1.GetOptions{}) require.NoError(t, err) - var written sourcev1alpha1.SaveRequest + var written sourcev1alpha1.RemoteSaveRequest require.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(got.Object, &written)) return written.Status } -func condition(status sourcev1alpha1.SaveRequestStatus, conditionType string) *metav1.Condition { +func condition(status sourcev1alpha1.RemoteSaveRequestStatus, conditionType string) *metav1.Condition { return apimeta.FindStatusCondition(status.Conditions, conditionType) } // A save attaches to the window of every GitTarget that mirrors its namespace, as its author found // from the source cluster's audit, and reports each commit and the apply it asked for. -func TestSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) { +func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) { finalizer := &saveFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, apply: git.SaveApply{ @@ -155,7 +155,7 @@ func TestSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) }, } author := fixedAuthor{user: git.UserInfo{Username: "alice@example.com"}, outcome: git.AttributionResolved} - request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) server, resource := saveServer(t, request, true, metav1.ConditionTrue, author, finalizer) status := serveOnce(t, server, resource, request) @@ -173,7 +173,7 @@ func TestSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) assert.Equal(t, "alice@example.com", status.Author) require.Len(t, status.Targets, 1) - assert.Equal(t, sourcev1alpha1.SaveRequestTarget{ + assert.Equal(t, sourcev1alpha1.RemoteSaveRequestTarget{ GitTarget: "team-a/app", Reason: saveReasonCommitted, Message: "the edits were committed and pushed", Branch: "main", Commit: "c0ffee", Applied: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", }, status.Targets[0]) @@ -186,9 +186,9 @@ func TestSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) // On a cluster with no audit stream a save names nobody, and attaches as such: to a window whose // edits name nobody. -func TestSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { +func TestRemoteSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}} - request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) server, resource := saveServer(t, request, true, metav1.ConditionUnknown, fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) @@ -203,9 +203,9 @@ func TestSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { // On a cluster that sends audit events, a save no event names lost its fact: it is refused rather // than attached to the window that should stay unattributed. -func TestSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { +func TestRemoteSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { finalizer := &saveFinalizer{} - request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) server, resource := saveServer(t, request, true, metav1.ConditionTrue, fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) @@ -217,9 +217,9 @@ func TestSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { } // A save in a namespace no GitTarget mirrors has nothing to save. -func TestSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { +func TestRemoteSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { finalizer := &saveFinalizer{} - request := newSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) server, resource := saveServer(t, request, false, "", nil, finalizer) status := serveOnce(t, server, resource, request) @@ -229,9 +229,9 @@ func TestSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { } // A save that failed in a GitTarget fails, and says where. -func TestSaveRequest_AFailureFailsTheRequest(t *testing.T) { +func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} - request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) server, resource := saveServer(t, request, true, "", nil, finalizer) status := serveOnce(t, server, resource, request) @@ -245,21 +245,21 @@ func TestSaveRequest_AFailureFailsTheRequest(t *testing.T) { // A request is finished once Ready is decided, and, when it committed and asked to catch up, once // Applied is decided too. -func TestSaveRequest_Finished(t *testing.T) { +func TestRemoteSaveRequest_Finished(t *testing.T) { at := metav1.NewTime(time.Now().Add(-time.Hour)) ready := func(status metav1.ConditionStatus, reason string) metav1.Condition { return metav1.Condition{Type: "Ready", Status: status, Reason: reason, LastTransitionTime: at} } - request := newSaveRequest(sourcev1alpha1.CatchUpAfterPush) - _, finished := saveRequestFinished(request) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) + _, finished := remoteSaveRequestFinished(request) assert.False(t, finished, "no Ready yet") request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionUnknown, saveReasonSaving)} - _, finished = saveRequestFinished(request) + _, finished = remoteSaveRequestFinished(request) assert.False(t, finished, "still saving") request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonCommitted)} - _, finished = saveRequestFinished(request) + _, finished = remoteSaveRequestFinished(request) assert.False(t, finished, "committed, its apply not decided") request.Status.Conditions = append( @@ -271,28 +271,28 @@ func TestSaveRequest_Finished(t *testing.T) { LastTransitionTime: at, }, ) - finishedAt, finished := saveRequestFinished(request) + finishedAt, finished := remoteSaveRequestFinished(request) assert.True(t, finished) assert.Equal(t, at.Time, finishedAt) request.Spec.CatchUp = sourcev1alpha1.CatchUpAsDefinedOnClaim request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonNoWindow)} - _, finished = saveRequestFinished(request) + _, finished = remoteSaveRequestFinished(request) assert.True(t, finished) } -// The server serves the KubeProviders that enable SaveRequests, and stops serving one that no longer +// The server serves the KubeProviders that enable RemoteSaveRequests, and stops serving one that no longer // does. -func TestSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T) { +func TestRemoteSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T) { scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) enabled := &configv1alpha3.KubeProvider{ ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}, - Spec: configv1alpha3.KubeProviderSpec{SaveRequests: configv1alpha3.SaveRequestsEnabled}, + Spec: configv1alpha3.KubeProviderSpec{RemoteSaveRequests: configv1alpha3.RemoteSaveRequestsEnabled}, } disabled := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "other", Namespace: "providers"}} c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(enabled, disabled).Build() - server := &SaveRequestServer{ + server := &RemoteSaveRequestServer{ Client: c, Log: logr.Discard(), dynamicClient: func(context.Context, k8stypes.NamespacedName) (dynamic.Interface, error) { return nil, errors.New("no cluster in this test") @@ -305,7 +305,7 @@ func TestSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T) { assert.Len(t, server.watchers, 1) assert.Contains(t, server.watchers, saveProvider) - enabled.Spec.SaveRequests = configv1alpha3.SaveRequestsDisabled + enabled.Spec.RemoteSaveRequests = configv1alpha3.RemoteSaveRequestsDisabled require.NoError(t, c.Update(ctx, enabled)) server.syncProviders(ctx) assert.Empty(t, server.watchers) diff --git a/internal/watch/resource_policy.go b/internal/watch/resource_policy.go index c40b1f62..e7854998 100644 --- a/internal/watch/resource_policy.go +++ b/internal/watch/resource_policy.go @@ -25,7 +25,7 @@ func isDefaultResourceExcluded(group, resource string) bool { groupResourceKey("batch", "jobs"), groupResourceKey("batch", "cronjobs"), // A save is a request to GitOps Reverser, never state to mirror (save_requests.go). - groupResourceKey("source.configbutler.ai", "saverequests"): + groupResourceKey("source.configbutler.ai", "remotesaverequests"): return true default: return false diff --git a/internal/watch/target_watch.go b/internal/watch/target_watch.go index f64a74c1..e6ff0d9f 100644 --- a/internal/watch/target_watch.go +++ b/internal/watch/target_watch.go @@ -1320,7 +1320,7 @@ func (m *Manager) attachAuthor( Terminating: u.GetDeletionTimestamp() != nil, }) // Stamp the outcome even when no actor was named: an unresolved attribution is a fact the - // writer, the author_kind metric, and CommitRequest matching all need. Leaving it at the + // writer, the author_kind metric, and SaveRequest matching all need. Leaving it at the // zero value would say "attribution was never attempted", which is exactly the conflation // that made this loss invisible. event.Attribution = outcome diff --git a/internal/webhook/validate_operator_types_handler.go b/internal/webhook/validate_operator_types_handler.go index ef09336b..3ad1b138 100644 --- a/internal/webhook/validate_operator_types_handler.go +++ b/internal/webhook/validate_operator_types_handler.go @@ -19,7 +19,7 @@ import ( // ValidateOperatorTypesPath is the validating admission endpoint scoped to our own // operator CRDs. Today its one job is command authorship — capturing the submitter of a -// command kind (a CommitRequest) into Redis and always allowing — but it is the intended +// command kind (a SaveRequest) into Redis and always allowing — but it is the intended // home for per-our-type admission generally (e.g. config validation of WatchRule / // GitProvider / GitTarget later), which would be added as additional webhook-config // entries with their own rules and failurePolicy. Distinct from the broad observe-all @@ -41,7 +41,7 @@ const ( // case here plus one rule in the webhook config — the handler body does not change. func isCommandKind(gr metav1.GroupResource) bool { switch gr { - case metav1.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}: + case metav1.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}: return true // future: case metav1.GroupResource{Group: "configbutler.ai", Resource: ""}: default: @@ -59,11 +59,11 @@ type CommandAuthorRecorder interface { // ValidateOperatorTypesHandler is the admission handler for our operator CRDs. It dispatches on the // resource under review and does two unrelated things: // -// - For a COMMAND kind (a CommitRequest) it captures the authenticated submitter into the +// - For a COMMAND kind (a SaveRequest) it captures the authenticated submitter into the // CommandAuthorStore and always allows — pure observation with a single side effect (a Redis // upsert), never a rejection, so a user's command never depends on it succeeding (a missed // capture leaves the request without a claimed actor; see -// docs/spec/commitrequest-admission-authorship.md). +// docs/spec/saverequest-admission-authorship.md). // - For a WatchRule it validates the one cross-object rule admission can usefully give feedback // on: a second source namespace against a GitTarget that declared its folder namespace-free. // That one CAN reject, and it is the only thing here that does. @@ -137,7 +137,7 @@ func (h *ValidateOperatorTypesHandler) Handle(ctx context.Context, req admission } // Info (not V(1)) on purpose: command CREATEs are rare, and this line proves the // webhook was actually invoked and the author captured — the first thing to check - // when a CommitRequest unexpectedly reports AuthorAttributed=False. + // when a SaveRequest unexpectedly reports AuthorAttributed=False. log.Info("recorded command author at admission", "resource", gr.String(), "namespace", req.Namespace, "name", req.Name, "uid", uid, "author", author.Author) diff --git a/internal/webhook/validate_operator_types_handler_test.go b/internal/webhook/validate_operator_types_handler_test.go index 7d184517..bb871f9e 100644 --- a/internal/webhook/validate_operator_types_handler_test.go +++ b/internal/webhook/validate_operator_types_handler_test.go @@ -37,8 +37,8 @@ func (f *fakeCommandAuthorRecorder) RecordCommandAuthor( return f.err } -func commitRequestResource() metav1.GroupVersionResource { - return metav1.GroupVersionResource{Group: "configbutler.ai", Version: "v1alpha3", Resource: "commitrequests"} +func saveRequestResource() metav1.GroupVersionResource { + return metav1.GroupVersionResource{Group: "configbutler.ai", Version: "v1alpha3", Resource: "saverequests"} } // commandReview builds an admission.Request for a command CREATE with the given raw @@ -72,7 +72,7 @@ func TestValidateOperatorTypesHandler_RecordsCommandAuthor(t *testing.T) { }, } resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, user, nil)) + commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, user, nil)) assert.True(t, resp.Allowed) require.Len(t, rec.calls, 1) @@ -90,7 +90,7 @@ func TestValidateOperatorTypesHandler_NilStoreNoOp(t *testing.T) { h := &ValidateOperatorTypesHandler{} resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, + commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, nil)) assert.True(t, resp.Allowed, "a missing author store never blocks a command") @@ -102,7 +102,7 @@ func TestValidateOperatorTypesHandler_ServiceAccountSubmitter(t *testing.T) { user := authnv1.UserInfo{Username: "system:serviceaccount:team-a:deployer"} resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"sa-uid"}}`, user, nil)) + commandReview(saveRequestResource(), `{"metadata":{"uid":"sa-uid"}}`, user, nil)) assert.True(t, resp.Allowed) require.Len(t, rec.calls, 1) @@ -129,7 +129,7 @@ func TestValidateOperatorTypesHandler_DryRunIsNotRecorded(t *testing.T) { resp := h.Handle( context.Background(), commandReview( - commitRequestResource(), + saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, &dryRun, @@ -145,7 +145,7 @@ func TestValidateOperatorTypesHandler_MissingUIDIsNotRecorded(t *testing.T) { h := &ValidateOperatorTypesHandler{Store: rec} resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{}}`, authnv1.UserInfo{Username: "alice"}, nil)) + commandReview(saveRequestResource(), `{"metadata":{}}`, authnv1.UserInfo{Username: "alice"}, nil)) assert.True(t, resp.Allowed) assert.Empty(t, rec.calls, "without a uid the record cannot be keyed") @@ -156,7 +156,7 @@ func TestValidateOperatorTypesHandler_NoUserIsNotRecorded(t *testing.T) { h := &ValidateOperatorTypesHandler{Store: rec} resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{}, nil)) + commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{}, nil)) assert.True(t, resp.Allowed) assert.Empty(t, rec.calls, "an unauthenticated request names no author") @@ -171,7 +171,7 @@ func TestValidateOperatorTypesHandler_RecordErrorStillAllows(t *testing.T) { resp := h.Handle( context.Background(), commandReview( - commitRequestResource(), + saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, nil, diff --git a/test/e2e/Taskfile-e2e.yml b/test/e2e/Taskfile-e2e.yml index f30e5a50..42c6d658 100644 --- a/test/e2e/Taskfile-e2e.yml +++ b/test/e2e/Taskfile-e2e.yml @@ -1328,7 +1328,7 @@ tasks: - | CTX="{{.CTX}}" KUBECTL="{{.KUBECTL}}" bash hack/e2e/wait-crds-established.sh 60 \ crd/kubeproviders.configbutler.ai crd/clusterwatchrules.configbutler.ai \ - crd/commitrequests.configbutler.ai crd/gitproviders.configbutler.ai \ + crd/saverequests.configbutler.ai crd/gitproviders.configbutler.ai \ crd/gitbranches.configbutler.ai crd/gitreceivers.configbutler.ai \ crd/gittargets.configbutler.ai crd/watchrules.configbutler.ai \ crd/fluxclaims.configbutler.ai crd/argocdclaims.configbutler.ai diff --git a/test/e2e/bi_directional_echoes_e2e_test.go b/test/e2e/bi_directional_echoes_e2e_test.go index 3e1b6e4a..de4b5259 100644 --- a/test/e2e/bi_directional_echoes_e2e_test.go +++ b/test/e2e/bi_directional_echoes_e2e_test.go @@ -395,15 +395,15 @@ func echoScenarios(run func() *echoRun) { r.pushFromOutside("saved", "s0") r.eventuallyEverywhere("saved", "s0", echoApplyBudget) - By("saving an API edit with a CommitRequest while a Git-side push changes the same object") - name := applyCommitRequestWithGenerateName(r.testNs, "echo-save-", r.targetName, "e2e: save the edit") + By("saving an API edit with a SaveRequest while a Git-side push changes the same object") + name := applySaveRequestWithGenerateName(r.testNs, "echo-save-", r.targetName, "e2e: save the edit") r.editThroughAPI("saved", "s-save") r.pushFromOutside("saved", "s-git") Eventually(func(g Gomega) { - g.Expect(commitRequestCondition(g, r.testNs, name, "Ready")).To(Equal("True")) - g.Expect(commitRequestField(g, r.testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(BeElementOf(commitRequestReasonCommitted, "AlreadyPresent")) + g.Expect(saveRequestCondition(g, r.testNs, name, "Ready")).To(Equal("True")) + g.Expect(saveRequestField(g, r.testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(BeElementOf(saveRequestReasonCommitted, "AlreadyPresent")) }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.eventuallyEverywhere("saved", "s-save", echoApplyBudget) r.consistentlyEverywhere("saved", "s-save") diff --git a/test/e2e/cluster/audit/policy.yaml b/test/e2e/cluster/audit/policy.yaml index bfc033c8..8ac915be 100644 --- a/test/e2e/cluster/audit/policy.yaml +++ b/test/e2e/cluster/audit/policy.yaml @@ -52,8 +52,8 @@ rules: - delete - deletecollection # No resources listed = Matches everything else (including CRDs). - # This also captures `commitrequests.configbutler.ai` create events, which + # This also captures `saverequests.configbutler.ai` create events, which # drive the "finalize the open commit window now" path. If this catch-all # is ever narrowed, keep at least Metadata-level capture for - # `commitrequests` create events — the consumer reads the object body from + # `saverequests` create events — the consumer reads the object body from # the apiserver, so Metadata is sufficient. diff --git a/test/e2e/e2e_suite_test.go b/test/e2e/e2e_suite_test.go index a9031f13..c69cd47e 100644 --- a/test/e2e/e2e_suite_test.go +++ b/test/e2e/e2e_suite_test.go @@ -128,36 +128,36 @@ var _ = SynchronizedAfterSuite(func() {}, func() { // Diagnostic first, so its numbers reach the artifacts even when the gating // audit invariant below fails the suite. reportAttributionStats() - reportCommitRequestOutcomes() + reportSaveRequestOutcomes() assertNoAnomalousAuditOutcomes() }) -// commitRequestOutcomes is the bounded outcome set of gitopsreverser_commit_requests_total, in the +// saveRequestOutcomes is the bounded outcome set of gitopsreverser_save_requests_total, in the // order a reader wants them: the success, the two refusals, the no-op, then the fault. -var commitRequestOutcomes = []string{ +var saveRequestOutcomes = []string{ "committed", "window_mismatch", "no_window", "already_present", "failed", } -// reportCommitRequestOutcomes prints the run's CommitRequest outcome distribution. +// reportSaveRequestOutcomes prints the run's SaveRequest outcome distribution. // // DIAGNOSTIC, not a gate. A suite-wide "none failed" assertion would first have to establish that // the last save had been scraped, and until that is handled a green result would mean less than it // looks. The wiring claim this counter has to support is made where the evidence is local: the -// Commit Request spec asserts its own save appears, scoped to its own GitTarget. +// Save Request spec asserts its own save appears, scoped to its own GitTarget. // // The breakdown is the point. `committed` is what the happy path looks like and the only value // these specs produce: `window_mismatch` needs two authors contending for one branch's window, // which nothing here arranges. The per-run reset means these numbers are this run's. -func reportCommitRequestOutcomes() { - By("reporting the run's CommitRequest terminal outcomes") +func reportSaveRequestOutcomes() { + By("reporting the run's SaveRequest terminal outcomes") ensurePrometheusClient() verifyPrometheusAvailable() - for _, oc := range commitRequestOutcomes { + for _, oc := range saveRequestOutcomes { n, err := queryPrometheus(fmt.Sprintf( - `sum(max_over_time(gitopsreverser_commit_requests_total{outcome=%q}[2h])) or vector(0)`, oc)) + `sum(max_over_time(gitopsreverser_save_requests_total{outcome=%q}[2h])) or vector(0)`, oc)) if err == nil { - _, _ = fmt.Fprintf(GinkgoWriter, " commit request outcome %-16s = %.0f\n", oc, n) + _, _ = fmt.Fprintf(GinkgoWriter, " save request outcome %-16s = %.0f\n", oc, n) } } } diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index 8a85d720..f0d97649 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -359,14 +359,14 @@ data: }) // A save that asks to catch up is applied as soon as it is pushed, with catchUpAfter off, and its - // CommitRequest reports the apply: Applied=True with the revision Flux applied. - It("applies a save that asks to catch up, and reports it on the CommitRequest", func() { + // SaveRequest reports the apply: Applied=True with the revision Flux applied. + It("applies a save that asks to catch up, and reports it on the SaveRequest", func() { const configMap, request = "applier-converge-edit", "applier-save-catch-up" setFluxClaimCatchUpAfter(testNs, fluxName, "0s") By("saving an API edit with catchUp: AfterPush") _, err := kubectlRunWithStdin(testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: %s namespace: %s @@ -387,17 +387,17 @@ spec: By("the save is pushed, and Ready says only that") var commit string Eventually(func(g Gomega) { - g.Expect(commitRequestCondition(g, testNs, request, "Ready")).To(Equal("True")) - g.Expect(commitRequestField(g, testNs, request, "{.status.conditions[?(@.type==\"Ready\")].reason}")). - To(Equal(commitRequestReasonCommitted)) - commit = commitRequestField(g, testNs, request, "{.status.commit}") + g.Expect(saveRequestCondition(g, testNs, request, "Ready")).To(Equal("True")) + g.Expect(saveRequestField(g, testNs, request, "{.status.conditions[?(@.type==\"Ready\")].reason}")). + To(Equal(saveRequestReasonCommitted)) + commit = saveRequestField(g, testNs, request, "{.status.commit}") g.Expect(commit).NotTo(BeEmpty()) }, 2*time.Minute, 2*time.Second).Should(Succeed()) By("the applier applies it at once, and Applied reports the revision") Eventually(func(g Gomega) { - g.Expect(commitRequestCondition(g, testNs, request, "Applied")).To(Equal("True")) - g.Expect(commitRequestField(g, testNs, request, "{.status.appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(saveRequestCondition(g, testNs, request, "Applied")).To(Equal("True")) + g.Expect(saveRequestField(g, testNs, request, "{.status.appliedRevision}")).To(ContainSubstring(commit)) g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(commit)) g.Expect(readClaimOperation(g, testNs, fluxClaimResource, fluxName).Reason).To(Equal("CatchUp")) }, 2*time.Minute, 2*time.Second).Should(Succeed()) diff --git a/test/e2e/helpers.go b/test/e2e/helpers.go index 064617f1..5606b495 100644 --- a/test/e2e/helpers.go +++ b/test/e2e/helpers.go @@ -543,18 +543,18 @@ func dumpFailureDiagnostics() { _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get Kubernetes events: %s\n", err) } - By("Fetching CommitRequest diagnostics") - commitRequestsOutput, err := kubectlRun( + By("Fetching SaveRequest diagnostics") + saveRequestsOutput, err := kubectlRun( "get", - "commitrequests.configbutler.ai", + "saverequests.configbutler.ai", "--all-namespaces", "-o", "yaml", ) if err == nil { - _, _ = fmt.Fprintf(GinkgoWriter, "CommitRequests:\n%s", commitRequestsOutput) + _, _ = fmt.Fprintf(GinkgoWriter, "SaveRequests:\n%s", saveRequestsOutput) } else { - _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get CommitRequests: %s\n", err) + _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get SaveRequests: %s\n", err) } podNames, err := controllerPodNames() diff --git a/test/e2e/commit_request_e2e_test.go b/test/e2e/save_request_e2e_test.go similarity index 72% rename from test/e2e/commit_request_e2e_test.go rename to test/e2e/save_request_e2e_test.go index 8b8635db..2800bb31 100644 --- a/test/e2e/commit_request_e2e_test.go +++ b/test/e2e/save_request_e2e_test.go @@ -13,12 +13,12 @@ import ( . "github.com/onsi/gomega" ) -// The CommitRequest suite exercises the "save" signal: a CommitRequest +// The SaveRequest suite exercises the "save" signal: a SaveRequest // object finalizes the open commit window for a GitTarget immediately, instead // of waiting for the rolling silence timer. The GitProvider is configured with -// a deliberately long commitWindow so that, without the CommitRequest, the +// a deliberately long commitWindow so that, without the SaveRequest, the // edit would not be committed for minutes — observing the commit promptly is -// what proves the commit-request path works. +// what proves the save-request path works. // // Not Serial: this spec owns a dedicated Gitea repo (its own GitProvider → // GitTarget → namespace-scoped WatchRule), so the only writer to its main @@ -26,7 +26,7 @@ import ( // namespace. The HEAD/SHA assertions below therefore read back only this spec's // own commit; concurrent audit traffic for other GitTargets lands in other // repos and cannot move this repo's HEAD. See docs/spec/e2e-serial-registry.md. -var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Ordered, func() { +var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordered, func() { var ( testNs string repo *RepoArtifacts @@ -40,7 +40,7 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or const commitWindow = "300s" BeforeAll(func() { - p := setupWindowedDeploymentPipeline("commit-request", "e2e/commit-request-test", commitWindow) + p := setupWindowedDeploymentPipeline("save-request", "e2e/save-request-test", commitWindow) testNs, repo, gitProvName, gitTargetName, watchRuleName = p.ns, p.repo, p.gitProvider, p.gitTarget, p.watchRule }) @@ -50,11 +50,11 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or }) It("finalizes the open commit window on demand and reports the resulting SHA", func() { - basePath := "e2e/commit-request-test" + basePath := "e2e/save-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("commit-request-deploy-%d", seed) - commitRequestName := fmt.Sprintf("commit-request-save-%d", seed) - message := fmt.Sprintf("save: commit request from e2e seed %d", seed) + deployName := fmt.Sprintf("save-request-deploy-%d", seed) + saveRequestName := fmt.Sprintf("save-request-save-%d", seed) + message := fmt.Sprintf("save: save request from e2e seed %d", seed) By("creating a Deployment to open a commit window") applyScaleTestDeployment(testNs, deployName, 0) @@ -69,16 +69,16 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or "the open commit window must hold the edit; main must not exist until the window is finalized") }, 10*time.Second, 2*time.Second).Should(Succeed()) - By("creating a CommitRequest to finalize the open window now") - applyCommitRequest(testNs, commitRequestName, gitTargetName, message) + By("creating a SaveRequest to finalize the open window now") + applySaveRequest(testNs, saveRequestName, gitTargetName, message) - By("waiting for the CommitRequest to commit and push") + By("waiting for the SaveRequest to commit and push") var reportedSHA string Eventually(func(g Gomega) { - reportedSHA = expectCommitRequestCommitted(g, testNs, commitRequestName, + reportedSHA = expectSaveRequestCommitted(g, testNs, saveRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) - branch := commitRequestField(g, testNs, commitRequestName, "{.status.branch}") + branch := saveRequestField(g, testNs, saveRequestName, "{.status.branch}") g.Expect(branch).To(Equal("main")) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -116,11 +116,11 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or // is: the commit above is verified in Git, so a missing series is a broken exporter rather // than a spec that did not run. Both queries are scoped to this suite's own GitTarget, so // they need no run bookkeeping to isolate from the other parallel processes. - By("verifying this save reached the CommitRequest outcome counter") + By("verifying this save reached the SaveRequest outcome counter") ensurePrometheusClient() waitForMetric( fmt.Sprintf( - `sum(gitopsreverser_commit_requests_total{outcome="committed",`+ + `sum(gitopsreverser_save_requests_total{outcome="committed",`+ `gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, testNs, gitTargetName), func(v float64) bool { return v > 0 }, @@ -133,7 +133,7 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or waitForMetric( fmt.Sprintf( `sum(gitopsreverser_git_commit_windows_total{close_reason="max_duration",`+ - `timer_source="commit_request",gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, + `timer_source="save_request",gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, testNs, gitTargetName), func(v float64) bool { return v > 0 }, "the save's window is counted with the timer that closed it", @@ -159,20 +159,20 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or "the join series maps this GitTarget to the GitBranch its commits are labelled by", ) - By("cleaning up the test Deployment and CommitRequest") + By("cleaning up the test Deployment and SaveRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") }) // The companion path: once a branch HAS been established (the previous spec's finalize created // main), a fresh edit must STILL be held in the open window — an existing branch must not - // advance until a CommitRequest finalizes it. Together with the spec above this covers both + // advance until a SaveRequest finalizes it. Together with the spec above this covers both // cases of "defer committing as long as possible": an absent branch and an established one. It("holds a new edit in the open window without advancing an already-established branch", func() { - basePath := "e2e/commit-request-test" + basePath := "e2e/save-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("commit-request-hold-%d", seed) - commitRequestName := fmt.Sprintf("commit-request-hold-save-%d", seed) + deployName := fmt.Sprintf("save-request-hold-%d", seed) + saveRequestName := fmt.Sprintf("save-request-hold-save-%d", seed) message := fmt.Sprintf("save: held edit from e2e seed %d", seed) By("capturing the branch HEAD the previous spec established") @@ -192,10 +192,10 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or "the open commit window must hold the new edit; main must not advance until finalized") }, 10*time.Second, 2*time.Second).Should(Succeed()) - By("creating a CommitRequest and confirming the branch then advances with the held edit") - applyCommitRequest(testNs, commitRequestName, gitTargetName, message) + By("creating a SaveRequest and confirming the branch then advances with the held edit") + applySaveRequest(testNs, saveRequestName, gitTargetName, message) Eventually(func(g Gomega) { - expectCommitRequestCommitted(g, testNs, commitRequestName, + expectSaveRequestCommitted(g, testNs, saveRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) g.Expect(remoteBranchHead(g, repo.CheckoutDir)).NotTo(Equal(baseSHA), "finalizing must advance main past the previously-established HEAD") @@ -211,9 +211,9 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or fmt.Sprintf("the held Deployment should exist after finalize at %s", expected)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the held Deployment and CommitRequest") + By("cleaning up the held Deployment and SaveRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") }) // generateName authorship is now captured at admission, not from the audit @@ -221,26 +221,26 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or // validating admission runs, so the validate-operator-types webhook records the // submitter keyed by uid with no response-body name recovery (the old audit // generateName headache is gone, see - // docs/spec/commitrequest-admission-authorship.md). This spec proves a - // generateName CommitRequest still finalizes and becomes Ready. It is skipped in + // docs/spec/saverequest-admission-authorship.md). This spec proves a + // generateName SaveRequest still finalizes and becomes Ready. It is skipped in // configured-author mode, where the edit's window is committer-authored and the // named admission author would not match it end to end. - It("finalizes a CommitRequest created with metadata.generateName", func() { - basePath := "e2e/commit-request-test" + It("finalizes a SaveRequest created with metadata.generateName", func() { + basePath := "e2e/save-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("commit-request-gen-deploy-%d", seed) - commitRequestPrefix := fmt.Sprintf("commit-request-gen-%d-", seed) - message := fmt.Sprintf("save: generateName commit request from e2e seed %d", seed) + deployName := fmt.Sprintf("save-request-gen-deploy-%d", seed) + saveRequestPrefix := fmt.Sprintf("save-request-gen-%d-", seed) + message := fmt.Sprintf("save: generateName save request from e2e seed %d", seed) By("creating a Deployment to open a commit window") applyScaleTestDeployment(testNs, deployName, 0) - By("creating a CommitRequest with metadata.generateName") - generatedName := applyCommitRequestWithGenerateName(testNs, commitRequestPrefix, gitTargetName, message) + By("creating a SaveRequest with metadata.generateName") + generatedName := applySaveRequestWithGenerateName(testNs, saveRequestPrefix, gitTargetName, message) - By("waiting for the generated-name CommitRequest to commit") + By("waiting for the generated-name SaveRequest to commit") Eventually(func(g Gomega) { - expectCommitRequestCommitted(g, testNs, generatedName, + expectSaveRequestCommitted(g, testNs, generatedName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -261,24 +261,24 @@ var _ = Describe("Commit Request", Label("commit-request", "audit-consumer"), Or recentCommitDiagnostics(repo.CheckoutDir, basePath)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the generateName Deployment and CommitRequest") + By("cleaning up the generateName Deployment and SaveRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", generatedName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", generatedName, "--ignore-not-found=true") }) }) -// The UC2 suite exercises a `kubectl apply` bundle that includes a CommitRequest +// The UC2 suite exercises a `kubectl apply` bundle that includes a SaveRequest // as its FIRST document — the deliberately-hard ordering where the save intent -// arrives before the work it is meant to save (docs/spec/commitrequest-design.md). The request +// arrives before the work it is meant to save (docs/spec/saverequest-design.md). The request // waits (spec.window.attachTimeout) for the first Deployment to open a window, attaches, and // collects (spec.window.maxDuration from the attach) while the rest of the bundle arrives, so the -// whole bundle lands in ONE commit carrying the CommitRequest's message. +// whole bundle lands in ONE commit carrying the SaveRequest's message. // // Its own dedicated Gitea repo (own GitProvider → GitTarget → namespace-scoped // Deployment WatchRule) makes the one-commit assertion unambiguous: main does not // exist until the bundle is finalized, so the bundle commit is the only commit on // main. -var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-consumer"), Ordered, func() { +var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consumer"), Ordered, func() { var ( testNs string repo *RepoArtifacts @@ -288,11 +288,11 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c ) // A long spec.commit.window so the silence timer can never be what produces the - // commit — only the CommitRequest finalize may (A1). + // commit — only the SaveRequest finalize may (A1). const commitWindow = "300s" BeforeAll(func() { - p := setupWindowedDeploymentPipeline("commit-request-bundle", "e2e/commit-request-bundle", commitWindow) + p := setupWindowedDeploymentPipeline("save-request-bundle", "e2e/save-request-bundle", commitWindow) testNs, repo, gitProvName, gitTargetName, watchRuleName = p.ns, p.repo, p.gitProvider, p.gitTarget, p.watchRule }) @@ -301,11 +301,11 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c cleanupNamespace(testNs) }) - It("lands a kubectl-apply bundle (CommitRequest first) in one commit with the request's message", func() { - basePath := "e2e/commit-request-bundle" + It("lands a kubectl-apply bundle (SaveRequest first) in one commit with the request's message", func() { + basePath := "e2e/save-request-bundle" seed := GinkgoRandomSeed() message := fmt.Sprintf("bundle save: apply from e2e seed %d", seed) - commitRequestName := fmt.Sprintf("commit-request-bundle-save-%d", seed) + saveRequestName := fmt.Sprintf("save-request-bundle-save-%d", seed) deployNames := []string{ fmt.Sprintf("bundle-deploy-a-%d", seed), fmt.Sprintf("bundle-deploy-b-%d", seed), @@ -318,25 +318,25 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c "main must not exist before the bundle is finalized") }, 5*time.Second, 1*time.Second).Should(Succeed()) - By("applying a bundle whose FIRST document is a CommitRequest, then three Deployments") + By("applying a bundle whose FIRST document is a SaveRequest, then three Deployments") // The window is sized to comfortably exceed the bundle's per-type ingestion spread, so the // collection after the attach is deterministic. var bundle strings.Builder - bundle.WriteString(commitRequestManifest(testNs, commitRequestName, gitTargetName, message, "8s")) + bundle.WriteString(saveRequestManifest(testNs, saveRequestName, gitTargetName, message, "8s")) for _, name := range deployNames { bundle.WriteString("---\n") bundle.WriteString(deploymentManifest(testNs, name)) } _, err := kubectlRunWithStdin(testNs, bundle.String(), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply the CommitRequest+Deployments bundle") + Expect(err).NotTo(HaveOccurred(), "failed to apply the SaveRequest+Deployments bundle") - By("waiting for the bundle's CommitRequest to commit the collected window") + By("waiting for the bundle's SaveRequest to commit the collected window") var reportedSHA string Eventually(func(g Gomega) { - reportedSHA = expectCommitRequestCommitted(g, testNs, commitRequestName, + reportedSHA = expectSaveRequestCommitted(g, testNs, saveRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) - branch := commitRequestField(g, testNs, commitRequestName, "{.status.branch}") + branch := saveRequestField(g, testNs, saveRequestName, "{.status.branch}") g.Expect(branch).To(Equal("main")) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -345,7 +345,7 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c pullLatestRepoState(g, repo.CheckoutDir) // Exactly one commit on the fresh repo's main: the entire bundle — applied - // across the CommitRequest's attribution and the per-type Deployment stream — + // across the SaveRequest's attribution and the per-type Deployment stream — // collapsed into a single commit. g.Expect(mustCommitCount(repo.CheckoutDir)).To(Equal(1), "the whole bundle must land in exactly one commit\n%s", @@ -354,7 +354,7 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c subject, logErr := gitRun(repo.CheckoutDir, "log", "-1", "--pretty=%B") g.Expect(logErr).NotTo(HaveOccurred()) g.Expect(strings.TrimSpace(subject)).To(Equal(message), - "the single commit must carry the CommitRequest's message verbatim\n%s", + "the single commit must carry the SaveRequest's message verbatim\n%s", recentCommitDiagnostics(repo.CheckoutDir, basePath)) headSHA, shaErr := gitRun(repo.CheckoutDir, "rev-parse", "HEAD") @@ -372,20 +372,20 @@ var _ = Describe("Commit Request Bundle (UC2)", Label("commit-request", "audit-c } }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the bundle Deployments and CommitRequest") + By("cleaning up the bundle Deployments and SaveRequest") for _, name := range deployNames { _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", name, "--ignore-not-found=true") } - _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") }) }) -// commitRequestManifest renders a single CommitRequest document with an explicit message and a +// saveRequestManifest renders a single SaveRequest document with an explicit message and a // window that waits and then collects for the given Go duration each. It is used to build -// multi-document `kubectl apply` bundles where the CommitRequest is the first document. -func commitRequestManifest(namespace, name, gitTargetName, message, window string) string { +// multi-document `kubectl apply` bundles where the SaveRequest is the first document. +func saveRequestManifest(namespace, name, gitTargetName, message, window string) string { return fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: %s namespace: %s @@ -424,11 +424,11 @@ spec: `, name, namespace, name, name) } -// applyCommitRequestWithGenerateName creates a CommitRequest using +// applySaveRequestWithGenerateName creates a SaveRequest using // metadata.generateName and returns the server-allocated name. The spec creates its work and this -// CommitRequest back-to-back: authorship is settled synchronously at admission (no controller-side +// SaveRequest back-to-back: authorship is settled synchronously at admission (no controller-side // wait), so a zero wait would race the work's watch event and could resolve NoWindow before the -// window opens. Waiting to attach is the documented mechanism for a CommitRequest issued +// window opens. Waiting to attach is the documented mechanism for a SaveRequest issued // concurrently with its work (UC2). // // The wait is deliberately tight: it covers audit delivery (--audit-webhook-batch-max-wait=1s) and @@ -437,10 +437,10 @@ spec: // attributes its events in order, so a work event held behind another event that waits out the whole // grace fails the spec with NoWindow instead of passing slowly. Do not widen it to make such a // failure go away: that wait is the bug. -func applyCommitRequestWithGenerateName(namespace, prefix, gitTargetName, message string) string { +func applySaveRequestWithGenerateName(namespace, prefix, gitTargetName, message string) string { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: generateName: %s namespace: %s @@ -455,19 +455,19 @@ spec: out, err := kubectlRunWithStdin(namespace, manifest, "create", "-f", "-", "-o", "jsonpath={.metadata.name}") Expect(err).NotTo(HaveOccurred(), - fmt.Sprintf("failed to create CommitRequest with generateName=%s", prefix)) + fmt.Sprintf("failed to create SaveRequest with generateName=%s", prefix)) name := strings.TrimSpace(out) Expect(name).NotTo(BeEmpty(), "kubectl create must return the server-allocated name") Expect(name).To(HavePrefix(prefix), "the allocated name must start with the requested prefix") return name } -// applyCommitRequest creates a CommitRequest object that targets the given +// applySaveRequest creates a SaveRequest object that targets the given // GitTarget with an optional commit message. -func applyCommitRequest(namespace, name, gitTargetName, message string) { +func applySaveRequest(namespace, name, gitTargetName, message string) { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: %s namespace: %s @@ -477,37 +477,37 @@ spec: message: %q `, name, namespace, gitTargetName, message) _, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to apply CommitRequest %s/%s", namespace, name)) + Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to apply SaveRequest %s/%s", namespace, name)) } -// commitRequestField reads a jsonpath field off a CommitRequest object. -func commitRequestField(g Gomega, namespace, name, jsonPath string) string { - out, err := kubectlRunInNamespace(namespace, "get", "commitrequest", name, +// saveRequestField reads a jsonpath field off a SaveRequest object. +func saveRequestField(g Gomega, namespace, name, jsonPath string) string { + out, err := kubectlRunInNamespace(namespace, "get", "saverequest", name, "-o", "jsonpath="+jsonPath) - g.Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to read %s of CommitRequest %s", jsonPath, name)) + g.Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to read %s of SaveRequest %s", jsonPath, name)) return strings.TrimSpace(out) } -// commitRequestCondition reads the status of one condition off a CommitRequest. -func commitRequestCondition(g Gomega, namespace, name, conditionType string) string { - return commitRequestField(g, namespace, name, +// saveRequestCondition reads the status of one condition off a SaveRequest. +func saveRequestCondition(g Gomega, namespace, name, conditionType string) string { + return saveRequestField(g, namespace, name, fmt.Sprintf(`{.status.conditions[?(@.type=="%s")].status}`, conditionType)) } -// commitRequestReasonCommitted is the one Ready reason that means a commit actually happened. -// It mirrors crReasonCommitted in internal/controller/commitrequest_finalize.go. -const commitRequestReasonCommitted = "Committed" +// saveRequestReasonCommitted is the one Ready reason that means a commit actually happened. +// It mirrors crReasonCommitted in internal/controller/saverequest_finalize.go. +const saveRequestReasonCommitted = "Committed" -// commitRequestOutcome is the terminal shape of a CommitRequest: not just whether it finished, +// saveRequestOutcome is the terminal shape of a SaveRequest: not just whether it finished, // but WHICH ending it reached. // // Ready=True alone cannot answer that, and reading it as "it committed" is how this suite went // blind. A benign rejection — no window collected in the grace, a foreign author's window, a // change already present on the remote — is deliberately Ready=True with Pushed=False and an -// empty status.commit, so that kstatus reads Current rather than Failed (see rejectCommitRequest). +// empty status.commit, so that kstatus reads Current rather than Failed (see rejectSaveRequest). // A spec asserting Ready=True therefore PASSES on a request that committed nothing, then fails // seconds later on the empty sha with ": not to be empty" and no reason attached. -type commitRequestOutcome struct { +type saveRequestOutcome struct { Ready string Reason string Message string @@ -518,59 +518,59 @@ type commitRequestOutcome struct { } // String renders the outcome as the diagnostic the bare-sha assertion never produced. -func (o commitRequestOutcome) String() string { +func (o saveRequestOutcome) String() string { return fmt.Sprintf( - "CommitRequest status: Ready=%s (reason=%q) Stalled=%s Pushed=%s commit=%q branch=%q\n message: %s", + "SaveRequest status: Ready=%s (reason=%q) Stalled=%s Pushed=%s commit=%q branch=%q\n message: %s", o.Ready, o.Reason, o.Stalled, o.Pushed, o.Commit, o.Branch, o.Message) } // isTerminal reports whether the controller has stopped working on this request: Ready=True -// (committed or benignly rejected) or Stalled=True (failed). It mirrors commitRequestIsTerminal +// (committed or benignly rejected) or Stalled=True (failed). It mirrors saveRequestIsTerminal // in the controller, and it is what makes an early give-up safe. -func (o commitRequestOutcome) isTerminal() bool { +func (o saveRequestOutcome) isTerminal() bool { return o.Ready == "True" || o.Stalled == "True" } -// readCommitRequestOutcome reads the whole terminal picture in one API call, so every field in a +// readSaveRequestOutcome reads the whole terminal picture in one API call, so every field in a // failure report is from the same observation rather than five reads a poll apart. -func readCommitRequestOutcome(g Gomega, namespace, name string) commitRequestOutcome { +func readSaveRequestOutcome(g Gomega, namespace, name string) saveRequestOutcome { const readyPath = `{.status.conditions[?(@.type=="Ready")]` - return commitRequestOutcome{ - Ready: commitRequestField(g, namespace, name, readyPath+`.status}`), - Reason: commitRequestField(g, namespace, name, readyPath+`.reason}`), - Message: commitRequestField(g, namespace, name, readyPath+`.message}`), - Stalled: commitRequestCondition(g, namespace, name, "Stalled"), - Pushed: commitRequestCondition(g, namespace, name, "Pushed"), - Commit: commitRequestField(g, namespace, name, "{.status.commit}"), - Branch: commitRequestField(g, namespace, name, "{.status.branch}"), + return saveRequestOutcome{ + Ready: saveRequestField(g, namespace, name, readyPath+`.status}`), + Reason: saveRequestField(g, namespace, name, readyPath+`.reason}`), + Message: saveRequestField(g, namespace, name, readyPath+`.message}`), + Stalled: saveRequestCondition(g, namespace, name, "Stalled"), + Pushed: saveRequestCondition(g, namespace, name, "Pushed"), + Commit: saveRequestField(g, namespace, name, "{.status.commit}"), + Branch: saveRequestField(g, namespace, name, "{.status.branch}"), } } -// expectCommitRequestCommitted asserts the request reached the COMMITTED ending, and returns the +// expectSaveRequestCommitted asserts the request reached the COMMITTED ending, and returns the // commit it reported. // // It gives up early on any other terminal ending rather than polling to the timeout. A terminal // outcome is final — the controller will not revisit it — so continuing to re-read it cannot // change the result; it only delays the report and buries the reason. Failing at the moment the // ending is known means the message names it: "NoWindow" instead of an empty string. -func expectCommitRequestCommitted(g Gomega, namespace, name, diagnostics string) string { - outcome := readCommitRequestOutcome(g, namespace, name) - if outcome.isTerminal() && outcome.Reason != commitRequestReasonCommitted { +func expectSaveRequestCommitted(g Gomega, namespace, name, diagnostics string) string { + outcome := readSaveRequestOutcome(g, namespace, name) + if outcome.isTerminal() && outcome.Reason != saveRequestReasonCommitted { StopTrying(fmt.Sprintf( - "the CommitRequest reached a terminal outcome that is not a commit, so waiting longer "+ + "the SaveRequest reached a terminal outcome that is not a commit, so waiting longer "+ "cannot help.\n%s\n%s", outcome, diagnostics)).Now() } - g.Expect(outcome.Ready).To(Equal("True"), "CommitRequest is not Ready yet\n%s\n%s", outcome, diagnostics) - g.Expect(outcome.Reason).To(Equal(commitRequestReasonCommitted), + g.Expect(outcome.Ready).To(Equal("True"), "SaveRequest is not Ready yet\n%s\n%s", outcome, diagnostics) + g.Expect(outcome.Reason).To(Equal(saveRequestReasonCommitted), "Ready=True must mean a commit, not a benign rejection\n%s\n%s", outcome, diagnostics) g.Expect(outcome.Pushed).To(Equal("True"), - "a committed CommitRequest must report Pushed=True\n%s", outcome) - g.Expect(outcome.Commit).NotTo(BeEmpty(), "a committed CommitRequest must report status.commit\n%s", outcome) + "a committed SaveRequest must report Pushed=True\n%s", outcome) + g.Expect(outcome.Commit).NotTo(BeEmpty(), "a committed SaveRequest must report status.commit\n%s", outcome) return outcome.Commit } -// windowedDeploymentPipeline is the pipeline a CommitRequest suite saves through: a namespace and +// windowedDeploymentPipeline is the pipeline a SaveRequest suite saves through: a namespace and // Gitea repo of its own, a GitProvider, a GitTarget with a long spec.commit.window, and a WatchRule // on Deployments. type windowedDeploymentPipeline struct { diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index d3754817..73afc90f 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -36,8 +36,8 @@ import ( // the GitTarget mirrors that workspace, and its FluxClaim holds the Kustomization through the // install's own provider. It shows the startup converge landing in the workspace, an edit made // there withheld from the apply's revert by the REMOTE object's managedFields, two -// CommitRequests that take over their edits' windows (the workspace sends no audit facts) and -// report the apply, and a SaveRequest created in the workspace itself. +// SaveRequests that take over their edits' windows (the workspace sends no audit facts) and +// report the apply, and a RemoteSaveRequest created in the workspace itself. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), // and the kcp specs additionally Skip when kcp is not installed — so a default `task test-e2e` @@ -825,10 +825,10 @@ spec: applyCatchUpSave(testNs, name, p.targetName) r.editThroughAPI(timingConfigMap, value) Eventually(func(g Gomega) { - g.Expect(commitRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) - g.Expect(commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(Equal(commitRequestReasonCommitted)) - commits[name] = commitRequestField(g, testNs, name, "{.status.commit}") + g.Expect(saveRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) + g.Expect(saveRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal(saveRequestReasonCommitted)) + commits[name] = saveRequestField(g, testNs, name, "{.status.commit}") g.Expect(commits[name]).NotTo(BeEmpty()) }, 2*time.Minute, 2*time.Second).Should(Succeed()) committed, inGit := r.committedValue(Default, timingConfigMap) @@ -836,54 +836,54 @@ spec: Expect(committed).To(Equal(value), "the save's commit holds its edit") message, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commits[name]) Expect(err).NotTo(HaveOccurred()) - Expect(message).To(ContainSubstring("Author-Source: CommitRequest "+testNs+"/"+name), + Expect(message).To(ContainSubstring("Author-Source: SaveRequest "+testNs+"/"+name), "the commit says its author is the request's submitter") } last := commits[names[len(names)-1]] Eventually(func(g Gomega) { for _, name := range names { - applied := commitRequestCondition(g, testNs, name, "Applied") - reason := commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) + applied := saveRequestCondition(g, testNs, name, "Applied") + reason := saveRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) if applied == "False" { - StopTrying(fmt.Sprintf("CommitRequest %s is Applied=False (%s): only a commit the branch "+ + StopTrying(fmt.Sprintf("SaveRequest %s is Applied=False (%s): only a commit the branch "+ "lost may be", name, reason)).Now() } - g.Expect(applied).To(Equal("True"), "CommitRequest %s Applied (%s)", name, reason) + g.Expect(applied).To(Equal("True"), "SaveRequest %s Applied (%s)", name, reason) g.Expect(reason).To(Equal("Applied")) // A later save's commit holds the earlier one's, so one apply may settle both. - g.Expect(commitRequestField(g, testNs, name, "{.status.appliedRevision}")). + g.Expect(saveRequestField(g, testNs, name, "{.status.appliedRevision}")). To(Or(ContainSubstring(commits[name]), ContainSubstring(last))) } - g.Expect(commitRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). + g.Expect(saveRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). To(ContainSubstring(last)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.eventuallyEverywhere(timingConfigMap, "saved-2", echoApplyBudget) }) - // A save made in the workspace itself: a SaveRequest, served because the workspace's + // A save made in the workspace itself: a RemoteSaveRequest, served because the workspace's // KubeProvider enables them. It names no GitTarget; it saves the window of every GitTarget that // mirrors its namespace. kcp sends no audit facts, so the request names nobody, like the edit, // and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is // looked for first, for the whole attribution grace, so the edit is made once the request is // waiting for a window. - It("saves an edit made in the workspace with a SaveRequest created there", func() { - By("installing the SaveRequest CRD in the workspace, and serving the workspace's SaveRequests") - _, err := kcp.wsKubectl(hash, "apply", "-f", "config/crd/bases/source.configbutler.ai_saverequests.yaml") + It("saves an edit made in the workspace with a RemoteSaveRequest created there", func() { + By("installing the RemoteSaveRequest CRD in the workspace, and serving the workspace's RemoteSaveRequests") + _, err := kcp.wsKubectl(hash, "apply", "-f", "config/crd/bases/source.configbutler.ai_remotesaverequests.yaml") Expect(err).NotTo(HaveOccurred()) _, err = kcp.wsKubectl(hash, "wait", "--for=condition=Established", - "crd/saverequests.source.configbutler.ai", "--timeout=60s") + "crd/remotesaverequests.source.configbutler.ai", "--timeout=60s") Expect(err).NotTo(HaveOccurred()) _, err = kubectlRunInNamespace(testNs, "patch", "kubeprovider", ws, "--type=merge", "-p", - `{"spec":{"saveRequests":"Enabled"}}`) + `{"spec":{"remoteSaveRequests":"Enabled"}}`) Expect(err).NotTo(HaveOccurred()) - By("creating a SaveRequest in the workspace, and editing once it waits for a window") + By("creating a RemoteSaveRequest in the workspace, and editing once it waits for a window") const name, message = "sc-flux-workspace-save", "e2e: save from the workspace" - createWorkspaceSaveRequest(kcp, hash, testNs, name, message) + createWorkspaceRemoteSaveRequest(kcp, hash, testNs, name, message) saveField := func(g Gomega, jsonPath string) string { - out, getErr := kcp.wsKubectl(hash, "-n", testNs, "get", "saverequest", name, "-o", "jsonpath="+jsonPath) + out, getErr := kcp.wsKubectl(hash, "-n", testNs, "get", "remotesaverequest", name, "-o", "jsonpath="+jsonPath) g.Expect(getErr).NotTo(HaveOccurred()) return strings.TrimSpace(out) } @@ -920,16 +920,16 @@ spec: }) }) -// createWorkspaceSaveRequest creates the SaveRequest name in namespace of a workspace, asking to catch +// createWorkspaceRemoteSaveRequest creates the RemoteSaveRequest name in namespace of a workspace, asking to catch // up. It waits up to 30s for a window: an edit from a cluster with no audit stream reaches its window // only after the attribution grace. -func createWorkspaceSaveRequest(kcp *kcpTunnel, hash, namespace, name, message string) { +func createWorkspaceRemoteSaveRequest(kcp *kcpTunnel, hash, namespace, name, message string) { GinkgoHelper() - f, err := os.CreateTemp("", "e2e-saverequest-*.yaml") + f, err := os.CreateTemp("", "e2e-remotesaverequest-*.yaml") Expect(err).NotTo(HaveOccurred()) defer func() { _ = os.Remove(f.Name()) }() _, err = fmt.Fprintf(f, `apiVersion: source.configbutler.ai/v1alpha1 -kind: SaveRequest +kind: RemoteSaveRequest metadata: name: %s namespace: %s @@ -943,7 +943,7 @@ spec: Expect(err).NotTo(HaveOccurred()) Expect(f.Close()).To(Succeed()) _, err = kcp.wsKubectl(hash, "create", "-f", f.Name()) - Expect(err).NotTo(HaveOccurred(), "failed to create SaveRequest %s/%s in the workspace", namespace, name) + Expect(err).NotTo(HaveOccurred(), "failed to create RemoteSaveRequest %s/%s in the workspace", namespace, name) } // applyFluxIntoWorkspace creates, in flux-system, a GitRepository following main and a Kustomization @@ -994,13 +994,13 @@ spec: }, 3*time.Minute, 2*time.Second).Should(Succeed(), "the Kustomization must apply its folder into the workspace") } -// applyCatchUpSave creates the CommitRequest name on gitTarget, asking its applier to catch up once +// applyCatchUpSave creates the SaveRequest name on gitTarget, asking its applier to catch up once // it is pushed. It waits 45s for its window: a write from a cluster with no audit stream reaches the // window only after the attribution grace. func applyCatchUpSave(namespace, name, gitTarget string) { GinkgoHelper() _, err := kubectlRunWithStdin(namespace, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest +kind: SaveRequest metadata: name: %s namespace: %s @@ -1014,5 +1014,5 @@ spec: attachTimeout: "45s" maxDuration: "4s" `, name, namespace, gitTarget, name), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s", name) + Expect(err).NotTo(HaveOccurred(), "failed to create SaveRequest %s", name) } From 043274212c0e4b5066e077dcaaa07a657c66667a Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 19:19:06 +0000 Subject: [PATCH 18/33] feat(remotesaverequest): a GitTarget admits RemoteSaveRequests, and a save holds only its namespace's edits Three gaps an API review found in the source-cluster save: - A save was not confined to its namespace. A commit window groups by GitTarget and author, so a save from one namespace could commit the same author's edits in another, or, on a cluster without audit, anyone's. A RemoteSaveRequest now attaches only to a window whose writes are all in its namespace; once it has, a write anywhere else, cluster-scoped ones included, opens a window of its own. A window that already holds writes from elsewhere is left alone, and the save ends WindowOutsideNamespace. - A cluster-scoped watch matched every namespace: a collection with no namespace is both "cluster-scoped" and "every namespace". The match now reads the GitTarget's watched types and counts namespaced ones. - The GitTarget's owner could not consent. Enabling the requests on a KubeProvider made every mirroring GitTarget eligible, and a request steers a window's message, its timers and an apply. The provider's switch now only makes them reachable: GitTarget spec.allowedRemoteSaveRequests admits them by source namespace (names or "*") and an optional requestSelector, and admits none when omitted. A request a mirroring GitTarget does not admit is Ready=False, Stalled=True, NotAllowed. status.targets is status.gitTargets, a list keyed by the GitTarget's namespace and name. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../v1alpha1/remotesaverequest_types.go | 64 ++-- api/source/v1alpha1/zz_generated.deepcopy.go | 36 +-- api/v1alpha3/access_grants.go | 54 ++++ api/v1alpha3/access_grants_test.go | 44 +++ api/v1alpha3/gittarget_types.go | 19 ++ api/v1alpha3/kubeprovider_types.go | 17 +- api/v1alpha3/zz_generated.deepcopy.go | 32 ++ .../crd/bases/configbutler.ai_gittargets.yaml | 91 ++++++ .../bases/configbutler.ai_kubeproviders.yaml | 14 +- ...ce.configbutler.ai_remotesaverequests.yaml | 68 ++-- docs/configuration.md | 3 +- docs/future/source-side-save-plan.md | 47 ++- docs/rbac.md | 16 +- hack/crdfields/fields.yaml | 7 +- .../access_grants_admission_test.go | 19 ++ internal/controller/saverequest_finalize.go | 7 + internal/git/branch_worker.go | 1 + internal/git/open_window.go | 21 +- internal/git/save_request_attach.go | 39 +++ internal/git/save_request_attach_loop.go | 21 +- internal/git/save_request_scope_test.go | 146 +++++++++ internal/watch/remote_save_requests.go | 132 +++++--- internal/watch/remote_save_requests_test.go | 296 ++++++++++++++++-- test/e2e/source_cluster_e2e_test.go | 55 +++- 24 files changed, 1066 insertions(+), 183 deletions(-) create mode 100644 internal/git/save_request_scope_test.go diff --git a/api/source/v1alpha1/remotesaverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go index 40d388ae..182c6270 100644 --- a/api/source/v1alpha1/remotesaverequest_types.go +++ b/api/source/v1alpha1/remotesaverequest_types.go @@ -15,7 +15,8 @@ import ( // The spec is immutable, compared as SaveRequestSpec compares it: durations as durations, so a typed // client's re-spelling of "1m" as "1m0s" is not a change. -// RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It is immutable. +// RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It +// is immutable. // +kubebuilder:validation:XValidation:rule="has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="RemoteSaveRequest spec is immutable after creation" type RemoteSaveRequestSpec struct { // Message is the commit message this request asks for. Omitted, each @@ -57,8 +58,8 @@ type RemoteSaveRequestSpec struct { CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } -// CatchUpPolicy is when a RemoteSaveRequest's commit is applied by a GitTarget's -// applier. +// CatchUpPolicy is when a RemoteSaveRequest's commit is applied by a +// GitTarget's applier. type CatchUpPolicy string const ( @@ -68,9 +69,9 @@ const ( CatchUpAfterPush CatchUpPolicy = "AfterPush" ) -// RemoteSaveRequestWindow is how long a RemoteSaveRequest waits for a commit window, and -// when that window closes. A window is only ever opened by an edit; a -// request waits for one and never opens one. +// RemoteSaveRequestWindow is how long a RemoteSaveRequest waits for a commit +// window, and when that window closes. A window is only ever opened by an +// edit; a request waits for one and never opens one. // +kubebuilder:validation:XValidation:rule="duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="RemoteSaveRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" type RemoteSaveRequestWindow struct { @@ -117,19 +118,23 @@ type RemoteSaveRequestStatus struct { // +optional Author string `json:"author,omitempty"` - // Targets is the save in each GitTarget that mirrors this namespace. + // GitTargets is the save in each GitTarget that admitted this request. // +optional - // +listType=atomic + // +listType=map + // +listMapKey=namespace + // +listMapKey=name // +kubebuilder:validation:MaxItems=64 - Targets []RemoteSaveRequestTarget `json:"targets,omitempty"` + GitTargets []RemoteSaveRequestGitTarget `json:"gitTargets,omitempty"` // Conditions are the latest observations of this RemoteSaveRequest's state. // Ready is True once the request ended without error in every GitTarget, // with reason "Committed" when one of them committed, and otherwise the // reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - // "NoGitTarget" when no GitTarget mirrors this namespace, or - // "AuthorUnresolved" when this cluster sends audit events and none named - // the request's creator. Ready=False with Stalled=True is a failure. + // "WindowOutsideNamespace", "NoGitTarget" when no GitTarget mirrors this + // namespace, or "AuthorUnresolved" when this cluster sends audit events + // and none named the request's creator. Ready=False with Stalled=True is a + // failure, with reason "NotAllowed" when GitTargets mirror this namespace + // and none admits the request. // // Applied, with spec.catchUp "AfterPush", is True once every GitTarget's // applier applied a revision that holds its commit, or needed none, and @@ -142,16 +147,21 @@ type RemoteSaveRequestStatus struct { Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` } -// RemoteSaveRequestTarget is the save in one GitTarget. -type RemoteSaveRequestTarget struct { - // GitTarget is the GitTarget, as namespace/name on the cluster GitOps - // Reverser runs in. +// RemoteSaveRequestGitTarget is the save in one GitTarget. +type RemoteSaveRequestGitTarget struct { + // Namespace is the GitTarget's namespace, on the cluster GitOps Reverser + // runs in. + // +required + Namespace string `json:"namespace"` + + // Name is the GitTarget's name. // +required - GitTarget string `json:"gitTarget"` + Name string `json:"name"` // Reason is how the save ended in this GitTarget, as the Ready reasons - // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or - // "Failed". Empty while it runs. + // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", + // "WindowOutsideNamespace" when the open window held edits made outside + // this namespace, or "Failed". Empty while it runs. // +optional Reason string `json:"reason,omitempty"` @@ -186,13 +196,15 @@ type RemoteSaveRequestTarget struct { // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// RemoteSaveRequest asks GitOps Reverser to save the edits made in its namespace: -// the commit window of each GitTarget that mirrors this namespace gets its -// message and closes on its timers. Create it on the cluster the edits were -// made on, as JSON. Its creator is found from this cluster's audit events, -// as the edits' authors are, so it saves only its creator's edits; on a -// cluster with no audit it names nobody, and saves edits that name nobody. -// Its spec is immutable, and status reports each GitTarget's commit. +// RemoteSaveRequest asks GitOps Reverser to save the edits made in its +// namespace: the commit window of each GitTarget that mirrors this namespace +// and admits the request, in spec.allowedRemoteSaveRequests, gets its message +// and closes on its timers. Edits made elsewhere are never part of its save. +// Create it on the cluster the edits were made on, as JSON. Its creator is +// found from this cluster's audit events, as the edits' authors are, so it +// saves only its creator's edits; on a cluster with no audit it names +// nobody, and saves edits that name nobody. Its spec is immutable, and +// status reports each GitTarget's commit. type RemoteSaveRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/api/source/v1alpha1/zz_generated.deepcopy.go b/api/source/v1alpha1/zz_generated.deepcopy.go index 36ca5bd0..593256b0 100644 --- a/api/source/v1alpha1/zz_generated.deepcopy.go +++ b/api/source/v1alpha1/zz_generated.deepcopy.go @@ -38,6 +38,21 @@ func (in *RemoteSaveRequest) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *RemoteSaveRequestGitTarget) DeepCopyInto(out *RemoteSaveRequestGitTarget) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestGitTarget. +func (in *RemoteSaveRequestGitTarget) DeepCopy() *RemoteSaveRequestGitTarget { + if in == nil { + return nil + } + out := new(RemoteSaveRequestGitTarget) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestList) DeepCopyInto(out *RemoteSaveRequestList) { *out = *in @@ -93,9 +108,9 @@ func (in *RemoteSaveRequestSpec) DeepCopy() *RemoteSaveRequestSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestStatus) DeepCopyInto(out *RemoteSaveRequestStatus) { *out = *in - if in.Targets != nil { - in, out := &in.Targets, &out.Targets - *out = make([]RemoteSaveRequestTarget, len(*in)) + if in.GitTargets != nil { + in, out := &in.GitTargets, &out.GitTargets + *out = make([]RemoteSaveRequestGitTarget, len(*in)) copy(*out, *in) } if in.Conditions != nil { @@ -117,21 +132,6 @@ func (in *RemoteSaveRequestStatus) DeepCopy() *RemoteSaveRequestStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestTarget) DeepCopyInto(out *RemoteSaveRequestTarget) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestTarget. -func (in *RemoteSaveRequestTarget) DeepCopy() *RemoteSaveRequestTarget { - if in == nil { - return nil - } - out := new(RemoteSaveRequestTarget) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestWindow) DeepCopyInto(out *RemoteSaveRequestWindow) { *out = *in diff --git a/api/v1alpha3/access_grants.go b/api/v1alpha3/access_grants.go index 8518a336..19864d12 100644 --- a/api/v1alpha3/access_grants.go +++ b/api/v1alpha3/access_grants.go @@ -219,6 +219,60 @@ type KubeProviderApplierClaimGrant struct { ApplierNamespaces []ScopeNamespace `json:"applierNamespaces"` } +// SourceNamespace is a namespace of a source cluster, by exact name, or "*" +// for every namespace. +// +kubebuilder:validation:MaxLength=63 +// +kubebuilder:validation:XValidation:rule="self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$')",message="a source namespace is a namespace name, or '*' alone; variables are refused, because a source cluster's namespaces are not the control cluster's" +type SourceNamespace string + +// GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some +// namespaces of the source cluster to save a GitTarget's commit windows. +type GitTargetRemoteSaveRequestGrant struct { + // SourceNamespaces are the namespaces, on the cluster the GitTarget's + // spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: + // exact names, or "*" for every namespace the GitTarget mirrors. + // +required + // +listType=set + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=64 + SourceNamespaces []SourceNamespace `json:"sourceNamespaces"` + + // RequestSelector narrows this grant to the RemoteSaveRequests whose labels + // match. Omitted, it admits every request from sourceNamespaces. Whoever + // creates a request picks its labels, so this picks requests and proves + // nothing about who made them. + // +optional + RequestSelector *metav1.LabelSelector `json:"requestSelector,omitempty"` +} + +// AdmitsRemoteSaveRequest reports whether spec.allowedRemoteSaveRequests admits a RemoteSaveRequest +// in namespace, on the source cluster, carrying requestLabels. A selector that does not convert is an +// error, never a silent allow; it is returned only when no grant admits. +func (t *GitTarget) AdmitsRemoteSaveRequest(namespace string, requestLabels map[string]string) (bool, error) { + var errs []error + for i := range t.Spec.AllowedRemoteSaveRequests { + grant := &t.Spec.AllowedRemoteSaveRequests[i] + if !slices.ContainsFunc(grant.SourceNamespaces, func(ns SourceNamespace) bool { + return ns == "*" || string(ns) == namespace + }) { + continue + } + if grant.RequestSelector == nil { + return true, nil + } + selector, err := metav1.LabelSelectorAsSelector(grant.RequestSelector) + if err != nil { + errs = append(errs, fmt.Errorf("grant %d: requestSelector %s: %w", i, + metav1.FormatLabelSelector(grant.RequestSelector), err)) + continue + } + if selector.Matches(labels.Set(requestLabels)) { + return true, nil + } + } + return false, errors.Join(errs...) +} + // The API server defaults an omitted grant list on GitBranch and KubeProvider (the // +kubebuilder:default markers on their fields), so a list read from it is never nil. These mirror // those markers for an object that did not come from the API server, such as one built in memory or diff --git a/api/v1alpha3/access_grants_test.go b/api/v1alpha3/access_grants_test.go index cf3fc49c..1519c2c6 100644 --- a/api/v1alpha3/access_grants_test.go +++ b/api/v1alpha3/access_grants_test.go @@ -349,3 +349,47 @@ func TestGitBranch_GitProviderKey(t *testing.T) { branch.Spec.GitProviderRef.Namespace = "platform" assert.Equal(t, "platform/shared-repo", branch.GitProviderKey().String()) } + +// A GitTarget admits a RemoteSaveRequest only when one grant admits both its namespace and its +// labels; a grant whose selector does not convert admits nothing, and says why only when no other +// grant admits. +func TestGitTarget_AdmitsRemoteSaveRequest(t *testing.T) { + broken := &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ + {Key: "app", Operator: "Near"}, + }} + cart := &metav1.LabelSelector{MatchLabels: map[string]string{"app": "cart"}} + target := func(grants ...GitTargetRemoteSaveRequestGrant) *GitTarget { + return &GitTarget{Spec: GitTargetSpec{AllowedRemoteSaveRequests: grants}} + } + labels := map[string]string{"app": "cart"} + + admitted, err := target().AdmitsRemoteSaveRequest("shop", labels) + require.NoError(t, err) + assert.False(t, admitted, "nothing is admitted by default") + + admitted, err = target(GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"*"}}). + AdmitsRemoteSaveRequest("shop", labels) + require.NoError(t, err) + assert.True(t, admitted) + + admitted, err = target( + GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: cart}, + ). + AdmitsRemoteSaveRequest("shop", map[string]string{"app": "admin"}) + require.NoError(t, err) + assert.False(t, admitted, "its labels must match too") + + admitted, err = target( + GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: broken}, + ). + AdmitsRemoteSaveRequest("shop", labels) + require.Error(t, err) + assert.False(t, admitted) + + admitted, err = target( + GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: broken}, + GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: cart}, + ).AdmitsRemoteSaveRequest("shop", labels) + require.NoError(t, err, "another grant admits") + assert.True(t, admitted) +} diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 474a01c3..41593534 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -186,6 +186,25 @@ type GitTargetSpec struct { // +optional Commit *GitTargetCommitSpec `json:"commit,omitempty"` + // Consent lives on the GitTarget, not on the KubeProvider that serves the requests: the provider's + // owner lends a cluster, and whether a save made on it may steer this folder's commits, with its + // message, its timers and an apply, is the folder's owner's decision. Nothing is admitted by + // default, because a source cluster's namespaces are not the control cluster's, so there is no + // "Same" to default to. + + // AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster + // spec.kubeProviderRef names, that may save this GitTarget's commit + // windows. A request is admitted when one grant admits both its namespace + // and its labels. Omitted or empty, none is. A request that GitTargets + // mirror but none admits reports "NotAllowed". The KubeProvider must also + // set spec.remoteSaveRequests "Enabled". + // + // An admitted request saves only the edits made in its own namespace. + // +optional + // +listType=atomic + // +kubebuilder:validation:MaxItems=16 + AllowedRemoteSaveRequests []GitTargetRemoteSaveRequestGrant `json:"allowedRemoteSaveRequests,omitempty"` + // The scan keeps running while writes are suspended: freezing status.placement too would leave // a stale answer at the moment it costs most. A suspended target is Ready=True reason // Suspended, because not writing is the configured outcome rather than ill health. A local diff --git a/api/v1alpha3/kubeprovider_types.go b/api/v1alpha3/kubeprovider_types.go index da954183..663bde97 100644 --- a/api/v1alpha3/kubeprovider_types.go +++ b/api/v1alpha3/kubeprovider_types.go @@ -100,22 +100,25 @@ type KubeProviderSpec struct { // +optional Attribution *KubeProviderAttribution `json:"attribution,omitempty"` - // RemoteSaveRequests says whether GitOps Reverser serves the RemoteSaveRequests - // (source.configbutler.ai) created on this cluster. "Enabled" watches - // them, saves the commit windows of the GitTargets that mirror each one's - // namespace through this KubeProvider, and writes each one's status back. + // RemoteSaveRequests says whether GitOps Reverser serves the + // RemoteSaveRequests (source.configbutler.ai) created on this cluster. + // "Enabled" watches them, saves the commit windows of the GitTargets that + // mirror each one's namespace through this KubeProvider and admit it in + // spec.allowedRemoteSaveRequests, and writes each one's status back. // Defaults to "Disabled". // // "Enabled" writes to this cluster: the credential needs get, list, watch - // and delete on remotesaverequests, and update on remotesaverequests/status. The - // RemoteSaveRequest CRD must be installed on the cluster. + // and delete on remotesaverequests, and update on + // remotesaverequests/status. The RemoteSaveRequest CRD must be installed + // on the cluster. // +optional // +kubebuilder:validation:Enum=Disabled;Enabled // +kubebuilder:default=Disabled RemoteSaveRequests RemoteSaveRequestsMode `json:"remoteSaveRequests,omitempty"` } -// RemoteSaveRequestsMode is whether a KubeProvider's RemoteSaveRequests are served. +// RemoteSaveRequestsMode is whether a KubeProvider's RemoteSaveRequests are +// served. type RemoteSaveRequestsMode string const ( diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index cc3ec5d1..95be0da2 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -1169,6 +1169,31 @@ func (in *GitTargetPlacementStatus) DeepCopy() *GitTargetPlacementStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitTargetRemoteSaveRequestGrant) DeepCopyInto(out *GitTargetRemoteSaveRequestGrant) { + *out = *in + if in.SourceNamespaces != nil { + in, out := &in.SourceNamespaces, &out.SourceNamespaces + *out = make([]SourceNamespace, len(*in)) + copy(*out, *in) + } + if in.RequestSelector != nil { + in, out := &in.RequestSelector, &out.RequestSelector + *out = new(v1.LabelSelector) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetRemoteSaveRequestGrant. +func (in *GitTargetRemoteSaveRequestGrant) DeepCopy() *GitTargetRemoteSaveRequestGrant { + if in == nil { + return nil + } + out := new(GitTargetRemoteSaveRequestGrant) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetRetentionStatus) DeepCopyInto(out *GitTargetRetentionStatus) { *out = *in @@ -1223,6 +1248,13 @@ func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = new(GitTargetCommitSpec) (*in).DeepCopyInto(*out) } + if in.AllowedRemoteSaveRequests != nil { + in, out := &in.AllowedRemoteSaveRequests, &out.AllowedRemoteSaveRequests + *out = make([]GitTargetRemoteSaveRequestGrant, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetSpec. diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index 5bd40d06..3dbb019d 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -107,6 +107,97 @@ spec: spec: description: spec defines the desired state of GitTarget. properties: + allowedRemoteSaveRequests: + description: |- + AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster + spec.kubeProviderRef names, that may save this GitTarget's commit + windows. A request is admitted when one grant admits both its namespace + and its labels. Omitted or empty, none is. A request that GitTargets + mirror but none admits reports "NotAllowed". The KubeProvider must also + set spec.remoteSaveRequests "Enabled". + + An admitted request saves only the edits made in its own namespace. + items: + description: |- + GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some + namespaces of the source cluster to save a GitTarget's commit windows. + properties: + requestSelector: + description: |- + RequestSelector narrows this grant to the RemoteSaveRequests whose labels + match. Omitted, it admits every request from sourceNamespaces. Whoever + creates a request picks its labels, so this picks requests and proves + nothing about who made them. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + sourceNamespaces: + description: |- + SourceNamespaces are the namespaces, on the cluster the GitTarget's + spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: + exact names, or "*" for every namespace the GitTarget mirrors. + items: + description: |- + SourceNamespace is a namespace of a source cluster, by exact name, or "*" + for every namespace. + maxLength: 63 + type: string + x-kubernetes-validations: + - message: a source namespace is a namespace name, or '*' + alone; variables are refused, because a source cluster's + namespaces are not the control cluster's + rule: self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$') + maxItems: 64 + minItems: 1 + type: array + x-kubernetes-list-type: set + required: + - sourceNamespaces + type: object + maxItems: 16 + type: array + x-kubernetes-list-type: atomic applierClaimRef: description: |- ApplierClaimRef names the FluxClaim or ArgoCDClaim, in this namespace, that diff --git a/config/crd/bases/configbutler.ai_kubeproviders.yaml b/config/crd/bases/configbutler.ai_kubeproviders.yaml index e605af68..8d9d3c01 100644 --- a/config/crd/bases/configbutler.ai_kubeproviders.yaml +++ b/config/crd/bases/configbutler.ai_kubeproviders.yaml @@ -512,15 +512,17 @@ spec: remoteSaveRequests: default: Disabled description: |- - RemoteSaveRequests says whether GitOps Reverser serves the RemoteSaveRequests - (source.configbutler.ai) created on this cluster. "Enabled" watches - them, saves the commit windows of the GitTargets that mirror each one's - namespace through this KubeProvider, and writes each one's status back. + RemoteSaveRequests says whether GitOps Reverser serves the + RemoteSaveRequests (source.configbutler.ai) created on this cluster. + "Enabled" watches them, saves the commit windows of the GitTargets that + mirror each one's namespace through this KubeProvider and admit it in + spec.allowedRemoteSaveRequests, and writes each one's status back. Defaults to "Disabled". "Enabled" writes to this cluster: the credential needs get, list, watch - and delete on remotesaverequests, and update on remotesaverequests/status. The - RemoteSaveRequest CRD must be installed on the cluster. + and delete on remotesaverequests, and update on + remotesaverequests/status. The RemoteSaveRequest CRD must be installed + on the cluster. enum: - Disabled - Enabled diff --git a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml index 13aa37d8..78611ac3 100644 --- a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml +++ b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml @@ -35,13 +35,15 @@ spec: schema: openAPIV3Schema: description: |- - RemoteSaveRequest asks GitOps Reverser to save the edits made in its namespace: - the commit window of each GitTarget that mirrors this namespace gets its - message and closes on its timers. Create it on the cluster the edits were - made on, as JSON. Its creator is found from this cluster's audit events, - as the edits' authors are, so it saves only its creator's edits; on a - cluster with no audit it names nobody, and saves edits that name nobody. - Its spec is immutable, and status reports each GitTarget's commit. + RemoteSaveRequest asks GitOps Reverser to save the edits made in its + namespace: the commit window of each GitTarget that mirrors this namespace + and admits the request, in spec.allowedRemoteSaveRequests, gets its message + and closes on its timers. Edits made elsewhere are never part of its save. + Create it on the cluster the edits were made on, as JSON. Its creator is + found from this cluster's audit events, as the edits' authors are, so it + saves only its creator's edits; on a cluster with no audit it names + nobody, and saves edits that name nobody. Its spec is immutable, and + status reports each GitTarget's commit. properties: apiVersion: description: |- @@ -170,9 +172,11 @@ spec: Ready is True once the request ended without error in every GitTarget, with reason "Committed" when one of them committed, and otherwise the reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - "NoGitTarget" when no GitTarget mirrors this namespace, or - "AuthorUnresolved" when this cluster sends audit events and none named - the request's creator. Ready=False with Stalled=True is a failure. + "WindowOutsideNamespace", "NoGitTarget" when no GitTarget mirrors this + namespace, or "AuthorUnresolved" when this cluster sends audit events + and none named the request's creator. Ready=False with Stalled=True is a + failure, with reason "NotAllowed" when GitTargets mirror this namespace + and none admits the request. Applied, with spec.catchUp "AfterPush", is True once every GitTarget's applier applied a revision that holds its commit, or needed none, and @@ -235,16 +239,11 @@ spec: x-kubernetes-list-map-keys: - type x-kubernetes-list-type: map - observedGeneration: - description: ObservedGeneration is the latest generation observed - by GitOps Reverser. - format: int64 - type: integer - targets: - description: Targets is the save in each GitTarget that mirrors this - namespace. + gitTargets: + description: GitTargets is the save in each GitTarget that admitted + this request. items: - description: RemoteSaveRequestTarget is the save in one GitTarget. + description: RemoteSaveRequestGitTarget is the save in one GitTarget. properties: applied: description: |- @@ -262,26 +261,39 @@ spec: description: Commit is the commit the save made, once it is on the remote. type: string - gitTarget: - description: |- - GitTarget is the GitTarget, as namespace/name on the cluster GitOps - Reverser runs in. - type: string message: description: Message says more about the reason. type: string + name: + description: Name is the GitTarget's name. + type: string + namespace: + description: |- + Namespace is the GitTarget's namespace, on the cluster GitOps Reverser + runs in. + type: string reason: description: |- Reason is how the save ended in this GitTarget, as the Ready reasons - above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or - "Failed". Empty while it runs. + above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", + "WindowOutsideNamespace" when the open window held edits made outside + this namespace, or "Failed". Empty while it runs. type: string required: - - gitTarget + - name + - namespace type: object maxItems: 64 type: array - x-kubernetes-list-type: atomic + x-kubernetes-list-map-keys: + - namespace + - name + x-kubernetes-list-type: map + observedGeneration: + description: ObservedGeneration is the latest generation observed + by GitOps Reverser. + format: int64 + type: integer type: object required: - spec diff --git a/docs/configuration.md b/docs/configuration.md index 4eb8054b..ad7aefc5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -120,6 +120,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | | `commit.unattributedWindows` | `Keep` | How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | +| `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may save this target's windows. See [`RemoteSaveRequest`](#remotesaverequest) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | | `placement.useKustomize` | off | Keep the folder a kustomize folder. See [keeping the folder a kustomize folder](#keeping-the-folder-a-kustomize-folder-specplacementusekustomize) | @@ -195,7 +196,7 @@ The references are immutable: to hold another `Application`, delete the applier | `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | -| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit is pushed; `status.targets` reports it | +| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit is pushed; `status.gitTargets` reports it | diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index 2de45c0a..903d5cd4 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -9,15 +9,19 @@ It also plans the second step: letting a save take over a window whose writes na opt-in, for a source cluster that cannot send audit events at all. There it is the only way a save can put its author's name on the commit. -Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The source-side save is `RemoteSaveRequest` -(`source.configbutler.ai/v1alpha1`, `internal/watch/save_requests.go`), served on a KubeProvider with -`spec.remoteSaveRequests: Enabled`. The takeover is opt-in, as described below: the GitTarget's -`spec.commit.unattributedWindows` and the SaveRequest's `spec.window.author`, each `SavedBy` or -`TakeOver`. The open questions are answered as proposed: `RemoteSaveRequest` in its own group, a default -`attachTimeout` of `5s`, the reason `AuthorUnresolved`, and no extra queue for a second save on a -GitTarget beyond the worker's own (a window carries one request at a time). - -This revision takes in gitops-api's review of the first draft. +Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The two request kinds are a pair: +`SaveRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and +`RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/remote_save_requests.go`) on +the source cluster. A KubeProvider serves its cluster's requests with `spec.remoteSaveRequests: +Enabled`, and each GitTarget admits them in `spec.allowedRemoteSaveRequests` (none by default). A +`RemoteSaveRequest` saves only the edits made in its own namespace. The takeover is opt-in, as +described below: the GitTarget's `spec.commit.unattributedWindows` and the `SaveRequest`'s +`spec.window.author`, each `SavedBy` or `TakeOver`. The open questions are answered as proposed: its +own group, a default `attachTimeout` of `5s`, the reason `AuthorUnresolved`, and no extra queue for a +second save on a GitTarget beyond the worker's own (a window carries one request at a time). + +This revision takes in gitops-api's review of the first draft, and a later API review of the +implementation: see [After the API review](#after-the-api-review). ## Decisions @@ -275,3 +279,28 @@ ways forward before the branch merges: 5. Names for the takeover: `spec.commit.unattributedWindows` on the GitTarget and `spec.window.author` on the `SaveRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. 6. `cf3b97be`: turn it into the opt-in on this branch, or take it out and build the takeover later? + +## After the API review + +A review of the implementation found three things, all changed: + +1. **A save was not confined to its namespace.** A window groups by GitTarget and author, not by + namespace, so a save from `shop` could commit the same author's edits in `billing`, or, with no + audit, anyone's. A `RemoteSaveRequest` now attaches only to a window whose writes are all in its + namespace, and once attached, a write anywhere else opens a window of its own. A window that + already holds writes from elsewhere is left alone: `WindowOutsideNamespace`. Grouping every window + by namespace stays rejected; the confinement applies only to the window a remote save holds. +2. **A cluster-scoped watch matched every namespace.** A collection with no namespace is either + cluster-scoped or every namespace, and the match did not tell them apart. It now reads the + GitTarget's watched types and counts only namespaced ones. +3. **The GitTarget's owner could not consent.** Enabling the requests on a KubeProvider made every + GitTarget mirroring the namespace eligible, and a request steers a window's message, its timers + and an apply. The provider's switch still makes them reachable; `GitTarget.spec.allowedRemoteSaveRequests` + admits them, by source namespace (names or `"*"`; no `Same`, since the namespaces are another + cluster's) and an optional `requestSelector`. Omitted, it admits none: `NotAllowed`. Labels pick + requests and prove nothing about who made them. + +The status list became `status.gitTargets`, keyed by `namespace` and `name`, and the names became the +`SaveRequest`/`RemoteSaveRequest` pair. A `SaveRequest` on the control cluster gets no grant list: its +`spec.gitTargetRef` is a local reference, so the namespace's RBAC already decides who may save, and +`spec.commit.unattributedWindows` bounds what it may claim. diff --git a/docs/rbac.md b/docs/rbac.md index eac91d1e..57444f9e 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -89,12 +89,16 @@ cluster-wide `list`/`watch` of its types, and gets a clean 403 if the credential ### `remotesaverequests` on a source cluster A source credential only reads, with one opt-in exception. A `KubeProvider` with -`spec.remoteSaveRequests: Enabled` serves the [`RemoteSaveRequest`s](configuration.md#remotesaverequest) created on its -cluster, and writes their status back there. Its credential then needs, in the -`source.configbutler.ai` group, `get`, `list`, `watch` and `delete` on `remotesaverequests` (`delete` for -the finished-request TTL), and `update` on `remotesaverequests/status`. Nothing else on a source cluster is -written. The manager role carries the same rules for the cluster the operator runs in, for an -`inCluster` provider. +`spec.remoteSaveRequests: Enabled` serves the +[`RemoteSaveRequest`s](configuration.md#remotesaverequest) created on its cluster, and writes their +status back there. Its credential then needs, in the `source.configbutler.ai` group, `get`, `list`, +`watch` and `delete` on `remotesaverequests` (`delete` for the finished-request TTL), and `update` on +`remotesaverequests/status`. Nothing else on a source cluster is written. The manager role carries +the same rules for the cluster the operator runs in, for an `inCluster` provider. + +Whoever may create a `RemoteSaveRequest` in a namespace may save that namespace's edits, for every +`GitTarget` whose `spec.allowedRemoteSaveRequests` admits it. That grant, not the provider's switch, +is the `GitTarget` owner's consent. ## The operator does not read Secrets wholesale diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 76bbac9c..195d53e1 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -163,6 +163,11 @@ kinds: How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) + - path: allowedRemoteSaveRequests + default: none admitted + doc: >- + The `RemoteSaveRequest`s, by source namespace and labels, that may save this target's + windows. See [`RemoteSaveRequest`](#remotesaverequest) - path: placement.byType default: the built-in path doc: >- @@ -343,4 +348,4 @@ kinds: - path: catchUp doc: >- `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit - is pushed; `status.targets` reports it + is pushed; `status.gitTargets` reports it diff --git a/internal/controller/access_grants_admission_test.go b/internal/controller/access_grants_admission_test.go index 97cd1baf..e6b693e9 100644 --- a/internal/controller/access_grants_admission_test.go +++ b/internal/controller/access_grants_admission_test.go @@ -120,6 +120,25 @@ var _ = Describe("Access grants at admission", func() { refused(kubeProvider("allowedApplierClaims", consumers(same)), "applierNamespaces: Required value") }) + It("takes a GitTarget's RemoteSaveRequest grants by source namespace name or *, and nothing else", func() { + gitTarget := func(grants ...any) *unstructured.Unstructured { + return object("GitTarget", map[string]any{ + "gitBranchRef": map[string]any{"name": "b"}, "path": "apps", + "kubeProviderRef": map[string]any{"name": "k"}, "allowedRemoteSaveRequests": grants, + }) + } + from := func(namespaces ...any) map[string]any { return map[string]any{"sourceNamespaces": namespaces} } + accepted(gitTarget(from("shop", "billing"))) + accepted(gitTarget(from("*"))) + labelled := from("shop") + labelled["requestSelector"] = map[string]any{"matchLabels": map[string]any{"app": "cart"}} + accepted(gitTarget(labelled)) + refused(gitTarget(from("{consumer.namespace}")), "variables are refused") + refused(gitTarget(from("Shop")), "a source namespace is a namespace name") + refused(gitTarget(from()), "sourceNamespaces") + refused(gitTarget(map[string]any{}), "sourceNamespaces: Required value") + }) + // consumerNamespaces is required on every grant type and from is required inside it, with no default // for either, so a grant that leaves the "who" out admits nobody instead of its holder's namespace. // The block's own rules cannot make it required, so each list is checked here. diff --git a/internal/controller/saverequest_finalize.go b/internal/controller/saverequest_finalize.go index 1fa6d4bc..14bdd7a4 100644 --- a/internal/controller/saverequest_finalize.go +++ b/internal/controller/saverequest_finalize.go @@ -64,6 +64,9 @@ func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { return crOutcomeWindowMismatch case git.FinalizeAlreadyPresent: return crOutcomeAlreadyPresent + case git.FinalizeWindowOutsideNamespace: + // Only a RemoteSaveRequest is confined to a namespace, so a SaveRequest never ends this way. + return crOutcomeFailed default: return crOutcomeFailed } @@ -308,6 +311,10 @@ func applyFinalizeResultToStatus( case git.FinalizeAlreadyPresent: // The change already matches the remote, so the commit was dropped. resolveNothingToCommit(cr, result, crReasonAlreadyPresent, alreadyPresentMessage) + case git.FinalizeWindowOutsideNamespace: + // Only a RemoteSaveRequest is confined to a namespace (AttachSaveRequest.SourceNamespace), so + // this outcome is a bug here, as an unknown one is. + failSaveRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) default: // An empty or unknown outcome with no error is a bug, not a benign // rejection; fail it so it is not silently hidden. diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 98e53268..f435eb96 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1314,6 +1314,7 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { l.w.Log.Info("Window identity change forces finalize before appending event", "windowAuthor", l.openWindow.Author, "eventAuthor", event.UserInfo.Username, + "windowSourceNamespace", l.openWindow.sourceNamespace, "operation", event.Operation, "resource", event.Identifier.String(), "eventTarget", event.GitTargetNamespace+"/"+event.GitTargetName) diff --git a/internal/git/open_window.go b/internal/git/open_window.go index 087f3e42..d31acfc2 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -52,6 +52,15 @@ type openWindow struct { claim UnattributedClaim claimedBy UserInfo + // namespace is the source namespace of the window's writes while they all share one, "" for a + // cluster-scoped write; spansNamespaces is set once two writes' namespaces differ. + namespace string + spansNamespaces bool + // sourceNamespace confines the window to the writes of one source namespace once a request that + // saves only those attached (AttachSaveRequest.SourceNamespace): a write anywhere else cannot + // join it. Empty confines nothing. + sourceNamespace string + // timers decide when the window closes. They are the GitTarget's, snapshotted when the window // opened, until a SaveRequest attaches and replaces them with its own. timers windowTimers @@ -96,6 +105,7 @@ func newOpenWindow(e Event, writer eventContentWriter) *openWindow { GitTargetNamespace: e.GitTargetNamespace, pathToEvent: make(map[string]Event), writer: writer, + namespace: e.Identifier.Namespace, openedAt: time.Now(), } } @@ -107,13 +117,22 @@ func (w *openWindow) canAppend(e Event) bool { return e.Attribution == w.Attribution && e.UserInfo.Username == w.Author && e.GitTargetName == w.GitTarget && - e.GitTargetNamespace == w.GitTargetNamespace + e.GitTargetNamespace == w.GitTargetNamespace && + (w.sourceNamespace == "" || e.Identifier.Namespace == w.sourceNamespace) +} + +// holdsOnly reports whether every write in the window was made in namespace. +func (w *openWindow) holdsOnly(namespace string) bool { + return !w.spansNamespaces && w.namespace == namespace } // add records an event in the window. If the path was already present the // event replaces the previous one (last-write-wins inside a single window), which // it returns; otherwise pathOrder is extended. func (w *openWindow) add(e Event) (Event, bool) { + if e.Identifier.Namespace != w.namespace { + w.spansNamespaces = true + } key := windowPathKey(e, w.writer) replaced, exists := w.pathToEvent[key] if !exists { diff --git a/internal/git/save_request_attach.go b/internal/git/save_request_attach.go index d6e0381a..8988f10c 100644 --- a/internal/git/save_request_attach.go +++ b/internal/git/save_request_attach.go @@ -53,6 +53,10 @@ const ( // produced no diff — the change already matches the remote, so no commit was // made (loop prevention). Resolved at finalize, never waiting on a push. FinalizeAlreadyPresent FinalizeOutcome = "AlreadyPresent" + // FinalizeWindowOutsideNamespace means the request's own window was open during its wait but held + // writes made outside the one namespace the request saves (AttachSaveRequest.SourceNamespace), so + // it was left untouched and the wait ran out without a window this request could attach to. + FinalizeWindowOutsideNamespace FinalizeOutcome = "WindowOutsideNamespace" ) // SaveRequestPhase is where an unresolved SaveRequest stands on the worker. It is what the @@ -155,6 +159,11 @@ type AttachSaveRequest struct { // the GitTarget's KubeProvider has never received an audit fact. Once it has, a write naming no // author means its fact went missing, and that window stays another's. Unattributed UnattributedClaim + // SourceNamespace confines the save to the writes made in one namespace of the source cluster: the + // request attaches only to a window whose writes are all there, and once it has, a write anywhere + // else, cluster-scoped ones included, opens a window of its own. A RemoteSaveRequest sets its own + // namespace; empty confines nothing. + SourceNamespace string } // UnattributedClaim is what a SaveRequest may claim of a commit window whose writes name no author. @@ -212,6 +221,8 @@ type pendingSaveRequest struct { // unattributed is what the request may claim of a window whose writes name no author // (AttachSaveRequest.Unattributed). unattributed UnattributedClaim + // sourceNamespace confines the save to one source namespace (AttachSaveRequest.SourceNamespace). + sourceNamespace string // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. @@ -231,6 +242,9 @@ type pendingSaveRequest struct { // instant check at expiry would miss the common case and report the refusal as "nothing was // pending" — intermittently, which is worse than never. sawForeignWindow bool + // sawWiderWindow is set when the request's own window was open during its wait but held writes + // outside its sourceNamespace. Sticky for the reason sawForeignWindow is. + sawWiderWindow bool } // expiryOutcome is the terminal outcome for a request whose attach timeout ran out without it @@ -246,6 +260,9 @@ func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { if p.sawForeignWindow { return FinalizeWindowMismatch } + if p.sawWiderWindow { + return FinalizeWindowOutsideNamespace + } return FinalizeNoOpenWindow } @@ -271,6 +288,12 @@ func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { // would let one author's request finalize another's window. Comparing both costs nothing and // keeps "bob never attaches to alice's window" true regardless of what the outcomes say. func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { + return p.identifiesWindow(w) && p.withinScope(w) +} + +// identifiesWindow is matchesWindow without the namespace confinement: the window is on the +// request's GitTarget and is its author's, or one it may claim. +func (p *pendingSaveRequest) identifiesWindow(w *openWindow) bool { if p == nil || w == nil { return false } @@ -280,6 +303,22 @@ func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.claims(w) } +// withinScope reports whether every write in the window is in the request's sourceNamespace, or the +// request confines nothing. +func (p *pendingSaveRequest) withinScope(w *openWindow) bool { + return p.sourceNamespace == "" || w.holdsOnly(p.sourceNamespace) +} + +// noteUnmatchedWindow records why the open window, which the request cannot attach to, is not its +// own: another author's or GitTarget's, or its own holding writes outside its namespace. +func (p *pendingSaveRequest) noteUnmatchedWindow(w *openWindow) { + if p.identifiesWindow(w) { + p.sawWiderWindow = true + return + } + p.sawForeignWindow = true +} + // claims reports whether the request matches a window on its GitTarget whose writes name no author // only because it may claim such a window: it names its submitter, and asked for more than its own // windows (AttachSaveRequest.Unattributed). diff --git a/internal/git/save_request_attach_loop.go b/internal/git/save_request_attach_loop.go index fdba8eaa..d7ee82b8 100644 --- a/internal/git/save_request_attach_loop.go +++ b/internal/git/save_request_attach_loop.go @@ -216,7 +216,8 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) commitEmpty: req.CommitEmpty, catchUpNow: req.CatchUpNow, - unattributed: req.Unattributed, + unattributed: req.Unattributed, + sourceNamespace: req.SourceNamespace, } l.pendingCRs[id] = pcr l.w.setSaveRequestPhase(id, PhaseWaitingForWindow) @@ -235,7 +236,7 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) // passed: with attachTimeout: 0s that is every request, so a foreign window open at // registration would go unseen and the request would resolve NoWindow — and, with // CommitEmpty, record an empty commit while another author's work is in flight. - pcr.sawForeignWindow = true + pcr.noteUnmatchedWindow(l.openWindow) return } if req.Attach == v1alpha3.AttachNext { @@ -295,7 +296,20 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { } now := time.Now() for _, pcr := range l.pendingCRs { - if !pcr.waiting(now) || pcr.sawForeignWindow || pcr.matchesWindow(l.openWindow) { + if !pcr.waiting(now) || pcr.matchesWindow(l.openWindow) { + continue + } + if pcr.identifiesWindow(l.openWindow) { + if !pcr.sawWiderWindow { + pcr.sawWiderWindow = true + l.w.Log.Info("SaveRequest is waiting on its window, which holds writes outside its namespace", + "request", pcr.id.Namespace+"/"+pcr.id.Name, + "sourceNamespace", pcr.sourceNamespace, + "requestTarget", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) + } + continue + } + if pcr.sawForeignWindow { continue } pcr.sawForeignWindow = true @@ -347,6 +361,7 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { } l.openWindow.timers = timers l.openWindow.lastWriteAt = now + l.openWindow.sourceNamespace = pcr.sourceNamespace if pcr.claims(l.openWindow) { l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} l.w.Log.Info("SaveRequest claimed a window whose writes name no author", diff --git a/internal/git/save_request_scope_test.go b/internal/git/save_request_scope_test.go new file mode 100644 index 00000000..f632e1e5 --- /dev/null +++ b/internal/git/save_request_scope_test.go @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "testing" + "time" + + gogit "github.com/go-git/go-git/v6" + "github.com/go-git/go-git/v6/plumbing" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// configMapEventIn is configMapTargetEvent for a ConfigMap in namespace, "" for a write the window +// keys like a cluster-scoped one. +func configMapEventIn(name, username, namespace string) Event { + event := configMapTargetEvent(name, username, crTarget) + event.Identifier.Namespace = namespace + event.Object.SetNamespace(namespace) + return event +} + +// scopedAttachReq is a request that saves only the writes made in namespace. +func scopedAttachReq(author, namespace string) *AttachSaveRequest { + req := attachReq(author, 60*time.Second) + req.SourceNamespace = namespace + return req +} + +// A request confined to a namespace attaches only to a window whose writes are all in it. The +// window's author and GitTarget still decide first: a window of someone else's is foreign, not wider. +func TestMatchesWindow_SourceNamespaceConfinesTheSave(t *testing.T) { + window := func(events ...Event) *openWindow { + w := newOpenWindow(events[0], &contentWriter{}) + for _, e := range events { + w.add(e) + } + return w + } + shop := configMapEventIn("a", "alice", "shop") + billing := configMapEventIn("b", "alice", "billing") + cluster := configMapEventIn("c", "alice", "") + request := func(namespace string) *pendingSaveRequest { + return &pendingSaveRequest{ + author: "alice", gitTargetName: crTarget, gitTargetNamespace: "default", sourceNamespace: namespace, + } + } + + assert.True(t, request("shop").matchesWindow(window(shop)), "every write is in its namespace") + assert.False(t, request("shop").matchesWindow(window(billing)), "every write is elsewhere") + assert.False(t, request("shop").matchesWindow(window(shop, billing)), "one write is elsewhere") + assert.False(t, request("shop").matchesWindow(window(cluster)), "a cluster-scoped write is in no namespace") + assert.True(t, request("").matchesWindow(window(shop, billing, cluster)), "no confinement") + + wider := request("shop") + wider.noteUnmatchedWindow(window(shop, billing)) + assert.Equal(t, FinalizeWindowOutsideNamespace, wider.expiryOutcome()) + + bob := configMapEventIn("d", "bob", "shop") + foreign := request("shop") + foreign.noteUnmatchedWindow(window(bob)) + assert.Equal(t, FinalizeWindowMismatch, foreign.expiryOutcome(), "bob's window is foreign") +} + +// Once a confined request attached, a write by the same author in another namespace does not join +// its window: the window closes with the request's message and only its namespace's write, and the +// other write opens a window of its own. +func TestAttach_AConfinedSaveKeepsOtherNamespacesOutOfItsCommit(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, crTarget, crTarget) + + loop := newBranchWorkerEventLoop(worker, time.Hour) + defer loop.stopTimers() + + req := scopedAttachReq("alice", "shop") + req.Message = "save: the shop" + serviceAttach(loop, req) + + loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{configMapEventIn("cart", "alice", "shop")}}}) + loop.endWake(0) + require.NotNil(t, loop.openWindow) + require.NotNil(t, loop.openWindow.pendingCR, "the request attached to the shop window") + + loop.handleQueueItem(WorkItem{Request: &WriteRequest{ + Events: []Event{configMapEventIn("invoice", "alice", "billing")}, + }}) + loop.endWake(0) + + res, ok := outcome(t, worker) + require.True(t, ok, "the billing write closed the shop window") + require.NoError(t, res.Err) + require.Equal(t, FinalizeCommitted, res.Outcome) + require.NotNil(t, loop.openWindow, "the billing write opened a window of its own") + assert.Nil(t, loop.openWindow.pendingCR, "and the request is not in it") + assert.Equal(t, "billing", loop.openWindow.namespace) + + repo, err := gogit.PlainOpen(worker.repoPath()) + require.NoError(t, err) + commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) + require.NoError(t, err) + assert.Equal(t, "save: the shop", commit.Message) + stats, err := commit.Stats() + require.NoError(t, err) + changed := make([]string, 0, len(stats)) + for _, stat := range stats { + changed = append(changed, stat.Name) + } + assert.Contains(t, changed, crTarget+"/shop/configmaps/cart.yaml") + assert.NotContains( + t, + changed, + crTarget+"/billing/configmaps/invoice.yaml", + "billing's write is not the shop's save", + ) +} + +// A confined request leaves its author's window alone when that window already holds writes from +// elsewhere, and says so when its wait runs out. +func TestAttach_AConfinedSaveLeavesAWiderWindowAlone(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, crTarget, crTarget) + + loop := newBranchWorkerEventLoop(worker, time.Hour) + loop.lastPushAt = time.Now() + defer loop.stopTimers() + + loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{ + configMapEventIn("cart", "alice", "shop"), + configMapEventIn("invoice", "alice", "billing"), + }}}) + require.NotNil(t, loop.openWindow) + + serviceAttach(loop, scopedAttachReq("alice", "shop")) + require.Nil(t, loop.openWindow.pendingCR, "a window holding billing's write is not the shop's to save") + + forceDue(loop) + loop.endWake(0) + + res, ok := outcome(t, worker) + require.True(t, ok) + require.NoError(t, res.Err) + assert.Equal(t, FinalizeWindowOutsideNamespace, res.Outcome) + require.NotNil(t, loop.openWindow, "the window is left open") + assert.Len(t, loop.openWindow.orderedEvents(), 2) +} diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go index 73f04ba8..34712651 100644 --- a/internal/watch/remote_save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -31,10 +31,17 @@ import ( // A RemoteSaveRequest is a save made on a source cluster, next to the edits it saves // (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose -// spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way the -// SaveRequest controller serves a SaveRequest: it attaches the request to the commit window of -// every GitTarget that mirrors the request's namespace from that cluster, polls the branch workers -// for the outcomes, and writes them back to the request's status on the source cluster. +// spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way +// the SaveRequest controller serves a SaveRequest: it attaches the request to the commit window of +// every GitTarget that mirrors the request's namespace from that cluster and admits it in +// spec.allowedRemoteSaveRequests, polls the branch workers for the outcomes, and writes them back to +// the request's status on the source cluster. The provider's switch makes the requests reachable; +// the GitTarget's grant is its owner's consent to being steered by them. +// +// A request saves only the edits made in its own namespace: the worker attaches it only to a window +// whose writes are all there, and keeps every other write out of that window once it has +// (git.AttachSaveRequest.SourceNamespace). A window groups by author and GitTarget, not by +// namespace, so without that a save in one namespace would commit edits made in another. // // The request's author is found as the edits' authors are, from the source cluster's audit facts, so // a save attaches only to its creator's windows by the ordinary author rule. On a cluster that sends @@ -76,9 +83,11 @@ const ( saveReasonCommitted = "Committed" saveReasonNoWindow = "NoWindow" saveReasonWindowMismatch = "WindowMismatch" + saveReasonOutsideNamespace = "WindowOutsideNamespace" saveReasonAlreadyPresent = "AlreadyPresent" saveReasonFailed = "Failed" saveReasonNoGitTarget = "NoGitTarget" + saveReasonNotAllowed = "NotAllowed" saveReasonAuthorUnresolved = "AuthorUnresolved" saveReasonSaving = "Saving" saveReasonApplying = "Applying" @@ -148,7 +157,8 @@ func (s *RemoteSaveRequestServer) syncProviders(ctx context.Context) { want := map[k8stypes.NamespacedName]string{} for i := range providers.Items { provider := &providers.Items[i] - if provider.DeletionTimestamp == nil && provider.Spec.RemoteSaveRequests == configv1alpha3.RemoteSaveRequestsEnabled { + if provider.DeletionTimestamp == nil && + provider.Spec.RemoteSaveRequests == configv1alpha3.RemoteSaveRequestsEnabled { want[provider.Key()] = provider.AuditRoute() } } @@ -205,7 +215,11 @@ func (s *RemoteSaveRequestServer) run(ctx context.Context, watcher *remoteSaveRe } // session lists the cluster's RemoteSaveRequests and watches them, until the watch ends. -func (s *RemoteSaveRequestServer) session(ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher) error { +func (s *RemoteSaveRequestServer) session( + ctx context.Context, + log logr.Logger, + watcher *remoteSaveRequestWatcher, +) error { dc, err := s.clusterClient(ctx, watcher.provider) if err != nil { return err @@ -260,12 +274,21 @@ func (s *RemoteSaveRequestServer) clusterClient( // consider serves a request that has not finished and is not served yet, and deletes one that // finished longer than the TTL ago. func (s *RemoteSaveRequestServer) consider( - ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher, resource dynamic.NamespaceableResourceInterface, + ctx context.Context, + log logr.Logger, + watcher *remoteSaveRequestWatcher, + resource dynamic.NamespaceableResourceInterface, u *unstructured.Unstructured, ) { var request sourcev1alpha1.RemoteSaveRequest if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, &request); err != nil { - log.Info("Cannot read a RemoteSaveRequest", "remoteSaveRequest", u.GetNamespace()+"/"+u.GetName(), "err", err.Error()) + log.Info( + "Cannot read a RemoteSaveRequest", + "remoteSaveRequest", + u.GetNamespace()+"/"+u.GetName(), + "err", + err.Error(), + ) return } if finishedAt, finished := remoteSaveRequestFinished(&request); finished { @@ -273,8 +296,9 @@ func (s *RemoteSaveRequestServer) consider( if err := resource.Namespace(request.Namespace).Delete(ctx, request.Name, metav1.DeleteOptions{ Preconditions: &metav1.Preconditions{UID: &request.UID}, }); err != nil && !apierrors.IsNotFound(err) { - log.V(1).Info("Cannot delete a finished RemoteSaveRequest", "remoteSaveRequest", request.Namespace+"/"+request.Name, - "err", err.Error()) + log.V(1). + Info("Cannot delete a finished RemoteSaveRequest", "remoteSaveRequest", request.Namespace+"/"+request.Name, + "err", err.Error()) } } return @@ -341,21 +365,30 @@ func (s *RemoteSaveRequestServer) serve( return } - targets, err := s.mirroringTargets(ctx, watcher.provider, request.Namespace) + targets, mirrored, err := s.admittingTargets(ctx, watcher.provider, request) if err != nil { - log.Info("Cannot find the GitTargets that mirror a RemoteSaveRequest's namespace; will retry", "err", err.Error()) + log.Info("Cannot find the GitTargets that admit a RemoteSaveRequest; will retry", "err", err.Error()) return } if len(targets) == 0 { + if mirrored { + message := "a GitTarget mirrors this namespace from this cluster, but none admits this request " + + "in spec.allowedRemoteSaveRequests" + setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonNotAllowed, message) + setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonNotAllowed, message) + write() + log.Info("Refused a RemoteSaveRequest no GitTarget admits") + return + } setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonNoGitTarget, "no GitTarget mirrors this namespace from this cluster, so there is nothing to save") write() return } - status.Targets = make([]sourcev1alpha1.RemoteSaveRequestTarget, len(targets)) + status.GitTargets = make([]sourcev1alpha1.RemoteSaveRequestGitTarget, len(targets)) attaches := make([]git.AttachSaveRequest, len(targets)) for i, target := range targets { - status.Targets[i] = sourcev1alpha1.RemoteSaveRequestTarget{GitTarget: target.String()} + status.GitTargets[i] = sourcev1alpha1.RemoteSaveRequestGitTarget{Namespace: target.Namespace, Name: target.Name} attaches[i] = remoteSaveRequestAttach(request, target, author.Username, attribution) } setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, @@ -402,36 +435,52 @@ func (s *RemoteSaveRequestServer) sentAuditFacts(ctx context.Context, provider k return apimeta.IsStatusConditionTrue(kp.Status.Conditions, "AuditFactsReceived") } -// mirroringTargets are the GitTargets that mirror namespace from the KubeProvider: they name it, and -// one of their running streams watches that namespace, or every namespace. -func (s *RemoteSaveRequestServer) mirroringTargets( - ctx context.Context, provider k8stypes.NamespacedName, namespace string, -) ([]types.ResourceReference, error) { +// admittingTargets are the GitTargets a request saves: they name the KubeProvider, mirror the +// request's namespace, and admit the request in spec.allowedRemoteSaveRequests. mirrored reports +// whether any GitTarget mirrors the namespace, admitting or not. +func (s *RemoteSaveRequestServer) admittingTargets( + ctx context.Context, provider k8stypes.NamespacedName, request *sourcev1alpha1.RemoteSaveRequest, +) ([]types.ResourceReference, bool, error) { var targets configv1alpha3.GitTargetList if err := s.Client.List(ctx, &targets); err != nil { - return nil, err + return nil, false, err } var out []types.ResourceReference + mirrored := false for i := range targets.Items { target := &targets.Items[i] if target.DeletionTimestamp != nil || target.KubeProviderKey() != provider { continue } ref := types.NewResourceReference(target.Name, target.Namespace) - if s.Manager.mirrorsNamespace(ref, namespace) { + if !s.Manager.mirrorsNamespace(ref, request.Namespace) { + continue + } + mirrored = true + admitted, err := target.AdmitsRemoteSaveRequest(request.Namespace, request.Labels) + if err != nil { + s.Log.Info("A GitTarget's spec.allowedRemoteSaveRequests cannot be read; it admits nothing", + "gitTarget", ref.String(), "err", err.Error()) + } + if admitted { out = append(out, ref) } } sort.Slice(out, func(i, j int) bool { return out[i].String() < out[j].String() }) - return out, nil + return out, mirrored, nil } -// mirrorsNamespace reports whether a running stream of gitDest watches namespace, or every namespace. +// mirrorsNamespace reports whether gitDest mirrors a namespaced type in namespace: a WatchRule +// watches the type there, or a rule watches it across every namespace. A cluster-scoped type is in +// no namespace, so a GitTarget that mirrors only those mirrors none. func (m *Manager) mirrorsNamespace(gitDest types.ResourceReference, namespace string) bool { - m.streamProgressMu.Lock() - defer m.streamProgressMu.Unlock() - for collection := range m.streamProgress[gitDest.Key()] { - if collection.Namespace == namespace || collection.Namespace == "" { + for _, wt := range m.residentWatchedTypeTable(gitDest).Types { + if !wt.Namespaced { + continue + } + _, named := wt.NamespaceScopes[namespace] + _, everywhere := wt.NamespaceScopes[""] + if named || everywhere { return true } } @@ -457,6 +506,7 @@ func remoteSaveRequestAttach( AttachTimeout: defaultSaveAttachTimeout, MaxDuration: defaultSaveMaxDuration, CatchUpNow: request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush, + SourceNamespace: request.Namespace, } if window := request.Spec.Window; window != nil { if window.AttachTimeout != nil { @@ -496,7 +546,7 @@ func (s *RemoteSaveRequestServer) finalize( } resolved[i] = true remaining-- - recordSaveOutcome(&status.Targets[i], result) + recordSaveOutcome(&status.GitTargets[i], result) } if remaining == 0 { break @@ -511,7 +561,7 @@ func (s *RemoteSaveRequestServer) finalize( } // recordSaveOutcome records one GitTarget's outcome on its target status. -func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestTarget, result git.FinalizeResult) { +func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestGitTarget, result git.FinalizeResult) { target.Branch = result.Branch if result.Err != nil { target.Reason, target.Message = saveReasonFailed, result.Err.Error() @@ -530,6 +580,9 @@ func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestTarget, result gi case git.FinalizeWindowMismatch: target.Reason = saveReasonWindowMismatch target.Message = "only another author's commit window was open" + case git.FinalizeWindowOutsideNamespace: + target.Reason = saveReasonOutsideNamespace + target.Message = "the open commit window held edits made outside this namespace, so it was left alone" case git.FinalizeAlreadyPresent: target.Reason = saveReasonAlreadyPresent target.Message = "the edits were in Git already" @@ -541,21 +594,21 @@ func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestTarget, result gi // setSaveReady sets Ready from the targets' outcomes: a failure in any fails the request, a commit in // any is Committed, and otherwise the first target's reason says why nothing was committed. func setSaveReady(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { - for _, target := range status.Targets { + for _, target := range status.GitTargets { if target.Reason == saveReasonFailed { - message := target.GitTarget + ": " + target.Message + message := gitTargetName(target) + ": " + target.Message setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonFailed, message) setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonFailed, message) return } } - for _, target := range status.Targets { + for _, target := range status.GitTargets { if target.Reason == saveReasonCommitted { setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonCommitted, target.Message) return } } - first := status.Targets[0] + first := status.GitTargets[0] setSaveCondition(request, status, "Ready", metav1.ConditionTrue, first.Reason, first.Message) } @@ -587,7 +640,7 @@ func (s *RemoteSaveRequestServer) pollApplies( ) bool { pending := false for i := range attaches { - target := &status.Targets[i] + target := &status.GitTargets[i] if target.Reason != saveReasonCommitted || saveApplySettled(target.Applied) { continue } @@ -620,20 +673,25 @@ func saveApplySettled(reason string) bool { func setSaveApplied(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { reason, condition := saveReasonApplied, metav1.ConditionTrue message := "every applier applied a revision that holds the commit, or needed no apply" - for _, target := range status.Targets { + for _, target := range status.GitTargets { switch target.Applied { case git.SaveApplyCommitDropped: setSaveCondition(request, status, "Applied", metav1.ConditionFalse, target.Applied, - target.GitTarget+": the branch no longer holds the commit") + gitTargetName(target)+": the branch no longer holds the commit") return case "Lost": reason, condition = "Lost", metav1.ConditionUnknown - message = target.GitTarget + ": GitOps Reverser does not know how the apply went" + message = gitTargetName(target) + ": GitOps Reverser does not know how the apply went" } } setSaveCondition(request, status, "Applied", condition, reason, message) } +// gitTargetName is a target status's GitTarget as namespace/name, for a message. +func gitTargetName(target sourcev1alpha1.RemoteSaveRequestGitTarget) string { + return target.Namespace + "/" + target.Name +} + func readyReason(status *sourcev1alpha1.RemoteSaveRequestStatus) string { if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil { return ready.Reason diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go index 3518c3d7..9887ea92 100644 --- a/internal/watch/remote_save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -13,6 +13,7 @@ import ( "github.com/go-logr/logr" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" @@ -70,34 +71,72 @@ func (a fixedAuthor) ResolveAuthor(context.Context, AuthorQuery) (git.UserInfo, var saveProvider = k8stypes.NamespacedName{Namespace: "providers", Name: "workspace"} -// saveServer is a server for the KubeProvider providers/workspace, whose GitTarget team-a/app mirrors -// the source namespace "shop" when mirrored is set, with a RemoteSaveRequest shop/save on the source cluster. +// saveSetup is what a save is served against: the GitTarget team-a/app of the KubeProvider +// providers/workspace, the type it mirrors, the grants it holds, and what the source cluster's audit +// says. +type saveSetup struct { + // mirrors is the type the GitTarget mirrors; nil mirrors nothing. + mirrors *WatchedType + grants []configv1alpha3.GitTargetRemoteSaveRequestGrant + auditFacts metav1.ConditionStatus + author AuthorResolver +} + +// configMapsIn is ConfigMaps watched in namespace, or in every namespace for "". +func configMapsIn(namespace string) *WatchedType { + return &WatchedType{ + GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, + Namespaced: true, + Scope: configv1alpha3.ResourceScopeNamespaced, + NamespaceScopes: map[string]string{namespace: ""}, + } +} + +// grantFrom admits the RemoteSaveRequests of namespaces. +func grantFrom(namespaces ...configv1alpha3.SourceNamespace) configv1alpha3.GitTargetRemoteSaveRequestGrant { + return configv1alpha3.GitTargetRemoteSaveRequestGrant{SourceNamespaces: namespaces} +} + +// shopSetup is a GitTarget that mirrors ConfigMaps in "shop" and admits that namespace's requests. +func shopSetup(auditFacts metav1.ConditionStatus, author AuthorResolver) saveSetup { + return saveSetup{ + mirrors: configMapsIn("shop"), + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, + auditFacts: auditFacts, + author: author, + } +} + +// saveServer is a server for setup, with request on the source cluster. func saveServer( - t *testing.T, request *sourcev1alpha1.RemoteSaveRequest, mirrored bool, auditFacts metav1.ConditionStatus, - author AuthorResolver, finalizer *saveFinalizer, + t *testing.T, request *sourcev1alpha1.RemoteSaveRequest, setup saveSetup, finalizer *saveFinalizer, ) (*RemoteSaveRequestServer, dynamic.ResourceInterface) { t.Helper() scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) provider := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} - if auditFacts != "" { + if setup.auditFacts != "" { provider.Status.Conditions = []metav1.Condition{ - {Type: "AuditFactsReceived", Status: auditFacts, Reason: "Test"}, + {Type: "AuditFactsReceived", Status: setup.auditFacts, Reason: "Test"}, } } target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a"}, Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, + KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, + AllowedRemoteSaveRequests: setup.grants, }, } c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, target). WithStatusSubresource(&configv1alpha3.KubeProvider{}).Build() - manager := &Manager{AuthorResolver: author} - if mirrored { - manager.trackStreamProgress(types.NewResourceReference("app", "team-a"), - targetWatchKey{GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, Namespace: "shop"}) + manager := &Manager{AuthorResolver: setup.author} + if setup.mirrors != nil { + gitDest := types.NewResourceReference("app", "team-a") + manager.ensureWatchedTypeStore() + manager.watchedTypes.tables = map[string]WatchedTypeTable{ + gitDest.Key(): {GitDest: gitDest, Types: []WatchedType{*setup.mirrors}}, + } } fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(request) require.NoError(t, err) @@ -106,7 +145,10 @@ func saveServer( obj.SetKind("RemoteSaveRequest") dc := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), map[schema.GroupVersionResource]string{remoteSaveRequestGVR(): "RemoteSaveRequestList"}, obj) - server := &RemoteSaveRequestServer{Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard()} + server := &RemoteSaveRequestServer{ + Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard(), + dynamicClient: func(context.Context, k8stypes.NamespacedName) (dynamic.Interface, error) { return dc, nil }, + } return server, dc.Resource(remoteSaveRequestGVR()).Namespace("shop") } @@ -126,7 +168,10 @@ func newRemoteSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1. // serveOnce serves the request and returns its status as written to the source cluster. func serveOnce( - t *testing.T, server *RemoteSaveRequestServer, resource dynamic.ResourceInterface, request *sourcev1alpha1.RemoteSaveRequest, + t *testing.T, + server *RemoteSaveRequestServer, + resource dynamic.ResourceInterface, + request *sourcev1alpha1.RemoteSaveRequest, ) sourcev1alpha1.RemoteSaveRequestStatus { t.Helper() watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} @@ -156,7 +201,7 @@ func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *test } author := fixedAuthor{user: git.UserInfo{Username: "alice@example.com"}, outcome: git.AttributionResolved} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - server, resource := saveServer(t, request, true, metav1.ConditionTrue, author, finalizer) + server, resource := saveServer(t, request, shopSetup(metav1.ConditionTrue, author), finalizer) status := serveOnce(t, server, resource, request) @@ -170,13 +215,14 @@ func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *test assert.Equal(t, 3*time.Second, attach.AttachTimeout) assert.True(t, attach.CatchUpNow) assert.Equal(t, "uid-save@team-a/app", attach.UID, "one identity per GitTarget") + assert.Equal(t, "shop", attach.SourceNamespace, "it saves only the edits made in its namespace") assert.Equal(t, "alice@example.com", status.Author) - require.Len(t, status.Targets, 1) - assert.Equal(t, sourcev1alpha1.RemoteSaveRequestTarget{ - GitTarget: "team-a/app", Reason: saveReasonCommitted, Message: "the edits were committed and pushed", + require.Len(t, status.GitTargets, 1) + assert.Equal(t, sourcev1alpha1.RemoteSaveRequestGitTarget{ + Namespace: "team-a", Name: "app", Reason: saveReasonCommitted, Message: "the edits were committed and pushed", Branch: "main", Commit: "c0ffee", Applied: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", - }, status.Targets[0]) + }, status.GitTargets[0]) require.NotNil(t, condition(status, "Ready")) assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) @@ -189,8 +235,8 @@ func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *test func TestRemoteSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, true, metav1.ConditionUnknown, - fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) + server, resource := saveServer(t, request, + shopSetup(metav1.ConditionUnknown, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) status := serveOnce(t, server, resource, request) @@ -206,8 +252,8 @@ func TestRemoteSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { func TestRemoteSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { finalizer := &saveFinalizer{} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, true, metav1.ConditionTrue, - fixedAuthor{outcome: git.AttributionUnresolved}, finalizer) + server, resource := saveServer(t, request, + shopSetup(metav1.ConditionTrue, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) status := serveOnce(t, server, resource, request) @@ -216,23 +262,140 @@ func TestRemoteSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testin assert.Equal(t, saveReasonAuthorUnresolved, condition(status, "Ready").Reason) } -// A save in a namespace no GitTarget mirrors has nothing to save. +// A save in a namespace no GitTarget mirrors has nothing to save. A cluster-scoped type is in no +// namespace, so a GitTarget mirroring only one mirrors none, whatever its grant admits. func TestRemoteSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { - finalizer := &saveFinalizer{} + clusterRoles := &WatchedType{ + GVR: schema.GroupVersionResource{ + Group: "rbac.authorization.k8s.io", + Version: "v1", + Resource: "clusterroles", + }, + Scope: configv1alpha3.ResourceScopeCluster, + NamespaceScopes: map[string]string{"": ""}, + } + everything := []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("*")} + for name, setup := range map[string]saveSetup{ + "nothing mirrored": {grants: everything}, + "another namespace": {mirrors: configMapsIn("billing"), grants: everything}, + "only a cluster-scoped type": {mirrors: clusterRoles, grants: everything}, + } { + t.Run(name, func(t *testing.T) { + finalizer := &saveFinalizer{} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, setup, finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Empty(t, finalizer.attaches) + assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) + assert.Equal(t, saveReasonNoGitTarget, condition(status, "Ready").Reason) + }) + } +} + +// A GitTarget that mirrors the namespace saves a request only when one of its grants admits both the +// request's namespace and its labels; nothing is admitted by default. +func TestRemoteSaveRequest_TheGitTargetMustAdmitIt(t *testing.T) { + selector := func(labels map[string]string) *metav1.LabelSelector { + return &metav1.LabelSelector{MatchLabels: labels} + } + shopLabelled := func(labels map[string]string) configv1alpha3.GitTargetRemoteSaveRequestGrant { + grant := grantFrom("shop") + grant.RequestSelector = selector(labels) + return grant + } + tests := map[string]struct { + grants []configv1alpha3.GitTargetRemoteSaveRequestGrant + admitted bool + }{ + "no grant": {}, + "another namespace": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("billing")}}, + "its namespace": { + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, + admitted: true, + }, + "every namespace": { + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("*")}, + admitted: true, + }, + "its namespace and labels": { + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{shopLabelled(map[string]string{"app": "cart"})}, + admitted: true, + }, + "its namespace, other labels": { + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{shopLabelled(map[string]string{"app": "admin"})}, + }, + "one complete grant of two": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{ + {SourceNamespaces: []configv1alpha3.SourceNamespace{"billing"}}, + shopLabelled(map[string]string{"app": "cart"}), + }, admitted: true}, + "a selector that does not convert admits nothing": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{{ + SourceNamespaces: []configv1alpha3.SourceNamespace{"shop"}, + RequestSelector: &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ + {Key: "app", Operator: "Near", Values: []string{"cart"}}, + }}, + }}}, + } + for name, tt := range tests { + t.Run(name, func(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request.Labels = map[string]string{"app": "cart"} + setup := shopSetup("", nil) + setup.grants = tt.grants + server, resource := saveServer(t, request, setup, finalizer) + + status := serveOnce(t, server, resource, request) + + if tt.admitted { + assert.Len(t, finalizer.attaches, 1) + assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) + return + } + assert.Empty(t, finalizer.attaches, "nothing is attached") + assert.Empty(t, status.GitTargets) + assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) + assert.Equal(t, saveReasonNotAllowed, condition(status, "Ready").Reason) + assert.Contains(t, condition(status, "Ready").Message, "spec.allowedRemoteSaveRequests") + assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) + }) + } +} + +// A GitTarget that watches a type across every namespace mirrors the request's. +func TestRemoteSaveRequest_AWatchOfEveryNamespaceMirrorsIt(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, false, "", nil, finalizer) + setup := shopSetup("", nil) + setup.mirrors = configMapsIn("") + server, resource := saveServer(t, request, setup, finalizer) status := serveOnce(t, server, resource, request) - assert.Empty(t, finalizer.attaches) - assert.Equal(t, saveReasonNoGitTarget, condition(status, "Ready").Reason) + assert.Len(t, finalizer.attaches, 1) + assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) +} + +// A window holding edits from elsewhere is left alone, and the save says why. +func TestRemoteSaveRequest_AWindowOutsideTheNamespaceIsReported(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeWindowOutsideNamespace}} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) + server, resource := saveServer(t, request, shopSetup("", nil), finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) + assert.Equal(t, saveReasonOutsideNamespace, condition(status, "Ready").Reason) + assert.Contains(t, condition(status, "Ready").Message, "outside this namespace") + assert.Nil(t, condition(status, "Applied"), "nothing was committed to apply") } // A save that failed in a GitTarget fails, and says where. func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - server, resource := saveServer(t, request, true, "", nil, finalizer) + server, resource := saveServer(t, request, shopSetup("", nil), finalizer) status := serveOnce(t, server, resource, request) @@ -311,3 +474,80 @@ func TestRemoteSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T assert.Empty(t, server.watchers) server.stopAll() } + +// A session serves every request it lists, and every one its watch delivers, writing each outcome +// back to the source cluster. +func TestRemoteSaveRequest_ASessionServesWhatItListsAndWatches(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, shopSetup("", nil), finalizer) + watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} + readyReasonOf := func(name string) string { + got, err := resource.Get(context.Background(), name, metav1.GetOptions{}) + if err != nil { + return "" + } + var written sourcev1alpha1.RemoteSaveRequest + if runtime.DefaultUnstructuredConverter.FromUnstructured(got.Object, &written) != nil { + return "" + } + return readyReason(&written.Status) + } + + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan error, 1) + go func() { done <- server.session(ctx, logr.Discard(), watcher) }() + + assert.Eventually(t, func() bool { return readyReasonOf("save") == saveReasonCommitted }, + 5*time.Second, 20*time.Millisecond, "the listed request is served") + + later := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + later.Name, later.UID = "later", "uid-later" + fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(later) + require.NoError(t, err) + obj := &unstructured.Unstructured{Object: fields} + obj.SetAPIVersion(sourcev1alpha1.GroupVersion.String()) + obj.SetKind("RemoteSaveRequest") + _, err = resource.Create(context.Background(), obj, metav1.CreateOptions{}) + require.NoError(t, err) + assert.Eventually(t, func() bool { return readyReasonOf("later") == saveReasonCommitted }, + 5*time.Second, 20*time.Millisecond, "the watched request is served") + + cancel() + require.NoError(t, <-done) +} + +// A finished request is deleted once it is older than the TTL, and kept before that or without one; +// it is never served again. +func TestRemoteSaveRequest_AFinishedRequestIsDeletedAfterTheTTL(t *testing.T) { + for name, tt := range map[string]struct { + ttl time.Duration + deleted bool + }{ + "past the TTL": {ttl: time.Minute, deleted: true}, + "within the TTL": {ttl: 2 * time.Hour}, + "no TTL": {}, + } { + t.Run(name, func(t *testing.T) { + finalizer := &saveFinalizer{} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request.Status.Conditions = []metav1.Condition{{ + Type: "Ready", Status: metav1.ConditionTrue, Reason: saveReasonNoWindow, + LastTransitionTime: metav1.NewTime(time.Now().Add(-time.Hour)), + }} + server, resource := saveServer(t, request, shopSetup("", nil), finalizer) + server.TTL = tt.ttl + watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} + dc, err := server.clusterClient(context.Background(), saveProvider) + require.NoError(t, err) + current, err := resource.Get(context.Background(), request.Name, metav1.GetOptions{}) + require.NoError(t, err) + + server.consider(context.Background(), logr.Discard(), watcher, dc.Resource(remoteSaveRequestGVR()), current) + + _, err = resource.Get(context.Background(), request.Name, metav1.GetOptions{}) + assert.Equal(t, tt.deleted, apierrors.IsNotFound(err), "deleted") + assert.Empty(t, finalizer.attaches, "a finished request is never served again") + }) + } +} diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 73afc90f..985008ac 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -863,14 +863,20 @@ spec: }) // A save made in the workspace itself: a RemoteSaveRequest, served because the workspace's - // KubeProvider enables them. It names no GitTarget; it saves the window of every GitTarget that - // mirrors its namespace. kcp sends no audit facts, so the request names nobody, like the edit, + // KubeProvider enables them, and saving only for a GitTarget that admits it. It names no + // GitTarget; it saves the window of every admitting GitTarget that mirrors its namespace, first + // refused while the GitTarget admits none. kcp sends no audit facts, so the request names nobody, like the edit, // and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is // looked for first, for the whole attribution grace, so the edit is made once the request is // waiting for a window. It("saves an edit made in the workspace with a RemoteSaveRequest created there", func() { By("installing the RemoteSaveRequest CRD in the workspace, and serving the workspace's RemoteSaveRequests") - _, err := kcp.wsKubectl(hash, "apply", "-f", "config/crd/bases/source.configbutler.ai_remotesaverequests.yaml") + _, err := kcp.wsKubectl( + hash, + "apply", + "-f", + "config/crd/bases/source.configbutler.ai_remotesaverequests.yaml", + ) Expect(err).NotTo(HaveOccurred()) _, err = kcp.wsKubectl(hash, "wait", "--for=condition=Established", "crd/remotesaverequests.source.configbutler.ai", "--timeout=60s") @@ -878,15 +884,39 @@ spec: _, err = kubectlRunInNamespace(testNs, "patch", "kubeprovider", ws, "--type=merge", "-p", `{"spec":{"remoteSaveRequests":"Enabled"}}`) Expect(err).NotTo(HaveOccurred()) + requestField := func(g Gomega, name, jsonPath string) string { + out, getErr := kcp.wsKubectl( + hash, + "-n", + testNs, + "get", + "remotesaverequest", + name, + "-o", + "jsonpath="+jsonPath, + ) + g.Expect(getErr).NotTo(HaveOccurred()) + return strings.TrimSpace(out) + } + + By("refusing a RemoteSaveRequest while the GitTarget admits none") + const refused = "sc-flux-workspace-save-refused" + createWorkspaceRemoteSaveRequest(kcp, hash, testNs, refused, "e2e: not admitted") + Eventually(func(g Gomega) { + g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("False")) + g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal("NotAllowed")) + }, 2*time.Minute, time.Second).Should(Succeed()) + + By("admitting the workspace namespace's RemoteSaveRequests on the GitTarget") + _, err = kubectlRunInNamespace(testNs, "patch", "gittarget", p.targetName, "--type=merge", "-p", + fmt.Sprintf(`{"spec":{"allowedRemoteSaveRequests":[{"sourceNamespaces":[%q]}]}}`, testNs)) + Expect(err).NotTo(HaveOccurred()) By("creating a RemoteSaveRequest in the workspace, and editing once it waits for a window") const name, message = "sc-flux-workspace-save", "e2e: save from the workspace" createWorkspaceRemoteSaveRequest(kcp, hash, testNs, name, message) - saveField := func(g Gomega, jsonPath string) string { - out, getErr := kcp.wsKubectl(hash, "-n", testNs, "get", "remotesaverequest", name, "-o", "jsonpath="+jsonPath) - g.Expect(getErr).NotTo(HaveOccurred()) - return strings.TrimSpace(out) - } + saveField := func(g Gomega, jsonPath string) string { return requestField(g, name, jsonPath) } Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Saving")) }, 2*time.Minute, time.Second).Should(Succeed(), "the server picks the request up and waits for a window") @@ -897,8 +927,9 @@ spec: Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("True")) g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Committed")) - g.Expect(saveField(g, "{.status.targets[0].gitTarget}")).To(Equal(testNs + "/" + p.targetName)) - commit = saveField(g, "{.status.targets[0].commit}") + g.Expect(saveField(g, "{.status.gitTargets[0].namespace}")).To(Equal(testNs)) + g.Expect(saveField(g, "{.status.gitTargets[0].name}")).To(Equal(p.targetName)) + commit = saveField(g, "{.status.gitTargets[0].commit}") g.Expect(commit).NotTo(BeEmpty()) g.Expect(saveField(g, "{.status.author}")).To(BeEmpty(), "kcp sends no audit facts") }, 2*time.Minute, 2*time.Second).Should(Succeed()) @@ -911,8 +942,8 @@ spec: Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Applied")].status}`)).To(Equal("True")) - g.Expect(saveField(g, "{.status.targets[0].applied}")).To(Equal("Applied")) - g.Expect(saveField(g, "{.status.targets[0].appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(saveField(g, "{.status.gitTargets[0].applied}")).To(Equal("Applied")) + g.Expect(saveField(g, "{.status.gitTargets[0].appliedRevision}")).To(ContainSubstring(commit)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(commit)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.consistentlyEverywhere(timingConfigMap, "saved-3") From 219b6eac36632e9b242d09a0cf7962341ea33073 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 20:16:05 +0000 Subject: [PATCH 19/33] feat(saverequest): name a GitTarget in another namespace, which admits it in spec.allowedSaveRequests A SaveRequest's spec.gitTargetRef takes a namespace, so a request can save a GitTarget outside its own namespace. The GitTarget consents in spec.allowedSaveRequests, a grant list like the others: omitted, it is one from: Same grant, so a request in the GitTarget's namespace saves as before; [] admits nobody. A request no grant admits fails at first sight with reason NotAllowed. spec.commit.unattributedWindows moves into each grant, so a request from another namespace claims of a window whose writes name no author only what the grant admitting it allows; admitted by several, it may claim the most any of them allows. A RemoteSaveRequest is no longer confined to its namespace's edits: a commit window groups by GitTarget and author, and the GitTarget's spec.allowedRemoteSaveRequests is what authorizes saving its creator's whole window there. WindowOutsideNamespace is gone. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../v1alpha1/remotesaverequest_types.go | 29 ++- api/v1alpha3/access_grants.go | 85 +++++++- api/v1alpha3/access_grants_defaults_test.go | 10 + api/v1alpha3/gittarget_types.go | 43 ++-- api/v1alpha3/saverequest_types.go | 24 ++- api/v1alpha3/zz_generated.deepcopy.go | 23 +++ .../crd/bases/configbutler.ai_gittargets.yaml | 195 +++++++++++++++--- .../bases/configbutler.ai_saverequests.yaml | 15 +- ...ce.configbutler.ai_remotesaverequests.yaml | 29 ++- docs/architecture.md | 3 +- docs/configuration.md | 74 +++++-- docs/future/source-side-save-plan.md | 37 ++-- docs/rbac.md | 6 +- hack/crdfields/fields.yaml | 12 +- internal/authz/saverequest_admission.go | 34 +++ internal/authz/saverequest_admission_test.go | 76 +++++++ .../access_grants_admission_test.go | 8 + .../duration_fields_admission_test.go | 6 +- .../grant_defaults_apiserver_test.go | 25 +++ internal/controller/saverequest_apply.go | 2 +- internal/controller/saverequest_controller.go | 103 ++++++--- .../controller/saverequest_controller_test.go | 6 +- .../saverequest_controller_unit_test.go | 73 ++++++- internal/controller/saverequest_finalize.go | 20 +- .../saverequest_outcome_metric_test.go | 10 +- internal/git/branch_worker.go | 1 - internal/git/open_window.go | 21 +- internal/git/save_request_attach.go | 39 ---- internal/git/save_request_attach_loop.go | 21 +- internal/git/save_request_scope_test.go | 146 ------------- internal/watch/remote_save_requests.go | 12 +- internal/watch/remote_save_requests_test.go | 65 ++++-- test/e2e/source_cluster_e2e_test.go | 17 +- 33 files changed, 820 insertions(+), 450 deletions(-) create mode 100644 internal/authz/saverequest_admission.go create mode 100644 internal/authz/saverequest_admission_test.go delete mode 100644 internal/git/save_request_scope_test.go diff --git a/api/source/v1alpha1/remotesaverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go index 182c6270..ed8f6558 100644 --- a/api/source/v1alpha1/remotesaverequest_types.go +++ b/api/source/v1alpha1/remotesaverequest_types.go @@ -130,9 +130,9 @@ type RemoteSaveRequestStatus struct { // Ready is True once the request ended without error in every GitTarget, // with reason "Committed" when one of them committed, and otherwise the // reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - // "WindowOutsideNamespace", "NoGitTarget" when no GitTarget mirrors this - // namespace, or "AuthorUnresolved" when this cluster sends audit events - // and none named the request's creator. Ready=False with Stalled=True is a + // "NoGitTarget" when no GitTarget mirrors this namespace, or + // "AuthorUnresolved" when this cluster sends audit events and none named + // the request's creator. Ready=False with Stalled=True is a // failure, with reason "NotAllowed" when GitTargets mirror this namespace // and none admits the request. // @@ -159,9 +159,8 @@ type RemoteSaveRequestGitTarget struct { Name string `json:"name"` // Reason is how the save ended in this GitTarget, as the Ready reasons - // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", - // "WindowOutsideNamespace" when the open window held edits made outside - // this namespace, or "Failed". Empty while it runs. + // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or + // "Failed". Empty while it runs. // +optional Reason string `json:"reason,omitempty"` @@ -196,15 +195,15 @@ type RemoteSaveRequestGitTarget struct { // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// RemoteSaveRequest asks GitOps Reverser to save the edits made in its -// namespace: the commit window of each GitTarget that mirrors this namespace -// and admits the request, in spec.allowedRemoteSaveRequests, gets its message -// and closes on its timers. Edits made elsewhere are never part of its save. -// Create it on the cluster the edits were made on, as JSON. Its creator is -// found from this cluster's audit events, as the edits' authors are, so it -// saves only its creator's edits; on a cluster with no audit it names -// nobody, and saves edits that name nobody. Its spec is immutable, and -// status reports each GitTarget's commit. +// RemoteSaveRequest asks GitOps Reverser to save its creator's edits: the +// commit window of each GitTarget that mirrors this namespace and admits the +// request, in spec.allowedRemoteSaveRequests, gets its message and closes on +// its timers. The window holds the creator's edits anywhere that GitTarget +// mirrors, not only in this namespace. Create it on the cluster the edits +// were made on, as JSON. Its creator is found from this cluster's audit +// events, as the edits' authors are, so it saves only its creator's edits; on +// a cluster with no audit it names nobody, and saves edits that name nobody. +// Its spec is immutable, and status reports each GitTarget's commit. type RemoteSaveRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/api/v1alpha3/access_grants.go b/api/v1alpha3/access_grants.go index 19864d12..c66ee4d5 100644 --- a/api/v1alpha3/access_grants.go +++ b/api/v1alpha3/access_grants.go @@ -219,6 +219,72 @@ type KubeProviderApplierClaimGrant struct { ApplierNamespaces []ScopeNamespace `json:"applierNamespaces"` } +// GitTargetSaveRequestGrant admits the SaveRequests of some namespaces to save +// a GitTarget's commit windows, and bounds what they may claim of a window +// whose writes name no author. +type GitTargetSaveRequestGrant struct { + // ConsumerNamespaces are the namespaces whose SaveRequests this grant + // admits. from: "Same" is this GitTarget's namespace. + // +required + ConsumerNamespaces ConsumerNamespaces `json:"consumerNamespaces"` + + // UnattributedWindows says how much an admitted SaveRequest may claim of a + // commit window whose writes name no author, when it asks in + // spec.window.author. It acts only while the GitTarget's KubeProvider has + // never received an audit fact; on a cluster that has, a write naming no + // author is never claimed. Defaults to "Keep". + // + // - "Keep": a request attaches only to its creator's own window. + // - "SavedBy": a request may give such a window its message. The commit + // names no author, and its message ends with a Saved-By line naming the + // request's creator. + // - "TakeOver": a request may also take such a window over. Its creator + // authors the commit, and the message ends with an Author-Source line. + // + // Every write that names no author shares one window, whoever made it, so + // a takeover names the creator as the author of all of them. Allow it only + // where one person edits the mirrored namespaces at a time. A request + // admitted by several grants may claim the most any of them allows. + // +optional + // +kubebuilder:validation:Enum=Keep;SavedBy;TakeOver + // +kubebuilder:default=Keep + UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` +} + +// SaveRequestAdmission is what a GitTarget's allowedSaveRequests give a +// SaveRequest. +// +kubebuilder:object:generate=false +type SaveRequestAdmission struct { + // Admitted reports whether some grant admits the request's namespace. + Admitted bool + // UnattributedWindows is the most the admitting grants allow the request to + // claim of a window whose writes name no author. + UnattributedWindows UnattributedWindowPolicy +} + +// AdmitsSaveRequest reports what spec.allowedSaveRequests gives a SaveRequest in namespace, by the +// labels that namespace carries in the control cluster. +func (t *GitTarget) AdmitsSaveRequest(namespace string, nsLabels map[string]string) (SaveRequestAdmission, error) { + grants := t.saveRequestGrants() + admitting, err := admitAll(consumersOf(grants), t.Namespace, namespace, nsLabels) + if err != nil || len(admitting) == 0 { + return SaveRequestAdmission{}, err + } + out := SaveRequestAdmission{Admitted: true, UnattributedWindows: UnattributedWindowsKeep} + for _, i := range admitting { + policy := grants[i].UnattributedWindows + if policy != "" && !out.UnattributedWindows.Allows(WindowAuthor(policy)) { + out.UnattributedWindows = policy + } + } + return out, nil +} + +// DescribeAllowedSaveRequests names the namespaces spec.allowedSaveRequests admits. +func (t *GitTarget) DescribeAllowedSaveRequests() string { + return describeAdmitted(consumersOf(t.saveRequestGrants()), t.Namespace) +} + // SourceNamespace is a namespace of a source cluster, by exact name, or "*" // for every namespace. // +kubebuilder:validation:MaxLength=63 @@ -273,7 +339,7 @@ func (t *GitTarget) AdmitsRemoteSaveRequest(namespace string, requestLabels map[ return false, errors.Join(errs...) } -// The API server defaults an omitted grant list on GitBranch and KubeProvider (the +// The API server defaults an omitted grant list on GitBranch, KubeProvider and GitTarget (the // +kubebuilder:default markers on their fields), so a list read from it is never nil. These mirror // those markers for an object that did not come from the API server, such as one built in memory or // read through a fake client: nil is the default, and only a non-nil empty list admits nobody. @@ -303,6 +369,15 @@ func DefaultKubeProviderApplierClaimGrants() []KubeProviderApplierClaimGrant { }} } +// DefaultGitTargetSaveRequestGrants is GitTarget.spec.allowedSaveRequests when it is omitted: +// SaveRequests in the GitTarget's own namespace, claiming nothing of an unattributed window. +func DefaultGitTargetSaveRequestGrants() []GitTargetSaveRequestGrant { + return []GitTargetSaveRequestGrant{{ + ConsumerNamespaces: ConsumerNamespaces{From: GrantFromSame}, + UnattributedWindows: UnattributedWindowsKeep, + }} +} + // orDefault is grants, or def when grants is nil (omitted rather than empty). func orDefault[G any](grants []G, def func() []G) []G { if grants == nil { @@ -326,6 +401,14 @@ func (p *KubeProvider) applierClaimGrants() []KubeProviderApplierClaimGrant { return orDefault(p.Spec.AllowedApplierClaims, DefaultKubeProviderApplierClaimGrants) } +// saveRequestGrants is spec.allowedSaveRequests as the API server serves it. +func (t *GitTarget) saveRequestGrants() []GitTargetSaveRequestGrant { + return orDefault(t.Spec.AllowedSaveRequests, DefaultGitTargetSaveRequestGrants) +} + +// consumers is the admitting half of a GitTargetSaveRequestGrant. +func (g *GitTargetSaveRequestGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } + // consumers is the admitting half of a GitProviderGitBranchGrant. func (g *GitProviderGitBranchGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } diff --git a/api/v1alpha3/access_grants_defaults_test.go b/api/v1alpha3/access_grants_defaults_test.go index affd4dc9..649f19c5 100644 --- a/api/v1alpha3/access_grants_defaults_test.go +++ b/api/v1alpha3/access_grants_defaults_test.go @@ -42,6 +42,10 @@ func TestGrantDefaultsMatchCRDs(t *testing.T) { assertCRDDefault(t, "configbutler.ai_kubeproviders.yaml", "allowedApplierClaims", DefaultKubeProviderApplierClaimGrants()) }) + t.Run("GitTarget allowedSaveRequests", func(t *testing.T) { + assertCRDDefault(t, "configbutler.ai_gittargets.yaml", "allowedSaveRequests", + DefaultGitTargetSaveRequestGrants()) + }) } // assertCRDDefault asserts that the CRD's default for spec. decodes to want. @@ -73,6 +77,12 @@ func TestGrantLists_EmptySurvivesSerialization(t *testing.T) { } } + raw, err := json.Marshal(GitTargetSpec{AllowedSaveRequests: []GitTargetSaveRequestGrant{}}) + require.NoError(t, err) + var targetFields map[string]json.RawMessage + require.NoError(t, json.Unmarshal(raw, &targetFields)) + assert.JSONEq(t, "[]", string(targetFields["allowedSaveRequests"]), "GitTarget spec.allowedSaveRequests") + var decoded GitBranchSpec require.NoError(t, json.Unmarshal([]byte(`{"allowedGitTargets":[]}`), &decoded)) assert.NotNil(t, decoded.AllowedGitTargets, "a decoded [] stays distinguishable from omitted") diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 41593534..48ebeecb 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -186,6 +186,24 @@ type GitTargetSpec struct { // +optional Commit *GitTargetCommitSpec `json:"commit,omitempty"` + // AllowedSaveRequests says which SaveRequests may save this GitTarget's + // commit windows, by the namespace they are in, and how much each may + // claim of a window whose writes name no author: + // + // - Omitted: the SaveRequests in this GitTarget's namespace, as one grant + // with from: "Same" that claims nothing. + // - Empty list ([]): no SaveRequest. + // - A list of grants: the SaveRequests that at least one grant allows. + // + // A list replaces the default: the SaveRequests in this namespace keep + // access only if at least one grant allows them. A request no grant allows + // fails with reason "NotAllowed". + // +optional + // +listType=atomic + // +kubebuilder:validation:MaxItems=64 + // +kubebuilder:default={{consumerNamespaces: {from: Same}, unattributedWindows: Keep}} + AllowedSaveRequests []GitTargetSaveRequestGrant `json:"allowedSaveRequests"` + // Consent lives on the GitTarget, not on the KubeProvider that serves the requests: the provider's // owner lends a cluster, and whether a save made on it may steer this folder's commits, with its // message, its timers and an apply, is the folder's owner's decision. Nothing is admitted by @@ -199,7 +217,8 @@ type GitTargetSpec struct { // mirror but none admits reports "NotAllowed". The KubeProvider must also // set spec.remoteSaveRequests "Enabled". // - // An admitted request saves only the edits made in its own namespace. + // An admitted request saves its creator's whole commit window on this + // GitTarget, edits outside the request's namespace included. // +optional // +listType=atomic // +kubebuilder:validation:MaxItems=16 @@ -256,28 +275,6 @@ type GitTargetCommitSpec struct { // built-in one. // +optional Message *CommitMessageSpec `json:"message,omitempty"` - - // UnattributedWindows says how much a SaveRequest on this GitTarget may - // claim of a commit window whose writes name no author. It acts only while - // the GitTarget's KubeProvider has never received an audit fact; on a - // cluster that has, a write naming no author is never claimed. Defaults to - // "Keep". - // - // - "Keep": a request attaches only to its creator's own window. - // - "SavedBy": a request that asks may give such a window its message. The - // commit names no author, and its message ends with a Saved-By line - // naming the request's creator. - // - "TakeOver": a request that asks may also take such a window over. Its - // creator authors the commit, and the message ends with an Author-Source - // line. - // - // Every write that names no author shares one window, whoever made it, so - // a takeover names the creator as the author of all of them. Allow it only - // where one person edits the mirrored namespaces at a time. - // +optional - // +kubebuilder:validation:Enum=Keep;SavedBy;TakeOver - // +kubebuilder:default=Keep - UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` } // UnattributedWindowPolicy is how much a SaveRequest may claim of a commit diff --git a/api/v1alpha3/saverequest_types.go b/api/v1alpha3/saverequest_types.go index f334e902..cc7c53a5 100644 --- a/api/v1alpha3/saverequest_types.go +++ b/api/v1alpha3/saverequest_types.go @@ -5,6 +5,7 @@ package v1alpha3 import ( meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" ) // SaveRequestSpec defines the desired state of SaveRequest. The spec is @@ -25,11 +26,13 @@ import ( // +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="SaveRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.whenNothingToCommit) || self.whenNothingToCommit != 'CommitEmpty' || has(self.message)",message="whenNothingToCommit: CommitEmpty requires spec.message, because a message is all an empty commit records" type SaveRequestSpec struct { - // GitTargetRef names the GitTarget, in this namespace, whose commit window - // this request attaches to. + // GitTargetRef names the GitTarget whose commit window this request + // attaches to: in this namespace, or namespace names another. The + // GitTarget's spec.allowedSaveRequests must admit this namespace, or the + // request fails with reason "NotAllowed". // +required // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitTargetRef.name must not be empty" - GitTargetRef meta.LocalObjectReference `json:"gitTargetRef"` + GitTargetRef meta.NamespacedObjectReference `json:"gitTargetRef"` // Message is the commit message this request asks for. Omitted, the // GitTarget's spec.commit.message.liveTemplate words the commit. @@ -160,8 +163,9 @@ type SaveRequestWindow struct { // to the creator's own window. "SavedBy" may also give its message to a // window whose writes name no author, naming no author. "TakeOver" may // also take such a window over, and the creator authors the commit. The - // GitTarget's spec.commit.unattributedWindows must allow what is asked, or - // the request fails with reason TakeOverNotAllowed. Defaults to "Own". + // unattributedWindows of a grant in the GitTarget's spec.allowedSaveRequests + // that admits this namespace must allow what is asked, or the request fails + // with reason TakeOverNotAllowed. Defaults to "Own". // +optional // +kubebuilder:validation:Enum=Own;SavedBy;TakeOver // +kubebuilder:default=Own @@ -320,6 +324,16 @@ type SaveRequestList struct { Items []SaveRequest `json:"items"` } +// GitTargetKey is the GitTarget spec.gitTargetRef names: in the request's namespace unless the ref +// names another. +func (s *SaveRequest) GitTargetKey() types.NamespacedName { + namespace := s.Spec.GitTargetRef.Namespace + if namespace == "" { + namespace = s.Namespace + } + return types.NamespacedName{Namespace: namespace, Name: s.Spec.GitTargetRef.Name} +} + func init() { SchemeBuilder.Register(&SaveRequest{}, &SaveRequestList{}) } diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 95be0da2..75d47d97 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -1213,6 +1213,22 @@ func (in *GitTargetRetentionStatus) DeepCopy() *GitTargetRetentionStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitTargetSaveRequestGrant) DeepCopyInto(out *GitTargetSaveRequestGrant) { + *out = *in + in.ConsumerNamespaces.DeepCopyInto(&out.ConsumerNamespaces) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetSaveRequestGrant. +func (in *GitTargetSaveRequestGrant) DeepCopy() *GitTargetSaveRequestGrant { + if in == nil { + return nil + } + out := new(GitTargetSaveRequestGrant) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = *in @@ -1248,6 +1264,13 @@ func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = new(GitTargetCommitSpec) (*in).DeepCopyInto(*out) } + if in.AllowedSaveRequests != nil { + in, out := &in.AllowedSaveRequests, &out.AllowedSaveRequests + *out = make([]GitTargetSaveRequestGrant, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } if in.AllowedRemoteSaveRequests != nil { in, out := &in.AllowedRemoteSaveRequests, &out.AllowedRemoteSaveRequests *out = make([]GitTargetRemoteSaveRequestGrant, len(*in)) diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index 3dbb019d..02dff40b 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -116,7 +116,8 @@ spec: mirror but none admits reports "NotAllowed". The KubeProvider must also set spec.remoteSaveRequests "Enabled". - An admitted request saves only the edits made in its own namespace. + An admitted request saves its creator's whole commit window on this + GitTarget, edits outside the request's namespace included. items: description: |- GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some @@ -198,6 +199,173 @@ spec: maxItems: 16 type: array x-kubernetes-list-type: atomic + allowedSaveRequests: + default: + - consumerNamespaces: + from: Same + unattributedWindows: Keep + description: |- + AllowedSaveRequests says which SaveRequests may save this GitTarget's + commit windows, by the namespace they are in, and how much each may + claim of a window whose writes name no author: + + - Omitted: the SaveRequests in this GitTarget's namespace, as one grant + with from: "Same" that claims nothing. + - Empty list ([]): no SaveRequest. + - A list of grants: the SaveRequests that at least one grant allows. + + A list replaces the default: the SaveRequests in this namespace keep + access only if at least one grant allows them. A request no grant allows + fails with reason "NotAllowed". + items: + description: |- + GitTargetSaveRequestGrant admits the SaveRequests of some namespaces to save + a GitTarget's commit windows, and bounds what they may claim of a window + whose writes name no author. + properties: + consumerNamespaces: + description: |- + ConsumerNamespaces are the namespaces whose SaveRequests this grant + admits. from: "Same" is this GitTarget's namespace. + properties: + from: + description: |- + From selects which namespaces this grant allows consumers from: + + - "Same": the namespace of the object that holds the grant. + - "Names": the namespaces listed in names. + - "Selector": the namespaces whose labels match selector. + - "All": every namespace. + enum: + - Same + - Names + - Selector + - All + type: string + names: + description: 'Names are the namespaces allowed with from: + "Names", by exact name.' + items: + description: |- + GrantNamespace is a namespace a grant admits, by exact name. A variable and + "*" are refused. + maxLength: 63 + type: string + x-kubernetes-validations: + - message: 'names: a variable is refused, because names + are matched against the consumer''s namespace and + ''{consumer.namespace}'' would admit every consumer; + use from: All to admit every namespace, or from: Same + for the namespace of the object holding the grant' + rule: self != '{consumer.namespace}' && self != '{consumer.name}' + - message: 'names: ''*'' is refused; use from: All to + admit every namespace' + rule: self != '*' + - message: names holds namespace names (DNS labels) + rule: self == '{consumer.namespace}' || self == '{consumer.name}' + || self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$') + maxItems: 64 + minItems: 1 + type: array + x-kubernetes-list-type: set + selector: + description: |- + Selector matches the labels of the namespaces allowed with from: + "Selector". It must not be empty: to allow every namespace, use from: + "All". + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + required: + - from + type: object + x-kubernetes-validations: + - message: 'from: Names needs names' + rule: self.from != 'Names' || has(self.names) + - message: 'names goes only with from: Names' + rule: '!has(self.names) || self.from == ''Names''' + - message: 'from: Selector needs a selector; to admit every + namespace, use from: All' + rule: self.from != 'Selector' || has(self.selector) + - message: 'selector goes only with from: Selector' + rule: '!has(self.selector) || self.from == ''Selector''' + - message: 'selector: {} is refused; to admit every namespace, + use from: All' + rule: '!has(self.selector) || (has(self.selector.matchLabels) + && size(self.selector.matchLabels) > 0) || (has(self.selector.matchExpressions) + && size(self.selector.matchExpressions) > 0)' + unattributedWindows: + default: Keep + description: |- + UnattributedWindows says how much an admitted SaveRequest may claim of a + commit window whose writes name no author, when it asks in + spec.window.author. It acts only while the GitTarget's KubeProvider has + never received an audit fact; on a cluster that has, a write naming no + author is never claimed. Defaults to "Keep". + + - "Keep": a request attaches only to its creator's own window. + - "SavedBy": a request may give such a window its message. The commit + names no author, and its message ends with a Saved-By line naming the + request's creator. + - "TakeOver": a request may also take such a window over. Its creator + authors the commit, and the message ends with an Author-Source line. + + Every write that names no author shares one window, whoever made it, so + a takeover names the creator as the author of all of them. Allow it only + where one person edits the mirrored namespaces at a time. A request + admitted by several grants may claim the most any of them allows. + enum: + - Keep + - SavedBy + - TakeOver + type: string + required: + - consumerNamespaces + type: object + maxItems: 64 + type: array + x-kubernetes-list-type: atomic applierClaimRef: description: |- ApplierClaimRef names the FluxClaim or ArgoCDClaim, in this namespace, that @@ -282,31 +450,6 @@ spec: one that fails at commit time commits the message as written. type: string type: object - unattributedWindows: - default: Keep - description: |- - UnattributedWindows says how much a SaveRequest on this GitTarget may - claim of a commit window whose writes name no author. It acts only while - the GitTarget's KubeProvider has never received an audit fact; on a - cluster that has, a write naming no author is never claimed. Defaults to - "Keep". - - - "Keep": a request attaches only to its creator's own window. - - "SavedBy": a request that asks may give such a window its message. The - commit names no author, and its message ends with a Saved-By line - naming the request's creator. - - "TakeOver": a request that asks may also take such a window over. Its - creator authors the commit, and the message ends with an Author-Source - line. - - Every write that names no author shares one window, whoever made it, so - a takeover names the creator as the author of all of them. Allow it only - where one person edits the mirrored namespaces at a time. - enum: - - Keep - - SavedBy - - TakeOver - type: string window: description: |- Window sets when this GitTarget's commit windows close. A window opens on an diff --git a/config/crd/bases/configbutler.ai_saverequests.yaml b/config/crd/bases/configbutler.ai_saverequests.yaml index dbe5a4aa..021177e0 100644 --- a/config/crd/bases/configbutler.ai_saverequests.yaml +++ b/config/crd/bases/configbutler.ai_saverequests.yaml @@ -100,12 +100,18 @@ spec: type: string gitTargetRef: description: |- - GitTargetRef names the GitTarget, in this namespace, whose commit window - this request attaches to. + GitTargetRef names the GitTarget whose commit window this request + attaches to: in this namespace, or namespace names another. The + GitTarget's spec.allowedSaveRequests must admit this namespace, or the + request fails with reason "NotAllowed". properties: name: description: Name of the referent. type: string + namespace: + description: Namespace of the referent, when not specified it + acts as LocalObjectReference. + type: string required: - name type: object @@ -190,8 +196,9 @@ spec: to the creator's own window. "SavedBy" may also give its message to a window whose writes name no author, naming no author. "TakeOver" may also take such a window over, and the creator authors the commit. The - GitTarget's spec.commit.unattributedWindows must allow what is asked, or - the request fails with reason TakeOverNotAllowed. Defaults to "Own". + unattributedWindows of a grant in the GitTarget's spec.allowedSaveRequests + that admits this namespace must allow what is asked, or the request fails + with reason TakeOverNotAllowed. Defaults to "Own". enum: - Own - SavedBy diff --git a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml index 78611ac3..387fc82e 100644 --- a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml +++ b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml @@ -35,15 +35,15 @@ spec: schema: openAPIV3Schema: description: |- - RemoteSaveRequest asks GitOps Reverser to save the edits made in its - namespace: the commit window of each GitTarget that mirrors this namespace - and admits the request, in spec.allowedRemoteSaveRequests, gets its message - and closes on its timers. Edits made elsewhere are never part of its save. - Create it on the cluster the edits were made on, as JSON. Its creator is - found from this cluster's audit events, as the edits' authors are, so it - saves only its creator's edits; on a cluster with no audit it names - nobody, and saves edits that name nobody. Its spec is immutable, and - status reports each GitTarget's commit. + RemoteSaveRequest asks GitOps Reverser to save its creator's edits: the + commit window of each GitTarget that mirrors this namespace and admits the + request, in spec.allowedRemoteSaveRequests, gets its message and closes on + its timers. The window holds the creator's edits anywhere that GitTarget + mirrors, not only in this namespace. Create it on the cluster the edits + were made on, as JSON. Its creator is found from this cluster's audit + events, as the edits' authors are, so it saves only its creator's edits; on + a cluster with no audit it names nobody, and saves edits that name nobody. + Its spec is immutable, and status reports each GitTarget's commit. properties: apiVersion: description: |- @@ -172,9 +172,9 @@ spec: Ready is True once the request ended without error in every GitTarget, with reason "Committed" when one of them committed, and otherwise the reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - "WindowOutsideNamespace", "NoGitTarget" when no GitTarget mirrors this - namespace, or "AuthorUnresolved" when this cluster sends audit events - and none named the request's creator. Ready=False with Stalled=True is a + "NoGitTarget" when no GitTarget mirrors this namespace, or + "AuthorUnresolved" when this cluster sends audit events and none named + the request's creator. Ready=False with Stalled=True is a failure, with reason "NotAllowed" when GitTargets mirror this namespace and none admits the request. @@ -275,9 +275,8 @@ spec: reason: description: |- Reason is how the save ended in this GitTarget, as the Ready reasons - above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", - "WindowOutsideNamespace" when the open window held edits made outside - this namespace, or "Failed". Empty while it runs. + above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or + "Failed". Empty while it runs. type: string required: - name diff --git a/docs/architecture.md b/docs/architecture.md index a9137d98..02cc8ed1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1781,7 +1781,8 @@ immediately. That is not a failure: when the author state and GitTarget match, first come first served, and replaces the window's timers with the request's own, so time spent waiting never shortens the collection. A request that names its submitter may also claim a window on its `GitTarget` whose writes name no author, when it asks - (`spec.window.author`), the `GitTarget` allows it (`spec.commit.unattributedWindows`), and the + (`spec.window.author`), the grant admitting it allows it (`unattributedWindows` in the `GitTarget`'s + `spec.allowedSaveRequests`), and the `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`). With `SavedBy` the window takes the request's message, the commit names no author, and the message ends with a `Saved-By` trailer; with `TakeOver` the submitter authors the commit, and the message ends diff --git a/docs/configuration.md b/docs/configuration.md index ad7aefc5..9716e99e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -119,7 +119,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | -| `commit.unattributedWindows` | `Keep` | How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | +| `allowedSaveRequests` | `[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"Keep"}]` | The namespaces whose `SaveRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See [who may save](#who-may-save-specallowedsaverequests) | | `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may save this target's windows. See [`RemoteSaveRequest`](#remotesaverequest) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | @@ -178,7 +178,7 @@ The references are immutable: to hold another `Application`, delete the applier | Field | Default | What it does | |---|---|---| -| `gitTargetRef` | **required** | The target whose commit window the request attaches to | +| `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedSaveRequests` must admit this one | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | | `window.author` | `Own` | Whose window to attach to: `Own`, or also claim a window whose writes name no author with `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | @@ -2964,13 +2964,14 @@ Use it sparingly. ## `SaveRequest` -`SaveRequest` is a one-shot "save" signal for a same-namespace `GitTarget`. It does not create or -change watch rules. It attaches to its author's commit window, gives that window its message, and -decides when the window closes, instead of the target's timers. +`SaveRequest` is a one-shot "save" signal for a `GitTarget`. It does not create or change watch +rules. It attaches to its author's commit window, gives that window its message, and decides when +the window closes, instead of the target's timers. The important fields are: -- `spec.gitTargetRef.name`: the target whose window to attach to +- `spec.gitTargetRef`: the target whose window to attach to, by `name`, in the request's own + namespace unless `namespace` names another; see [who may save](#who-may-save-specallowedsaverequests) - `spec.message`: optional literal commit message, preserved verbatim - `spec.window`: which window to attach to and when it closes; see [the request window](#the-request-window-specwindow) @@ -3069,6 +3070,38 @@ attach to it: the outcome is `WindowMismatch` no matter how long the request wai that sends no audit events at all, see [saving where writes name no author](#saving-where-writes-name-no-author). +### Who may save: `spec.allowedSaveRequests` + +A `GitTarget` says which namespaces' `SaveRequest`s may save its windows in +`spec.allowedSaveRequests`, a grant list like the others (see [the grants](#the-grants)): + +- **Omitted:** the `SaveRequest`s in the `GitTarget`'s own namespace, as one grant with + `from: Same` that claims nothing of a window whose writes name no author. +- **`[]`:** no `SaveRequest`. +- **A list:** the `SaveRequest`s at least one grant admits. The list replaces the default, so the + own namespace keeps access only if a grant admits it. + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: shared + namespace: platform +spec: + allowedSaveRequests: + - consumerNamespaces: {from: Same} + - consumerNamespaces: + from: Selector + selector: + matchLabels: + platform.example.com/tenant: "true" + unattributedWindows: SavedBy +``` + +A request no grant admits fails when it is first seen, before it attaches: `Ready=False`, +`Stalled=True`, reason `NotAllowed`. Each grant also sets `unattributedWindows` for the requests it +admits, described below; a request admitted by several grants may claim the most any of them allows. + ### Saving where writes name no author On a source cluster that sends no audit events, no write names its author, so every write lands in @@ -3076,8 +3109,8 @@ a window that names no actor. A request whose submitter was captured at admissio such a window, and its save ends `WindowMismatch`. Two settings let it claim one, and both default to claiming nothing: -- the `GitTarget`'s `spec.commit.unattributedWindows` says how much a request on it may claim: - `Keep` (the default), `SavedBy`, or `TakeOver`; +- the `unattributedWindows` of the grant in the `GitTarget`'s `spec.allowedSaveRequests` that admits + the request's namespace says how much it may claim: `Keep` (the default), `SavedBy`, or `TakeOver`; - the request's `spec.window.author` says what this save asks for: `Own` (the default), `SavedBy`, or `TakeOver`. @@ -3087,8 +3120,8 @@ claiming nothing: | `SavedBy` | gives the window its message; the commit names no author, and its message ends with `Saved-By: ` | | `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: SaveRequest /; no audit record names who made these changes` | -`TakeOver` on the `GitTarget` allows both claims, and `SavedBy` allows only `SavedBy`. A request that -asks for more than its `GitTarget` allows fails when it is first seen, before it attaches: +A grant's `TakeOver` allows both claims, and `SavedBy` allows only `SavedBy`. A request that asks for +more than the grants admitting it allow fails when it is first seen, before it attaches: `Ready=False`, `Stalled=True`, reason `TakeOverNotAllowed`. Both claims act only while the `GitTarget`'s `KubeProvider` has never received an audit fact: its @@ -3225,8 +3258,8 @@ never deleted. A `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next to the edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It -names no `GitTarget`: it saves the edits made in its own namespace, in the commit window of every -`GitTarget` that mirrors that namespace from that cluster and allows it. Each window gets its message +names no `GitTarget`: it saves its creator's commit window in every `GitTarget` that mirrors its +namespace from that cluster and allows it. Each window gets its message and closes on its timers, as with a [`SaveRequest`](#saverequest). ```yaml @@ -3278,14 +3311,11 @@ its labels, so a selector proves nothing about who made it. ### What a save holds -A `RemoteSaveRequest` saves only the edits made in its own namespace. It attaches to a window whose -writes are all in that namespace; while it is attached, a write anywhere else, including a -cluster-scoped one, does not join the window and opens one of its own. A window that already holds -writes from elsewhere is left alone, and the save ends `WindowOutsideNamespace`. Save from a -`SaveRequest` on the cluster GitOps Reverser runs in to commit such a window under one message. - -A `GitTarget` that mirrors only cluster-scoped types mirrors no namespace, so no `RemoteSaveRequest` -saves it. +A commit window holds one author's edits to one `GitTarget`, wherever that `GitTarget` mirrors them. +An admitted `RemoteSaveRequest` therefore saves its creator's whole window, edits outside the +request's own namespace included: a grant in `allowedRemoteSaveRequests` authorizes exactly that. The +request's namespace only decides which `GitTarget`s it reaches. A `GitTarget` that mirrors only +cluster-scoped types mirrors no namespace, so no `RemoteSaveRequest` reaches it. ### Who the save is from @@ -3297,7 +3327,7 @@ loses the save's author without an error. | The source cluster | The save | | --- | --- | | sends audit events, and one names the creator | attaches to the creator's windows; `status.author` names them | -| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window in its namespace whose edits name nobody; the commit's author stays unresolved | +| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window whose edits name nobody; the commit's author stays unresolved | | sends audit events, but none names the creator | is refused, `Ready=True` with reason `AuthorUnresolved`: its event went missing, and attaching it would claim the window that should stay unattributed. Save again | `spec.window.attachTimeout` (default `5s`) counts from when GitOps Reverser picks the request up, @@ -3309,7 +3339,7 @@ window as for a `SaveRequest`. `status.gitTargets` lists each `GitTarget` the save was admitted to, by `namespace` and `name` on the cluster GitOps Reverser runs in, with the save's `reason` there (`Committed`, `NoWindow`, -`AlreadyPresent`, `WindowMismatch`, `WindowOutsideNamespace` or `Failed`), its `branch` and `commit`, +`AlreadyPresent`, `WindowMismatch` or `Failed`), its `branch` and `commit`, and, with `catchUp: AfterPush`, its `applied` reason and `appliedRevision`. - **Ready** is `True` once the save ended without error in every `GitTarget`, with reason `Committed` diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index 903d5cd4..238dc086 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -12,13 +12,15 @@ put its author's name on the commit. Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The two request kinds are a pair: `SaveRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/remote_save_requests.go`) on -the source cluster. A KubeProvider serves its cluster's requests with `spec.remoteSaveRequests: -Enabled`, and each GitTarget admits them in `spec.allowedRemoteSaveRequests` (none by default). A -`RemoteSaveRequest` saves only the edits made in its own namespace. The takeover is opt-in, as -described below: the GitTarget's `spec.commit.unattributedWindows` and the `SaveRequest`'s -`spec.window.author`, each `SavedBy` or `TakeOver`. The open questions are answered as proposed: its -own group, a default `attachTimeout` of `5s`, the reason `AuthorUnresolved`, and no extra queue for a -second save on a GitTarget beyond the worker's own (a window carries one request at a time). +the source cluster. Each `GitTarget` admits them in a grant list: `spec.allowedSaveRequests` (by +control-cluster namespace; omitted, its own namespace) and `spec.allowedRemoteSaveRequests` (by source +namespace and labels; omitted, none). A KubeProvider also serves its cluster's remote requests only +with `spec.remoteSaveRequests: Enabled`. The takeover is opt-in, as described below, with one change: +the `GitTarget`'s bound moved from `spec.commit.unattributedWindows` into each `allowedSaveRequests` +grant, so a request from another namespace claims only what its own grant allows. The open questions +are answered as proposed: its own group, a default `attachTimeout` of `5s`, the reason +`AuthorUnresolved`, and no extra queue for a second save on a GitTarget beyond the worker's own (a +window carries one request at a time). This revision takes in gitops-api's review of the first draft, and a later API review of the implementation: see [After the API review](#after-the-api-review). @@ -282,14 +284,15 @@ ways forward before the branch merges: ## After the API review -A review of the implementation found three things, all changed: +A review of the implementation found three things: -1. **A save was not confined to its namespace.** A window groups by GitTarget and author, not by - namespace, so a save from `shop` could commit the same author's edits in `billing`, or, with no - audit, anyone's. A `RemoteSaveRequest` now attaches only to a window whose writes are all in its - namespace, and once attached, a write anywhere else opens a window of its own. A window that - already holds writes from elsewhere is left alone: `WindowOutsideNamespace`. Grouping every window - by namespace stays rejected; the confinement applies only to the window a remote save holds. +1. **A remote save is not confined to its namespace.** A window groups by GitTarget and author, not by + namespace, so a save from `shop` commits the same author's edits in `billing` too, or, with no + audit, anyone's. The review offered two answers: confine the save, or let the GitTarget's grant + authorize the whole window. The grant is the answer: an admitted `RemoteSaveRequest` saves its + creator's whole window on that GitTarget, and the docs say so. Confining it was built and taken out + again: a namespace-shaped exception to windows that are not namespace-shaped, with an outcome of its + own (`WindowOutsideNamespace`) that the person saving could do nothing about. 2. **A cluster-scoped watch matched every namespace.** A collection with no namespace is either cluster-scoped or every namespace, and the match did not tell them apart. It now reads the GitTarget's watched types and counts only namespaced ones. @@ -301,6 +304,6 @@ A review of the implementation found three things, all changed: requests and prove nothing about who made them. The status list became `status.gitTargets`, keyed by `namespace` and `name`, and the names became the -`SaveRequest`/`RemoteSaveRequest` pair. A `SaveRequest` on the control cluster gets no grant list: its -`spec.gitTargetRef` is a local reference, so the namespace's RBAC already decides who may save, and -`spec.commit.unattributedWindows` bounds what it may claim. +`SaveRequest`/`RemoteSaveRequest` pair. A `SaveRequest` then gained a cross-namespace +`spec.gitTargetRef`, and with it `GitTarget.spec.allowedSaveRequests`: a grant list like the others, +defaulting to one `from: Same` grant, whose grants also carry the `unattributedWindows` bound. diff --git a/docs/rbac.md b/docs/rbac.md index 57444f9e..1c871abf 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -96,9 +96,11 @@ status back there. Its credential then needs, in the `source.configbutler.ai` gr `remotesaverequests/status`. Nothing else on a source cluster is written. The manager role carries the same rules for the cluster the operator runs in, for an `inCluster` provider. -Whoever may create a `RemoteSaveRequest` in a namespace may save that namespace's edits, for every +Whoever may create a `RemoteSaveRequest` in a namespace may save their own commit window on every `GitTarget` whose `spec.allowedRemoteSaveRequests` admits it. That grant, not the provider's switch, -is the `GitTarget` owner's consent. +is the `GitTarget` owner's consent. On the cluster GitOps Reverser runs in, whoever may create a +`SaveRequest` in a namespace may save on every `GitTarget` whose `spec.allowedSaveRequests` admits +that namespace. ## The operator does not read Secrets wholesale diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 195d53e1..d633e4ef 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -158,11 +158,11 @@ kinds: - path: commit.message default: the built-in templates doc: How commits are phrased. See [commit messages](commit-messages.md) - - path: commit.unattributedWindows + - path: allowedSaveRequests doc: >- - How much a `SaveRequest` may claim of a window whose writes name no author, on a cluster - that never sent an audit fact: `Keep`, `SavedBy` or `TakeOver`. See - [saving where writes name no author](#saving-where-writes-name-no-author) + The namespaces whose `SaveRequest`s may save this target's windows, and how much each may + claim of a window whose writes name no author. See + [who may save](#who-may-save-specallowedsaverequests) - path: allowedRemoteSaveRequests default: none admitted doc: >- @@ -305,7 +305,9 @@ kinds: title: "`SaveRequest` (namespaced): save now" fields: - path: gitTargetRef - doc: The target whose commit window the request attaches to + doc: >- + The target whose commit window the request attaches to: in this namespace, or `namespace` + names another, whose `allowedSaveRequests` must admit this one - path: message default: the target's templates doc: Commit message, committed verbatim unless `requestTemplate` frames it diff --git a/internal/authz/saverequest_admission.go b/internal/authz/saverequest_admission.go new file mode 100644 index 00000000..6d38627c --- /dev/null +++ b/internal/authz/saverequest_admission.go @@ -0,0 +1,34 @@ +// SPDX-License-Identifier: Apache-2.0 + +package authz + +import ( + "context" + + "sigs.k8s.io/controller-runtime/pkg/client" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" +) + +// SaveRequestAdmitted reports whether request may save the commit windows of target, the GitTarget +// its spec.gitTargetRef names: a grant in target's spec.allowedSaveRequests admits the request's +// namespace, by the labels that namespace carries. The admission also says how much the request may +// claim of a window whose writes name no author. A non-NotFound read of the Namespace is returned as +// err, so the caller retries instead of refusing a request on a transient failure. +func SaveRequestAdmitted( + ctx context.Context, + reader client.Reader, + request *configv1alpha3.SaveRequest, + target *configv1alpha3.GitTarget, +) (Decision, configv1alpha3.SaveRequestAdmission, error) { + nsLabels, err := namespaceLabels(ctx, reader, request.Namespace) + if err != nil { + return Decision{}, configv1alpha3.SaveRequestAdmission{}, err + } + admission, selErr := target.AdmitsSaveRequest(request.Namespace, nsLabels) + if !admission.Admitted { + return notAdmitted("GitTarget", client.ObjectKeyFromObject(target), target.DescribeAllowedSaveRequests(), + "spec.allowedSaveRequests", request.Namespace, selErr), admission, nil + } + return Decision{Allowed: true}, admission, nil +} diff --git a/internal/authz/saverequest_admission_test.go b/internal/authz/saverequest_admission_test.go new file mode 100644 index 00000000..9d95c09d --- /dev/null +++ b/internal/authz/saverequest_admission_test.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package authz + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/fake" + "sigs.k8s.io/controller-runtime/pkg/client/interceptor" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" +) + +// A GitTarget admits a SaveRequest by the namespace it is in and that namespace's labels, and gives it +// the most any admitting grant allows of a window whose writes name no author. +func TestSaveRequestAdmitted(t *testing.T) { + tenant := &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{ + Name: testNS, Labels: map[string]string{"platform.example.com/tenant": "true"}, + }} + request := &configv1alpha3.SaveRequest{ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: testNS}} + target := func(grants ...configv1alpha3.GitTargetSaveRequestGrant) *configv1alpha3.GitTarget { + return &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "shared", Namespace: platformNS}, + Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: grants}, + } + } + bySelector := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetSaveRequestGrant { + return configv1alpha3.GitTargetSaveRequestGrant{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{ + From: configv1alpha3.GrantFromSelector, + Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"platform.example.com/tenant": "true"}}, + }, + UnattributedWindows: policy, + } + } + all := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetSaveRequestGrant { + return configv1alpha3.GitTargetSaveRequestGrant{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, + UnattributedWindows: policy, + } + } + reader := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithObjects(tenant).Build() + + decision, _, err := SaveRequestAdmitted(context.Background(), reader, request, target()) + require.NoError(t, err) + assert.False(t, decision.Allowed, "the default admits only the GitTarget's own namespace") + assert.Equal(t, ReasonNamespaceNotAuthorized, decision.Reason) + assert.Contains(t, decision.Message, "spec.allowedSaveRequests") + + decision, admission, err := SaveRequestAdmitted(context.Background(), reader, request, + target(bySelector(configv1alpha3.UnattributedWindowsSavedBy))) + require.NoError(t, err) + assert.True(t, decision.Allowed, "the namespace's labels match") + assert.Equal(t, configv1alpha3.UnattributedWindowsSavedBy, admission.UnattributedWindows) + + _, admission, err = SaveRequestAdmitted(context.Background(), reader, request, + target(all(configv1alpha3.UnattributedWindowsKeep), bySelector(configv1alpha3.UnattributedWindowsTakeOver))) + require.NoError(t, err) + assert.Equal(t, configv1alpha3.UnattributedWindowsTakeOver, admission.UnattributedWindows, + "the most any admitting grant allows") + + failing := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithInterceptorFuncs(interceptor.Funcs{ + Get: func(context.Context, client.WithWatch, client.ObjectKey, client.Object, ...client.GetOption) error { + return errors.New("apiserver unavailable") + }, + }).Build() + _, _, err = SaveRequestAdmitted(context.Background(), failing, request, target(all(""))) + require.Error(t, err, "a namespace that cannot be read is retried, never refused") +} diff --git a/internal/controller/access_grants_admission_test.go b/internal/controller/access_grants_admission_test.go index e6b693e9..dc981ca1 100644 --- a/internal/controller/access_grants_admission_test.go +++ b/internal/controller/access_grants_admission_test.go @@ -159,6 +159,14 @@ var _ = Describe("Access grants at admission", func() { {"KubeProvider allowedApplierClaims", func(g map[string]any) *unstructured.Unstructured { return kubeProvider("allowedApplierClaims", g) }, map[string]any{"applierNamespaces": []any{"flux-system"}}}, + {"GitTarget allowedSaveRequests", + func(g map[string]any) *unstructured.Unstructured { + return object("GitTarget", map[string]any{ + "gitBranchRef": map[string]any{"name": "b"}, "path": "apps", + "kubeProviderRef": map[string]any{"name": "k"}, "allowedSaveRequests": []any{g}, + }) + }, + map[string]any{"unattributedWindows": "SavedBy"}}, } { It("refuses a "+list.name+" grant without a complete consumerNamespaces block", func() { refused(list.build(maps.Clone(list.scope)), "consumerNamespaces: Required value") diff --git a/internal/controller/duration_fields_admission_test.go b/internal/controller/duration_fields_admission_test.go index 8144c995..d638e212 100644 --- a/internal/controller/duration_fields_admission_test.go +++ b/internal/controller/duration_fields_admission_test.go @@ -198,7 +198,7 @@ var _ = Describe("Duration fields", func() { omitted := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-omitted", Namespace: "default"}, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, + GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, }, } Expect(k8sClient.Create(ctx, omitted)).To(Succeed()) @@ -215,7 +215,7 @@ var _ = Describe("Duration fields", func() { zero := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-zeros", Namespace: "default"}, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, + GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, Window: &configbutleraiv1alpha3.SaveRequestWindow{ AttachTimeout: &metav1.Duration{}, IdleTimeout: &metav1.Duration{}, @@ -309,7 +309,7 @@ var _ = Describe("Duration fields", func() { request := &configbutleraiv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "any-target"}, + GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, Message: "save", Window: &configbutleraiv1alpha3.SaveRequestWindow{ MaxDuration: &metav1.Duration{Duration: time.Minute}, diff --git a/internal/controller/grant_defaults_apiserver_test.go b/internal/controller/grant_defaults_apiserver_test.go index d767e175..2c468543 100644 --- a/internal/controller/grant_defaults_apiserver_test.go +++ b/internal/controller/grant_defaults_apiserver_test.go @@ -102,4 +102,29 @@ func TestGrantLists_APIServerDefaultsOmittedAndKeepsEmpty(t *testing.T) { require.NotNil(t, obj.Spec.AllowedApplierClaims) assert.Empty(t, obj.Spec.AllowedApplierClaims) }) + + target := func(name string, grants []configbutleraiv1alpha3.GitTargetSaveRequestGrant) *configbutleraiv1alpha3.GitTarget { + gt := &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: name}, + Spec: configbutleraiv1alpha3.GitTargetSpec{Path: "apps", AllowedSaveRequests: grants}, + } + gt.Spec.GitBranchRef.Name = "main" + gt.Spec.KubeProviderRef.Name = "local" + return gt + } + + t.Run("an omitted GitTarget allowedSaveRequests reads back as the default", func(t *testing.T) { + obj := target("omitted", nil) + require.NoError(t, c.Create(ctx, obj)) + require.NoError(t, c.Get(ctx, client.ObjectKeyFromObject(obj), obj)) + assert.Equal(t, configbutleraiv1alpha3.DefaultGitTargetSaveRequestGrants(), obj.Spec.AllowedSaveRequests) + }) + + t.Run("an empty GitTarget allowedSaveRequests stays empty through an update", func(t *testing.T) { + obj := target("deny-all", []configbutleraiv1alpha3.GitTargetSaveRequestGrant{}) + require.NoError(t, c.Create(ctx, obj)) + relabel(t, obj) + require.NotNil(t, obj.Spec.AllowedSaveRequests, "an update must not turn [] into the default") + assert.Empty(t, obj.Spec.AllowedSaveRequests) + }) } diff --git a/internal/controller/saverequest_apply.go b/internal/controller/saverequest_apply.go index a2a6cfd8..254035c6 100644 --- a/internal/controller/saverequest_apply.go +++ b/internal/controller/saverequest_apply.go @@ -52,7 +52,7 @@ func (r *SaveRequestReconciler) followApply( log := logf.FromContext(ctx).WithName("SaveRequestReconciler") attach := git.AttachSaveRequest{ Namespace: cr.Namespace, Name: cr.Name, UID: string(cr.UID), - GitTargetName: cr.Spec.GitTargetRef.Name, GitTargetNamespace: cr.Namespace, + GitTargetName: cr.GitTargetKey().Name, GitTargetNamespace: cr.GitTargetKey().Namespace, } apply, known, err := r.Finalizer.LookupSaveRequestApply(ctx, attach) if err != nil { diff --git a/internal/controller/saverequest_controller.go b/internal/controller/saverequest_controller.go index 8ea1a6f3..cbaa9d84 100644 --- a/internal/controller/saverequest_controller.go +++ b/internal/controller/saverequest_controller.go @@ -19,6 +19,7 @@ import ( logf "sigs.k8s.io/controller-runtime/pkg/log" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/authz" "github.com/ConfigButler/gitops-reverser/internal/git" "github.com/ConfigButler/gitops-reverser/internal/queue" ) @@ -230,13 +231,9 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) // decision is final — there is no wait and no requeue for the author. author, attribution := r.attributeAuthor(ctx, saveRequest) - // Refused only at first sight, before any attach was sent: a GitTarget's policy lowered later - // does not fail a request the worker may already hold, which keeps what it was first sent. - claim, claimErr := r.unattributedClaim(ctx, saveRequest) - if claimErr != nil && findCondition(saveRequest.Status.Conditions, ConditionTypeReady) == nil { - r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, claimErr, attribution, - resolvedSaveRequestTarget(saveRequest)) - return ctrl.Result{}, nil + claim, refused, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) + if err != nil || refused { + return ctrl.Result{}, err } if err := r.stampFirstSightConditions(ctx, saveRequest, attribution); err != nil { @@ -264,8 +261,8 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) UID: string(saveRequest.UID), Author: author.Author, Attribution: attribution.gitOutcome(), - GitTargetName: saveRequest.Spec.GitTargetRef.Name, - GitTargetNamespace: saveRequest.Namespace, + GitTargetName: saveRequest.GitTargetKey().Name, + GitTargetNamespace: saveRequest.GitTargetKey().Namespace, Message: saveRequest.Spec.Message, Attach: window.attach, AttachTimeout: window.attachTimeout, @@ -283,22 +280,69 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) return r.attachSaveRequest(ctx, log, req, saveRequest, attribution, attach) } +// admitAtFirstSight runs admitSaveRequest, and fails a request it refuses, but only at first sight, +// before any attach was sent: a GitTarget's grants lowered later do not fail a request the worker +// may already hold, which keeps what it was first sent. refused reports a request now failed; err is +// a read to retry. +func (r *SaveRequestReconciler) admitAtFirstSight( + ctx context.Context, + log logr.Logger, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, +) (git.UnattributedClaim, bool, error) { + claim, err := r.admitSaveRequest(ctx, saveRequest) + if err == nil { + return claim, false, nil + } + if !errors.Is(err, errSaveRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed) { + return claim, false, err + } + if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) != nil { + return claim, false, nil + } + r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, + resolvedSaveRequestTarget(saveRequest)) + return claim, true, nil +} + // errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author -// than its GitTarget's spec.commit.unattributedWindows allows. It is terminal: the spec is immutable. +// than the grants admitting it allow. It is terminal: the spec is immutable. var errTakeOverNotAllowed = errors.New("spec.window.author asks for more than the GitTarget allows") -// unattributedClaim decides what the request may claim of a window on its GitTarget whose writes -// name no author (git.AttachSaveRequest.Unattributed). It claims what spec.window.author asks for, -// and only while the GitTarget's KubeProvider has never received an audit fact: its -// AuditFactsReceived condition is not True, so no write watched on that cluster can name its author. -// On a cluster that has sent one, a write naming no author means its fact went missing, and nothing -// is claimed. A request that asks for more than the GitTarget's spec.commit.unattributedWindows -// allows is errTakeOverNotAllowed. A GitTarget or KubeProvider that cannot be read claims nothing, -// and the request attaches by author alone. -func (r *SaveRequestReconciler) unattributedClaim( +// errSaveRequestNotAllowed is a request whose GitTarget's spec.allowedSaveRequests admits no grant for +// its namespace. It is terminal: the spec is immutable, so a later grant reaches only a new request. +var errSaveRequestNotAllowed = errors.New("the GitTarget does not admit SaveRequests from this namespace") + +// admitSaveRequest decides whether the request may save its GitTarget's windows, and what it may claim +// of a window whose writes name no author (git.AttachSaveRequest.Unattributed). +// +// A grant in the GitTarget's spec.allowedSaveRequests must admit the request's namespace, or the +// request is errSaveRequestNotAllowed. It then claims what spec.window.author asks for, and only while +// the GitTarget's KubeProvider has never received an audit fact: its AuditFactsReceived condition is +// not True, so no write watched on that cluster can name its author. On a cluster that has sent one, a +// write naming no author means its fact went missing, and nothing is claimed. A request that asks for +// more than the admitting grants' unattributedWindows allow is errTakeOverNotAllowed. +// +// A GitTarget that does not exist decides nothing here: the attach path reports it. Any other read +// error is returned as is, so the request is retried rather than refused. +func (r *SaveRequestReconciler) admitSaveRequest( ctx context.Context, saveRequest *configbutleraiv1alpha3.SaveRequest, ) (git.UnattributedClaim, error) { + var target configbutleraiv1alpha3.GitTarget + if err := r.Get(ctx, saveRequest.GitTargetKey(), &target); err != nil { + if apierrors.IsNotFound(err) { + return git.UnattributedClaimNone, nil + } + return git.UnattributedClaimNone, fmt.Errorf("read GitTarget %s: %w", saveRequest.GitTargetKey(), err) + } + decision, admission, err := authz.SaveRequestAdmitted(ctx, r.Client, saveRequest, &target) + if err != nil { + return git.UnattributedClaimNone, err + } + if !decision.Allowed { + return git.UnattributedClaimNone, fmt.Errorf("%w: %s", errSaveRequestNotAllowed, decision.Message) + } ask := configbutleraiv1alpha3.WindowAuthorOwn if window := saveRequest.Spec.Window; window != nil && window.Author != "" { ask = window.Author @@ -306,19 +350,10 @@ func (r *SaveRequestReconciler) unattributedClaim( if ask == configbutleraiv1alpha3.WindowAuthorOwn { return git.UnattributedClaimNone, nil } - var target configbutleraiv1alpha3.GitTarget - if err := r.Get(ctx, client.ObjectKey{ - Namespace: saveRequest.Namespace, Name: saveRequest.Spec.GitTargetRef.Name, - }, &target); err != nil { - return git.UnattributedClaimNone, nil //nolint:nilerr // a missing GitTarget is the attach path's to report - } - policy := configbutleraiv1alpha3.UnattributedWindowsKeep - if target.Spec.Commit != nil && target.Spec.Commit.UnattributedWindows != "" { - policy = target.Spec.Commit.UnattributedWindows - } - if !policy.Allows(ask) { - return git.UnattributedClaimNone, fmt.Errorf("%w: spec.window.author is %q, and GitTarget %s has "+ - "spec.commit.unattributedWindows %q", errTakeOverNotAllowed, ask, target.Name, policy) + if !admission.UnattributedWindows.Allows(ask) { + return git.UnattributedClaimNone, fmt.Errorf("%w: spec.window.author is %q, and the grants of GitTarget %s "+ + "that admit namespace %q allow unattributedWindows %q", errTakeOverNotAllowed, ask, + saveRequest.GitTargetKey(), saveRequest.Namespace, admission.UnattributedWindows) } key := target.KubeProviderKey() if key.Name == "" { @@ -359,7 +394,7 @@ func (r *SaveRequestReconciler) attachSaveRequest( if result.Err != nil { log.Error(result.Err, "SaveRequest finalize failed", - "gitTarget", saveRequest.Spec.GitTargetRef.Name, "name", req.NamespacedName) + "gitTarget", saveRequest.GitTargetKey().String(), "name", req.NamespacedName) } r.writeTerminalStatus(ctx, log, saveRequest, result, result.Err, attribution, resolvedSaveRequestTarget(saveRequest)) @@ -739,7 +774,7 @@ func (r *SaveRequestReconciler) writeTerminalStatus( "commit", current.Status.Commit, "outcome", result.Outcome, "finalizeError", finalizeError, - "gitTarget", current.Spec.GitTargetRef.Name, + "gitTarget", current.GitTargetKey().String(), "age", time.Since(current.CreationTimestamp.Time).String()) return } diff --git a/internal/controller/saverequest_controller_test.go b/internal/controller/saverequest_controller_test.go index 18cde54b..e7ffe49b 100644 --- a/internal/controller/saverequest_controller_test.go +++ b/internal/controller/saverequest_controller_test.go @@ -30,7 +30,7 @@ var _ = Describe("SaveRequest controller", func() { Namespace: namespace, }, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{ + GitTargetRef: meta.NamespacedObjectReference{ Name: "team-a-config", }, Message: "increase checkout API memory", @@ -58,7 +58,7 @@ var _ = Describe("SaveRequest controller", func() { Namespace: namespace, }, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{ + GitTargetRef: meta.NamespacedObjectReference{ Name: "team-a-config", }, }, @@ -114,7 +114,7 @@ var _ = Describe("SaveRequest message schema", func() { Namespace: namespace, }, Spec: configbutleraiv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "team-a-config"}, + GitTargetRef: meta.NamespacedObjectReference{Name: "team-a-config"}, Message: message, }, } diff --git a/internal/controller/saverequest_controller_unit_test.go b/internal/controller/saverequest_controller_unit_test.go index 93d8e9c2..6a3f4f2c 100644 --- a/internal/controller/saverequest_controller_unit_test.go +++ b/internal/controller/saverequest_controller_unit_test.go @@ -12,6 +12,7 @@ import ( meta "github.com/fluxcd/pkg/apis/meta" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -108,7 +109,7 @@ func newSaveRequest(name string) *configv1alpha3.SaveRequest { CreationTimestamp: metav1.Now(), }, Spec: configv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "team-a-config"}, + GitTargetRef: meta.NamespacedObjectReference{Name: "team-a-config"}, Message: "save: " + name, }, } @@ -147,6 +148,7 @@ func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...clien t.Helper() scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) + require.NoError(t, corev1.AddToScheme(scheme)) builder := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(objects...). @@ -813,7 +815,7 @@ func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { UID: "cr-uid", }, Spec: configv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: ""}, + GitTargetRef: meta.NamespacedObjectReference{Name: ""}, }, } @@ -884,13 +886,19 @@ func TestSaveRequestReconciler_LiteralValidation(t *testing.T) { // asks for more than its GitTarget allows fails at once, before anything is attached. func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { target := func(policy configv1alpha3.UnattributedWindowPolicy) *configv1alpha3.GitTarget { - return &configv1alpha3.GitTarget{ + gt := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{ KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, - Commit: &configv1alpha3.GitTargetCommitSpec{UnattributedWindows: policy}, }, } + if policy != "" { + gt.Spec.AllowedSaveRequests = []configv1alpha3.GitTargetSaveRequestGrant{{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromSame}, + UnattributedWindows: policy, + }} + } + return gt } provider := func(facts metav1.ConditionStatus) *configv1alpha3.KubeProvider { p := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} @@ -998,3 +1006,60 @@ func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaud }) } } + +// A SaveRequest may name a GitTarget in another namespace, which saves for it only when a grant in its +// spec.allowedSaveRequests admits the request's namespace. Omitted, the list admits the GitTarget's own +// namespace alone; a refused request fails at once, before anything is attached. +func TestSaveRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) { + platformTarget := func(grants []configv1alpha3.GitTargetSaveRequestGrant) *configv1alpha3.GitTarget { + return &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "shared", Namespace: "platform"}, + Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: grants}, + } + } + fromNames := func(names ...configv1alpha3.GrantNamespace) []configv1alpha3.GitTargetSaveRequestGrant { + return []configv1alpha3.GitTargetSaveRequestGrant{{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromNames, Names: names}, + }} + } + cases := []struct { + name string + grants []configv1alpha3.GitTargetSaveRequestGrant + admitted bool + }{ + {"the default admits only the GitTarget's own namespace", nil, false}, + {"a grant for another namespace", fromNames("team-b"), false}, + {"a grant for the request's namespace", fromNames("default"), true}, + {"every namespace", []configv1alpha3.GitTargetSaveRequestGrant{{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, + }}, true}, + {"an empty list admits nobody", []configv1alpha3.GitTargetSaveRequestGrant{}, false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + cr := newSaveRequest("save") + cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "shared", Namespace: "platform"} + c := newSaveRequestClient(t, nil, cr, platformTarget(tc.grants)) + f := &fakeFinalizer{} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + + reconcileSaveRequest(t, r, "save") + + if tc.admitted { + require.NotEmpty(t, f.calls, "the request must be attached") + assert.Equal(t, "platform", f.calls[0].GitTargetNamespace) + assert.Equal(t, "shared", f.calls[0].GitTargetName) + return + } + assert.Empty(t, f.calls, "a refused request is never attached") + var got configv1alpha3.SaveRequest + require.NoError(t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got)) + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionFalse, ready.Status) + assert.Equal(t, crReasonNotAllowed, ready.Reason) + assert.Contains(t, ready.Message, "spec.allowedSaveRequests") + assert.True(t, apimeta.IsStatusConditionTrue(got.Status.Conditions, ConditionTypeStalled)) + }) + } +} diff --git a/internal/controller/saverequest_finalize.go b/internal/controller/saverequest_finalize.go index 14bdd7a4..ef7c1dc4 100644 --- a/internal/controller/saverequest_finalize.go +++ b/internal/controller/saverequest_finalize.go @@ -29,6 +29,7 @@ const ( crReasonAuthorCaptureDisabled = "AuthorCaptureDisabled" crReasonGitTargetRefPruned = "GitTargetRefPruned" crReasonTakeOverNotAllowed = "TakeOverNotAllowed" + crReasonNotAllowed = "NotAllowed" ) // The bounded `outcome` values on gitopsreverser_save_requests_total. @@ -64,9 +65,6 @@ func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { return crOutcomeWindowMismatch case git.FinalizeAlreadyPresent: return crOutcomeAlreadyPresent - case git.FinalizeWindowOutsideNamespace: - // Only a RemoteSaveRequest is confined to a namespace, so a SaveRequest never ends this way. - return crOutcomeFailed default: return crOutcomeFailed } @@ -93,10 +91,11 @@ type saveRequestTarget struct { // needed to bound the label — the plan's rule that metric recording must not block terminal status // on another failing call. // -// The namespace is the REQUEST's, not a field: a SaveRequest names a same-namespace GitTarget, -// which is the same assumption ServiceSaveRequest makes when it builds the object key. +// The key is SaveRequest.GitTargetKey, the GitTarget the attach names, which is the key +// ServiceSaveRequest gets. func resolvedSaveRequestTarget(cr *configv1alpha3.SaveRequest) saveRequestTarget { - return saveRequestTarget{namespace: cr.Namespace, name: cr.Spec.GitTargetRef.Name} + key := cr.GitTargetKey() + return saveRequestTarget{namespace: key.Namespace, name: key.Name} } // recordSaveRequestOutcome increments the terminal-outcome counter once. @@ -285,8 +284,11 @@ func applyFinalizeResultToStatus( if finalizeErr != nil { reason := crReasonFinalizeFailed - if errors.Is(finalizeErr, errTakeOverNotAllowed) { + switch { + case errors.Is(finalizeErr, errTakeOverNotAllowed): reason = crReasonTakeOverNotAllowed + case errors.Is(finalizeErr, errSaveRequestNotAllowed): + reason = crReasonNotAllowed } failSaveRequest(cr, reason, finalizeErr.Error()) return @@ -311,10 +313,6 @@ func applyFinalizeResultToStatus( case git.FinalizeAlreadyPresent: // The change already matches the remote, so the commit was dropped. resolveNothingToCommit(cr, result, crReasonAlreadyPresent, alreadyPresentMessage) - case git.FinalizeWindowOutsideNamespace: - // Only a RemoteSaveRequest is confined to a namespace (AttachSaveRequest.SourceNamespace), so - // this outcome is a bug here, as an unknown one is. - failSaveRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) default: // An empty or unknown outcome with no error is a bug, not a benign // rejection; fail it so it is not silently hidden. diff --git a/internal/controller/saverequest_outcome_metric_test.go b/internal/controller/saverequest_outcome_metric_test.go index 2b68623d..ff6fc277 100644 --- a/internal/controller/saverequest_outcome_metric_test.go +++ b/internal/controller/saverequest_outcome_metric_test.go @@ -60,7 +60,7 @@ func TestSaveRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { for _, target := range []string{"team-a-config", "team-b-config"} { cr := newSaveRequest("save-" + target) - cr.Spec.GitTargetRef = meta.LocalObjectReference{Name: target} + cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target} c := newSaveRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, @@ -108,7 +108,7 @@ func TestSaveRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { require.NoError(t, err) cr := newSaveRequest("save-bogus") - cr.Spec.GitTargetRef = meta.LocalObjectReference{Name: "no-such-target"} + cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "no-such-target"} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(cr.Spec).resolveTimeout())) c := newSaveRequestClient(t, nil, cr) @@ -296,7 +296,7 @@ func TestSaveRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { pruned := &configv1alpha3.SaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "save-pruned", Namespace: "default", UID: types.UID("uid-save-pruned")}, Spec: configv1alpha3.SaveRequestSpec{ - GitTargetRef: meta.LocalObjectReference{Name: ""}, + GitTargetRef: meta.NamespacedObjectReference{Name: ""}, Message: "save: pruned", }, } @@ -361,10 +361,10 @@ func TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { }, } fresh := newSaveRequest("save-fresh") - fresh.Spec.GitTargetRef = meta.LocalObjectReference{Name: target.Name} + fresh.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target.Name} fresh.CreationTimestamp = metav1.Now() stale := newSaveRequest("save-stale") - stale.Spec.GitTargetRef = meta.LocalObjectReference{Name: target.Name} + stale.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target.Name} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. stale.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(stale.Spec).resolveTimeout())) c := newSaveRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index f435eb96..98e53268 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1314,7 +1314,6 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { l.w.Log.Info("Window identity change forces finalize before appending event", "windowAuthor", l.openWindow.Author, "eventAuthor", event.UserInfo.Username, - "windowSourceNamespace", l.openWindow.sourceNamespace, "operation", event.Operation, "resource", event.Identifier.String(), "eventTarget", event.GitTargetNamespace+"/"+event.GitTargetName) diff --git a/internal/git/open_window.go b/internal/git/open_window.go index d31acfc2..087f3e42 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -52,15 +52,6 @@ type openWindow struct { claim UnattributedClaim claimedBy UserInfo - // namespace is the source namespace of the window's writes while they all share one, "" for a - // cluster-scoped write; spansNamespaces is set once two writes' namespaces differ. - namespace string - spansNamespaces bool - // sourceNamespace confines the window to the writes of one source namespace once a request that - // saves only those attached (AttachSaveRequest.SourceNamespace): a write anywhere else cannot - // join it. Empty confines nothing. - sourceNamespace string - // timers decide when the window closes. They are the GitTarget's, snapshotted when the window // opened, until a SaveRequest attaches and replaces them with its own. timers windowTimers @@ -105,7 +96,6 @@ func newOpenWindow(e Event, writer eventContentWriter) *openWindow { GitTargetNamespace: e.GitTargetNamespace, pathToEvent: make(map[string]Event), writer: writer, - namespace: e.Identifier.Namespace, openedAt: time.Now(), } } @@ -117,22 +107,13 @@ func (w *openWindow) canAppend(e Event) bool { return e.Attribution == w.Attribution && e.UserInfo.Username == w.Author && e.GitTargetName == w.GitTarget && - e.GitTargetNamespace == w.GitTargetNamespace && - (w.sourceNamespace == "" || e.Identifier.Namespace == w.sourceNamespace) -} - -// holdsOnly reports whether every write in the window was made in namespace. -func (w *openWindow) holdsOnly(namespace string) bool { - return !w.spansNamespaces && w.namespace == namespace + e.GitTargetNamespace == w.GitTargetNamespace } // add records an event in the window. If the path was already present the // event replaces the previous one (last-write-wins inside a single window), which // it returns; otherwise pathOrder is extended. func (w *openWindow) add(e Event) (Event, bool) { - if e.Identifier.Namespace != w.namespace { - w.spansNamespaces = true - } key := windowPathKey(e, w.writer) replaced, exists := w.pathToEvent[key] if !exists { diff --git a/internal/git/save_request_attach.go b/internal/git/save_request_attach.go index 8988f10c..d6e0381a 100644 --- a/internal/git/save_request_attach.go +++ b/internal/git/save_request_attach.go @@ -53,10 +53,6 @@ const ( // produced no diff — the change already matches the remote, so no commit was // made (loop prevention). Resolved at finalize, never waiting on a push. FinalizeAlreadyPresent FinalizeOutcome = "AlreadyPresent" - // FinalizeWindowOutsideNamespace means the request's own window was open during its wait but held - // writes made outside the one namespace the request saves (AttachSaveRequest.SourceNamespace), so - // it was left untouched and the wait ran out without a window this request could attach to. - FinalizeWindowOutsideNamespace FinalizeOutcome = "WindowOutsideNamespace" ) // SaveRequestPhase is where an unresolved SaveRequest stands on the worker. It is what the @@ -159,11 +155,6 @@ type AttachSaveRequest struct { // the GitTarget's KubeProvider has never received an audit fact. Once it has, a write naming no // author means its fact went missing, and that window stays another's. Unattributed UnattributedClaim - // SourceNamespace confines the save to the writes made in one namespace of the source cluster: the - // request attaches only to a window whose writes are all there, and once it has, a write anywhere - // else, cluster-scoped ones included, opens a window of its own. A RemoteSaveRequest sets its own - // namespace; empty confines nothing. - SourceNamespace string } // UnattributedClaim is what a SaveRequest may claim of a commit window whose writes name no author. @@ -221,8 +212,6 @@ type pendingSaveRequest struct { // unattributed is what the request may claim of a window whose writes name no author // (AttachSaveRequest.Unattributed). unattributed UnattributedClaim - // sourceNamespace confines the save to one source namespace (AttachSaveRequest.SourceNamespace). - sourceNamespace string // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. @@ -242,9 +231,6 @@ type pendingSaveRequest struct { // instant check at expiry would miss the common case and report the refusal as "nothing was // pending" — intermittently, which is worse than never. sawForeignWindow bool - // sawWiderWindow is set when the request's own window was open during its wait but held writes - // outside its sourceNamespace. Sticky for the reason sawForeignWindow is. - sawWiderWindow bool } // expiryOutcome is the terminal outcome for a request whose attach timeout ran out without it @@ -260,9 +246,6 @@ func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { if p.sawForeignWindow { return FinalizeWindowMismatch } - if p.sawWiderWindow { - return FinalizeWindowOutsideNamespace - } return FinalizeNoOpenWindow } @@ -288,12 +271,6 @@ func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { // would let one author's request finalize another's window. Comparing both costs nothing and // keeps "bob never attaches to alice's window" true regardless of what the outcomes say. func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { - return p.identifiesWindow(w) && p.withinScope(w) -} - -// identifiesWindow is matchesWindow without the namespace confinement: the window is on the -// request's GitTarget and is its author's, or one it may claim. -func (p *pendingSaveRequest) identifiesWindow(w *openWindow) bool { if p == nil || w == nil { return false } @@ -303,22 +280,6 @@ func (p *pendingSaveRequest) identifiesWindow(w *openWindow) bool { return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.claims(w) } -// withinScope reports whether every write in the window is in the request's sourceNamespace, or the -// request confines nothing. -func (p *pendingSaveRequest) withinScope(w *openWindow) bool { - return p.sourceNamespace == "" || w.holdsOnly(p.sourceNamespace) -} - -// noteUnmatchedWindow records why the open window, which the request cannot attach to, is not its -// own: another author's or GitTarget's, or its own holding writes outside its namespace. -func (p *pendingSaveRequest) noteUnmatchedWindow(w *openWindow) { - if p.identifiesWindow(w) { - p.sawWiderWindow = true - return - } - p.sawForeignWindow = true -} - // claims reports whether the request matches a window on its GitTarget whose writes name no author // only because it may claim such a window: it names its submitter, and asked for more than its own // windows (AttachSaveRequest.Unattributed). diff --git a/internal/git/save_request_attach_loop.go b/internal/git/save_request_attach_loop.go index d7ee82b8..fdba8eaa 100644 --- a/internal/git/save_request_attach_loop.go +++ b/internal/git/save_request_attach_loop.go @@ -216,8 +216,7 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) commitEmpty: req.CommitEmpty, catchUpNow: req.CatchUpNow, - unattributed: req.Unattributed, - sourceNamespace: req.SourceNamespace, + unattributed: req.Unattributed, } l.pendingCRs[id] = pcr l.w.setSaveRequestPhase(id, PhaseWaitingForWindow) @@ -236,7 +235,7 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) // passed: with attachTimeout: 0s that is every request, so a foreign window open at // registration would go unseen and the request would resolve NoWindow — and, with // CommitEmpty, record an empty commit while another author's work is in flight. - pcr.noteUnmatchedWindow(l.openWindow) + pcr.sawForeignWindow = true return } if req.Attach == v1alpha3.AttachNext { @@ -296,20 +295,7 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { } now := time.Now() for _, pcr := range l.pendingCRs { - if !pcr.waiting(now) || pcr.matchesWindow(l.openWindow) { - continue - } - if pcr.identifiesWindow(l.openWindow) { - if !pcr.sawWiderWindow { - pcr.sawWiderWindow = true - l.w.Log.Info("SaveRequest is waiting on its window, which holds writes outside its namespace", - "request", pcr.id.Namespace+"/"+pcr.id.Name, - "sourceNamespace", pcr.sourceNamespace, - "requestTarget", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) - } - continue - } - if pcr.sawForeignWindow { + if !pcr.waiting(now) || pcr.sawForeignWindow || pcr.matchesWindow(l.openWindow) { continue } pcr.sawForeignWindow = true @@ -361,7 +347,6 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { } l.openWindow.timers = timers l.openWindow.lastWriteAt = now - l.openWindow.sourceNamespace = pcr.sourceNamespace if pcr.claims(l.openWindow) { l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} l.w.Log.Info("SaveRequest claimed a window whose writes name no author", diff --git a/internal/git/save_request_scope_test.go b/internal/git/save_request_scope_test.go deleted file mode 100644 index f632e1e5..00000000 --- a/internal/git/save_request_scope_test.go +++ /dev/null @@ -1,146 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package git - -import ( - "testing" - "time" - - gogit "github.com/go-git/go-git/v6" - "github.com/go-git/go-git/v6/plumbing" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -// configMapEventIn is configMapTargetEvent for a ConfigMap in namespace, "" for a write the window -// keys like a cluster-scoped one. -func configMapEventIn(name, username, namespace string) Event { - event := configMapTargetEvent(name, username, crTarget) - event.Identifier.Namespace = namespace - event.Object.SetNamespace(namespace) - return event -} - -// scopedAttachReq is a request that saves only the writes made in namespace. -func scopedAttachReq(author, namespace string) *AttachSaveRequest { - req := attachReq(author, 60*time.Second) - req.SourceNamespace = namespace - return req -} - -// A request confined to a namespace attaches only to a window whose writes are all in it. The -// window's author and GitTarget still decide first: a window of someone else's is foreign, not wider. -func TestMatchesWindow_SourceNamespaceConfinesTheSave(t *testing.T) { - window := func(events ...Event) *openWindow { - w := newOpenWindow(events[0], &contentWriter{}) - for _, e := range events { - w.add(e) - } - return w - } - shop := configMapEventIn("a", "alice", "shop") - billing := configMapEventIn("b", "alice", "billing") - cluster := configMapEventIn("c", "alice", "") - request := func(namespace string) *pendingSaveRequest { - return &pendingSaveRequest{ - author: "alice", gitTargetName: crTarget, gitTargetNamespace: "default", sourceNamespace: namespace, - } - } - - assert.True(t, request("shop").matchesWindow(window(shop)), "every write is in its namespace") - assert.False(t, request("shop").matchesWindow(window(billing)), "every write is elsewhere") - assert.False(t, request("shop").matchesWindow(window(shop, billing)), "one write is elsewhere") - assert.False(t, request("shop").matchesWindow(window(cluster)), "a cluster-scoped write is in no namespace") - assert.True(t, request("").matchesWindow(window(shop, billing, cluster)), "no confinement") - - wider := request("shop") - wider.noteUnmatchedWindow(window(shop, billing)) - assert.Equal(t, FinalizeWindowOutsideNamespace, wider.expiryOutcome()) - - bob := configMapEventIn("d", "bob", "shop") - foreign := request("shop") - foreign.noteUnmatchedWindow(window(bob)) - assert.Equal(t, FinalizeWindowMismatch, foreign.expiryOutcome(), "bob's window is foreign") -} - -// Once a confined request attached, a write by the same author in another namespace does not join -// its window: the window closes with the request's message and only its namespace's write, and the -// other write opens a window of its own. -func TestAttach_AConfinedSaveKeepsOtherNamespacesOutOfItsCommit(t *testing.T) { - worker, _, _ := setupCommitPushSplitWorker(t) - createPlainGitTarget(t, worker, crTarget, crTarget) - - loop := newBranchWorkerEventLoop(worker, time.Hour) - defer loop.stopTimers() - - req := scopedAttachReq("alice", "shop") - req.Message = "save: the shop" - serviceAttach(loop, req) - - loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{configMapEventIn("cart", "alice", "shop")}}}) - loop.endWake(0) - require.NotNil(t, loop.openWindow) - require.NotNil(t, loop.openWindow.pendingCR, "the request attached to the shop window") - - loop.handleQueueItem(WorkItem{Request: &WriteRequest{ - Events: []Event{configMapEventIn("invoice", "alice", "billing")}, - }}) - loop.endWake(0) - - res, ok := outcome(t, worker) - require.True(t, ok, "the billing write closed the shop window") - require.NoError(t, res.Err) - require.Equal(t, FinalizeCommitted, res.Outcome) - require.NotNil(t, loop.openWindow, "the billing write opened a window of its own") - assert.Nil(t, loop.openWindow.pendingCR, "and the request is not in it") - assert.Equal(t, "billing", loop.openWindow.namespace) - - repo, err := gogit.PlainOpen(worker.repoPath()) - require.NoError(t, err) - commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) - require.NoError(t, err) - assert.Equal(t, "save: the shop", commit.Message) - stats, err := commit.Stats() - require.NoError(t, err) - changed := make([]string, 0, len(stats)) - for _, stat := range stats { - changed = append(changed, stat.Name) - } - assert.Contains(t, changed, crTarget+"/shop/configmaps/cart.yaml") - assert.NotContains( - t, - changed, - crTarget+"/billing/configmaps/invoice.yaml", - "billing's write is not the shop's save", - ) -} - -// A confined request leaves its author's window alone when that window already holds writes from -// elsewhere, and says so when its wait runs out. -func TestAttach_AConfinedSaveLeavesAWiderWindowAlone(t *testing.T) { - worker, _, _ := setupCommitPushSplitWorker(t) - createPlainGitTarget(t, worker, crTarget, crTarget) - - loop := newBranchWorkerEventLoop(worker, time.Hour) - loop.lastPushAt = time.Now() - defer loop.stopTimers() - - loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{ - configMapEventIn("cart", "alice", "shop"), - configMapEventIn("invoice", "alice", "billing"), - }}}) - require.NotNil(t, loop.openWindow) - - serviceAttach(loop, scopedAttachReq("alice", "shop")) - require.Nil(t, loop.openWindow.pendingCR, "a window holding billing's write is not the shop's to save") - - forceDue(loop) - loop.endWake(0) - - res, ok := outcome(t, worker) - require.True(t, ok) - require.NoError(t, res.Err) - assert.Equal(t, FinalizeWindowOutsideNamespace, res.Outcome) - require.NotNil(t, loop.openWindow, "the window is left open") - assert.Len(t, loop.openWindow.orderedEvents(), 2) -} diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go index 34712651..474aa8f2 100644 --- a/internal/watch/remote_save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -38,10 +38,9 @@ import ( // the request's status on the source cluster. The provider's switch makes the requests reachable; // the GitTarget's grant is its owner's consent to being steered by them. // -// A request saves only the edits made in its own namespace: the worker attaches it only to a window -// whose writes are all there, and keeps every other write out of that window once it has -// (git.AttachSaveRequest.SourceNamespace). A window groups by author and GitTarget, not by -// namespace, so without that a save in one namespace would commit edits made in another. +// A commit window groups by author and GitTarget, not by namespace, so an admitted request saves its +// author's whole window on that GitTarget, edits outside the request's namespace included. The +// GitTarget's grant is what authorizes that. // // The request's author is found as the edits' authors are, from the source cluster's audit facts, so // a save attaches only to its creator's windows by the ordinary author rule. On a cluster that sends @@ -83,7 +82,6 @@ const ( saveReasonCommitted = "Committed" saveReasonNoWindow = "NoWindow" saveReasonWindowMismatch = "WindowMismatch" - saveReasonOutsideNamespace = "WindowOutsideNamespace" saveReasonAlreadyPresent = "AlreadyPresent" saveReasonFailed = "Failed" saveReasonNoGitTarget = "NoGitTarget" @@ -506,7 +504,6 @@ func remoteSaveRequestAttach( AttachTimeout: defaultSaveAttachTimeout, MaxDuration: defaultSaveMaxDuration, CatchUpNow: request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush, - SourceNamespace: request.Namespace, } if window := request.Spec.Window; window != nil { if window.AttachTimeout != nil { @@ -580,9 +577,6 @@ func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestGitTarget, result case git.FinalizeWindowMismatch: target.Reason = saveReasonWindowMismatch target.Message = "only another author's commit window was open" - case git.FinalizeWindowOutsideNamespace: - target.Reason = saveReasonOutsideNamespace - target.Message = "the open commit window held edits made outside this namespace, so it was left alone" case git.FinalizeAlreadyPresent: target.Reason = saveReasonAlreadyPresent target.Message = "the edits were in Git already" diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go index 9887ea92..38e40088 100644 --- a/internal/watch/remote_save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -5,6 +5,7 @@ package watch import ( "context" "errors" + "fmt" "sync" "testing" "time" @@ -215,7 +216,6 @@ func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *test assert.Equal(t, 3*time.Second, attach.AttachTimeout) assert.True(t, attach.CatchUpNow) assert.Equal(t, "uid-save@team-a/app", attach.UID, "one identity per GitTarget") - assert.Equal(t, "shop", attach.SourceNamespace, "it saves only the edits made in its namespace") assert.Equal(t, "alice@example.com", status.Author) require.Len(t, status.GitTargets, 1) @@ -377,20 +377,6 @@ func TestRemoteSaveRequest_AWatchOfEveryNamespaceMirrorsIt(t *testing.T) { assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) } -// A window holding edits from elsewhere is left alone, and the save says why. -func TestRemoteSaveRequest_AWindowOutsideTheNamespaceIsReported(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeWindowOutsideNamespace}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - server, resource := saveServer(t, request, shopSetup("", nil), finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) - assert.Equal(t, saveReasonOutsideNamespace, condition(status, "Ready").Reason) - assert.Contains(t, condition(status, "Ready").Message, "outside this namespace") - assert.Nil(t, condition(status, "Applied"), "nothing was committed to apply") -} - // A save that failed in a GitTarget fails, and says where. func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} @@ -551,3 +537,52 @@ func TestRemoteSaveRequest_AFinishedRequestIsDeletedAfterTheTTL(t *testing.T) { }) } } + +// Each worker outcome is recorded on its GitTarget's status with the reason the API documents. +func TestRemoteSaveRequest_RecordsEachOutcome(t *testing.T) { + for result, want := range map[*git.FinalizeResult]string{ + {Outcome: git.FinalizeCommitted, Commit: "c0ffee"}: saveReasonCommitted, + {Outcome: git.FinalizeNoOpenWindow}: saveReasonNoWindow, + {Outcome: git.FinalizeWindowMismatch}: saveReasonWindowMismatch, + {Outcome: git.FinalizeAlreadyPresent}: saveReasonAlreadyPresent, + {Outcome: "Unknown"}: saveReasonFailed, + {Err: errors.New("boom")}: saveReasonFailed, + {Err: git.ErrSaveRequestWithdrawn}: saveReasonFailed, + } { + var target sourcev1alpha1.RemoteSaveRequestGitTarget + recordSaveOutcome(&target, *result) + assert.Equal(t, want, target.Reason, "%+v", *result) + assert.NotEmpty(t, target.Message) + } +} + +// Applied is False only for a commit a branch lost, Unknown when an apply was lost track of, and True +// otherwise. +func TestRemoteSaveRequest_AppliedSummarizesTheTargets(t *testing.T) { + for name, tt := range map[string]struct { + applied []string + status metav1.ConditionStatus + reason string + }{ + "all applied": {[]string{git.SaveApplyApplied, git.SaveApplyNothingToApply}, metav1.ConditionTrue, saveReasonApplied}, + "one lost": {[]string{git.SaveApplyApplied, "Lost"}, metav1.ConditionUnknown, "Lost"}, + "one commit dropped": {[]string{"Lost", git.SaveApplyCommitDropped}, metav1.ConditionFalse, git.SaveApplyCommitDropped}, + } { + t.Run(name, func(t *testing.T) { + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) + status := &sourcev1alpha1.RemoteSaveRequestStatus{} + for i, applied := range tt.applied { + status.GitTargets = append(status.GitTargets, sourcev1alpha1.RemoteSaveRequestGitTarget{ + Namespace: "team-a", Name: fmt.Sprintf("app-%d", i), Reason: saveReasonCommitted, Applied: applied, + }) + } + setSaveApplied(request, status) + got := condition(*status, "Applied") + require.NotNil(t, got) + assert.Equal(t, tt.status, got.Status) + assert.Equal(t, tt.reason, got.Reason) + assert.True(t, saveApplySettled(tt.applied[0])) + }) + } + assert.False(t, saveApplySettled("Applying")) +} diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 985008ac..a7c71be6 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -808,15 +808,22 @@ spec: // is made as soon as the first is pushed, so the two may share one apply. // // No write in the workspace names its author, since kcp sends no audit facts, and the request's - // submitter is captured at admission in this cluster. The GitTarget allows a takeover, each - // request asks for one, and the workspace's KubeProvider has never received a fact, so each - // request takes over the window of its edit: its submitter authors the commit, and the message + // submitter is captured at admission in this cluster. The GitTarget's grant for its own namespace + // allows a takeover, each request asks for one, and the workspace's KubeProvider has never + // received a fact, so each request takes over the window of its edit: its submitter authors the commit, and the message // ends with a trailer that says so. Every event from the workspace waits out the attribution // grace first, so the requests wait longer than that for their window. It("takes over the window of an edit in the workspace, and reports the apply on each of two saves", func() { verifyResourceCondition("kubeprovider", ws, testNs, "AuditFactsReceived", "Unknown", "", "") - _, err := kubectlRunInNamespace(testNs, "patch", "gittarget", p.targetName, "--type=merge", "-p", - `{"spec":{"commit":{"unattributedWindows":"TakeOver"}}}`) + _, err := kubectlRunInNamespace( + testNs, + "patch", + "gittarget", + p.targetName, + "--type=merge", + "-p", + `{"spec":{"allowedSaveRequests":[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"TakeOver"}]}}`, + ) Expect(err).NotTo(HaveOccurred()) names := []string{"sc-flux-save-1", "sc-flux-save-2"} commits := map[string]string{} From e0a5b8befd19880191ee3354012e0b759ae11124 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 20:50:42 +0000 Subject: [PATCH 20/33] feat(remotesaverequest): name the GitTarget, refuse a second save in progress, and admit by GitTarget UID A RemoteSaveRequest names its GitTarget in a required spec.gitTargetRef (namespace and name on the cluster GitOps Reverser runs in) and saves that one. Reaching every admitting GitTarget that mirrors its namespace made an unchanged request do more whenever a GitTarget was added, and discovery that had not caught up after a start reported NoGitTarget. The GitTarget must mirror the cluster the request came from and admit it in spec.allowedRemoteSaveRequests; status is flat (commit, appliedRevision, Ready, Applied). A request served again after a restart keeps the author found when it was first served, since its status writes moved its resourceVersion, and keeps an outcome already decided, following only its apply. A second save of one author on one GitTarget, made while the first still waits for a window or collects one, is refused at once with SaveInProgress: a window carries one message, and the second would lose its own without a word. A save for the Next window is not refused by a collecting one, which it closes first. save_requests_total gains outcome save_in_progress. A SaveRequest records the UID of the GitTarget that admitted it in status.gitTargetUID. Nothing is attached before one did, so a GitTarget created after the request under the same name must admit it; only a refusal by the GitTarget that admitted it is set aside, for a request an attach may already have reached. docs/design/concurrent-author-windows.md proposes a window per author, open side by side, for the split a second person's edit causes. Co-Authored-By: Claude Opus 5.5 (1M context) --- .../v1alpha1/remotesaverequest_types.go | 129 +++--- api/source/v1alpha1/zz_generated.deepcopy.go | 36 +- api/v1alpha3/saverequest_types.go | 6 + .../bases/configbutler.ai_saverequests.yaml | 6 + ...ce.configbutler.ai_remotesaverequests.yaml | 158 ++++--- docs/INDEX.md | 1 + docs/configuration.md | 76 ++-- docs/design/concurrent-author-windows.md | 145 ++++++ docs/future/source-side-save-plan.md | 81 ++-- docs/interpreting-metrics.md | 2 +- docs/rbac.md | 2 +- hack/crdfields/fields.yaml | 14 +- internal/controller/saverequest_controller.go | 111 ++++- .../saverequest_controller_unit_test.go | 92 ++++ internal/controller/saverequest_finalize.go | 8 + internal/git/save_request_attach.go | 12 + internal/git/save_request_attach_loop.go | 32 ++ internal/git/save_request_attach_test.go | 105 ++++- internal/watch/remote_save_requests.go | 427 ++++++++---------- internal/watch/remote_save_requests_test.go | 318 +++++++------ test/e2e/source_cluster_e2e_test.go | 30 +- 21 files changed, 1087 insertions(+), 704 deletions(-) create mode 100644 docs/design/concurrent-author-windows.md diff --git a/api/source/v1alpha1/remotesaverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go index ed8f6558..159af3cd 100644 --- a/api/source/v1alpha1/remotesaverequest_types.go +++ b/api/source/v1alpha1/remotesaverequest_types.go @@ -8,21 +8,29 @@ import ( // A RemoteSaveRequest is the source-cluster twin of a configbutler.ai SaveRequest, for the people who edit // a source cluster and cannot reach the cluster GitOps Reverser runs in. It is its own kind, in its own -// group, rather than a SaveRequest with another schema: it names no GitTarget, because its creator -// cannot see them, its author comes from the source cluster's audit events rather than from admission, -// and it is installed alone. docs/future/source-side-save-plan.md has the reasoning. +// group, rather than a SaveRequest with another schema: its GitTarget is on another cluster than the +// request, its author comes from the source cluster's audit events rather than from admission, and it +// is installed alone. docs/future/source-side-save-plan.md has the reasoning. // // The spec is immutable, compared as SaveRequestSpec compares it: durations as durations, so a typed // client's re-spelling of "1m" as "1m0s" is not a change. // RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It // is immutable. -// +kubebuilder:validation:XValidation:rule="has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="RemoteSaveRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="RemoteSaveRequest spec is immutable after creation" type RemoteSaveRequestSpec struct { - // Message is the commit message this request asks for. Omitted, each + // GitTargetRef names the GitTarget whose commit window this request + // attaches to, on the cluster GitOps Reverser runs in. Its namespace is a + // namespace of that cluster, not of this one. The GitTarget must mirror + // this cluster, and admit this request in spec.allowedRemoteSaveRequests, + // or the request fails with reason "NotAllowed". + // +required + GitTargetRef GitTargetReference `json:"gitTargetRef"` + + // Message is the commit message this request asks for. Omitted, the // GitTarget's spec.commit.message.liveTemplate words the commit. // - // The text is never evaluated as a template. If a GitTarget sets + // The text is never evaluated as a template. If the GitTarget sets // spec.commit.message.requestTemplate, that template wraps this text; // otherwise the text is committed as written. // @@ -46,18 +54,34 @@ type RemoteSaveRequestSpec struct { // +kubebuilder:default={attachTimeout: "5s", maxDuration: "2s"} Window *RemoteSaveRequestWindow `json:"window,omitempty"` - // CatchUp decides when the applier of each GitTarget is asked to apply - // the branch after this request pushes a commit. "AsDefinedOnClaim" - // leaves it to the applier claim's spec.catchUpAfter. "AfterPush" asks - // right after the push, and status.targets reports how it went. The - // cluster already holds the saved objects. Defaults to - // "AsDefinedOnClaim". + // CatchUp decides when the GitTarget's applier is asked to apply the + // branch after this request pushes a commit. "AsDefinedOnClaim" leaves it + // to the applier claim's spec.catchUpAfter. "AfterPush" asks right after + // the push, and the Applied condition reports how it went. The cluster + // already holds the saved objects. Defaults to "AsDefinedOnClaim". // +optional // +kubebuilder:validation:Enum=AsDefinedOnClaim;AfterPush // +kubebuilder:default=AsDefinedOnClaim CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } +// GitTargetReference names a GitTarget on the cluster GitOps Reverser runs +// in. +type GitTargetReference struct { + // Namespace is the GitTarget's namespace, on the cluster GitOps Reverser + // runs in. + // +required + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=63 + Namespace string `json:"namespace"` + + // Name is the GitTarget's name. + // +required + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` +} + // CatchUpPolicy is when a RemoteSaveRequest's commit is applied by a // GitTarget's applier. type CatchUpPolicy string @@ -118,55 +142,26 @@ type RemoteSaveRequestStatus struct { // +optional Author string `json:"author,omitempty"` - // GitTargets is the save in each GitTarget that admitted this request. - // +optional - // +listType=map - // +listMapKey=namespace - // +listMapKey=name - // +kubebuilder:validation:MaxItems=64 - GitTargets []RemoteSaveRequestGitTarget `json:"gitTargets,omitempty"` - // Conditions are the latest observations of this RemoteSaveRequest's state. - // Ready is True once the request ended without error in every GitTarget, - // with reason "Committed" when one of them committed, and otherwise the - // reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - // "NoGitTarget" when no GitTarget mirrors this namespace, or - // "AuthorUnresolved" when this cluster sends audit events and none named - // the request's creator. Ready=False with Stalled=True is a - // failure, with reason "NotAllowed" when GitTargets mirror this namespace - // and none admits the request. // - // Applied, with spec.catchUp "AfterPush", is True once every GitTarget's - // applier applied a revision that holds its commit, or needed none, and - // False only when a branch lost the commit. It never changes Ready. + // Ready is True once the request ended without error: reason "Committed", + // or, with nothing committed, "NoWindow", "AlreadyPresent", + // "WindowMismatch", or "AuthorUnresolved" when this cluster sends audit + // events and none named the request's creator. Ready=False with + // Stalled=True is a failure: "NotAllowed" when the GitTarget does not + // admit the request or mirrors another cluster, "GitTargetNotFound", + // "SaveInProgress" when an earlier save of the same author on the + // GitTarget was still in progress, or "Failed". + // + // Applied, with spec.catchUp "AfterPush", is True once the applier applied + // a revision that holds the commit, or needed none, and False only when + // the branch lost the commit. It never changes Ready. // +optional // +listType=map // +listMapKey=type // +patchMergeKey=type // +patchStrategy=merge Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` -} - -// RemoteSaveRequestGitTarget is the save in one GitTarget. -type RemoteSaveRequestGitTarget struct { - // Namespace is the GitTarget's namespace, on the cluster GitOps Reverser - // runs in. - // +required - Namespace string `json:"namespace"` - - // Name is the GitTarget's name. - // +required - Name string `json:"name"` - - // Reason is how the save ended in this GitTarget, as the Ready reasons - // above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or - // "Failed". Empty while it runs. - // +optional - Reason string `json:"reason,omitempty"` - - // Message says more about the reason. - // +optional - Message string `json:"message,omitempty"` // Branch is the branch the save was committed on. // +optional @@ -176,13 +171,8 @@ type RemoteSaveRequestGitTarget struct { // +optional Commit string `json:"commit,omitempty"` - // Applied is how the apply the save asked for went, with spec.catchUp - // "AfterPush": "Applying", "Retrying", "Applied", "NothingToApply", - // "NoApplierClaim", "ApplierGone", "CommitDropped" or "Lost". - // +optional - Applied string `json:"applied,omitempty"` - - // AppliedRevision is the revision the applier applied. + // AppliedRevision is the revision the applier applied, with spec.catchUp + // "AfterPush". It holds the commit, and may be a later one. // +optional AppliedRevision string `json:"appliedRevision,omitempty"` } @@ -191,19 +181,20 @@ type RemoteSaveRequestGitTarget struct { // +kubebuilder:subresource:status // +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` // +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` +// +kubebuilder:printcolumn:name="GitTarget",type=string,JSONPath=`.spec.gitTargetRef.name` // +kubebuilder:printcolumn:name="Author",type=string,JSONPath=`.status.author` +// +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.commit`,priority=1 // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// RemoteSaveRequest asks GitOps Reverser to save its creator's edits: the -// commit window of each GitTarget that mirrors this namespace and admits the -// request, in spec.allowedRemoteSaveRequests, gets its message and closes on -// its timers. The window holds the creator's edits anywhere that GitTarget -// mirrors, not only in this namespace. Create it on the cluster the edits -// were made on, as JSON. Its creator is found from this cluster's audit -// events, as the edits' authors are, so it saves only its creator's edits; on -// a cluster with no audit it names nobody, and saves edits that name nobody. -// Its spec is immutable, and status reports each GitTarget's commit. +// RemoteSaveRequest asks GitOps Reverser to save its creator's edits in the +// GitTarget spec.gitTargetRef names: the creator's commit window there gets +// its message and closes on its timers. The window holds the creator's edits +// anywhere that GitTarget mirrors. Create it on the cluster the edits were +// made on, as JSON. Its creator is found from this cluster's audit events, as +// the edits' authors are, so it saves only its creator's edits; on a cluster +// with no audit it names nobody, and saves edits that name nobody. Its spec is +// immutable, and status reports the commit. type RemoteSaveRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/api/source/v1alpha1/zz_generated.deepcopy.go b/api/source/v1alpha1/zz_generated.deepcopy.go index 593256b0..6e0cfc5f 100644 --- a/api/source/v1alpha1/zz_generated.deepcopy.go +++ b/api/source/v1alpha1/zz_generated.deepcopy.go @@ -11,6 +11,21 @@ import ( runtime "k8s.io/apimachinery/pkg/runtime" ) +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitTargetReference) DeepCopyInto(out *GitTargetReference) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetReference. +func (in *GitTargetReference) DeepCopy() *GitTargetReference { + if in == nil { + return nil + } + out := new(GitTargetReference) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequest) DeepCopyInto(out *RemoteSaveRequest) { *out = *in @@ -38,21 +53,6 @@ func (in *RemoteSaveRequest) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestGitTarget) DeepCopyInto(out *RemoteSaveRequestGitTarget) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestGitTarget. -func (in *RemoteSaveRequestGitTarget) DeepCopy() *RemoteSaveRequestGitTarget { - if in == nil { - return nil - } - out := new(RemoteSaveRequestGitTarget) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestList) DeepCopyInto(out *RemoteSaveRequestList) { *out = *in @@ -88,6 +88,7 @@ func (in *RemoteSaveRequestList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestSpec) DeepCopyInto(out *RemoteSaveRequestSpec) { *out = *in + out.GitTargetRef = in.GitTargetRef if in.Window != nil { in, out := &in.Window, &out.Window *out = new(RemoteSaveRequestWindow) @@ -108,11 +109,6 @@ func (in *RemoteSaveRequestSpec) DeepCopy() *RemoteSaveRequestSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RemoteSaveRequestStatus) DeepCopyInto(out *RemoteSaveRequestStatus) { *out = *in - if in.GitTargets != nil { - in, out := &in.GitTargets, &out.GitTargets - *out = make([]RemoteSaveRequestGitTarget, len(*in)) - copy(*out, *in) - } if in.Conditions != nil { in, out := &in.Conditions, &out.Conditions *out = make([]v1.Condition, len(*in)) diff --git a/api/v1alpha3/saverequest_types.go b/api/v1alpha3/saverequest_types.go index cc7c53a5..9dfe91d6 100644 --- a/api/v1alpha3/saverequest_types.go +++ b/api/v1alpha3/saverequest_types.go @@ -262,6 +262,12 @@ type SaveRequestStatus struct { // +patchMergeKey=type Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` + // GitTargetUID is the UID of the GitTarget whose spec.allowedSaveRequests + // admitted this request. A GitTarget with another UID, such as one created + // after the request under the same name, must admit it again. + // +optional + GitTargetUID string `json:"gitTargetUID,omitempty"` + // Branch is the branch the commit is made on. Set once the request ends. // +optional Branch string `json:"branch,omitempty"` diff --git a/config/crd/bases/configbutler.ai_saverequests.yaml b/config/crd/bases/configbutler.ai_saverequests.yaml index 021177e0..f4455d38 100644 --- a/config/crd/bases/configbutler.ai_saverequests.yaml +++ b/config/crd/bases/configbutler.ai_saverequests.yaml @@ -357,6 +357,12 @@ spec: x-kubernetes-list-map-keys: - type x-kubernetes-list-type: map + gitTargetUID: + description: |- + GitTargetUID is the UID of the GitTarget whose spec.allowedSaveRequests + admitted this request. A GitTarget with another UID, such as one created + after the request under the same name, must admit it again. + type: string observedGeneration: description: ObservedGeneration is the latest generation observed by the controller. diff --git a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml index 387fc82e..be2bdf6e 100644 --- a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml +++ b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml @@ -21,9 +21,16 @@ spec: - jsonPath: .status.conditions[?(@.type=="Ready")].reason name: Reason type: string + - jsonPath: .spec.gitTargetRef.name + name: GitTarget + type: string - jsonPath: .status.author name: Author type: string + - jsonPath: .status.commit + name: Commit + priority: 1 + type: string - jsonPath: .status.conditions[?(@.type=="Applied")].status name: Applied priority: 1 @@ -35,15 +42,14 @@ spec: schema: openAPIV3Schema: description: |- - RemoteSaveRequest asks GitOps Reverser to save its creator's edits: the - commit window of each GitTarget that mirrors this namespace and admits the - request, in spec.allowedRemoteSaveRequests, gets its message and closes on - its timers. The window holds the creator's edits anywhere that GitTarget - mirrors, not only in this namespace. Create it on the cluster the edits - were made on, as JSON. Its creator is found from this cluster's audit - events, as the edits' authors are, so it saves only its creator's edits; on - a cluster with no audit it names nobody, and saves edits that name nobody. - Its spec is immutable, and status reports each GitTarget's commit. + RemoteSaveRequest asks GitOps Reverser to save its creator's edits in the + GitTarget spec.gitTargetRef names: the creator's commit window there gets + its message and closes on its timers. The window holds the creator's edits + anywhere that GitTarget mirrors. Create it on the cluster the edits were + made on, as JSON. Its creator is found from this cluster's audit events, as + the edits' authors are, so it saves only its creator's edits; on a cluster + with no audit it names nobody, and saves edits that name nobody. Its spec is + immutable, and status reports the commit. properties: apiVersion: description: |- @@ -70,22 +76,45 @@ spec: catchUp: default: AsDefinedOnClaim description: |- - CatchUp decides when the applier of each GitTarget is asked to apply - the branch after this request pushes a commit. "AsDefinedOnClaim" - leaves it to the applier claim's spec.catchUpAfter. "AfterPush" asks - right after the push, and status.targets reports how it went. The - cluster already holds the saved objects. Defaults to - "AsDefinedOnClaim". + CatchUp decides when the GitTarget's applier is asked to apply the + branch after this request pushes a commit. "AsDefinedOnClaim" leaves it + to the applier claim's spec.catchUpAfter. "AfterPush" asks right after + the push, and the Applied condition reports how it went. The cluster + already holds the saved objects. Defaults to "AsDefinedOnClaim". enum: - AsDefinedOnClaim - AfterPush type: string + gitTargetRef: + description: |- + GitTargetRef names the GitTarget whose commit window this request + attaches to, on the cluster GitOps Reverser runs in. Its namespace is a + namespace of that cluster, not of this one. The GitTarget must mirror + this cluster, and admit this request in spec.allowedRemoteSaveRequests, + or the request fails with reason "NotAllowed". + properties: + name: + description: Name is the GitTarget's name. + maxLength: 253 + minLength: 1 + type: string + namespace: + description: |- + Namespace is the GitTarget's namespace, on the cluster GitOps Reverser + runs in. + maxLength: 63 + minLength: 1 + type: string + required: + - name + - namespace + type: object message: description: |- - Message is the commit message this request asks for. Omitted, each + Message is the commit message this request asks for. Omitted, the GitTarget's spec.commit.message.liveTemplate words the commit. - The text is never evaluated as a template. If a GitTarget sets + The text is never evaluated as a template. If the GitTarget sets spec.commit.message.requestTemplate, that template wraps this text; otherwise the text is committed as written. @@ -150,37 +179,53 @@ spec: || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) - message: spec.window.idleTimeout must not exceed maxDuration rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' + required: + - gitTargetRef type: object x-kubernetes-validations: - message: RemoteSaveRequest spec is immutable after creation - rule: has(self.message) == has(oldSelf.message) && (!has(self.message) - || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) - && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) - || self.catchUp == oldSelf.catchUp) + rule: self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) + == has(oldSelf.message) && (!has(self.message) || self.message == + oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) + == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == + oldSelf.catchUp) status: description: status defines the observed state of RemoteSaveRequest. properties: + appliedRevision: + description: |- + AppliedRevision is the revision the applier applied, with spec.catchUp + "AfterPush". It holds the commit, and may be a later one. + type: string author: description: |- Author is who created this request, as this cluster's audit events name them. Empty when they name nobody: then the request saves only edits that name nobody either. type: string + branch: + description: Branch is the branch the save was committed on. + type: string + commit: + description: Commit is the commit the save made, once it is on the + remote. + type: string conditions: description: |- Conditions are the latest observations of this RemoteSaveRequest's state. - Ready is True once the request ended without error in every GitTarget, - with reason "Committed" when one of them committed, and otherwise the - reason of the first: "NoWindow", "AlreadyPresent", "WindowMismatch", - "NoGitTarget" when no GitTarget mirrors this namespace, or - "AuthorUnresolved" when this cluster sends audit events and none named - the request's creator. Ready=False with Stalled=True is a - failure, with reason "NotAllowed" when GitTargets mirror this namespace - and none admits the request. - Applied, with spec.catchUp "AfterPush", is True once every GitTarget's - applier applied a revision that holds its commit, or needed none, and - False only when a branch lost the commit. It never changes Ready. + Ready is True once the request ended without error: reason "Committed", + or, with nothing committed, "NoWindow", "AlreadyPresent", + "WindowMismatch", or "AuthorUnresolved" when this cluster sends audit + events and none named the request's creator. Ready=False with + Stalled=True is a failure: "NotAllowed" when the GitTarget does not + admit the request or mirrors another cluster, "GitTargetNotFound", + "SaveInProgress" when an earlier save of the same author on the + GitTarget was still in progress, or "Failed". + + Applied, with spec.catchUp "AfterPush", is True once the applier applied + a revision that holds the commit, or needed none, and False only when + the branch lost the commit. It never changes Ready. items: description: Condition contains details for one aspect of the current state of this API Resource. @@ -239,55 +284,6 @@ spec: x-kubernetes-list-map-keys: - type x-kubernetes-list-type: map - gitTargets: - description: GitTargets is the save in each GitTarget that admitted - this request. - items: - description: RemoteSaveRequestGitTarget is the save in one GitTarget. - properties: - applied: - description: |- - Applied is how the apply the save asked for went, with spec.catchUp - "AfterPush": "Applying", "Retrying", "Applied", "NothingToApply", - "NoApplierClaim", "ApplierGone", "CommitDropped" or "Lost". - type: string - appliedRevision: - description: AppliedRevision is the revision the applier applied. - type: string - branch: - description: Branch is the branch the save was committed on. - type: string - commit: - description: Commit is the commit the save made, once it is - on the remote. - type: string - message: - description: Message says more about the reason. - type: string - name: - description: Name is the GitTarget's name. - type: string - namespace: - description: |- - Namespace is the GitTarget's namespace, on the cluster GitOps Reverser - runs in. - type: string - reason: - description: |- - Reason is how the save ended in this GitTarget, as the Ready reasons - above: "Committed", "NoWindow", "AlreadyPresent", "WindowMismatch", or - "Failed". Empty while it runs. - type: string - required: - - name - - namespace - type: object - maxItems: 64 - type: array - x-kubernetes-list-map-keys: - - namespace - - name - x-kubernetes-list-type: map observedGeneration: description: ObservedGeneration is the latest generation observed by GitOps Reverser. diff --git a/docs/INDEX.md b/docs/INDEX.md index 61fabdfd..7ee538c3 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -153,6 +153,7 @@ so you never have to read a proposal to find out it already shipped. | [`watches-labels-simplification.md`](design/watches-labels-simplification.md) | **step 2, implemented.** After the collection rename and operation-filter removal, native per-rule `objectSelector` (#146). Kubernetes is authoritative: `Always` compares selected identities with all managed Git identities in the type/namespace scope, without comparing Git labels. Full support includes supported Kustomize layouts from the first release, plus label-exit attribution (now selector-agnostic, see the attribution spec) and ordered producer replacement. | | [`label-selection-follow-ups.md`](design/label-selection-follow-ups.md) | **design, open.** What the label-selection review rounds left open, with the evidence for each: real deletions in a selected collection wait the full 3s attribution grace (the watch cannot tell them from a label exit, measured), the resync fence that stops overlapping collections from coalescing (kept on purpose), same-second overlap ties, and one store rebuild per object coming back. Also lists what was fixed on the branch | | [`attribution-removal-wait-options.md`](design/attribution-removal-wait-options.md) | a removal now waits for evidence about the DELETION rather than accepting the object's last write, which stopped it naming whoever last edited the object as the author of a deletion they did not perform. Enumerates the eight situations a resolution can be in and shows the cost is concentrated in exactly one: a removal for which no delete fact will ever arrive (a graceful pod delete, a status-only removal, a type the audit policy skips) spends the whole grace to return the answer it had at t=0, measured at ~3.1s against ~70ms when evidence is present. Prices five options against that, and recommends a per-route watermark — stop waiting once the fact stream has demonstrably moved past this event — over a second timeout flag whose right value lives in the API server's config rather than ours. Open: the decision, and how common the case is outside the e2e suite | +| [`concurrent-author-windows.md`](design/concurrent-author-windows.md) | **proposal, not built.** A branch worker holds one commit window at a time, so a second person's edit closes the first person's window and splits their change into two commits. Proposes a window per GitTarget and author, open side by side, with one rule that keeps every path's history in order: a write to a path another open window holds closes that window first (`path_conflict`). Branch-wide closes (foreign commit, apply, shutdown, buffer limit) close every window. Commit order becomes close order. A second save of one author stays refused (`SaveInProgress`). Open: what `buffer_limit` closes, windows of several GitTargets side by side, a cap on open windows, and a save cut short by a path conflict | | [`commitrequest-save-wait-options.md`](design/commitrequest-save-wait-options.md) | **design, decided** — a five-phase plan, reported from a live cluster: a `SaveRequest` can expire before the write it exists to publish arrives, because the close delay defaulted to `0` while the write is still held head-of-line waiting for an audit fact, so the save resolves `Ready=True`/`NoWindowInGrace` and the edit commits seconds later under the target's `liveTemplate`. The decision is to repair the default (`spec.closeDelay`, now a duration string defaulting to `"2s"`, **shipped as phase 0**) and then let a request **name the write it is waiting for**, reusing the `(uid, resourceVersion)` identity the attribution join already keys on, including its settled `ExactCapable` rule. Records why the fact index cannot BE the wait (it sits upstream of the watch event, so a fact-keyed wait fires before the window opens) and what to take from it anyway — the register-then-check waiter discipline, and a best-effort retrospective oracle bounded by a TUNABLE TTL. Twelve situations, and the page is deliberate about the two it cannot fix cheaply: row 11 (a saturated queue is rejected in the WATCH path and never reaches the worker, so no worker-side wait can diagnose it) is an accepted, named gap rather than a promise, and row 10 is fixed only by a hook in the fidelity-drop branch that marks a wait terminally UNSATISFIABLE — satisfying before that gate would arm a finalize for a write that never commits. Carries a **preconditions chapter** for saves on today's build (audit policy first, then delay sizing, fidelity warm-up, save-button debounce, and assert on the commit rather than on `Ready`). `CommitterActivity` is deferred, not recommended: by its own arrival-not-existence rule it cannot satisfy the already-arrived case. Open: situation 12 (one actor, two saves, one window — blocks phase 2), whether the fact-index lookup earns its coupling, and how an unsatisfied wait is graded | | [`commit-timing-surface.md`](design/commit-timing-surface.md) | **built.** One configuration surface for commit windows and saves, an in-place `v1alpha3` break with one tested upgrade procedure: only writes open a window; a `window` block (`idleTimeout`, `maxDuration`, target default `1m`) on both kinds, which a `SaveRequest` replaces whole for the window it attaches to; `attach: CurrentOrNext \| Next` with `attachTimeout`; opt-in `whenNothingToCommit: CommitEmpty` that keeps the cause in the reason. Precise clock and zero rules. Supersedes the save-wait timer model. Open: the target's `maxDuration` value | | [`watch-and-catalog-architecture.md`](design/watch-and-catalog-architecture.md) | the target three-layer watch model — **needs a human call before building** | diff --git a/docs/configuration.md b/docs/configuration.md index 9716e99e..84dd8410 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -120,7 +120,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | | `allowedSaveRequests` | `[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"Keep"}]` | The namespaces whose `SaveRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See [who may save](#who-may-save-specallowedsaverequests) | -| `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may save this target's windows. See [`RemoteSaveRequest`](#remotesaverequest) | +| `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and save its windows. See [`RemoteSaveRequest`](#remotesaverequest) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | | `placement.useKustomize` | off | Keep the folder a kustomize folder. See [keeping the folder a kustomize folder](#keeping-the-folder-a-kustomize-folder-specplacementusekustomize) | @@ -192,11 +192,12 @@ The references are immutable: to hold another `Application`, delete the applier | Field | Default | What it does | |---|---|---| -| `message` | each target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | +| `gitTargetRef` | **required** | The `GitTarget` to save, by `namespace` and `name` on the cluster GitOps Reverser runs in. Its `allowedRemoteSaveRequests` must admit this request | +| `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | -| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit is pushed; `status.gitTargets` reports it | +| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed; the `Applied` condition reports it | @@ -3161,6 +3162,7 @@ reason, without changing Git. | The writes already matched Git | no commit, `AlreadyPresent` | empty commit, `AlreadyPresent` | | No eligible window before `attachTimeout` | no commit, `NoWindow` | empty commit, `NoWindow` | | Only another author's window was open | no commit, `WindowMismatch` | no commit, `WindowMismatch` | +| An earlier save of the same author on the `GitTarget` was still in progress | `SaveInProgress`, at once | `SaveInProgress`, at once | | The target is suspended, or its render fidelity is not established | `FinalizeFailed`, the cause | `FinalizeFailed`, the cause | | The commit or the empty commit failed | `FinalizeFailed` | `FinalizeFailed` | | The push failed | `WaitingForPush`, with the cause, until a push lands; `FinalizeFailed` if the worker stops first | the same | @@ -3186,6 +3188,14 @@ request produced a pushed commit: [`CommitEmpty`](#when-there-is-nothing-to-commit), `NoWindow` and `AlreadyPresent` can come with a recorded empty commit. A failed finalize is `Ready=False` with reason `FinalizeFailed`. + `Ready=False` with reason `SaveInProgress` means an earlier request of the same author on the same + `GitTarget` was still waiting for a commit window, or collecting one, when this one arrived. A + window carries one message, so the two would race for it and one message would be lost without a + word; the later request is refused at once instead. Save again with a new request once the earlier + one is `Ready`. A request for the `Next` window is not refused by one that is collecting: it closes + that window under the earlier request's message first. Saves by different authors never race, + since each author has windows of its own. + `AlreadyPresent` means **the remote confirmed there was nothing to add**, so it is reported after the push rather than when the local plan found no difference. That costs a few seconds and buys a true answer: a competing Git-side edit can turn the same captured object into a real commit when @@ -3258,21 +3268,27 @@ never deleted. A `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next to the edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It -names no `GitTarget`: it saves its creator's commit window in every `GitTarget` that mirrors its -namespace from that cluster and allows it. Each window gets its message -and closes on its timers, as with a [`SaveRequest`](#saverequest). +names one `GitTarget` on the cluster GitOps Reverser runs in, and saves its creator's commit window +there: the window gets its message and closes on its timers, as with a [`SaveRequest`](#saverequest). ```yaml apiVersion: source.configbutler.ai/v1alpha1 kind: RemoteSaveRequest metadata: - name: save-now + name: save-cart namespace: shop spec: + gitTargetRef: + namespace: platform + name: shop message: "fix(shop): raise the cart limit" catchUp: AfterPush ``` +`spec.gitTargetRef` is required, and both its fields are: `namespace` is a namespace of the cluster +GitOps Reverser runs in, not of the source cluster. The people creating requests need no access to +`GitTarget`s; their tooling supplies the address. + ### Serving the `RemoteSaveRequest`s of a cluster Three things turn it on, each on purpose: @@ -3287,9 +3303,10 @@ Three things turn it on, each on purpose: `list`, `watch` and `delete` on `remotesaverequests`, and `update` on `remotesaverequests/status`, in the `source.configbutler.ai` group. A `RemoteSaveRequest` is never mirrored, whatever a `WatchRule` selects. -3. **Grant it on each `GitTarget`, in `spec.allowedRemoteSaveRequests`.** The provider switch makes +3. **Grant it on the `GitTarget`, in `spec.allowedRemoteSaveRequests`.** The provider switch makes the requests reachable; the `GitTarget`'s owner decides which of them may save its windows. An - omitted or empty list admits none. + omitted or empty list admits none. The `GitTarget` must also mirror the cluster the request was + made on: its `spec.kubeProviderRef` names the `KubeProvider` that served the request. ```yaml apiVersion: configbutler.ai/v1alpha3 @@ -3304,18 +3321,20 @@ spec: ``` A request is admitted when one grant admits both its namespace and its labels. `sourceNamespaces` are -namespaces of the cluster `spec.kubeProviderRef` names, by exact name, or `"*"` for every namespace the -`GitTarget` mirrors. `requestSelector` is optional, and narrows the grant to the requests whose labels -match: use it to pick which of several `GitTarget`s a request saves. Whoever creates a request picks -its labels, so a selector proves nothing about who made it. +namespaces of the cluster `spec.kubeProviderRef` names, by exact name, or `"*"` for every namespace. +`requestSelector` is optional, and narrows the grant to the requests whose labels match. Whoever +creates a request picks its labels, so a selector proves nothing about who made it. ### What a save holds A commit window holds one author's edits to one `GitTarget`, wherever that `GitTarget` mirrors them. An admitted `RemoteSaveRequest` therefore saves its creator's whole window, edits outside the request's own namespace included: a grant in `allowedRemoteSaveRequests` authorizes exactly that. The -request's namespace only decides which `GitTarget`s it reaches. A `GitTarget` that mirrors only -cluster-scoped types mirrors no namespace, so no `RemoteSaveRequest` reaches it. +request's namespace decides only whether a grant admits it. + +As with a `SaveRequest`, a second request of the same author on the same `GitTarget`, made while the +first still waits for a window or collects one, is refused at once with `SaveInProgress`: a window +carries one message, and the two would race for it. ### Who the save is from @@ -3337,18 +3356,21 @@ window as for a `SaveRequest`. ### Its status -`status.gitTargets` lists each `GitTarget` the save was admitted to, by `namespace` and `name` on the -cluster GitOps Reverser runs in, with the save's `reason` there (`Committed`, `NoWindow`, -`AlreadyPresent`, `WindowMismatch` or `Failed`), its `branch` and `commit`, -and, with `catchUp: AfterPush`, its `applied` reason and `appliedRevision`. - -- **Ready** is `True` once the save ended without error in every `GitTarget`, with reason `Committed` - when one of them committed, and otherwise the first one's reason, `NoGitTarget` when no `GitTarget` - mirrors the namespace, or `AuthorUnresolved`. `Ready=False` with `Stalled=True` is a failure, - reason `NotAllowed` when `GitTarget`s mirror the namespace but none of them admits the request. -- **Applied**, with `catchUp: AfterPush`, is `True` once every applier applied a revision that holds - the commit, or needed none, `False` only when a branch lost the commit, and `Unknown` with reason - `Lost` when GitOps Reverser restarted before it learned how an apply went. +`status.author`, `status.branch` and `status.commit` say who saved and what was pushed, and, with +`catchUp: AfterPush`, `status.appliedRevision` what the applier applied. + +- **Ready** is `True` once the save ended without error: reason `Committed`, or, with nothing + committed, `NoWindow`, `AlreadyPresent`, `WindowMismatch` or `AuthorUnresolved`. `Ready=False` with + `Stalled=True` is a failure: `NotAllowed` when the `GitTarget` does not admit the request or + mirrors another cluster, `GitTargetNotFound` when it did not exist within `attachTimeout`, + `SaveInProgress`, or `Failed`. +- **Applied**, with `catchUp: AfterPush`, is `True` once the applier applied a revision that holds + the commit, or needed none, `False` only when the branch lost the commit, and `Unknown` with reason + `Lost` when GitOps Reverser restarted before it learned how the apply went. + +A request is served again after GitOps Reverser restarts. It keeps what its status already says: the +author found when it was first served, and an outcome already decided, in which case only its apply, +if it asked for one, is followed. A finished `RemoteSaveRequest` is deleted `--save-request-ttl` (default `48h`) after it finished; `0` keeps it. diff --git a/docs/design/concurrent-author-windows.md b/docs/design/concurrent-author-windows.md new file mode 100644 index 00000000..ee48f3aa --- /dev/null +++ b/docs/design/concurrent-author-windows.md @@ -0,0 +1,145 @@ +# Design: one commit window per author, open at the same time + +Status: **proposal, not built.** Raised by gitops-api's review of the source-side save (PR #456): +with saves created straight from a browser, a second person's edit splits the first person's change +into two commits. This page sets out the change that would fix that, what it costs, and what is +still open. + +## The problem + +A branch worker holds **one** open commit window at a time (`branchWorkerEventLoop.openWindow`). A +window belongs to one author on one `GitTarget`, and a write that is anyone else's cannot join it +(`openWindow.canAppend`). It closes the window first, with close reason `identity_change`, and opens +its own. + +```mermaid +sequenceDiagram + participant A as Alice + participant B as Bob + participant W as Branch worker + A->>W: edit cm/a + Note over W: window 1 opens: Alice + B->>W: edit cm/b + Note over W: window 1 closes (identity_change), window 2 opens: Bob + A->>W: edit cm/c + Note over W: window 2 closes, window 3 opens: Alice + A->>W: SaveRequest "fix(shop): cart limit" + Note over W: attaches to window 3 only +``` + +Alice's change is three files and one save. Git gets two commits for her: `cm/a` under the generated +message, `cm/c` under hers. Nothing fails; the history is just wrong about what belongs together. + +gitops-api avoids it today by serializing every save and its edits behind one lock in its server +(`commitGate`). A client that creates `RemoteSaveRequest`s from the browser has no such lock, so the +split comes back. + +Two saves by **one** author on one `GitTarget` are a different problem, and it is already handled. +A window carries one message, so the second save is refused with `SaveInProgress` while the first +still waits for a window or collects one. That stays true under this proposal, which changes only +how different authors interleave. + +## The proposal + +Keep a window per **(GitTarget, author, attribution class)** instead of one per worker. A write joins +the open window with its identity, or opens one. Writes by others no longer close it. Each window +keeps its own timers, its own attached request and its own message. + +```mermaid +sequenceDiagram + participant A as Alice + participant B as Bob + participant W as Branch worker + A->>W: edit cm/a + Note over W: Alice's window opens + B->>W: edit cm/b + Note over W: Bob's window opens; Alice's stays open + A->>W: edit cm/c + Note over W: joins Alice's window + A->>W: SaveRequest + Note over W: Alice's window: cm/a + cm/c, her message +``` + +Writes that name no author keep sharing one window, as today: they have no author to tell apart. + +## What has to stay true + +### One path, one order + +Today the single window gives a simple guarantee: commits land in the order their writes arrived, +so the last write to a path is the one Git ends with. With several open windows, that breaks unless +something keeps it: + +1. Alice edits `cm/x` (her window). +2. Bob edits `cm/x` after her (his window). +3. Bob's window closes first, then Alice's: Git ends with Alice's **older** `cm/x`. + +**Rule: a write to a path another open window holds closes that window first.** Bob's edit of `cm/x` +closes Alice's window, so her commit lands before his. The close reason is new, `path_conflict`. +The path is the window's own key (`windowPathKey`: the Git path, after placement), so a delete, a +rename to the same document, and a namespaced and a cluster-scoped object that map to one file are +all caught. Two authors editing the same object are therefore split exactly as today; two editing +different objects are not. + +### Everything that closes "the window" closes all of them + +The other close reasons are about the branch, not about one author, and each must close every open +window, oldest first: + +| Reason | Why all of them | +| --- | --- | +| `foreign_commit` | the branch moved under every window | +| `apply_due`, `resync_before_apply` | an apply must not run with unpublished edits from anyone | +| `shutdown` | nothing survives the loop | +| `buffer_limit` | the cap is the worker's memory, not one author's | + +`buffer_limit` could instead close only the largest or oldest window. Closing all is simpler and the +cap is rarely reached; this is an open question below. + +### Commit order is close order + +Commits land in the order their windows close, not the order their first writes arrived. With Alice +open since 10:00 and Bob since 10:01, Bob's commit can come first. Together with the path rule above, +no path ever goes back in time, but `git log` no longer reads as one timeline of edits. That is the +point of the change, and it needs saying in `docs/configuration.md` when it ships. + +### Saves + +- A save attaches to the open window of its identity, by lookup instead of comparison with the one + window. `matchesWindow` keeps its rules. +- `WindowMismatch` becomes rare: it meant "only another author's window was open". With every author + allowed a window, it remains only for the window a save may not claim: an unattributed one, without + `SavedBy` or `TakeOver`. +- `SaveInProgress` is unchanged: it is already per author and `GitTarget`. +- The takeover (`TakeOver`, `SavedBy`) claims the unattributed window, which stays one window. + +## What it costs + +- **The loop's state.** `openWindow` becomes a map with an opening order, the close timer becomes the + earliest of all windows' deadlines, and every path that reads `l.openWindow` today (attach, + finalize, the apply window, resync cut-off, metrics) has to say which window, or all. +- **Memory.** One window per active author. The byte cap stays global, so the bound does not change. +- **Tests.** The commit-window, attach and apply-window suites assume one window. Most of them keep + their meaning with one author; the interleaving cases need new ones, the path-conflict rule first. +- **Metrics.** `git_commit_windows_total` gains `close_reason="path_conflict"`. A gauge of open + windows would show how often authors overlap at all. + +## Alternatives + +1. **Keep one window, and let clients serialize.** What gitops-api does today. Correct, and costs a + server on the save path, which is what `RemoteSaveRequest` set out to remove. +2. **Per-author windows without the path rule.** Simpler, and wrong: a path can go back in time + with no error. +3. **Merge concurrent windows into one commit with several authors.** Git has one author per commit; + `Co-authored-by` trailers would blur exactly the attribution the product exists for. + +## Open questions + +1. Does `buffer_limit` close every window, or only the one that crossed it? +2. Should `identity_change` remain, for a write of another `GitTarget` on the same branch, or do + windows of different `GitTarget`s also stay open side by side? The proposal says side by side; + the per-`GitTarget` encryption and bootstrap that made one window per target necessary are per + window already. +3. Is a cap on open windows needed beyond the byte cap, such as one per `GitTarget`? +4. When a save's own window is closed early by `path_conflict`, it commits with the save's message + and less than the person meant to save. Is that acceptable, or should the save report it? diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index 238dc086..77991d80 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -12,15 +12,14 @@ put its author's name on the commit. Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The two request kinds are a pair: `SaveRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/remote_save_requests.go`) on -the source cluster. Each `GitTarget` admits them in a grant list: `spec.allowedSaveRequests` (by -control-cluster namespace; omitted, its own namespace) and `spec.allowedRemoteSaveRequests` (by source -namespace and labels; omitted, none). A KubeProvider also serves its cluster's remote requests only -with `spec.remoteSaveRequests: Enabled`. The takeover is opt-in, as described below, with one change: -the `GitTarget`'s bound moved from `spec.commit.unattributedWindows` into each `allowedSaveRequests` -grant, so a request from another namespace claims only what its own grant allows. The open questions -are answered as proposed: its own group, a default `attachTimeout` of `5s`, the reason -`AuthorUnresolved`, and no extra queue for a second save on a GitTarget beyond the worker's own (a -window carries one request at a time). +the source cluster. Each names its GitTarget, which admits it in a grant list: +`spec.allowedSaveRequests` (by control-cluster namespace; omitted, its own namespace) and +`spec.allowedRemoteSaveRequests` (by source namespace and labels; omitted, none). A KubeProvider +also serves its cluster's remote requests only with `spec.remoteSaveRequests: Enabled`, and the +GitTarget must mirror that cluster. The takeover is opt-in, as described below, bounded per grant. +The open questions are answered: its own group, a default `attachTimeout` of `5s`, the reason +`AuthorUnresolved`, and a second save of one author on one GitTarget is refused while the first is in +progress (`SaveInProgress`). This revision takes in gitops-api's review of the first draft, and a later API review of the implementation: see [After the API review](#after-the-api-review). @@ -100,7 +99,7 @@ wait on an extra hop, and add a second status to keep in step. Reusing `SaveRequest` with a different schema would be worse than a separate kind: -- **Another shape.** It names no GitTarget, and its status lists one or more of them. +- **Another shape.** It names a GitTarget on another cluster than the one it is created on. - **Another trust model.** Its author comes from audit; a `SaveRequest`'s comes from admission. - **One name for two things** invites the confusion that `tenant-` already costs gitops-api. - **Installed alone.** A workspace gets one small CRD instead of the whole `configbutler.ai` set, so @@ -111,10 +110,11 @@ Reusing `SaveRequest` with a different schema would be worse than a separate kin ### Which GitTarget it saves -The people creating it cannot see GitTargets, so it names none. It saves for the GitTargets that -mirror its own namespace from that KubeProvider. Usually that is one: in gitops-api, each -`repo-` namespace maps to exactly one GitTarget. When several mirror the namespace, into -several folders, it saves each one's window, and its status lists each GitTarget with its outcome. +It names one, in a required `spec.gitTargetRef` of `namespace` and `name` on the control cluster. +The people creating it need no access to GitTargets: their tooling supplies the address. This draft +had it name none and save every GitTarget mirroring its namespace; review showed that an unchanged +request then does more whenever another GitTarget is added, and that its caller cannot say "this one +only". See [After the API review](#after-the-api-review). ### When its wait starts @@ -155,10 +155,14 @@ turns it on, and the KubeProvider's docs say what it adds. - **Create it as JSON.** gitops-api found that kcp drops audit events whose request body is YAML. A save sent as YAML would lose its author without any error. A browser sends JSON anyway; the docs must say it for everyone else. -- **One save at a time per GitTarget.** gitops-api queues saves in its server (`commitGate`). If the - browser creates them directly, that queue is gone. The worker already lets a window carry at most - one request, and a second waits for the next window (`attachWaitingSaveRequests`). Whether that - is enough, or the operator should hold a save until the one before it has resolved, is open. +- **One save at a time per GitTarget and author.** gitops-api queues saves in its server + (`commitGate`). If the browser creates them directly, that queue is gone. A window carries one + message, so a second save of the same author on the same GitTarget, made while the first still + waits for a window or collects one, is refused at once with `SaveInProgress`, rather than losing + its message to the first in silence. The client saves again once the first resolved. A second + person's edit that lands while a window is open still closes that window early: windows are one at + a time per branch worker, whoever writes. That is + [a design question of its own](../design/concurrent-author-windows.md). ## The takeover @@ -192,8 +196,11 @@ do. Only the GitTarget's owner knows which it is, so the GitTarget decides. ### What the GitTarget allows -One field on the GitTarget, an enum, because there are two levels of trust worth offering. Proposed: -`spec.commit.unattributedWindows`. +An enum, because there are two levels of trust worth offering: `unattributedWindows`, on each grant +in the GitTarget's `spec.allowedSaveRequests`. A request claims only what the grant admitting its +namespace allows; admitted by several, it may claim the most any of them allows. (The first version +was one field, `spec.commit.unattributedWindows`, for requests in the GitTarget's own namespace; +it moved into the grants when a `SaveRequest` could name a GitTarget in another namespace.) - **`"Keep"`** (the default): a save attaches only to its author's window, as today. On a cluster with no audit, a save from the control cluster finds no window of its own. @@ -213,10 +220,8 @@ One field on the GitTarget, an enum, because there are two levels of trust worth Author-Source: SaveRequest team-a/save-1; no audit record names who made these changes ``` -A request's `spec.gitTargetRef` is a local reference, so the request is always in its GitTarget's -namespace, and a plain field is enough. A grant on the KubeProvider was considered and set aside: -the provider's owner usually does not know who edits which namespace, and the grant convention's -default ("omitted means this namespace") is the wrong one here. +A grant on the KubeProvider was considered and set aside: the provider's owner usually does not know +who edits which namespace. The GitTarget's owner does, and grants it per consuming namespace. ### Three conditions, all required @@ -224,7 +229,7 @@ default ("omitted means this namespace") is the wrong one here. flowchart TB Start["A save finds an open window on its GitTarget
whose writes name no author"] --> Q1{"1. Did the save ask
for more than its own window?"} Q1 -->|no| Own["Its own windows only"] - Q1 -->|yes| Q2{"2. Does the GitTarget's
unattributedWindows allow what it asked?"} + Q1 -->|yes| Q2{"2. Does the grant admitting it
allow what it asked (unattributedWindows)?"} Q2 -->|no| Refuse["Refused: Ready=False, Stalled=True
reason TakeOverNotAllowed"] Q2 -->|yes| Q3{"3. Has the KubeProvider ever
received an audit fact?"} Q3 -->|yes| Own @@ -233,9 +238,10 @@ flowchart TB 1. **The save asks for it.** An enum under `spec.window` beside `attach`, matching the GitTarget's: `spec.window.author`, `"Own"` (the default), `"SavedBy"` or `"TakeOver"`. A save asks for what it - wants each time, and the GitTarget bounds it: `"TakeOver"` allows a save that asks for - `"SavedBy"`, and not the other way round. -2. **The GitTarget allows it.** A save that asks for more than the GitTarget allows fails at once, + wants each time, and the grant bounds it: `"TakeOver"` allows a save that asks for `"SavedBy"`, + and not the other way round. +2. **The GitTarget's grant allows it.** A save that asks for more than the grants admitting it allow + fails at once, when it is first reconciled: `Ready=False`, `Stalled=True`, reason `TakeOverNotAllowed`. The spec is immutable, so it cannot recover by itself. 3. **The KubeProvider has never received an audit fact** (`AuditFactsReceived` is not `True`). On a @@ -303,7 +309,18 @@ A review of the implementation found three things: cluster's) and an optional `requestSelector`. Omitted, it admits none: `NotAllowed`. Labels pick requests and prove nothing about who made them. -The status list became `status.gitTargets`, keyed by `namespace` and `name`, and the names became the -`SaveRequest`/`RemoteSaveRequest` pair. A `SaveRequest` then gained a cross-namespace -`spec.gitTargetRef`, and with it `GitTarget.spec.allowedSaveRequests`: a grant list like the others, -defaulting to one `from: Same` grant, whose grants also carry the `unattributedWindows` bound. +A second review then found that implicit target selection still made a save broader than its caller +may intend: a request reached every admitting GitTarget mirroring its namespace, so adding a +GitTarget changed what an unchanged request did. A `RemoteSaveRequest` now names its GitTarget in a +required `spec.gitTargetRef`, saves that one, and reports a flat status (`status.commit`, +`status.appliedRevision`, `Ready`, `Applied`). The request's namespace decides only admission. That +also removed a cold-start failure: with no named target, discovery that had not caught up reported +`NoGitTarget`. The same review fixed two recovery paths: a request served again after a restart +keeps the author found when it was first served (its status writes moved its resourceVersion, so its +audit fact no longer matches), and keeps an outcome already decided, following only its apply. + +A `SaveRequest` gained a cross-namespace `spec.gitTargetRef`, and with it +`GitTarget.spec.allowedSaveRequests`: a grant list like the others, defaulting to one `from: Same` +grant, whose grants also carry the `unattributedWindows` bound. The GitTarget that admitted a request +is recorded by UID (`status.gitTargetUID`), so a GitTarget created after the request under the same +name must admit it before anything is attached. diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index 19ac66b9..a3a33796 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -216,7 +216,7 @@ boundary, the commit, the push. Background: | `git_next_retry_timestamp_seconds` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | When the next attempt at the pending writes is due, as a Unix timestamp. No sample while no retry is scheduled. | | `git_materialization_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | Attempts to commit decided writes that stopped before any push: `reason` is `unreachable` or `parent_unavailable`. Nothing is lost; the writes wait for the retry. A failure `git_pushes_total` cannot see, because no push cycle started. | | `git_branch_targets` | gauge | `gitbranch_namespace`, `gitbranch_name`, `gittarget_namespace`, `gittarget_name`, `source_cluster` | Always 1. The **join** between the GitTarget-labeled half of the pipeline and the GitBranch-labeled half: the `GitTarget`'s `spec.gitBranchRef`. Published per configured GitTarget, whether or not its GitBranch exists or its worker runs. | -| `save_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `SaveRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | +| `save_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `SaveRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `save_in_progress` (refused: an earlier save of the same author on the target was in progress) / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | | `placements_total` | counter | `source`, `disposition`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new document at a resolved path. | | `placement_refusals_total` | counter | `reason`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new resource the writer declined. Every increment is a resource **absent** from the mirror. | | `placement_kustomization_entries_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | `added` / `no_change` / `failed`. | diff --git a/docs/rbac.md b/docs/rbac.md index 1c871abf..4f2a4709 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -96,7 +96,7 @@ status back there. Its credential then needs, in the `source.configbutler.ai` gr `remotesaverequests/status`. Nothing else on a source cluster is written. The manager role carries the same rules for the cluster the operator runs in, for an `inCluster` provider. -Whoever may create a `RemoteSaveRequest` in a namespace may save their own commit window on every +Whoever may create a `RemoteSaveRequest` in a namespace may save their own commit window on any `GitTarget` whose `spec.allowedRemoteSaveRequests` admits it. That grant, not the provider's switch, is the `GitTarget` owner's consent. On the cluster GitOps Reverser runs in, whoever may create a `SaveRequest` in a namespace may save on every `GitTarget` whose `spec.allowedSaveRequests` admits diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index d633e4ef..e423b579 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -166,8 +166,8 @@ kinds: - path: allowedRemoteSaveRequests default: none admitted doc: >- - The `RemoteSaveRequest`s, by source namespace and labels, that may save this target's - windows. See [`RemoteSaveRequest`](#remotesaverequest) + The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and + save its windows. See [`RemoteSaveRequest`](#remotesaverequest) - path: placement.byType default: the built-in path doc: >- @@ -337,8 +337,12 @@ kinds: - kind: RemoteSaveRequest title: "`RemoteSaveRequest` (namespaced, on a source cluster): save the edits made here" fields: + - path: gitTargetRef + doc: >- + The `GitTarget` to save, by `namespace` and `name` on the cluster GitOps Reverser runs in. + Its `allowedRemoteSaveRequests` must admit this request - path: message - default: each target's templates + default: the target's templates doc: Commit message, committed verbatim unless `requestTemplate` frames it - path: window.attachTimeout doc: How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) @@ -349,5 +353,5 @@ kinds: doc: Close the attached window this long after the attach - path: catchUp doc: >- - `AfterPush` asks each `GitTarget`'s applier to apply the branch tip right after the commit - is pushed; `status.gitTargets` reports it + `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit + is pushed; the `Applied` condition reports it diff --git a/internal/controller/saverequest_controller.go b/internal/controller/saverequest_controller.go index cbaa9d84..ac90537c 100644 --- a/internal/controller/saverequest_controller.go +++ b/internal/controller/saverequest_controller.go @@ -57,6 +57,9 @@ type CommandAuthorLookup interface { // windowMismatchMessage explains the author-bound refusal: an open window // existed but was not this requester's, so it was deliberately left alone. +const saveInProgressMessage = "an earlier save of the same author on this GitTarget was still waiting for " + + "or collecting a commit window, which carries one message; save again once it resolved" + const windowMismatchMessage = "the open commit window belongs to a different author or GitTarget; " + "nothing was committed for this request" @@ -231,8 +234,8 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) // decision is final — there is no wait and no requeue for the author. author, attribution := r.attributeAuthor(ctx, saveRequest) - claim, refused, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) - if err != nil || refused { + claim, admitted, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) + if err != nil || admitted == saveRequestRefused { return ctrl.Result{}, err } @@ -272,37 +275,86 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) CatchUpNow: saveRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, Unattributed: claim, } - if time.Since(saveRequest.CreationTimestamp.Time) >= window.resolveTimeout() { + return r.sendSaveRequest(ctx, log, req, saveRequest, attribution, attach, + time.Since(saveRequest.CreationTimestamp.Time) >= window.resolveTimeout(), + admitted == saveRequestTargetMissing) +} + +// sendSaveRequest sends the attach, or the withdraw once the request is past its bound, or, while +// its GitTarget does not exist and never admitted it, neither: nothing is attached before a GitTarget +// admitted the request, so one created later is asked first, on a later poll. +func (r *SaveRequestReconciler) sendSaveRequest( + ctx context.Context, + log logr.Logger, + req ctrl.Request, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + attach git.AttachSaveRequest, + pastTheBound, targetMissing bool, +) (ctrl.Result, error) { + switch { + case pastTheBound: // Past the bound the controller only withdraws: an attach sent now would sit behind the // withdraw on the worker's FIFO and could register the request after it was cancelled. return r.withdrawSaveRequest(ctx, log, req, saveRequest, attribution, attach) + case targetMissing: + return r.pollSaveRequest(ctx, saveRequest, attribution, git.PhaseWaitingForWorker, "") } return r.attachSaveRequest(ctx, log, req, saveRequest, attribution, attach) } -// admitAtFirstSight runs admitSaveRequest, and fails a request it refuses, but only at first sight, -// before any attach was sent: a GitTarget's grants lowered later do not fail a request the worker -// may already hold, which keeps what it was first sent. refused reports a request now failed; err is -// a read to retry. +// saveRequestAdmission is how admitAtFirstSight left a request. +type saveRequestAdmission int + +const ( + // saveRequestAdmitted is a request its GitTarget admitted, now or before. + saveRequestAdmitted saveRequestAdmission = iota + // saveRequestTargetMissing is a request whose GitTarget does not exist and never admitted it. + saveRequestTargetMissing + // saveRequestRefused is a request now failed. + saveRequestRefused +) + +// admitAtFirstSight runs admitSaveRequest, records the GitTarget that admitted the request, and fails +// a request it refuses. A refusal is final unless that same GitTarget, by UID, admitted the request +// before: then its grants were lowered after an attach may have been sent, and the worker, which may +// hold the request, keeps what it was first sent. A GitTarget that did not exist when the request was +// first seen, or was recreated since, admitted nothing, so its refusal stands. err is a read or write +// to retry. func (r *SaveRequestReconciler) admitAtFirstSight( ctx context.Context, log logr.Logger, saveRequest *configbutleraiv1alpha3.SaveRequest, attribution saveRequestAttribution, -) (git.UnattributedClaim, bool, error) { - claim, err := r.admitSaveRequest(ctx, saveRequest) - if err == nil { - return claim, false, nil - } - if !errors.Is(err, errSaveRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed) { - return claim, false, err - } - if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) != nil { - return claim, false, nil +) (git.UnattributedClaim, saveRequestAdmission, error) { + claim, targetUID, err := r.admitSaveRequest(ctx, saveRequest) + recorded := saveRequest.Status.GitTargetUID + switch { + case err == nil && targetUID == "": + if recorded != "" { + // Its GitTarget is gone since it admitted the request: the attach path reports that. + return claim, saveRequestAdmitted, nil + } + return claim, saveRequestTargetMissing, r.stampFirstSightConditions(ctx, saveRequest, attribution) + case err == nil: + if recorded == targetUID { + return claim, saveRequestAdmitted, nil + } + // Recorded before any attach is sent for this GitTarget, so a later refusal can tell the two + // apart. At first sight the in-progress stamp that follows writes it. + saveRequest.Status.GitTargetUID = targetUID + if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) == nil { + return claim, saveRequestAdmitted, nil + } + return claim, saveRequestAdmitted, r.Status().Update(ctx, saveRequest) + case !errors.Is(err, errSaveRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed): + return claim, saveRequestAdmitted, err + case recorded == targetUID: + return git.UnattributedClaimNone, saveRequestAdmitted, nil } r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, resolvedSaveRequestTarget(saveRequest)) - return claim, true, nil + return claim, saveRequestRefused, nil } // errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author @@ -323,20 +375,31 @@ var errSaveRequestNotAllowed = errors.New("the GitTarget does not admit SaveRequ // write naming no author means its fact went missing, and nothing is claimed. A request that asks for // more than the admitting grants' unattributedWindows allow is errTakeOverNotAllowed. // -// A GitTarget that does not exist decides nothing here: the attach path reports it. Any other read -// error is returned as is, so the request is retried rather than refused. +// A GitTarget that does not exist admits nothing yet, and is reported with an empty UID. Any other +// read error is returned as is, so the request is retried rather than refused. func (r *SaveRequestReconciler) admitSaveRequest( ctx context.Context, saveRequest *configbutleraiv1alpha3.SaveRequest, -) (git.UnattributedClaim, error) { +) (git.UnattributedClaim, string, error) { var target configbutleraiv1alpha3.GitTarget if err := r.Get(ctx, saveRequest.GitTargetKey(), &target); err != nil { if apierrors.IsNotFound(err) { - return git.UnattributedClaimNone, nil + return git.UnattributedClaimNone, "", nil } - return git.UnattributedClaimNone, fmt.Errorf("read GitTarget %s: %w", saveRequest.GitTargetKey(), err) + return git.UnattributedClaimNone, "", fmt.Errorf("read GitTarget %s: %w", saveRequest.GitTargetKey(), err) } - decision, admission, err := authz.SaveRequestAdmitted(ctx, r.Client, saveRequest, &target) + uid := string(target.UID) + claim, err := r.claimOn(ctx, saveRequest, &target) + return claim, uid, err +} + +// claimOn is admitSaveRequest for a GitTarget that exists. +func (r *SaveRequestReconciler) claimOn( + ctx context.Context, + saveRequest *configbutleraiv1alpha3.SaveRequest, + target *configbutleraiv1alpha3.GitTarget, +) (git.UnattributedClaim, error) { + decision, admission, err := authz.SaveRequestAdmitted(ctx, r.Client, saveRequest, target) if err != nil { return git.UnattributedClaimNone, err } diff --git a/internal/controller/saverequest_controller_unit_test.go b/internal/controller/saverequest_controller_unit_test.go index 6a3f4f2c..ec01c33c 100644 --- a/internal/controller/saverequest_controller_unit_test.go +++ b/internal/controller/saverequest_controller_unit_test.go @@ -141,11 +141,15 @@ func withInProgress(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest Type: ConditionTypeReconciling, Status: metav1.ConditionTrue, Reason: string(git.PhaseWaitingForWindow), Message: "in progress", }) + // Admitted when it was first seen, by the GitTarget withGitTargets gives it. + key := cr.GitTargetKey() + cr.Status.GitTargetUID = "uid-" + key.Namespace + "-" + key.Name return cr } func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...client.Object) client.WithWatch { t.Helper() + objects = withGitTargets(objects) scheme := runtime.NewScheme() require.NoError(t, configv1alpha3.AddToScheme(scheme)) require.NoError(t, corev1.AddToScheme(scheme)) @@ -159,6 +163,36 @@ func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...clien return builder.Build() } +// withGitTargets gives each GitTarget a UID, as the API server would, and, when objects hold no +// GitTarget at all, adds the one each SaveRequest names: a request is only attached once a GitTarget +// admitted it. A test that passes GitTargets of its own gets only those. +func withGitTargets(objects []client.Object) []client.Object { + present := map[client.ObjectKey]bool{} + for _, obj := range objects { + if target, ok := obj.(*configv1alpha3.GitTarget); ok { + present[client.ObjectKeyFromObject(target)] = true + if target.UID == "" { + target.UID = types.UID("uid-" + target.Namespace + "-" + target.Name) + } + } + } + if len(present) > 0 { + return objects + } + for _, obj := range objects { + request, ok := obj.(*configv1alpha3.SaveRequest) + if !ok || request.Spec.GitTargetRef.Name == "" || present[request.GitTargetKey()] { + continue + } + key := request.GitTargetKey() + present[key] = true + objects = append(objects, &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{ + Name: key.Name, Namespace: key.Namespace, UID: types.UID("uid-" + key.Namespace + "-" + key.Name), + }}) + } + return objects +} + func reconcileSaveRequest(t *testing.T, r *SaveRequestReconciler, name string) ctrl.Result { t.Helper() res, err := r.Reconcile(context.Background(), ctrl.Request{ @@ -1063,3 +1097,61 @@ func TestSaveRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) }) } } + +// A request whose GitTarget does not exist is not attached: it waits, and once a GitTarget by that +// name appears it must admit the request first. Its earlier in-progress conditions grant nothing. +func TestSaveRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testing.T) { + cr := newSaveRequest("save") + cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "later", Namespace: "platform"} + c := newSaveRequestClient(t, nil, cr, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "unrelated", Namespace: "platform"}, + }) + f := &fakeFinalizer{resolved: true, result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + + res := reconcileSaveRequest(t, r, "save") + assert.Empty(t, f.calls, "nothing is attached before a GitTarget admitted the request") + assert.Positive(t, res.RequeueAfter, "it waits for the GitTarget") + got := fetchSaveRequest(t, c, "save") + require.NotNil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady), "it reports progress") + assert.Empty(t, got.Status.GitTargetUID) + + require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "later", Namespace: "platform", UID: "uid-later"}, + Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: []configv1alpha3.GitTargetSaveRequestGrant{}}, + })) + reconcileSaveRequest(t, r, "save") + + assert.Empty(t, f.calls, "the GitTarget that appeared admits nobody") + got = fetchSaveRequest(t, c, "save") + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionFalse, ready.Status) + assert.Equal(t, crReasonNotAllowed, ready.Reason) +} + +// The GitTarget that admitted a request is recorded by UID, and a refusal by that same GitTarget +// afterwards, once its grants were lowered, does not fail a request an attach may already have reached. +func TestSaveRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { + cr := newSaveRequest("save") + target := &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}} + c := newSaveRequestClient(t, nil, cr, target) + f := &fakeFinalizer{} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + + reconcileSaveRequest(t, r, "save") + require.Len(t, f.calls, 1) + got := fetchSaveRequest(t, c, "save") + assert.Equal(t, string(target.UID), got.Status.GitTargetUID) + + require.NoError(t, c.Get(context.Background(), client.ObjectKeyFromObject(target), target)) + target.Spec.AllowedSaveRequests = []configv1alpha3.GitTargetSaveRequestGrant{} + require.NoError(t, c.Update(context.Background(), target)) + reconcileSaveRequest(t, r, "save") + + assert.Len(t, f.calls, 2, "the attach is sent again, as before the grant was lowered") + got = fetchSaveRequest(t, c, "save") + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.NotEqual(t, crReasonNotAllowed, ready.Reason) +} diff --git a/internal/controller/saverequest_finalize.go b/internal/controller/saverequest_finalize.go index ef7c1dc4..880089b1 100644 --- a/internal/controller/saverequest_finalize.go +++ b/internal/controller/saverequest_finalize.go @@ -30,6 +30,7 @@ const ( crReasonGitTargetRefPruned = "GitTargetRefPruned" crReasonTakeOverNotAllowed = "TakeOverNotAllowed" crReasonNotAllowed = "NotAllowed" + crReasonSaveInProgress = "SaveInProgress" ) // The bounded `outcome` values on gitopsreverser_save_requests_total. @@ -43,6 +44,7 @@ const ( crOutcomeNoWindow = "no_window" crOutcomeWindowMismatch = "window_mismatch" crOutcomeAlreadyPresent = "already_present" + crOutcomeSaveInProgress = "save_in_progress" crOutcomeFailed = "failed" ) @@ -65,6 +67,8 @@ func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { return crOutcomeWindowMismatch case git.FinalizeAlreadyPresent: return crOutcomeAlreadyPresent + case git.FinalizeSaveInProgress: + return crOutcomeSaveInProgress default: return crOutcomeFailed } @@ -313,6 +317,10 @@ func applyFinalizeResultToStatus( case git.FinalizeAlreadyPresent: // The change already matches the remote, so the commit was dropped. resolveNothingToCommit(cr, result, crReasonAlreadyPresent, alreadyPresentMessage) + case git.FinalizeSaveInProgress: + // Refused, not failed by the worker: the spec is immutable, so the caller saves again with a + // new request once the earlier one resolved. + failSaveRequest(cr, crReasonSaveInProgress, saveInProgressMessage) default: // An empty or unknown outcome with no error is a bug, not a benign // rejection; fail it so it is not silently hidden. diff --git a/internal/git/save_request_attach.go b/internal/git/save_request_attach.go index d6e0381a..87eaa372 100644 --- a/internal/git/save_request_attach.go +++ b/internal/git/save_request_attach.go @@ -53,6 +53,11 @@ const ( // produced no diff — the change already matches the remote, so no commit was // made (loop prevention). Resolved at finalize, never waiting on a push. FinalizeAlreadyPresent FinalizeOutcome = "AlreadyPresent" + // FinalizeSaveInProgress means an earlier request of the same author on the same GitTarget was + // still waiting for a window, or collecting the open one, when this one was registered. Both + // would race for one window, which carries one message, so this one is refused at once rather + // than left to lose its message to the first, or its edits to the first's commit. + FinalizeSaveInProgress FinalizeOutcome = "SaveInProgress" ) // SaveRequestPhase is where an unresolved SaveRequest stands on the worker. It is what the @@ -280,6 +285,13 @@ func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.claims(w) } +// sameSaver reports whether two requests would save one window: the same GitTarget and the same +// author, by the comparison matchesWindow makes. +func (p *pendingSaveRequest) sameSaver(other *pendingSaveRequest) bool { + return p.gitTargetName == other.gitTargetName && p.gitTargetNamespace == other.gitTargetNamespace && + p.author == other.author && p.attribution.NamesActor() == other.attribution.NamesActor() +} + // claims reports whether the request matches a window on its GitTarget whose writes name no author // only because it may claim such a window: it names its submitter, and asked for more than its own // windows (AttachSaveRequest.Unattributed). diff --git a/internal/git/save_request_attach_loop.go b/internal/git/save_request_attach_loop.go index fdba8eaa..1f9997cf 100644 --- a/internal/git/save_request_attach_loop.go +++ b/internal/git/save_request_attach_loop.go @@ -218,6 +218,13 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) unattributed: req.Unattributed, } + if earlier := l.saveInProgress(pcr, req.Attach); earlier != nil { + l.w.recordSaveRequestOutcome(id, FinalizeResult{Branch: l.w.Branch, Outcome: FinalizeSaveInProgress}) + l.w.Log.Info("SaveRequest refused: an earlier save of the same author on the GitTarget is in progress", + "request", id.Namespace+"/"+id.Name, "earlier", earlier.id.Namespace+"/"+earlier.id.Name, + "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName) + return + } l.pendingCRs[id] = pcr l.w.setSaveRequestPhase(id, PhaseWaitingForWindow) l.w.Log.Info("SaveRequest registered with worker", @@ -252,6 +259,31 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) } } +// saveInProgress returns the earlier request a newly registered one would race for one window, or nil. +// That is one of the same author on the same GitTarget (sameSaver) that is still waiting for a window, +// or that is collecting the open one. A request that asks for the Next window does not race one that +// is collecting: it closes that window under the earlier request's message first. A request already +// waiting for its push holds no window, so it races nothing. +func (l *branchWorkerEventLoop) saveInProgress( + pcr *pendingSaveRequest, attach v1alpha3.AttachPolicy, +) *pendingSaveRequest { + now := time.Now() + for _, earlier := range l.pendingCRs { + if !earlier.sameSaver(pcr) { + continue + } + if earlier.waiting(now) { + return earlier + } + collecting := earlier.attached && l.openWindow != nil && l.openWindow.pendingCR != nil && + *l.openWindow.pendingCR == earlier.id + if collecting && attach != v1alpha3.AttachNext { + return earlier + } + } + return nil +} + // serviceSaveRequests runs after every loop wake: note foreign windows, resolve waiting requests // whose attach deadline passed, attach the next waiting request to an open window, and re-arm the // attach timer. diff --git a/internal/git/save_request_attach_test.go b/internal/git/save_request_attach_test.go index 12214818..b7176e4c 100644 --- a/internal/git/save_request_attach_test.go +++ b/internal/git/save_request_attach_test.go @@ -503,36 +503,91 @@ func TestAttach_ForeignWindowClosingBeforeExpiryIsStillAMismatch(t *testing.T) { "bob was refused by a window that has since closed; the refusal still happened") } -// A window bob COULD have claimed, already taken by another of his own saves, is contention rather -// than a mismatch: it resolves on the next window, and calling it a mismatch would blame a -// queueing delay on the wrong cause. -func TestAttach_ClaimedSameAuthorWindowIsNotAMismatch(t *testing.T) { - worker, _, _ := setupCommitPushSplitWorker(t) - createPlainGitTarget(t, worker, "team-a", "team-a") +// A second save of one author on one GitTarget, made while the first still waits for a window or +// collects one, would race it for that window, which carries one message: the second is refused at +// once, SaveInProgress, instead of losing its message or its edits to the first. It is not a +// mismatch, and it holds nothing. +func TestAttach_ASecondSaveOfTheSameAuthorIsRefusedWhileTheFirstIsInProgress(t *testing.T) { + second := func() *AttachSaveRequest { + req := attachReq("alice", 60*time.Second) + req.Name, req.UID = "save-second", "uid-save-second" + return req + } + secondOutcome := func(t *testing.T, w *BranchWorker) (FinalizeResult, bool) { + t.Helper() + return w.LookupSaveRequestOutcome("default", "save-second", "uid-save-second") + } - loop := newBranchWorkerEventLoop(worker, time.Hour) - loop.lastPushAt = time.Now() - defer loop.stopTimers() + t.Run("while the first collects the window", func(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + loop := newBranchWorkerEventLoop(worker, time.Hour) + loop.lastPushAt = time.Now() + defer loop.stopTimers() + loop.handleQueueItem(WorkItem{Request: &WriteRequest{ + Events: []Event{configMapTargetEvent("cm", "alice", "team-a")}, + }}) + serviceAttach(loop, attachReq("alice", 60*time.Second)) + require.NotNil(t, loop.openWindow.pendingCR, "precondition: the first save collects the window") - loop.handleQueueItem(WorkItem{Request: &WriteRequest{ - Events: []Event{configMapTargetEvent("cm", "alice", "team-a")}, - }}) - require.NotNil(t, loop.openWindow) + serviceAttach(loop, second()) - // Alice's first save claims the window; her second one has to wait for the next. - first := attachReq("alice", 60*time.Second) - serviceAttach(loop, first) - require.NotNil(t, loop.openWindow.pendingCR, "precondition: the window is claimed") + res, ok := secondOutcome(t, worker) + require.True(t, ok) + assert.Equal(t, FinalizeSaveInProgress, res.Outcome) + assert.Len(t, loop.pendingCRs, 1, "the refused save holds nothing") + }) + + t.Run("while the first waits for a window", func(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + loop := newBranchWorkerEventLoop(worker, time.Hour) + defer loop.stopTimers() + serviceAttach(loop, attachReq("alice", 60*time.Second)) - second := attachReq("alice", 60*time.Second) - second.Name = "save-second" - second.UID = "uid-save-second" - serviceAttach(loop, second) + serviceAttach(loop, second()) + + res, ok := secondOutcome(t, worker) + require.True(t, ok) + assert.Equal(t, FinalizeSaveInProgress, res.Outcome) + }) - pending := loop.pendingCRs[saveRequestID{Namespace: second.Namespace, Name: second.Name, UID: second.UID}] - require.NotNil(t, pending) - assert.False(t, pending.sawForeignWindow, - "a window this author could have claimed is contention, never a mismatch") + t.Run("another author's save is not refused", func(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + loop := newBranchWorkerEventLoop(worker, time.Hour) + defer loop.stopTimers() + serviceAttach(loop, attachReq("alice", 60*time.Second)) + + bob := second() + bob.Author = "bob" + serviceAttach(loop, bob) + + _, resolved := secondOutcome(t, worker) + assert.False(t, resolved, "bob waits for his own window") + assert.Len(t, loop.pendingCRs, 2) + }) + + t.Run("a save for the Next window closes the collecting one instead", func(t *testing.T) { + worker, _, _ := setupCommitPushSplitWorker(t) + createPlainGitTarget(t, worker, "team-a", "team-a") + loop := newBranchWorkerEventLoop(worker, time.Hour) + loop.lastPushAt = time.Now() + defer loop.stopTimers() + loop.handleQueueItem(WorkItem{Request: &WriteRequest{ + Events: []Event{configMapTargetEvent("cm", "alice", "team-a")}, + }}) + serviceAttach(loop, attachReq("alice", 60*time.Second)) + require.NotNil(t, loop.openWindow.pendingCR, "precondition: the first save collects the window") + + next := second() + next.Attach = configv1alpha3.AttachNext + serviceAttach(loop, next) + + _, resolved := secondOutcome(t, worker) + assert.False(t, resolved, "it waits for the next window") + assert.Nil(t, loop.openWindow, "the first save's window was closed under its own message") + }) } // A window that opens AFTER the grace has already elapsed was never a refusal: the request waited diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go index 474aa8f2..12d052fe 100644 --- a/internal/watch/remote_save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -6,7 +6,6 @@ import ( "context" "errors" "fmt" - "sort" "sync" "time" @@ -26,21 +25,19 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/git" "github.com/ConfigButler/gitops-reverser/internal/queue" - "github.com/ConfigButler/gitops-reverser/internal/types" ) // A RemoteSaveRequest is a save made on a source cluster, next to the edits it saves // (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose // spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way -// the SaveRequest controller serves a SaveRequest: it attaches the request to the commit window of -// every GitTarget that mirrors the request's namespace from that cluster and admits it in -// spec.allowedRemoteSaveRequests, polls the branch workers for the outcomes, and writes them back to -// the request's status on the source cluster. The provider's switch makes the requests reachable; -// the GitTarget's grant is its owner's consent to being steered by them. +// the SaveRequest controller serves a SaveRequest: it attaches the request to its creator's commit +// window on the GitTarget its spec.gitTargetRef names, polls the branch worker for the outcome, and +// writes it back to the request's status on the source cluster. // -// A commit window groups by author and GitTarget, not by namespace, so an admitted request saves its -// author's whole window on that GitTarget, edits outside the request's namespace included. The -// GitTarget's grant is what authorizes that. +// Each party decides one thing. The request names the GitTarget; the KubeProvider it was served +// through says which cluster it came from, and the GitTarget must mirror that one; the GitTarget's +// spec.allowedRemoteSaveRequests admits the request's namespace and labels; and the window, which +// groups by author and GitTarget, decides which edits are saved, wherever that GitTarget mirrors them. // // The request's author is found as the edits' authors are, from the source cluster's audit facts, so // a save attaches only to its creator's windows by the ordinary author rule. On a cluster that sends @@ -48,6 +45,10 @@ import ( // whose KubeProvider has received an audit fact, a request that names nobody means its fact went // missing; it is refused rather than attached, so it cannot claim the window that should stay // unattributed. +// +// A request is served again after a restart or a dropped watch. Its status then carries what the +// first serving decided: its author, found once against the resourceVersion it was created with, +// which its status writes have moved since; and its outcome, which the worker may no longer know. // +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests,verbs=get;list;watch;delete // +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests/status,verbs=get;update @@ -77,19 +78,20 @@ const ( defaultSaveMaxDuration = 2 * time.Second ) -// RemoteSaveRequest reasons, on the Ready condition and each target. +// RemoteSaveRequest reasons, on the Ready and Applied conditions. const ( - saveReasonCommitted = "Committed" - saveReasonNoWindow = "NoWindow" - saveReasonWindowMismatch = "WindowMismatch" - saveReasonAlreadyPresent = "AlreadyPresent" - saveReasonFailed = "Failed" - saveReasonNoGitTarget = "NoGitTarget" - saveReasonNotAllowed = "NotAllowed" - saveReasonAuthorUnresolved = "AuthorUnresolved" - saveReasonSaving = "Saving" - saveReasonApplying = "Applying" - saveReasonApplied = "Applied" + saveReasonCommitted = "Committed" + saveReasonNoWindow = "NoWindow" + saveReasonWindowMismatch = "WindowMismatch" + saveReasonAlreadyPresent = "AlreadyPresent" + saveReasonSaveInProgress = "SaveInProgress" + saveReasonFailed = "Failed" + saveReasonNotAllowed = "NotAllowed" + saveReasonGitTargetNotFound = "GitTargetNotFound" + saveReasonAuthorUnresolved = "AuthorUnresolved" + saveReasonSaving = "Saving" + saveReasonApplying = "Applying" + saveReasonLost = "Lost" ) // SaveFinalizer is what serves a save once it is attached: the SaveRequest service of the event @@ -332,29 +334,36 @@ func remoteSaveRequestFinished(request *sourcev1alpha1.RemoteSaveRequest) (time. } // serve takes one request from attribution to its outcome, and to its apply when it asked to catch -// up, writing each step to its status. +// up, writing each step to its status. A request whose Ready is already decided, served again after a +// restart, only has its apply left to follow. func (s *RemoteSaveRequestServer) serve( ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher, resource dynamic.ResourceInterface, request *sourcev1alpha1.RemoteSaveRequest, ) { - log = log.WithValues("remoteSaveRequest", request.Namespace+"/"+request.Name) + ref := request.Spec.GitTargetRef + log = log.WithValues("remoteSaveRequest", request.Namespace+"/"+request.Name, + "gitTarget", ref.Namespace+"/"+ref.Name) status := request.Status.DeepCopy() write := func() { if err := writeRemoteSaveRequestStatus(ctx, resource, request, status); err != nil { log.Info("Cannot write the RemoteSaveRequest's status", "err", err.Error()) } } + if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil && + ready.Status != metav1.ConditionUnknown { + if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && ready.Reason == saveReasonCommitted { + s.followApply(ctx, request, status, remoteSaveRequestAttach(request, status.Author, ""), write) + } + return + } if err := git.ValidateLiteralCommitMessage(request.Spec.Message); err != nil { - setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonFailed, err.Error()) - setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonFailed, err.Error()) + failSave(request, status, saveReasonFailed, err.Error()) write() return } - author, attribution := s.resolveSaveAuthor(ctx, watcher.route, request) - status.Author = author.Username - if !attribution.NamesActor() && attribution != git.AttributionNotAttempted && - s.sentAuditFacts(ctx, watcher.provider) { + author, attribution, refused := s.saveAuthor(ctx, watcher, request, status) + if refused { setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonAuthorUnresolved, "no audit event of this cluster names who created this request, though the cluster sends audit "+ "events: its event went missing, so it saves nothing. Save again") @@ -362,57 +371,57 @@ func (s *RemoteSaveRequestServer) serve( log.Info("Refused a RemoteSaveRequest whose author no audit event names, on a cluster that sends them") return } - - targets, mirrored, err := s.admittingTargets(ctx, watcher.provider, request) + status.Author = author + reason, message, err := s.admit(ctx, watcher, request) if err != nil { - log.Info("Cannot find the GitTargets that admit a RemoteSaveRequest; will retry", "err", err.Error()) + log.Info("Cannot read the RemoteSaveRequest's GitTarget; it is served again on the next listing", + "err", err.Error()) return } - if len(targets) == 0 { - if mirrored { - message := "a GitTarget mirrors this namespace from this cluster, but none admits this request " + - "in spec.allowedRemoteSaveRequests" - setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonNotAllowed, message) - setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonNotAllowed, message) - write() - log.Info("Refused a RemoteSaveRequest no GitTarget admits") - return - } - setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonNoGitTarget, - "no GitTarget mirrors this namespace from this cluster, so there is nothing to save") + if reason != "" { + failSave(request, status, reason, message) write() + log.Info("Refused a RemoteSaveRequest", "reason", reason) return } - status.GitTargets = make([]sourcev1alpha1.RemoteSaveRequestGitTarget, len(targets)) - attaches := make([]git.AttachSaveRequest, len(targets)) - for i, target := range targets { - status.GitTargets[i] = sourcev1alpha1.RemoteSaveRequestGitTarget{Namespace: target.Namespace, Name: target.Name} - attaches[i] = remoteSaveRequestAttach(request, target, author.Username, attribution) - } + + attach := remoteSaveRequestAttach(request, author, attribution) setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, "waiting for the edits to reach a commit window") write() - s.finalize(ctx, request, status, attaches) + if !s.finalize(ctx, request, status, attach) { + return + } write() - log.Info("RemoteSaveRequest saved", "reason", apimeta.FindStatusCondition(status.Conditions, "Ready").Reason, - "author", status.Author) + log.Info("RemoteSaveRequest saved", "reason", readyReason(status), "author", status.Author) if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && readyReason(status) == saveReasonCommitted { - s.followApplies(ctx, request, status, attaches, write) + s.followApply(ctx, request, status, attach, write) } } -// resolveSaveAuthor finds the request's creator from the source cluster's audit facts, as a watched -// edit's author is found. -func (s *RemoteSaveRequestServer) resolveSaveAuthor( - ctx context.Context, route string, request *sourcev1alpha1.RemoteSaveRequest, -) (git.UserInfo, git.AttributionOutcome) { +// saveAuthor is the request's creator, found from the source cluster's audit facts as a watched edit's +// author is. refused reports a request no fact names on a provider that has received audit facts. +// +// A request already being saved was attributed when it was first served, and its status records the +// answer. It is not looked up again: the status writes since have moved its resourceVersion, so its +// audit fact would no longer match, and on an audited cluster it would be refused for that. +func (s *RemoteSaveRequestServer) saveAuthor( + ctx context.Context, watcher *remoteSaveRequestWatcher, request *sourcev1alpha1.RemoteSaveRequest, + status *sourcev1alpha1.RemoteSaveRequestStatus, +) (string, git.AttributionOutcome, bool) { + if apimeta.FindStatusCondition(status.Conditions, "Ready") != nil { + if status.Author != "" { + return status.Author, git.AttributionResolved, false + } + return "", git.AttributionUnresolved, false + } if s.Manager == nil || s.Manager.AuthorResolver == nil { - return git.UserInfo{}, git.AttributionNotAttempted + return "", git.AttributionNotAttempted, false } - return s.Manager.AuthorResolver.ResolveAuthor(ctx, AuthorQuery{ - AuditRoute: route, + user, attribution := s.Manager.AuthorResolver.ResolveAuthor(ctx, AuthorQuery{ + AuditRoute: watcher.route, GVR: remoteSaveRequestGVR(), UID: request.UID, ResourceVersion: request.ResourceVersion, @@ -421,6 +430,9 @@ func (s *RemoteSaveRequestServer) resolveSaveAuthor( Name: request.Name, ExactCapable: true, }) + refused := !attribution.NamesActor() && attribution != git.AttributionNotAttempted && + s.sentAuditFacts(ctx, watcher.provider) + return user.Username, attribution, refused } // sentAuditFacts reports whether the KubeProvider has ever received an audit fact: its @@ -433,72 +445,62 @@ func (s *RemoteSaveRequestServer) sentAuditFacts(ctx context.Context, provider k return apimeta.IsStatusConditionTrue(kp.Status.Conditions, "AuditFactsReceived") } -// admittingTargets are the GitTargets a request saves: they name the KubeProvider, mirror the -// request's namespace, and admit the request in spec.allowedRemoteSaveRequests. mirrored reports -// whether any GitTarget mirrors the namespace, admitting or not. -func (s *RemoteSaveRequestServer) admittingTargets( - ctx context.Context, provider k8stypes.NamespacedName, request *sourcev1alpha1.RemoteSaveRequest, -) ([]types.ResourceReference, bool, error) { - var targets configv1alpha3.GitTargetList - if err := s.Client.List(ctx, &targets); err != nil { - return nil, false, err - } - var out []types.ResourceReference - mirrored := false - for i := range targets.Items { - target := &targets.Items[i] - if target.DeletionTimestamp != nil || target.KubeProviderKey() != provider { - continue +// admit decides whether the GitTarget the request names takes it: it exists, mirrors the cluster the +// request was served from, and admits the request's namespace and labels in +// spec.allowedRemoteSaveRequests. A refusal is a reason and a message; err is a read to retry. A +// GitTarget that does not exist is waited for up to the request's attachTimeout, since it may be +// created alongside the request. +func (s *RemoteSaveRequestServer) admit( + ctx context.Context, watcher *remoteSaveRequestWatcher, request *sourcev1alpha1.RemoteSaveRequest, +) (string, string, error) { + ref := request.Spec.GitTargetRef + key := k8stypes.NamespacedName{Namespace: ref.Namespace, Name: ref.Name} + deadline := time.Now().Add(remoteSaveRequestAttach(request, "", "").AttachTimeout) + var target configv1alpha3.GitTarget + for { + err := s.Client.Get(ctx, key, &target) + if err == nil && target.DeletionTimestamp == nil { + break } - ref := types.NewResourceReference(target.Name, target.Namespace) - if !s.Manager.mirrorsNamespace(ref, request.Namespace) { - continue + if err != nil && !apierrors.IsNotFound(err) { + return "", "", err } - mirrored = true - admitted, err := target.AdmitsRemoteSaveRequest(request.Namespace, request.Labels) - if err != nil { - s.Log.Info("A GitTarget's spec.allowedRemoteSaveRequests cannot be read; it admits nothing", - "gitTarget", ref.String(), "err", err.Error()) + if !time.Now().Before(deadline) { + return saveReasonGitTargetNotFound, fmt.Sprintf("GitTarget %s does not exist", key), nil } - if admitted { - out = append(out, ref) + select { + case <-ctx.Done(): + return "", "", ctx.Err() + case <-time.After(remoteSaveRequestPoll): } } - sort.Slice(out, func(i, j int) bool { return out[i].String() < out[j].String() }) - return out, mirrored, nil -} - -// mirrorsNamespace reports whether gitDest mirrors a namespaced type in namespace: a WatchRule -// watches the type there, or a rule watches it across every namespace. A cluster-scoped type is in -// no namespace, so a GitTarget that mirrors only those mirrors none. -func (m *Manager) mirrorsNamespace(gitDest types.ResourceReference, namespace string) bool { - for _, wt := range m.residentWatchedTypeTable(gitDest).Types { - if !wt.Namespaced { - continue - } - _, named := wt.NamespaceScopes[namespace] - _, everywhere := wt.NamespaceScopes[""] - if named || everywhere { - return true + if target.KubeProviderKey() != watcher.provider { + return saveReasonNotAllowed, fmt.Sprintf("GitTarget %s mirrors another cluster than this one", key), nil + } + admitted, err := target.AdmitsRemoteSaveRequest(request.Namespace, request.Labels) + if !admitted { + message := fmt.Sprintf("GitTarget %s does not admit RemoteSaveRequests from namespace %q, with these "+ + "labels, in spec.allowedRemoteSaveRequests", key, request.Namespace) + if err != nil { + message += fmt.Sprintf(" (a grant's requestSelector is invalid: %v)", err) } + return saveReasonNotAllowed, message, nil } - return false + return "", "", nil } -// remoteSaveRequestAttach is the attach of a request to one GitTarget's window. Its identity carries the GitTarget, -// so one request saving several GitTargets on one branch worker is several requests there. +// remoteSaveRequestAttach is the attach of a request to its GitTarget's window. func remoteSaveRequestAttach( - request *sourcev1alpha1.RemoteSaveRequest, target types.ResourceReference, author string, - attribution git.AttributionOutcome, + request *sourcev1alpha1.RemoteSaveRequest, author string, attribution git.AttributionOutcome, ) git.AttachSaveRequest { attach := git.AttachSaveRequest{ Namespace: request.Namespace, Name: request.Name, - UID: string(request.UID) + "@" + target.String(), + UID: string(request.UID), Author: author, Attribution: attribution, - GitTargetName: target.Name, - GitTargetNamespace: target.Namespace, + GitTargetName: request.Spec.GitTargetRef.Name, + GitTargetNamespace: request.Spec.GitTargetRef.Namespace, Message: request.Spec.Message, Attach: configv1alpha3.AttachCurrentOrNext, AttachTimeout: defaultSaveAttachTimeout, @@ -520,170 +522,119 @@ func remoteSaveRequestAttach( return attach } -// finalize polls every GitTarget's outcome until each one resolved, withdrawing those still unheld past -// the request's bound, and records them on status. +// finalize polls the outcome until the worker resolved the request, withdrawing it once it is past its +// bound and still unheld, and records it on status. It reports false when ctx ended first. func (s *RemoteSaveRequestServer) finalize( ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attaches []git.AttachSaveRequest, -) { - bound := time.Now().Add(attaches[0].AttachTimeout + attaches[0].MaxDuration + remoteSaveRequestPushAllowance) - resolved := make([]bool, len(attaches)) - for remaining := len(attaches); remaining > 0; { - for i := range attaches { - if resolved[i] { - continue - } - ask := s.Finalizer.ServiceSaveRequest - if time.Now().After(bound) { - ask = s.Finalizer.WithdrawSaveRequest - } - result, done, err := ask(ctx, attaches[i]) - if err != nil || !done { - continue - } - resolved[i] = true - remaining-- - recordSaveOutcome(&status.GitTargets[i], result) + attach git.AttachSaveRequest, +) bool { + bound := time.Now().Add(attach.AttachTimeout + attach.MaxDuration + remoteSaveRequestPushAllowance) + for { + ask := s.Finalizer.ServiceSaveRequest + if time.Now().After(bound) { + ask = s.Finalizer.WithdrawSaveRequest } - if remaining == 0 { - break + if result, done, err := ask(ctx, attach); err == nil && done { + recordSaveOutcome(request, status, result) + return true } select { case <-ctx.Done(): - return + return false case <-time.After(remoteSaveRequestPoll): } } - setSaveReady(request, status) } -// recordSaveOutcome records one GitTarget's outcome on its target status. -func recordSaveOutcome(target *sourcev1alpha1.RemoteSaveRequestGitTarget, result git.FinalizeResult) { - target.Branch = result.Branch +// recordSaveOutcome records the worker's outcome: Ready, the branch and the commit. +func recordSaveOutcome( + request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, + result git.FinalizeResult, +) { + status.Branch = result.Branch if result.Err != nil { - target.Reason, target.Message = saveReasonFailed, result.Err.Error() + message := result.Err.Error() if errors.Is(result.Err, git.ErrSaveRequestWithdrawn) { - target.Message = "the save was not served in time and was withdrawn" + message = "the save was not served in time and was withdrawn" } + failSave(request, status, saveReasonFailed, message) return } + ready := func(reason, message string) { + setSaveCondition(request, status, "Ready", metav1.ConditionTrue, reason, message) + } switch result.Outcome { case git.FinalizeCommitted: - target.Reason, target.Commit = saveReasonCommitted, result.Commit - target.Message = "the edits were committed and pushed" + status.Commit = result.Commit + ready(saveReasonCommitted, "the edits were committed and pushed") case git.FinalizeNoOpenWindow: - target.Reason = saveReasonNoWindow - target.Message = "no edit of this request's author reached a commit window before attachTimeout" + ready(saveReasonNoWindow, "no edit of this request's author reached a commit window before attachTimeout") case git.FinalizeWindowMismatch: - target.Reason = saveReasonWindowMismatch - target.Message = "only another author's commit window was open" + ready(saveReasonWindowMismatch, "only another author's commit window was open") case git.FinalizeAlreadyPresent: - target.Reason = saveReasonAlreadyPresent - target.Message = "the edits were in Git already" + ready(saveReasonAlreadyPresent, "the edits were in Git already") + case git.FinalizeSaveInProgress: + failSave(request, status, saveReasonSaveInProgress, "an earlier save of the same author on this GitTarget "+ + "was still waiting for or collecting a commit window, which carries one message; save again once it "+ + "resolved") default: - target.Reason, target.Message = saveReasonFailed, "unexpected outcome: "+string(result.Outcome) + failSave(request, status, saveReasonFailed, "unexpected outcome: "+string(result.Outcome)) } } -// setSaveReady sets Ready from the targets' outcomes: a failure in any fails the request, a commit in -// any is Committed, and otherwise the first target's reason says why nothing was committed. -func setSaveReady(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { - for _, target := range status.GitTargets { - if target.Reason == saveReasonFailed { - message := gitTargetName(target) + ": " + target.Message - setSaveCondition(request, status, "Ready", metav1.ConditionFalse, saveReasonFailed, message) - setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, saveReasonFailed, message) - return - } - } - for _, target := range status.GitTargets { - if target.Reason == saveReasonCommitted { - setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonCommitted, target.Message) - return - } - } - first := status.GitTargets[0] - setSaveCondition(request, status, "Ready", metav1.ConditionTrue, first.Reason, first.Message) +// failSave records a request that ends without saving and cannot recover: Ready=False, Stalled=True. +func failSave( + request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, reason, message string, +) { + setSaveCondition(request, status, "Ready", metav1.ConditionFalse, reason, message) + setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, reason, message) } -// followApplies polls the apply each committed save asked for until every one is settled, and records -// them; a save its worker knows nothing of for remoteSaveRequestApplyLost is lost. -func (s *RemoteSaveRequestServer) followApplies( +// followApply polls the apply the committed save asked for until it is settled, and records it on the +// Applied condition; a save its worker knows nothing of for remoteSaveRequestApplyLost is Lost. +func (s *RemoteSaveRequestServer) followApply( ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attaches []git.AttachSaveRequest, write func(), + attach git.AttachSaveRequest, write func(), ) { - setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, - "waiting for the appliers to apply the commit") - write() + if apimeta.FindStatusCondition(status.Conditions, "Applied") == nil { + setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, + "waiting for the applier to apply the commit") + write() + } started := time.Now() - for s.pollApplies(ctx, status, attaches, time.Since(started) > remoteSaveRequestApplyLost) { + for { + apply, known, err := s.Finalizer.LookupSaveRequestApply(ctx, attach) + switch { + case err == nil && known && (apply.Done || apply.Applied): + status.AppliedRevision = apply.AppliedRevision + setSaveApplied(request, status, apply.Reason) + write() + return + case err == nil && !known && time.Since(started) > remoteSaveRequestApplyLost: + setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonLost, + "GitOps Reverser does not know how the apply went") + write() + return + } select { case <-ctx.Done(): return case <-time.After(remoteSaveRequestPoll): } } - setSaveApplied(request, status) - write() -} - -// pollApplies asks once for the apply of every committed target not settled yet, records it, and -// reports whether one is still pending. Once lost, a save its worker knows nothing of is lost. -func (s *RemoteSaveRequestServer) pollApplies( - ctx context.Context, status *sourcev1alpha1.RemoteSaveRequestStatus, attaches []git.AttachSaveRequest, lost bool, -) bool { - pending := false - for i := range attaches { - target := &status.GitTargets[i] - if target.Reason != saveReasonCommitted || saveApplySettled(target.Applied) { - continue - } - apply, known, err := s.Finalizer.LookupSaveRequestApply(ctx, attaches[i]) - switch { - case err == nil && known: - target.Applied, target.AppliedRevision = apply.Reason, apply.AppliedRevision - pending = pending || !apply.Done && !apply.Applied - case err == nil && lost: - target.Applied = "Lost" - default: - pending = true - } - } - return pending } -// saveApplySettled reports whether a target's apply reason changes no more. -func saveApplySettled(reason string) bool { - switch reason { - case git.SaveApplyApplied, git.SaveApplyNothingToApply, git.SaveApplyNoApplier, git.SaveApplyApplierGone, - git.SaveApplyCommitDropped, "Lost": - return true - } - return false -} - -// setSaveApplied sets Applied from the committed targets' applies: False only for a commit a branch -// lost, Unknown when one was lost track of, and True otherwise. -func setSaveApplied(request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus) { - reason, condition := saveReasonApplied, metav1.ConditionTrue - message := "every applier applied a revision that holds the commit, or needed no apply" - for _, target := range status.GitTargets { - switch target.Applied { - case git.SaveApplyCommitDropped: - setSaveCondition(request, status, "Applied", metav1.ConditionFalse, target.Applied, - gitTargetName(target)+": the branch no longer holds the commit") - return - case "Lost": - reason, condition = "Lost", metav1.ConditionUnknown - message = gitTargetName(target) + ": GitOps Reverser does not know how the apply went" - } +// setSaveApplied records a settled apply: False only for a commit the branch lost, True otherwise. +func setSaveApplied( + request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, reason string, +) { + if reason == git.SaveApplyCommitDropped { + setSaveCondition(request, status, "Applied", metav1.ConditionFalse, reason, + "the branch no longer holds the commit") + return } - setSaveCondition(request, status, "Applied", condition, reason, message) -} - -// gitTargetName is a target status's GitTarget as namespace/name, for a message. -func gitTargetName(target sourcev1alpha1.RemoteSaveRequestGitTarget) string { - return target.Namespace + "/" + target.Name + setSaveCondition(request, status, "Applied", metav1.ConditionTrue, reason, + "the applier applied a revision that holds the commit, or needed no apply") } func readyReason(status *sourcev1alpha1.RemoteSaveRequestStatus) string { diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go index 38e40088..1f08cdbe 100644 --- a/internal/watch/remote_save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -5,7 +5,6 @@ package watch import ( "context" "errors" - "fmt" "sync" "testing" "time" @@ -23,12 +22,12 @@ import ( k8stypes "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/dynamic" dynamicfake "k8s.io/client-go/dynamic/fake" + "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/client/fake" sourcev1alpha1 "github.com/ConfigButler/gitops-reverser/api/source/v1alpha1" configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/types" ) // saveFinalizer answers every attach and apply lookup the way the test sets, and records them. @@ -72,36 +71,26 @@ func (a fixedAuthor) ResolveAuthor(context.Context, AuthorQuery) (git.UserInfo, var saveProvider = k8stypes.NamespacedName{Namespace: "providers", Name: "workspace"} -// saveSetup is what a save is served against: the GitTarget team-a/app of the KubeProvider -// providers/workspace, the type it mirrors, the grants it holds, and what the source cluster's audit -// says. +// saveSetup is what a save is served against: the GitTarget team-a/app, when it exists, its +// KubeProvider and the grants it holds, and what the source cluster's audit says. type saveSetup struct { - // mirrors is the type the GitTarget mirrors; nil mirrors nothing. - mirrors *WatchedType + // noTarget leaves team-a/app out. + noTarget bool + // provider is the GitTarget's KubeProvider; empty is providers/workspace, the one serving the request. + provider string grants []configv1alpha3.GitTargetRemoteSaveRequestGrant auditFacts metav1.ConditionStatus author AuthorResolver } -// configMapsIn is ConfigMaps watched in namespace, or in every namespace for "". -func configMapsIn(namespace string) *WatchedType { - return &WatchedType{ - GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, - Namespaced: true, - Scope: configv1alpha3.ResourceScopeNamespaced, - NamespaceScopes: map[string]string{namespace: ""}, - } -} - // grantFrom admits the RemoteSaveRequests of namespaces. func grantFrom(namespaces ...configv1alpha3.SourceNamespace) configv1alpha3.GitTargetRemoteSaveRequestGrant { return configv1alpha3.GitTargetRemoteSaveRequestGrant{SourceNamespaces: namespaces} } -// shopSetup is a GitTarget that mirrors ConfigMaps in "shop" and admits that namespace's requests. +// shopSetup is a GitTarget that admits the requests of the namespace "shop". func shopSetup(auditFacts metav1.ConditionStatus, author AuthorResolver) saveSetup { return saveSetup{ - mirrors: configMapsIn("shop"), grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, auditFacts: auditFacts, author: author, @@ -121,24 +110,24 @@ func saveServer( {Type: "AuditFactsReceived", Status: setup.auditFacts, Reason: "Test"}, } } - target := &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a"}, - Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, - AllowedRemoteSaveRequests: setup.grants, - }, + objects := []client.Object{provider} + if !setup.noTarget { + providerName := setup.provider + if providerName == "" { + providerName = "workspace" + } + objects = append(objects, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a"}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{Name: providerName, Namespace: "providers"}, + AllowedRemoteSaveRequests: setup.grants, + }, + }) } - c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(provider, target). + c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(objects...). WithStatusSubresource(&configv1alpha3.KubeProvider{}).Build() manager := &Manager{AuthorResolver: setup.author} - if setup.mirrors != nil { - gitDest := types.NewResourceReference("app", "team-a") - manager.ensureWatchedTypeStore() - manager.watchedTypes.tables = map[string]WatchedTypeTable{ - gitDest.Key(): {GitDest: gitDest, Types: []WatchedType{*setup.mirrors}}, - } - } fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(request) require.NoError(t, err) obj := &unstructured.Unstructured{Object: fields} @@ -157,8 +146,9 @@ func newRemoteSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1. return &sourcev1alpha1.RemoteSaveRequest{ ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: "shop", UID: "uid-save", Generation: 1}, Spec: sourcev1alpha1.RemoteSaveRequestSpec{ - Message: "save: from the editor", - CatchUp: catchUp, + GitTargetRef: sourcev1alpha1.GitTargetReference{Namespace: "team-a", Name: "app"}, + Message: "save: from the editor", + CatchUp: catchUp, Window: &sourcev1alpha1.RemoteSaveRequestWindow{ AttachTimeout: &metav1.Duration{Duration: 3 * time.Second}, MaxDuration: &metav1.Duration{Duration: time.Second}, @@ -188,9 +178,9 @@ func condition(status sourcev1alpha1.RemoteSaveRequestStatus, conditionType stri return apimeta.FindStatusCondition(status.Conditions, conditionType) } -// A save attaches to the window of every GitTarget that mirrors its namespace, as its author found -// from the source cluster's audit, and reports each commit and the apply it asked for. -func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *testing.T) { +// A save attaches to its creator's window on the GitTarget it names, as its author found from the +// source cluster's audit, and reports the commit and the apply it asked for. +func TestRemoteSaveRequest_SavesTheNamedGitTargetAndReportsTheApply(t *testing.T) { finalizer := &saveFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, apply: git.SaveApply{ @@ -215,19 +205,17 @@ func TestRemoteSaveRequest_SavesEachMirroringGitTargetAndReportsTheApply(t *test assert.Equal(t, "save: from the editor", attach.Message) assert.Equal(t, 3*time.Second, attach.AttachTimeout) assert.True(t, attach.CatchUpNow) - assert.Equal(t, "uid-save@team-a/app", attach.UID, "one identity per GitTarget") assert.Equal(t, "alice@example.com", status.Author) - require.Len(t, status.GitTargets, 1) - assert.Equal(t, sourcev1alpha1.RemoteSaveRequestGitTarget{ - Namespace: "team-a", Name: "app", Reason: saveReasonCommitted, Message: "the edits were committed and pushed", - Branch: "main", Commit: "c0ffee", Applied: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", - }, status.GitTargets[0]) + assert.Equal(t, "main", status.Branch) + assert.Equal(t, "c0ffee", status.Commit) + assert.Equal(t, "main@sha1:c0ffee", status.AppliedRevision) require.NotNil(t, condition(status, "Ready")) assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) require.NotNil(t, condition(status, "Applied")) assert.Equal(t, metav1.ConditionTrue, condition(status, "Applied").Status) + assert.Equal(t, git.SaveApplyApplied, condition(status, "Applied").Reason) } // On a cluster with no audit stream a save names nobody, and attaches as such: to a window whose @@ -262,40 +250,9 @@ func TestRemoteSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testin assert.Equal(t, saveReasonAuthorUnresolved, condition(status, "Ready").Reason) } -// A save in a namespace no GitTarget mirrors has nothing to save. A cluster-scoped type is in no -// namespace, so a GitTarget mirroring only one mirrors none, whatever its grant admits. -func TestRemoteSaveRequest_NoGitTargetMirrorsTheNamespace(t *testing.T) { - clusterRoles := &WatchedType{ - GVR: schema.GroupVersionResource{ - Group: "rbac.authorization.k8s.io", - Version: "v1", - Resource: "clusterroles", - }, - Scope: configv1alpha3.ResourceScopeCluster, - NamespaceScopes: map[string]string{"": ""}, - } - everything := []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("*")} - for name, setup := range map[string]saveSetup{ - "nothing mirrored": {grants: everything}, - "another namespace": {mirrors: configMapsIn("billing"), grants: everything}, - "only a cluster-scoped type": {mirrors: clusterRoles, grants: everything}, - } { - t.Run(name, func(t *testing.T) { - finalizer := &saveFinalizer{} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, setup, finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Empty(t, finalizer.attaches) - assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) - assert.Equal(t, saveReasonNoGitTarget, condition(status, "Ready").Reason) - }) - } -} - -// A GitTarget that mirrors the namespace saves a request only when one of its grants admits both the -// request's namespace and its labels; nothing is admitted by default. +// The GitTarget the request names takes it only when it exists, mirrors the cluster the request came +// from, and has a grant admitting both the request's namespace and its labels; nothing is admitted by +// default. func TestRemoteSaveRequest_TheGitTargetMustAdmitIt(t *testing.T) { selector := func(labels map[string]string) *metav1.LabelSelector { return &metav1.LabelSelector{MatchLabels: labels} @@ -305,79 +262,68 @@ func TestRemoteSaveRequest_TheGitTargetMustAdmitIt(t *testing.T) { grant.RequestSelector = selector(labels) return grant } + grants := func(g ...configv1alpha3.GitTargetRemoteSaveRequestGrant) []configv1alpha3.GitTargetRemoteSaveRequestGrant { + return g + } tests := map[string]struct { - grants []configv1alpha3.GitTargetRemoteSaveRequestGrant - admitted bool + setup saveSetup + reason string }{ - "no grant": {}, - "another namespace": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("billing")}}, - "its namespace": { - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, - admitted: true, - }, - "every namespace": { - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("*")}, - admitted: true, + "no grant": {setup: saveSetup{}, reason: saveReasonNotAllowed}, + "another namespace": { + setup: saveSetup{grants: grants(grantFrom("billing"))}, + reason: saveReasonNotAllowed, }, + "its namespace": {setup: saveSetup{grants: grants(grantFrom("shop"))}}, + "every namespace": {setup: saveSetup{grants: grants(grantFrom("*"))}}, "its namespace and labels": { - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{shopLabelled(map[string]string{"app": "cart"})}, - admitted: true, + setup: saveSetup{grants: grants(shopLabelled(map[string]string{"app": "cart"}))}, }, "its namespace, other labels": { - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{shopLabelled(map[string]string{"app": "admin"})}, + setup: saveSetup{grants: grants(shopLabelled(map[string]string{"app": "admin"}))}, + reason: saveReasonNotAllowed, + }, + "one complete grant of two": {setup: saveSetup{grants: grants( + grantFrom("billing"), shopLabelled(map[string]string{"app": "cart"}), + )}}, + "a selector that does not convert admits nothing": {setup: saveSetup{grants: grants( + configv1alpha3.GitTargetRemoteSaveRequestGrant{ + SourceNamespaces: []configv1alpha3.SourceNamespace{"shop"}, + RequestSelector: &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ + {Key: "app", Operator: "Near", Values: []string{"cart"}}, + }}, + }, + )}, reason: saveReasonNotAllowed}, + "a GitTarget that mirrors another cluster": { + setup: saveSetup{provider: "elsewhere", grants: grants(grantFrom("*"))}, reason: saveReasonNotAllowed, }, - "one complete grant of two": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{ - {SourceNamespaces: []configv1alpha3.SourceNamespace{"billing"}}, - shopLabelled(map[string]string{"app": "cart"}), - }, admitted: true}, - "a selector that does not convert admits nothing": {grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{{ - SourceNamespaces: []configv1alpha3.SourceNamespace{"shop"}, - RequestSelector: &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ - {Key: "app", Operator: "Near", Values: []string{"cart"}}, - }}, - }}}, + "no such GitTarget": {setup: saveSetup{noTarget: true}, reason: saveReasonGitTargetNotFound}, } for name, tt := range tests { t.Run(name, func(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) request.Labels = map[string]string{"app": "cart"} - setup := shopSetup("", nil) - setup.grants = tt.grants - server, resource := saveServer(t, request, setup, finalizer) + request.Spec.Window.AttachTimeout = &metav1.Duration{} + server, resource := saveServer(t, request, tt.setup, finalizer) status := serveOnce(t, server, resource, request) - if tt.admitted { + if tt.reason == "" { assert.Len(t, finalizer.attaches, 1) assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) return } assert.Empty(t, finalizer.attaches, "nothing is attached") - assert.Empty(t, status.GitTargets) assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) - assert.Equal(t, saveReasonNotAllowed, condition(status, "Ready").Reason) - assert.Contains(t, condition(status, "Ready").Message, "spec.allowedRemoteSaveRequests") + assert.Equal(t, tt.reason, condition(status, "Ready").Reason) + assert.Contains(t, condition(status, "Ready").Message, "team-a/app") assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) }) } } -// A GitTarget that watches a type across every namespace mirrors the request's. -func TestRemoteSaveRequest_AWatchOfEveryNamespaceMirrorsIt(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - setup := shopSetup("", nil) - setup.mirrors = configMapsIn("") - server, resource := saveServer(t, request, setup, finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Len(t, finalizer.attaches, 1) - assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) -} - -// A save that failed in a GitTarget fails, and says where. +// A save that failed fails the request, and says why. func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) @@ -387,11 +333,56 @@ func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) assert.Equal(t, saveReasonFailed, condition(status, "Ready").Reason) - assert.Contains(t, condition(status, "Ready").Message, "team-a/app: the GitTarget is suspended") + assert.Contains(t, condition(status, "Ready").Message, "the GitTarget is suspended") assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) assert.Nil(t, condition(status, "Applied"), "nothing was committed to apply") } +// A request served again, after a restart or a dropped watch, keeps what the first serving decided. +// One being saved keeps its author: its status writes moved its resourceVersion, so its audit fact can +// no longer be found. One already committed keeps its commit, and only its apply is followed. +func TestRemoteSaveRequest_ServedAgainKeepsWhatItDecided(t *testing.T) { + t.Run("one being saved keeps its author", func(t *testing.T) { + finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + request.Status.Author = "alice@example.com" + request.Status.Conditions = []metav1.Condition{ + {Type: "Ready", Status: metav1.ConditionUnknown, Reason: saveReasonSaving}, + } + server, resource := saveServer(t, request, + shopSetup(metav1.ConditionTrue, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) + + status := serveOnce(t, server, resource, request) + + require.Len(t, finalizer.attaches, 1, "it is not refused for a fact it can no longer match") + assert.Equal(t, "alice@example.com", finalizer.attaches[0].Author) + assert.Equal(t, git.AttributionResolved, finalizer.attaches[0].Attribution) + assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) + }) + + t.Run("one committed keeps its commit and follows its apply", func(t *testing.T) { + finalizer := &saveFinalizer{ + result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow}, + apply: git.SaveApply{Done: true, Reason: git.SaveApplyNothingToApply}, + } + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) + request.Status.Commit = "c0ffee" + request.Status.Conditions = []metav1.Condition{ + {Type: "Ready", Status: metav1.ConditionTrue, Reason: saveReasonCommitted}, + {Type: "Applied", Status: metav1.ConditionUnknown, Reason: saveReasonApplying}, + } + server, resource := saveServer(t, request, shopSetup("", nil), finalizer) + + status := serveOnce(t, server, resource, request) + + assert.Empty(t, finalizer.attaches, "a decided save is never attached again") + assert.Equal(t, "c0ffee", status.Commit) + assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) + assert.Equal(t, metav1.ConditionTrue, condition(status, "Applied").Status) + assert.Equal(t, git.SaveApplyNothingToApply, condition(status, "Applied").Reason) + }) +} + // A request is finished once Ready is decided, and, when it committed and asked to catch up, once // Applied is decided too. func TestRemoteSaveRequest_Finished(t *testing.T) { @@ -416,7 +407,7 @@ func TestRemoteSaveRequest_Finished(t *testing.T) { metav1.Condition{ Type: "Applied", Status: metav1.ConditionTrue, - Reason: saveReasonApplied, + Reason: git.SaveApplyApplied, LastTransitionTime: at, }, ) @@ -538,51 +529,46 @@ func TestRemoteSaveRequest_AFinishedRequestIsDeletedAfterTheTTL(t *testing.T) { } } -// Each worker outcome is recorded on its GitTarget's status with the reason the API documents. +// Each worker outcome is recorded with the reason the API documents, and the refusals and failures +// are Ready=False, Stalled=True. func TestRemoteSaveRequest_RecordsEachOutcome(t *testing.T) { - for result, want := range map[*git.FinalizeResult]string{ - {Outcome: git.FinalizeCommitted, Commit: "c0ffee"}: saveReasonCommitted, - {Outcome: git.FinalizeNoOpenWindow}: saveReasonNoWindow, - {Outcome: git.FinalizeWindowMismatch}: saveReasonWindowMismatch, - {Outcome: git.FinalizeAlreadyPresent}: saveReasonAlreadyPresent, - {Outcome: "Unknown"}: saveReasonFailed, - {Err: errors.New("boom")}: saveReasonFailed, - {Err: git.ErrSaveRequestWithdrawn}: saveReasonFailed, + for result, want := range map[*git.FinalizeResult]struct { + reason string + ready metav1.ConditionStatus + }{ + {Outcome: git.FinalizeCommitted, Commit: "c0ffee"}: {saveReasonCommitted, metav1.ConditionTrue}, + {Outcome: git.FinalizeNoOpenWindow}: {saveReasonNoWindow, metav1.ConditionTrue}, + {Outcome: git.FinalizeWindowMismatch}: {saveReasonWindowMismatch, metav1.ConditionTrue}, + {Outcome: git.FinalizeAlreadyPresent}: {saveReasonAlreadyPresent, metav1.ConditionTrue}, + {Outcome: git.FinalizeSaveInProgress}: {saveReasonSaveInProgress, metav1.ConditionFalse}, + {Outcome: "Unknown"}: {saveReasonFailed, metav1.ConditionFalse}, + {Err: errors.New("boom")}: {saveReasonFailed, metav1.ConditionFalse}, + {Err: git.ErrSaveRequestWithdrawn}: {saveReasonFailed, metav1.ConditionFalse}, } { - var target sourcev1alpha1.RemoteSaveRequestGitTarget - recordSaveOutcome(&target, *result) - assert.Equal(t, want, target.Reason, "%+v", *result) - assert.NotEmpty(t, target.Message) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + status := &sourcev1alpha1.RemoteSaveRequestStatus{} + recordSaveOutcome(request, status, *result) + ready := condition(*status, "Ready") + require.NotNil(t, ready) + assert.Equal(t, want.reason, ready.Reason, "%+v", *result) + assert.Equal(t, want.ready, ready.Status, "%+v", *result) + assert.NotEmpty(t, ready.Message) + assert.Equal(t, want.ready == metav1.ConditionFalse, condition(*status, "Stalled") != nil, "%+v", *result) } } -// Applied is False only for a commit a branch lost, Unknown when an apply was lost track of, and True -// otherwise. -func TestRemoteSaveRequest_AppliedSummarizesTheTargets(t *testing.T) { - for name, tt := range map[string]struct { - applied []string - status metav1.ConditionStatus - reason string - }{ - "all applied": {[]string{git.SaveApplyApplied, git.SaveApplyNothingToApply}, metav1.ConditionTrue, saveReasonApplied}, - "one lost": {[]string{git.SaveApplyApplied, "Lost"}, metav1.ConditionUnknown, "Lost"}, - "one commit dropped": {[]string{"Lost", git.SaveApplyCommitDropped}, metav1.ConditionFalse, git.SaveApplyCommitDropped}, +// Applied is False only for a commit the branch lost, and True for every other settled apply. +func TestRemoteSaveRequest_AppliedIsFalseOnlyForADroppedCommit(t *testing.T) { + for reason, want := range map[string]metav1.ConditionStatus{ + git.SaveApplyApplied: metav1.ConditionTrue, + git.SaveApplyNothingToApply: metav1.ConditionTrue, + git.SaveApplyNoApplier: metav1.ConditionTrue, + git.SaveApplyCommitDropped: metav1.ConditionFalse, } { - t.Run(name, func(t *testing.T) { - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - status := &sourcev1alpha1.RemoteSaveRequestStatus{} - for i, applied := range tt.applied { - status.GitTargets = append(status.GitTargets, sourcev1alpha1.RemoteSaveRequestGitTarget{ - Namespace: "team-a", Name: fmt.Sprintf("app-%d", i), Reason: saveReasonCommitted, Applied: applied, - }) - } - setSaveApplied(request, status) - got := condition(*status, "Applied") - require.NotNil(t, got) - assert.Equal(t, tt.status, got.Status) - assert.Equal(t, tt.reason, got.Reason) - assert.True(t, saveApplySettled(tt.applied[0])) - }) + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) + status := &sourcev1alpha1.RemoteSaveRequestStatus{} + setSaveApplied(request, status, reason) + assert.Equal(t, want, condition(*status, "Applied").Status, reason) + assert.Equal(t, reason, condition(*status, "Applied").Reason) } - assert.False(t, saveApplySettled("Applying")) } diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index a7c71be6..930481f1 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -870,10 +870,9 @@ spec: }) // A save made in the workspace itself: a RemoteSaveRequest, served because the workspace's - // KubeProvider enables them, and saving only for a GitTarget that admits it. It names no - // GitTarget; it saves the window of every admitting GitTarget that mirrors its namespace, first - // refused while the GitTarget admits none. kcp sends no audit facts, so the request names nobody, like the edit, - // and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is + // KubeProvider enables them. It names the GitTarget, which must admit it: one is refused while + // the GitTarget admits none. kcp sends no audit facts, so the request names nobody, like the + // edit, and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is // looked for first, for the whole attribution grace, so the edit is made once the request is // waiting for a window. It("saves an edit made in the workspace with a RemoteSaveRequest created there", func() { @@ -908,7 +907,7 @@ spec: By("refusing a RemoteSaveRequest while the GitTarget admits none") const refused = "sc-flux-workspace-save-refused" - createWorkspaceRemoteSaveRequest(kcp, hash, testNs, refused, "e2e: not admitted") + createWorkspaceRemoteSaveRequest(kcp, hash, testNs, refused, "e2e: not admitted", testNs, p.targetName) Eventually(func(g Gomega) { g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("False")) g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].reason}`)). @@ -922,7 +921,7 @@ spec: By("creating a RemoteSaveRequest in the workspace, and editing once it waits for a window") const name, message = "sc-flux-workspace-save", "e2e: save from the workspace" - createWorkspaceRemoteSaveRequest(kcp, hash, testNs, name, message) + createWorkspaceRemoteSaveRequest(kcp, hash, testNs, name, message, testNs, p.targetName) saveField := func(g Gomega, jsonPath string) string { return requestField(g, name, jsonPath) } Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Saving")) @@ -934,9 +933,7 @@ spec: Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("True")) g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Committed")) - g.Expect(saveField(g, "{.status.gitTargets[0].namespace}")).To(Equal(testNs)) - g.Expect(saveField(g, "{.status.gitTargets[0].name}")).To(Equal(p.targetName)) - commit = saveField(g, "{.status.gitTargets[0].commit}") + commit = saveField(g, "{.status.commit}") g.Expect(commit).NotTo(BeEmpty()) g.Expect(saveField(g, "{.status.author}")).To(BeEmpty(), "kcp sends no audit facts") }, 2*time.Minute, 2*time.Second).Should(Succeed()) @@ -949,8 +946,8 @@ spec: Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Applied")].status}`)).To(Equal("True")) - g.Expect(saveField(g, "{.status.gitTargets[0].applied}")).To(Equal("Applied")) - g.Expect(saveField(g, "{.status.gitTargets[0].appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Applied")].reason}`)).To(Equal("Applied")) + g.Expect(saveField(g, "{.status.appliedRevision}")).To(ContainSubstring(commit)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(commit)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.consistentlyEverywhere(timingConfigMap, "saved-3") @@ -958,10 +955,10 @@ spec: }) }) -// createWorkspaceRemoteSaveRequest creates the RemoteSaveRequest name in namespace of a workspace, asking to catch -// up. It waits up to 30s for a window: an edit from a cluster with no audit stream reaches its window +// createWorkspaceRemoteSaveRequest creates the RemoteSaveRequest name in namespace of a workspace, saving the +// GitTarget targetNS/target and asking to catch up. It waits up to 30s for a window: an edit from a cluster with no audit stream reaches its window // only after the attribution grace. -func createWorkspaceRemoteSaveRequest(kcp *kcpTunnel, hash, namespace, name, message string) { +func createWorkspaceRemoteSaveRequest(kcp *kcpTunnel, hash, namespace, name, message, targetNS, target string) { GinkgoHelper() f, err := os.CreateTemp("", "e2e-remotesaverequest-*.yaml") Expect(err).NotTo(HaveOccurred()) @@ -972,12 +969,15 @@ metadata: name: %s namespace: %s spec: + gitTargetRef: + namespace: %s + name: %s message: %q catchUp: AfterPush window: attachTimeout: "30s" maxDuration: "4s" -`, name, namespace, message) +`, name, namespace, targetNS, target, message) Expect(err).NotTo(HaveOccurred()) Expect(f.Close()).To(Succeed()) _, err = kcp.wsKubectl(hash, "create", "-f", f.Name()) From fed6d5801a7e2e922203c8b66959fd176b56e921 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Mon, 26 Oct 2026 23:06:00 +0000 Subject: [PATCH 21/33] fix(saverequest): bind a save to the GitTarget that admitted it, and finish a lost apply An attach now carries the UID of the GitTarget that admitted the request. A GitTarget of that name with another UID replaced it and admitted nothing, so the router treats it as gone instead of handing the request to its worker. - RemoteSaveRequest: a request whose GitTarget is replaced while no worker holds it is admitted again by the replacement before it is attached, and is refused (NotAllowed) when its grants do not admit it. - SaveRequest: a replacement that refuses a request no longer fails it at once. The worker the first GitTarget sent it to answers first: a commit it made is reported, a request it holds is waited for, and the refusal is written only once the request is withdrawn or nothing held it. - RemoteSaveRequest: Applied=Unknown with reason Lost is a decided apply, as on SaveRequest, so the request finishes and expires after its TTL instead of being served again forever. Co-Authored-By: Claude Opus 5.5 (1M context) --- internal/controller/saverequest_controller.go | 64 ++++++++++--- .../saverequest_controller_unit_test.go | 71 ++++++++++++++ internal/git/save_request_attach.go | 4 + internal/watch/event_router.go | 8 +- internal/watch/event_router_test.go | 35 +++++++ internal/watch/remote_save_requests.go | 50 ++++++---- internal/watch/remote_save_requests_test.go | 95 ++++++++++++++++++- 7 files changed, 297 insertions(+), 30 deletions(-) diff --git a/internal/controller/saverequest_controller.go b/internal/controller/saverequest_controller.go index ac90537c..ada33e74 100644 --- a/internal/controller/saverequest_controller.go +++ b/internal/controller/saverequest_controller.go @@ -234,7 +234,7 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) // decision is final — there is no wait and no requeue for the author. author, attribution := r.attributeAuthor(ctx, saveRequest) - claim, admitted, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) + claim, admitted, refusal, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) if err != nil || admitted == saveRequestRefused { return ctrl.Result{}, err } @@ -266,6 +266,7 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) Attribution: attribution.gitOutcome(), GitTargetName: saveRequest.GitTargetKey().Name, GitTargetNamespace: saveRequest.GitTargetKey().Namespace, + GitTargetUID: saveRequest.Status.GitTargetUID, Message: saveRequest.Spec.Message, Attach: window.attach, AttachTimeout: window.attachTimeout, @@ -275,6 +276,9 @@ func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) CatchUpNow: saveRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, Unattributed: claim, } + if admitted == saveRequestReplaced { + return r.refuseOnceUnheld(ctx, log, req, saveRequest, attribution, attach, refusal) + } return r.sendSaveRequest(ctx, log, req, saveRequest, attribution, attach, time.Since(saveRequest.CreationTimestamp.Time) >= window.resolveTimeout(), admitted == saveRequestTargetMissing) @@ -313,48 +317,84 @@ const ( saveRequestTargetMissing // saveRequestRefused is a request now failed. saveRequestRefused + // saveRequestReplaced is a request a GitTarget refuses that replaced the one that admitted it. An + // attach may have been sent before, so it is refused only once no worker holds it. + saveRequestReplaced ) // admitAtFirstSight runs admitSaveRequest, records the GitTarget that admitted the request, and fails // a request it refuses. A refusal is final unless that same GitTarget, by UID, admitted the request // before: then its grants were lowered after an attach may have been sent, and the worker, which may // hold the request, keeps what it was first sent. A GitTarget that did not exist when the request was -// first seen, or was recreated since, admitted nothing, so its refusal stands. err is a read or write -// to retry. +// first seen admitted nothing, so its refusal stands. One that was recreated since admitted nothing +// either, but the GitTarget it replaced did: its refusal is saveRequestReplaced, returned with the +// refusal, and waits for the worker that may hold the request. err is a read or write to retry. func (r *SaveRequestReconciler) admitAtFirstSight( ctx context.Context, log logr.Logger, saveRequest *configbutleraiv1alpha3.SaveRequest, attribution saveRequestAttribution, -) (git.UnattributedClaim, saveRequestAdmission, error) { +) (git.UnattributedClaim, saveRequestAdmission, error, error) { claim, targetUID, err := r.admitSaveRequest(ctx, saveRequest) recorded := saveRequest.Status.GitTargetUID switch { case err == nil && targetUID == "": if recorded != "" { // Its GitTarget is gone since it admitted the request: the attach path reports that. - return claim, saveRequestAdmitted, nil + return claim, saveRequestAdmitted, nil, nil } - return claim, saveRequestTargetMissing, r.stampFirstSightConditions(ctx, saveRequest, attribution) + return claim, saveRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, saveRequest, attribution) case err == nil: if recorded == targetUID { - return claim, saveRequestAdmitted, nil + return claim, saveRequestAdmitted, nil, nil } // Recorded before any attach is sent for this GitTarget, so a later refusal can tell the two // apart. At first sight the in-progress stamp that follows writes it. saveRequest.Status.GitTargetUID = targetUID if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) == nil { - return claim, saveRequestAdmitted, nil + return claim, saveRequestAdmitted, nil, nil } - return claim, saveRequestAdmitted, r.Status().Update(ctx, saveRequest) + return claim, saveRequestAdmitted, nil, r.Status().Update(ctx, saveRequest) case !errors.Is(err, errSaveRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed): - return claim, saveRequestAdmitted, err + return claim, saveRequestAdmitted, nil, err case recorded == targetUID: - return git.UnattributedClaimNone, saveRequestAdmitted, nil + return git.UnattributedClaimNone, saveRequestAdmitted, nil, nil + case recorded != "": + return git.UnattributedClaimNone, saveRequestReplaced, err, nil } r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, resolvedSaveRequestTarget(saveRequest)) - return claim, saveRequestRefused, nil + return claim, saveRequestRefused, nil, nil +} + +// refuseOnceUnheld fails a request that a replacement of its GitTarget refuses. The GitTarget it +// replaced admitted the request, so an attach may have been sent, and the worker holding it may have +// committed it. That worker answers first: its outcome is reported, and a request it holds is waited +// for. The refusal is written only once the request is withdrawn, or nothing held it. +func (r *SaveRequestReconciler) refuseOnceUnheld( + ctx context.Context, + log logr.Logger, + req ctrl.Request, + saveRequest *configbutleraiv1alpha3.SaveRequest, + attribution saveRequestAttribution, + attach git.AttachSaveRequest, + refusal error, +) (ctrl.Result, error) { + result, resolved, withdrawErr := r.Finalizer.WithdrawSaveRequest(ctx, attach) + if withdrawErr != nil || !resolved { + if withdrawErr != nil { + log.V(1).Info("SaveRequest withdraw not yet serviceable; will retry", + "name", req.NamespacedName, "err", withdrawErr.Error()) + } + return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) + } + finalizeErr := result.Err + if errors.Is(finalizeErr, git.ErrSaveRequestWithdrawn) { + result, finalizeErr = git.FinalizeResult{}, refusal + } + r.writeTerminalStatus(ctx, log, saveRequest, result, finalizeErr, attribution, + resolvedSaveRequestTarget(saveRequest)) + return ctrl.Result{}, nil } // errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author diff --git a/internal/controller/saverequest_controller_unit_test.go b/internal/controller/saverequest_controller_unit_test.go index ec01c33c..16bd59a7 100644 --- a/internal/controller/saverequest_controller_unit_test.go +++ b/internal/controller/saverequest_controller_unit_test.go @@ -1141,6 +1141,7 @@ func TestSaveRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { reconcileSaveRequest(t, r, "save") require.Len(t, f.calls, 1) + assert.Equal(t, string(target.UID), f.calls[0].GitTargetUID, "the attach is bound to the admitting GitTarget") got := fetchSaveRequest(t, c, "save") assert.Equal(t, string(target.UID), got.Status.GitTargetUID) @@ -1155,3 +1156,73 @@ func TestSaveRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { require.NotNil(t, ready) assert.NotEqual(t, crReasonNotAllowed, ready.Reason) } + +// A GitTarget recreated with grants that refuse a request does not fail it while the worker the +// GitTarget it replaced sent it to may hold it: that worker answers first. A save it committed is +// reported as committed; one nothing held is refused; one it holds is waited for. +func TestSaveRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T) { + tests := []struct { + name string + withdraw git.FinalizeResult + resolved bool + wantReason string + wantCommit string + }{ + { + name: "the worker committed it", + withdraw: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, + resolved: true, + wantReason: crReasonCommitted, + wantCommit: "c0ffee", + }, + { + name: "nothing held it", + withdraw: git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, + resolved: true, + wantReason: crReasonNotAllowed, + }, + { + name: "the worker holds it", + withdraw: git.FinalizeResult{Phase: git.PhaseWaitingForPush}, + }, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + cr := newSaveRequest("save") + target := &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default", UID: "uid-first"}, + } + c := newSaveRequestClient(t, nil, cr, target) + f := &fakeFinalizer{} + r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + + reconcileSaveRequest(t, r, "save") + require.Len(t, f.calls, 1) + + require.NoError(t, c.Delete(context.Background(), target)) + refusing := configv1alpha3.GitTargetSpec{AllowedSaveRequests: []configv1alpha3.GitTargetSaveRequestGrant{}} + require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default", UID: "uid-second"}, + Spec: refusing, + })) + f.withdrawResult, f.withdrawResolved, f.withdrawPending = tc.withdraw, tc.resolved, true + res := reconcileSaveRequest(t, r, "save") + + assert.Len(t, f.calls, 1, "nothing is attached through the replacement") + require.Len(t, f.withdrawCalls, 1) + assert.Equal(t, "uid-first", f.withdrawCalls[0].GitTargetUID, "the worker is asked about the first one") + got := fetchSaveRequest(t, c, "save") + assert.Equal(t, "uid-first", got.Status.GitTargetUID) + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + if tc.wantReason == "" { + assert.Positive(t, res.RequeueAfter, "a held request is waited for") + assert.NotEqual(t, metav1.ConditionTrue, ready.Status) + assert.NotEqual(t, crReasonNotAllowed, ready.Reason) + return + } + assert.Equal(t, tc.wantReason, ready.Reason) + assert.Equal(t, tc.wantCommit, got.Status.Commit) + }) + } +} diff --git a/internal/git/save_request_attach.go b/internal/git/save_request_attach.go index 87eaa372..c068e057 100644 --- a/internal/git/save_request_attach.go +++ b/internal/git/save_request_attach.go @@ -136,6 +136,10 @@ type AttachSaveRequest struct { // GitTargetName / GitTargetNamespace scope the finalize to one GitTarget. GitTargetName string GitTargetNamespace string + // GitTargetUID is the UID of the GitTarget that admitted the request. A GitTarget of that name + // with another UID replaced it and admitted nothing, so it is never asked to take the request. + // Empty matches any. + GitTargetUID string // Message is the verbatim commit message to attach to the window. Empty keeps // the generated grouped-commit message. diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index d8057cd2..6744999e 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -193,7 +193,9 @@ func heldBecause(worker *git.BranchWorker, phase git.SaveRequestPhase) string { // The worker that holds the request comes first, whatever its GitTarget says now: a shared worker // outlives a deleted target, and a retired one finishes its last push after the manager stopped // listing it. Only a request no worker holds falls back to the GitTarget's current worker, which is -// nil while that worker has not started. A GitTarget that is gone is errGitTargetGone. +// nil while that worker has not started. A GitTarget that is gone is errGitTargetGone, and so is one +// that replaced the GitTarget that admitted the request (attach.GitTargetUID): the replacement admitted +// nothing. func (r *EventRouter) saveRequestWorker( ctx context.Context, attach git.AttachSaveRequest, @@ -213,6 +215,10 @@ func (r *EventRouter) saveRequestWorker( return nil, "", fmt.Errorf("get GitTarget %s/%s: %w", attach.GitTargetNamespace, attach.GitTargetName, err) } + if attach.GitTargetUID != "" && string(gitTarget.UID) != attach.GitTargetUID { + return nil, "", fmt.Errorf("%w: %s/%s was replaced", errGitTargetGone, + attach.GitTargetNamespace, attach.GitTargetName) + } worker, exists := r.WorkerManager.WorkerForTarget(&gitTarget) if exists { diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 79559442..5977430c 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -314,6 +314,41 @@ func TestGitPathRefusalReason(t *testing.T) { } } +// A GitTarget of the name an attach carries, but with another UID than the GitTarget that admitted +// the request, replaced it and admitted nothing: no worker of it is asked to take the request. +func TestServiceSaveRequest_AReplacedGitTargetIsGone(t *testing.T) { + scheme := eventRouterScheme(t) + gitTarget := &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a", UID: "uid-second"}, + Spec: configv1alpha3.GitTargetSpec{ + GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, + }, + } + client := fake.NewClientBuilder().WithScheme(scheme).WithObjects(gitTarget, testGitBranchObject()).Build() + workerManager := git.NewWorkerManager( + client, + logr.Discard(), + git.BranchWorkerLimits{}, + types.SensitiveResourcePolicy{}, + ) + router := NewEventRouter(workerManager, nil, client, logr.Discard()) + + attach := saveAttach("team-a-config", "team-a") + attach.GitTargetUID = "uid-first" + _, resolved, err := router.ServiceSaveRequest(context.Background(), attach) + require.ErrorIs(t, err, errGitTargetGone) + assert.False(t, resolved) + + result, resolved, err := router.WithdrawSaveRequest(context.Background(), attach) + require.NoError(t, err) + assert.True(t, resolved, "nothing holds it, and the replacement can never take it") + require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + + attach.GitTargetUID = "uid-second" + _, _, err = router.ServiceSaveRequest(context.Background(), attach) + require.NoError(t, err, "the GitTarget that admitted it is asked") +} + // TestServiceSaveRequest_NoWorkerKeepsTheRequestPending pins that a missing worker is not an // outcome. At startup a SaveRequest can reach the controller before its GitTarget's worker exists; // resolving it then would end the save before anything could collect its writes or record its diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go index 12d052fe..5f1f75c2 100644 --- a/internal/watch/remote_save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -316,7 +316,8 @@ func (s *RemoteSaveRequestServer) consider( } // remoteSaveRequestFinished reports whether a request has an outcome that changes no more, and since when: -// Ready decided, and, for one that asked to catch up and committed, Applied decided too. +// Ready decided, and, for one that asked to catch up and committed, Applied decided too. An apply +// reported Lost is decided: no worker will ever say more of it. func remoteSaveRequestFinished(request *sourcev1alpha1.RemoteSaveRequest) (time.Time, bool) { ready := apimeta.FindStatusCondition(request.Status.Conditions, "Ready") if ready == nil || ready.Status == metav1.ConditionUnknown { @@ -325,7 +326,7 @@ func remoteSaveRequestFinished(request *sourcev1alpha1.RemoteSaveRequest) (time. finishedAt := ready.LastTransitionTime.Time if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && ready.Reason == saveReasonCommitted { applied := apimeta.FindStatusCondition(request.Status.Conditions, "Applied") - if applied == nil || applied.Status == metav1.ConditionUnknown { + if applied == nil || (applied.Status == metav1.ConditionUnknown && applied.Reason != saveReasonLost) { return time.Time{}, false } finishedAt = applied.LastTransitionTime.Time @@ -372,7 +373,7 @@ func (s *RemoteSaveRequestServer) serve( return } status.Author = author - reason, message, err := s.admit(ctx, watcher, request) + reason, message, targetUID, err := s.admit(ctx, watcher, request) if err != nil { log.Info("Cannot read the RemoteSaveRequest's GitTarget; it is served again on the next listing", "err", err.Error()) @@ -386,11 +387,13 @@ func (s *RemoteSaveRequestServer) serve( } attach := remoteSaveRequestAttach(request, author, attribution) + attach.GitTargetUID = targetUID setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, "waiting for the edits to reach a commit window") write() - if !s.finalize(ctx, request, status, attach) { + readmit := func() (string, string, string, error) { return s.admit(ctx, watcher, request) } + if !s.finalize(ctx, request, status, attach, readmit) { return } write() @@ -447,12 +450,12 @@ func (s *RemoteSaveRequestServer) sentAuditFacts(ctx context.Context, provider k // admit decides whether the GitTarget the request names takes it: it exists, mirrors the cluster the // request was served from, and admits the request's namespace and labels in -// spec.allowedRemoteSaveRequests. A refusal is a reason and a message; err is a read to retry. A -// GitTarget that does not exist is waited for up to the request's attachTimeout, since it may be -// created alongside the request. +// spec.allowedRemoteSaveRequests. A refusal is a reason and a message; otherwise it returns the UID of +// the GitTarget that admitted the request, and err is a read to retry. A GitTarget that does not exist +// is waited for up to the request's attachTimeout, since it may be created alongside the request. func (s *RemoteSaveRequestServer) admit( ctx context.Context, watcher *remoteSaveRequestWatcher, request *sourcev1alpha1.RemoteSaveRequest, -) (string, string, error) { +) (string, string, string, error) { ref := request.Spec.GitTargetRef key := k8stypes.NamespacedName{Namespace: ref.Namespace, Name: ref.Name} deadline := time.Now().Add(remoteSaveRequestAttach(request, "", "").AttachTimeout) @@ -463,19 +466,19 @@ func (s *RemoteSaveRequestServer) admit( break } if err != nil && !apierrors.IsNotFound(err) { - return "", "", err + return "", "", "", err } if !time.Now().Before(deadline) { - return saveReasonGitTargetNotFound, fmt.Sprintf("GitTarget %s does not exist", key), nil + return saveReasonGitTargetNotFound, fmt.Sprintf("GitTarget %s does not exist", key), "", nil } select { case <-ctx.Done(): - return "", "", ctx.Err() + return "", "", "", ctx.Err() case <-time.After(remoteSaveRequestPoll): } } if target.KubeProviderKey() != watcher.provider { - return saveReasonNotAllowed, fmt.Sprintf("GitTarget %s mirrors another cluster than this one", key), nil + return saveReasonNotAllowed, fmt.Sprintf("GitTarget %s mirrors another cluster than this one", key), "", nil } admitted, err := target.AdmitsRemoteSaveRequest(request.Namespace, request.Labels) if !admitted { @@ -484,9 +487,9 @@ func (s *RemoteSaveRequestServer) admit( if err != nil { message += fmt.Sprintf(" (a grant's requestSelector is invalid: %v)", err) } - return saveReasonNotAllowed, message, nil + return saveReasonNotAllowed, message, "", nil } - return "", "", nil + return "", "", string(target.UID), nil } // remoteSaveRequestAttach is the attach of a request to its GitTarget's window. @@ -524,9 +527,13 @@ func remoteSaveRequestAttach( // finalize polls the outcome until the worker resolved the request, withdrawing it once it is past its // bound and still unheld, and records it on status. It reports false when ctx ended first. +// +// The GitTarget that admitted the request may be deleted while no worker holds it yet. Whatever has +// its name then is asked by readmit, as admit does, before the request is attached to it: a +// replacement admitted nothing, and its grants may not admit the request. func (s *RemoteSaveRequestServer) finalize( ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attach git.AttachSaveRequest, + attach git.AttachSaveRequest, readmit func() (string, string, string, error), ) bool { bound := time.Now().Add(attach.AttachTimeout + attach.MaxDuration + remoteSaveRequestPushAllowance) for { @@ -534,10 +541,21 @@ func (s *RemoteSaveRequestServer) finalize( if time.Now().After(bound) { ask = s.Finalizer.WithdrawSaveRequest } - if result, done, err := ask(ctx, attach); err == nil && done { + result, done, err := ask(ctx, attach) + if err == nil && done { recordSaveOutcome(request, status, result) return true } + if errors.Is(err, errGitTargetGone) { + reason, message, uid, admitErr := readmit() + if reason != "" { + failSave(request, status, reason, message) + return true + } + if admitErr == nil { + attach.GitTargetUID = uid + } + } select { case <-ctx.Done(): return false diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go index 1f08cdbe..b58075b3 100644 --- a/internal/watch/remote_save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -117,7 +117,7 @@ func saveServer( providerName = "workspace" } objects = append(objects, &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a"}, + ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a", UID: "uid-app"}, Spec: configv1alpha3.GitTargetSpec{ KubeProviderRef: meta.NamespacedObjectReference{Name: providerName, Namespace: "providers"}, AllowedRemoteSaveRequests: setup.grants, @@ -323,6 +323,86 @@ func TestRemoteSaveRequest_TheGitTargetMustAdmitIt(t *testing.T) { } } +// replacingFinalizer recreates the GitTarget team-a/app as replacement the first time a save is sent, +// before any worker holds it, and then answers as the router does: an attach bound to the GitTarget +// that was replaced finds it gone. +type replacingFinalizer struct { + saveFinalizer + + client client.Client + replacement *configv1alpha3.GitTarget +} + +func (f *replacingFinalizer) ServiceSaveRequest( + ctx context.Context, attach git.AttachSaveRequest, +) (git.FinalizeResult, bool, error) { + var current configv1alpha3.GitTarget + if err := f.client.Get(ctx, client.ObjectKey{Namespace: "team-a", Name: "app"}, ¤t); err != nil { + return git.FinalizeResult{}, false, err + } + if attach.GitTargetUID != string(current.UID) { + return git.FinalizeResult{}, false, errGitTargetGone + } + if f.replacement != nil { + if err := f.client.Delete(ctx, ¤t); err != nil { + return git.FinalizeResult{}, false, err + } + if err := f.client.Create(ctx, f.replacement); err != nil { + return git.FinalizeResult{}, false, err + } + f.replacement = nil + return git.FinalizeResult{Phase: git.PhaseWaitingForWorker}, false, nil + } + return f.saveFinalizer.ServiceSaveRequest(ctx, attach) +} + +// A GitTarget recreated while a save waits for its worker admitted nothing: the save is attached to it +// only once its own grants admit the request, and is refused when they do not. +func TestRemoteSaveRequest_AReplacedGitTargetMustAdmitItAgain(t *testing.T) { + tests := map[string]struct { + grants []configv1alpha3.GitTargetRemoteSaveRequestGrant + reason string + }{ + "the replacement admits it": { + grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, + reason: saveReasonCommitted, + }, + "the replacement admits nobody": {reason: saveReasonNotAllowed}, + } + for name, tt := range tests { + t.Run(name, func(t *testing.T) { + finalizer := &replacingFinalizer{ + saveFinalizer: saveFinalizer{ + result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}, + }, + replacement: &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a", UID: "uid-replacement"}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{ + Name: "workspace", Namespace: "providers", + }, + AllowedRemoteSaveRequests: tt.grants, + }, + }, + } + request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) + server, resource := saveServer(t, request, shopSetup("", nil), &finalizer.saveFinalizer) + finalizer.client = server.Client + server.Finalizer = finalizer + + status := serveOnce(t, server, resource, request) + + assert.Equal(t, tt.reason, condition(status, "Ready").Reason) + if tt.reason == saveReasonNotAllowed { + assert.Empty(t, finalizer.attaches, "nothing is attached through the replacement") + return + } + require.Len(t, finalizer.attaches, 1) + assert.Equal(t, "uid-replacement", finalizer.attaches[0].GitTargetUID) + }) + } +} + // A save that failed fails the request, and says why. func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} @@ -415,6 +495,19 @@ func TestRemoteSaveRequest_Finished(t *testing.T) { assert.True(t, finished) assert.Equal(t, at.Time, finishedAt) + applied := func(reason string) []metav1.Condition { + return []metav1.Condition{ready(metav1.ConditionTrue, saveReasonCommitted), { + Type: "Applied", Status: metav1.ConditionUnknown, Reason: reason, LastTransitionTime: at, + }} + } + request.Status.Conditions = applied(saveReasonApplying) + _, finished = remoteSaveRequestFinished(request) + assert.False(t, finished, "its apply is still followed") + request.Status.Conditions = applied(saveReasonLost) + finishedAt, finished = remoteSaveRequestFinished(request) + assert.True(t, finished, "an apply no worker knows of any more is decided") + assert.Equal(t, at.Time, finishedAt) + request.Spec.CatchUp = sourcev1alpha1.CatchUpAsDefinedOnClaim request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonNoWindow)} _, finished = remoteSaveRequestFinished(request) From 7a45f4644d41410d3af30a92b7f626af59382eb4 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 09:20:40 +0000 Subject: [PATCH 22/33] docs(review): one CommitRequest, no admission webhook The design for collapsing SaveRequest and RemoteSaveRequest into one CommitRequest kind served by two front ends over one core, with the admission webhook and takeover removed. configuration.md gains the table of what each remoteSaveRequests value does and what the credential needs, which the implementation replaces with the three-value table. Co-Authored-By: Claude Opus 5.5 --- docs/configuration.md | 14 +- .../20261027-save-request-simplification.md | 935 ++++++++++++++++++ 2 files changed, 945 insertions(+), 4 deletions(-) create mode 100644 docs/review/20261027-save-request-simplification.md diff --git a/docs/configuration.md b/docs/configuration.md index 84dd8410..75b0a7c2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -3299,15 +3299,21 @@ Three things turn it on, each on purpose: install it there too, for the clusters the operator mirrors with its own identity. 2. **Set `spec.remoteSaveRequests: Enabled` on the `KubeProvider`.** GitOps Reverser then watches the cluster's `RemoteSaveRequest`s through that provider, and writes each one's status back. It is the - one write GitOps Reverser makes on a source cluster: the provider's credential needs `get`, - `list`, `watch` and `delete` on `remotesaverequests`, and `update` on `remotesaverequests/status`, - in the `source.configbutler.ai` group. A `RemoteSaveRequest` is never mirrored, whatever a - `WatchRule` selects. + one write GitOps Reverser makes on a source cluster. A `RemoteSaveRequest` is never mirrored, + whatever a `WatchRule` selects. 3. **Grant it on the `GitTarget`, in `spec.allowedRemoteSaveRequests`.** The provider switch makes the requests reachable; the `GitTarget`'s owner decides which of them may save its windows. An omitted or empty list admits none. The `GitTarget` must also mirror the cluster the request was made on: its `spec.kubeProviderRef` names the `KubeProvider` that served the request. +What each value of `spec.remoteSaveRequests` does on the source cluster, and what the provider's +credential needs there, in the `source.configbutler.ai` group: + +| Value | What GitOps Reverser does | The credential needs | +| --- | --- | --- | +| `Disabled` (default) | nothing | nothing | +| `Enabled` | serves the requests and writes their status, and deletes a finished request `--save-request-ttl` after it finished (`0` keeps it) | `get`, `list`, `watch` and `delete` on `remotesaverequests`; `update` on `remotesaverequests/status` | + ```yaml apiVersion: configbutler.ai/v1alpha3 kind: GitTarget diff --git a/docs/review/20261027-save-request-simplification.md b/docs/review/20261027-save-request-simplification.md new file mode 100644 index 00000000..c959bd57 --- /dev/null +++ b/docs/review/20261027-save-request-simplification.md @@ -0,0 +1,935 @@ +# Review: one CommitRequest, no admission webhook + +Review of `feat/gitops-api-asks-converge-root-ca` (PR #456, at `70d07d91`), on one question: how +much of the `SaveRequest` / `RemoteSaveRequest` machinery can go, and in particular whether the +admission webhook can go entirely. + +Status: design decided on 2026-10-27, nothing built yet. The first version of this review proposed +one kind on the source cluster only, named `SaveRequest`. The decisions below replace that +proposal. + +## Decisions + +1. **One kind, named `CommitRequest`, in `configbutler.ai/v1alpha3`.** That is its name on `main`, + so the rename to `SaveRequest` (`b86dffbe`) is undone, which removes one breaking change. +2. **It is served in two ways, through one shared core.** + - **On the control cluster** it is an operator kind like any other. A controller in the + operator serves it with the operator's own identity, and a GitTarget admits it by + `consumerNamespaces`, default `from: Same`. + - **On any other source cluster** it is reached through the GitTarget's KubeProvider. That + KubeProvider sets `commitRequests` to `Reconcile` or `ReconcileAndDeleteFinished`, and the + GitTarget admits the request by `sourceNamespaces`. +3. **`spec.gitTargetRef` follows the API's reference convention.** An omitted namespace means the + request's own namespace, as in every other cross-namespace reference. On a source cluster, the + namespace is required, for the reason a ClusterWatchRule requires it: there is no + control-cluster namespace to default to. See [Naming the GitTarget](#naming-the-gittarget). +4. **The admission webhook goes completely.** That covers both handlers, the `/validate-all` e2e + endpoint, the command-author Redis store, the chart values, the manifests and the docs. + Takeover (`SavedBy`/`TakeOver`) goes with it, because only admission could name a taker. On + both paths, a request's author comes from audit facts, the same way an edit's author does. +5. **cert-manager stays**, but only as a prerequisite of attribution. See + [cert-manager](#cert-manager). +6. **GitOps Reverser never writes the CommitRequest CRD on a source cluster.** It reports the CRD's + state on the KubeProvider. Creating the CRD when it is absent is + [future work](#future-work-creating-the-crd), and updating it is ruled out for good. +7. **Several identities on one cluster are a goal**, so the path for other clusters works with + credentials scoped to namespaces. See + [Several identities on one cluster](#several-identities-on-one-cluster). + +## The design + +### The CommitRequest + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: CommitRequest +metadata: + name: raise-replicas + namespace: shop # on the control cluster, or on the GitTarget's source cluster +spec: + gitTargetRef: + namespace: team-shop # a control-cluster namespace; omitted, this request's own + # namespace, which only the control-cluster path accepts + name: shop + message: "Raise the replica count" + window: # defaults shown + attachTimeout: 5s + maxDuration: 2s + catchUp: AsDefinedOnClaim # or AfterPush +status: + author: jane@example.com # from audit; empty when no audit event names the creator + branch: main + commit: 3f2a… + conditions: [Ready, Stalled, Applied] +``` + +- **The spec and status** are today's `RemoteSaveRequest` ones, under the name and group of + `main`. `gitTargetRef` is a `meta.NamespacedObjectReference`, like every other cross-namespace + reference. +- **The phase reasons and the `save_requests_total` metric** come from today's control-cluster + reconciler. +- **Gone from the `main` version:** `spec.window.author`, because takeover is removed. +- **Kept from the `main` version:** `spec.window.attach` (`CurrentOrNext`/`Next`) and + `spec.whenNothingToCommit` (`Resolve`/`CommitEmpty`). Today only the control-cluster reconciler + has them. They move into the shared core, so a request on a source cluster gets them too. +- **Retention keeps `main`'s contract on both paths.** When a request finishes, it is stamped with + `configbutler.ai/delete-after`, and it is deleted once that time has passed. Removing the + annotation keeps the request. The source-cluster path's own TTL, which is computed from the + conditions, goes away. On a source cluster, deleting is opted into with + `commitRequests: ReconcileAndDeleteFinished` (see [The KubeProvider](#the-kubeprovider-speccommitrequests)). + +#### Naming the GitTarget + +Every reference to another namespace follows one rule: the namespace may be omitted, and then it is +the referring object's own namespace. The one exception is a ClusterWatchRule, which must name the +namespace because it has none of its own. + +| Reference | Type | An omitted namespace means | +| --- | --- | --- | +| GitTarget `gitBranchRef`, `kubeProviderRef` | `NamespacedObjectReference` | the GitTarget's namespace | +| GitBranch `gitProviderRef` | `NamespacedObjectReference` | the GitBranch's namespace | +| FluxClaim `kubeProviderRef`, `kustomizationRef`; ArgoCDClaim `kubeProviderRef`, `applicationRef` | `NamespacedObjectReference` | the claim's namespace | +| ClusterWatchRule `gitTargetRef` | `NamespacedObjectReference` | nothing: required, "a ClusterWatchRule has no namespace to default to" | +| WatchRule `gitTargetRef` | `LocalObjectReference` | always the WatchRule's namespace | +| `CommitRequest` `gitTargetRef` on `main` | `LocalObjectReference` | always the request's namespace | + +So `CommitRequest` follows the rule: + +- **On the control cluster, the namespace is optional**, and defaults to the request's namespace. + This is the behavior of `main` and of every other reference, and a single-cluster user writes + what they write today. +- **On a source cluster, the namespace is required**, for the ClusterWatchRule's reason. + - The request's namespace belongs to another cluster, so it is not a control-cluster namespace + to default to. + - Defaulting to "the control-cluster namespace with the same name" would turn a naming + coincidence into a reference. [definitions.md](../definitions.md) exists to prevent exactly + that. + - The KubeProvider's `{consumer.namespace}` does bridge the two sides by name, but only as a + variable someone writes on purpose, never as a silent default. + +**Where the rule is enforced.** Both clusters use the same CRD, and a CEL rule cannot know which +cluster it runs on, so the schema allows an omitted namespace. + +The source-cluster path does not pick up a request without a namespace, and it writes nothing to +it. It cannot refuse such a request, because it cannot know the refusal is true. A KubeProvider +whose kubeconfig points back at the control cluster sees the control cluster's requests, and there +an omitted namespace is valid. + +This is not a far-fetched case. `inCluster` is honored only for providers the installer lists in +`--in-cluster-kube-providers`, so a tenant who wants its own identity on the operator's cluster +needs exactly such a kubeconfig. That is the "several identities on one cluster" goal. A source +cluster user who forgets the namespace therefore sees no status. The field's doc says so, and +gitops-api always sets the namespace. + +**Rejected: a separate CRD variant for source clusters with the namespace required in CEL.** The +check would move into the schema. But one kind would then have two schemas, and the chart's +variant applied to a workspace would silently undo the check. + +### Which path serves a request + +The named GitTarget's KubeProvider alone decides which path serves a request: that path is the +request's **owner**. Two rules keep several observers of one request safe: + +1. **A path writes only what it knows to be true.** Only the owner serves a request and writes + its progress. An observer that is not the owner writes only `GitTargetNotFound`, after + `attachTimeout`, and only into an empty status. It writes nothing else, because nothing else is + true for every observer. +2. **Every status write is conditional on the resourceVersion its decision was made from.** On a + conflict, the writer reads the request again and decides again. Today's writer reads the object + again and overwrites it without deciding again + (`internal/watch/remote_save_requests.go:689-701`). An observer that sees a status it did not + write stops. So a `GitTargetNotFound` that was true when it was decided can land, but it never + overwrites an owner that has started. +3. **The owner claims a request by persisting its first status, and attaches only after that.** + The first write records the author and `Ready=Unknown`, reason `Saving`, and it is conditional + on the resourceVersion the owner read. Only when it succeeds does the owner attach to the + worker. If it conflicts and the fresh object already holds an outcome, such as a non-owner's + `GitTargetNotFound`, the owner stops without saving. + - Today's `serve` logs a failed status write and continues into `finalize` (l.348-352 and + l.391-393). Fixing the writer alone would therefore not be enough. + - **After the claim, a conflict re-applies the owner's outcome. It never restarts the save.** + Once the claim has landed, no other observer writes, by rule 1. So a later conflict comes only + from the owner's own writes, and the worker's outcome, once produced, is written onto the fresh + object as it is. + - **Ownership is structural, not "this process wrote the status".** After a restart, the owner + is still the path the GitTarget's KubeProvider picks. It finds its own `Saving` status and + resumes, as the server does today. + +| The request is on | The GitTarget's KubeProvider | What happens | +| --- | --- | --- | +| the control cluster | `inCluster` | **the control-cluster path serves it** | +| the control cluster | a kubeconfig, with either `Reconcile` value | left alone, because that kubeconfig may point back at the control cluster, and then its watcher serves the request | +| the control cluster | a kubeconfig, with `commitRequests: Ignore` | `NotAllowed`: "GitTarget … mirrors another cluster; create the CommitRequest there, and set `commitRequests: Reconcile` on its KubeProvider" | +| a source cluster that provider P watches | P, with a kubeconfig | **P's watcher serves it** | +| a source cluster that provider P watches | any other provider | left alone | +| a source cluster | `gitTargetRef` has no namespace | not picked up, and no status ([Naming the GitTarget](#naming-the-gittarget)) | +| either | the GitTarget does not exist | `GitTargetNotFound` after `attachTimeout`, by rule 1 and rule 2 | + +- **A control-cluster request for a GitTarget that mirrors another cluster is refused**, in every + install mode. Its author would come from the control cluster's audit, and the edits' authors + from the other cluster's. Those are two identity domains (F3), and the request would only work + while attribution is off. +- **"Local" means the provider's `inCluster` arm.** A kubeconfig that points back at the control + cluster counts as another cluster, because the operator cannot tell the difference. +- **Known gap.** A control-cluster request whose GitTarget's provider has `Reconcile` and points + somewhere else gets no status from anyone. The doc for the field says so. A cluster ID would + close the gap ([Future work](#future-work)). + +### What both paths share + +Once a path owns a request, the same core runs. These are its responsibilities, in order: + +1. **Admission** by `allowedCommitRequests`. A request no grant admits fails with `NotAllowed`. +2. **The author.** It comes from audit facts on the GitTarget's KubeProvider's route, from the + same index and with the same grace as the edits. + - **The core acquires the fact stream** for that route and `commitrequests`, as + `RemoteSaveRequestServer.run` does today (l.200). Today's control-cluster reconciler never + acquires one, because admission gave it the author. Without the stream, the fact follower + never reads the request's audit facts, so every save on that path would wait out the grace and + end without an author. + - **The wait runs whenever attribution is on**, and is skipped only when it is off (no + `AuthorResolver`). Whether a provider has received audit facts is checked after the wait, as + `sentAuditFacts` does today, and decides only `AuthorUnresolved`. Skipping the wait on a + provider that has not received a fact yet would lose the author while the provider's first + audit event is still in transit, or while its status condition lags. + - **The author is written to status in the first status write.** The lookup matches the + request's resourceVersion, and every status write changes it, so a later lookup could never + match. Today's server already does this (`status.Author` before the `Saving` write). +3. **Attach and outcomes:** one save in progress per author per GitTarget (`SaveInProgress`), the + UID binding, recovery after a restart, the phase reasons, the metric, and the worker's + `git.AttachSaveRequest`. +4. **Following the apply:** `catchUp: AfterPush` and the `Applied` condition. +5. **Retention:** stamping `configbutler.ai/delete-after`, and deleting once it has passed. + +Every status write follows rule 2 of [Which path serves a request](#which-path-serves-a-request). + +Each path feeds this core: + +- **The control-cluster path** is a controller-runtime reconciler. It uses the operator's cache, + RBAC and status client, and is cut down from today's `saverequest_controller.go`. +- **The source-cluster path** is today's `RemoteSaveRequestServer`. It runs a dynamic watch per + KubeProvider and writes status through that provider's credential. + +The core is written once, so F4 cannot come back. If the two paths ever need different logic +beyond picking up a request and writing its status, treat that as a design smell. + +**The default watch policy excludes `configbutler.ai/commitrequests`**, in place of +`source.configbutler.ai/remotesaverequests`. A request can sit in a namespace a WatchRule mirrors, +and it must never land in Git. + +What a save attaches to, in each install mode: + +| Install | Edit author | Request author | The request saves | +| --- | --- | --- | --- | +| Default (no attribution) | committer | nobody, no wait | the GitTarget's one unnamed window | +| Attribution on, a source cluster whose first audit event has not arrived | from audit, once it arrives | from audit, within the same grace | its creator's own window | +| Attribution on, single cluster | from audit | from audit | its creator's own window | +| kcp with audit (the managed product) | from audit | from audit | its creator's own window | +| A source cluster that sends no audit | nobody | nobody | the unnamed window: the message is kept, and the commit has no named author | + +Who may save is decided by `allowedCommitRequests`. Who authored the commit is decided by audit, or +by nobody. Neither is decided by admission. + +### The GitTarget: `spec.allowedCommitRequests` + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +spec: + allowedCommitRequests: + # The default when omitted: CommitRequests in this GitTarget's own namespace. + - consumerNamespaces: {from: Same} + # Requests on the control cluster, from other namespaces, by label: + - consumerNamespaces: + from: Selector + selector: {matchLabels: {team: shop}} + # Requests on the source cluster that spec.kubeProviderRef names: + - sourceNamespaces: ["*"] # every namespace this GitTarget mirrors +``` + +- **One list, and each grant sets exactly one of two fields.** A CEL rule enforces this. The two + fields are the ones the API already uses for each side: + - **`consumerNamespaces`** has the same type and meaning as in every other `allowed*` list: + control-cluster namespaces, with `from: Same | Names | Selector | All`. It admits requests on + the control-cluster path. + - **`sourceNamespaces`** has the same meaning as the KubeProvider's + `allowedGitTargets[].sourceNamespaces` and a WatchRule's `rules[].sourceNamespace`: namespaces + of the GitTarget's source cluster, as exact names or `"*"`. It admits requests on the + source-cluster path. + - Here `"*"` means every namespace this GitTarget mirrors. + - Variables are refused, because `{consumer.namespace}` would be the request's own namespace, + which bounds nothing. + - It can admit only namespaces the KubeProvider lends this GitTarget in `allowedGitTargets`, + the same bound its WatchRules have. +- **A grant on the side the GitTarget does not use admits nothing.** An `inCluster` GitTarget + never serves through `sourceNamespaces`, and any other GitTarget never serves through + `consumerNamespaces`. A `NotAllowed` message says which field applies. +- **Omitted, the list is `[{consumerNamespaces: {from: Same}}]`, and `[]` admits nobody.** This is + the pattern of the other lists: an API-server default, no `omitempty`, a Go mirror, and + `TestGrantDefaultsMatchCRDs`. A single-cluster user writes nothing, as on `main`. A user of + another cluster writes a `sourceNamespaces` grant, and that list replaces the default. +- **`requestSelector` is dropped.** It picked requests by labels their creator chooses, so it + proved nothing. It can come back later as an additive field. +- **`remote` is not used in any field name.** In this API, "remote" means the Git remote + (`status.remote`), and [definitions.md](../definitions.md) fixes "control cluster" and "source + cluster" as the two sides. + +### The KubeProvider: `spec.commitRequests` + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: KubeProvider +spec: + secretRef: {name: shop-workspace} # a kubeconfig Secret + commitRequests: ReconcileAndDeleteFinished # Ignore (default) | Reconcile | ReconcileAndDeleteFinished +``` + +Each value makes this cluster a bigger source of CommitRequests, and needs more of the credential +on `commitrequests` in the `configbutler.ai` group: + +| Value | What GitOps Reverser does | The credential needs | +| --- | --- | --- | +| `Ignore` (default) | nothing | nothing | +| `Reconcile` | serves the requests in the namespaces this KubeProvider lends its GitTargets, and writes their status. A finished request stays, and no `delete-after` is stamped on it, because nothing would act on it | `get`, `list`, `watch`; `update` on `commitrequests/status` | +| `ReconcileAndDeleteFinished` | as `Reconcile`, and also stamps `configbutler.ai/delete-after` when a request finishes, and deletes the request once that time has passed | as `Reconcile`, plus `patch` and `delete` | + +- **Why deletion is a separate value.** Deleting an object on someone else's cluster is a bigger + write than updating its status. The KubeProvider's owner lends the cluster, so the provider is + where that consent belongs, and the credential's RBAC follows it. gitops-api uses + `ReconcileAndDeleteFinished` for kcp workspaces. +- **Not `ReconcileAndPrune`.** [definitions.md](../definitions.md) reserves "prune" for + `GitTarget.spec.prune.mode`, which removes documents from a Git folder. +- **How long a finished request is kept stays installation-wide.** That is + `--save-request-ttl`, renamed back to `--commit-request-ttl`, its name on `main`: + default `48h`, and `0` stamps nothing in any mode. The annotation on a single request still + overrides it: remove it to keep the request, or edit it to move the deletion. +- **The annotation counts only while deletion is on.** If a provider goes from + `ReconcileAndDeleteFinished` back to `Reconcile`, annotations that were already stamped stay, but + nothing deletes those requests any more. +- **A delete the credential may not make** leaves the request in place. The + `CommitRequestsServed` message says so, and the condition stays `True`, because serving works. +- **The control cluster has no switch.** A CEL rule refuses every value but `Ignore` together with + `inCluster`. CommitRequests on the control cluster are always served, and finished ones are + deleted by the same flag and annotation, as on `main`. The operator owns that whole lifecycle, as + it does for every other kind. +- **I chose an enum over a boolean `reconcileCommitRequests: true`.** The API conventions avoid + booleans, and this third value is the reason: it was added without a second field. +- **It replaces `remoteSaveRequests: Enabled|Disabled`.** Today `Enabled` serves the requests and + also deletes them, as [configuration.md](../configuration.md#serving-the-remotesaverequests-of-a-cluster) + now lays out in a table of the same shape. The PR that builds this design replaces that table + with the one above. + +Both `Reconcile` values add one condition and one status field. A second condition comes in phase +2: + +- **`CommitRequestsServed`**: whether requests are being served, including whether the CRD is + there. +- **`status.commitRequests.namespaces`**: what this KubeProvider watches. +- **Phase 2: `CommitRequestCRDUpToDate`**, whether the CRD's schema matches the operator's. + +None of them exists on an `inCluster` provider. On the control cluster, the chart owns the CRD, as +it owns every other CRD, and the operator watches with its own permissions. + +#### Phase 1: the `CommitRequestsServed` condition + +This ships in the main PR. It needs only discovery and the watch itself. + +| Status | Reason | When | +| --- | --- | --- | +| True | `Watching` | the message says how many namespaces, or "every namespace" | +| True | `NothingToWatch` | no GitTarget mirroring through this KubeProvider is lent a namespace | +| False | `CRDNotInstalled` | discovery shows no `commitrequests.configbutler.ai` | +| False | `VersionNotServed` | the CRD is there, but does not serve `v1alpha3` | +| False | `Forbidden` | `list` or `watch` on CommitRequests was denied; the message names the namespaces | +| Unknown | `Unreachable` | discovery could not be read | + +- **Each message ends with its fix.** + - `CRDNotInstalled` and `VersionNotServed`: `kubectl apply --server-side -f` with the release's + `commitrequest-crd.yaml`. That is a new release asset. The existing `crds.yaml` holds every + CRD, and a source cluster needs only this one. + - `Forbidden`: the missing verbs and namespaces. + - `Unreachable`: the discovery error. + - `Watching` and `NothingToWatch` need no fix. +- **When it runs:** on every KubeProvider resync, and whenever the spec changes. A CRD applied by + hand therefore clears the condition without a restart. +- **Not part of `Ready`.** Mirroring works without it, as it does without `AuditFactsReceived`. +- **What it replaces.** Today a missing CRD shows up only as a log line, "The watch of + RemoteSaveRequests ended; reconnecting". +- **A printer column, `Saves`,** shows the reason, at `priority=1`. + +#### Phase 2: knowing whether the CRD is out of date + +This is its own PR. It is the "outdated" signal you asked for. It is deferred because it adds a +compatibility protocol, and because one part of it depends on kcp in a way that is not yet verified. + +**The revision line itself ships in phase 1**, in the type's doc comment, with its hash test. It is +a comment and a test, so it costs almost nothing. Without it, every CRD installed from phase 1's +release asset would carry no revision, and phase 2 could not judge any of them. + +The CRD carries a schema revision, and the operator compares it with its own. + +- **Where the revision is.** The last line of the kind's description reads + `Schema revision 7, compatible from revision 5.` kubebuilder renders the type's doc comment into + the CRD, so `kubectl explain commitrequest` prints the line. It is also in the cluster's published + OpenAPI v3 document. +- **How the operator reads it.** It reads `/openapi/v3/apis/configbutler.ai/v1alpha3`. The default + `system:discovery` role lets every authenticated identity read that document, so the check needs + no permission on CRDs and works with a credential scoped to a namespace. There is no fallback to + reading the CRD itself: an API that a kcp `APIBinding` provides has no CRD in the consuming + workspace ([kcp docs](https://docs.kcp.io/kcp/v0.28/concepts/apis/exporting-apis/)). +- **The gate.** The PR starts by verifying that kcp serves per-workspace OpenAPI v3, including for + APIs provided through an `APIBinding`. If it does not, the design changes before any code is + written. +- **The revision** goes up with every change to the generated CRD. A unit test hashes the CRD and + fails when the hash changes but the revision does not, the same way `hack/crdfields` guards the + field list. +- **The floor ("compatible from")** is raised by hand when a change breaks an older reader or + writer: a new required field, a removed field, or a changed meaning. This project makes hard + cuts, so the floor will often move with the revision, which is fine. +- **The comparison.** Take the operator's revision O and floor Fo, and the CRD's revision C: + - C equal to O: current. + - C below O: served when C is at least Fo. + - C above O: **never served**. +- **Why a newer CRD is never served.** The status writer replaces the whole `status` map, so an + older operator would erase every status field that a newer schema added. Serving a newer CRD + safely would need rules for preserving unknown fields, and tests for them. Refusing it costs one + operator upgrade, and that is the fix anyway. The CRD's own floor is therefore never read. +- **Rejected: comparing the served schema field by field.** It needs no bookkeeping, but it cannot + see changed validation, defaults or meaning, and a condition message cannot explain its result + well. + +| Status | Reason | When | Requests served? | +| --- | --- | --- | --- | +| True | `Current` | the CRD's revision is the operator's | yes | +| False | `Outdated` | older than the operator, but at or above the operator's floor | yes | +| False | `Incompatible` | older than the operator's floor | no | +| False | `NewerThanOperator` | newer than the operator; upgrade the operator | no | +| Unknown | `RevisionUnknown` | there is no revision line, so compatibility cannot be judged. This is not taken as evidence that the CRD came from elsewhere | yes | +| Unknown | `Unreadable` | the OpenAPI document could not be read | unchanged | + +- **`Outdated` keeps serving.** A user should not lose saves because a revision moved without + breaking anything. Only a crossed floor stops serving. +- **A reason that stops serving** also sets `CommitRequestsServed=False`, with reason + `CRDNotUsable`, pointing here. +- **The operator never updates the CRD**, whatever this condition says, for the reasons in + [Future work](#future-work-creating-the-crd). The condition is the signal, and a person acts on + it. + +#### When `commitRequests` is `Ignore` + +- **The conditions and `status.commitRequests` are removed.** + - A condition that would always say "not applicable" is noise. + - A `True` left over from before the switch would be wrong. + - `AuditFactsReceived` already works this way: it is absent while attribution is off. +- **The watches stop. The CRD and the requests on the cluster stay as they are.** A request created + there stays without status, unless another KubeProvider serves it. +- **Rejected: keep reporting the CRD read-only while the switch is off.** The check is cheap, but + it reports on something the KubeProvider's owner said it has nothing to do with. With several + providers on one cluster, every one of them would report the same state. + +### Several identities on one cluster + +On the control-cluster path, there is one identity: the operator's. This section is about the +source-cluster path. A KubeProvider is a cluster and an identity in one object, and several +KubeProviders can point at one cluster with different credentials, for example two kubeconfigs for +two teams' ServiceAccounts. + +| Concern | Today | With this design | +| --- | --- | --- | +| Which provider serves a request | every enabled provider tries, and the first one decides (F11) | the GitTarget's own provider only ([Which path serves a request](#which-path-serves-a-request)) | +| Watch scope | one cluster-wide list and watch, so a credential scoped to a namespace gets `Forbidden` | the namespaces the provider lends its GitTargets; a `Role` in each is enough | +| The CRD | cluster-scoped, so a credential scoped to a namespace cannot `get` it | read through OpenAPI, so every provider reports its state | +| Audit facts | providers in one namespace can share a route. A provider in another namespace that uses the same route is `Stalled`, reason `AuditRouteConflict` | unchanged | +| Duplicate work | each provider runs its own watch | each provider watches only its own slice; slices that overlap are watched twice | + +#### Chosen: the provider's lent namespaces set the scope + +A KubeProvider with `Reconcile` watches CommitRequests only in the namespaces it lends to the +GitTargets that mirror through it. These are the `sourceNamespaces` of the `spec.allowedGitTargets` +grants that admit each of those GitTargets, with `{consumer.namespace}` resolved per GitTarget. + +- **If any grant lends `"*"`**, the provider runs one watch across the cluster. Otherwise it runs + one watch per namespace. +- **`status.commitRequests.namespaces`** shows the result, so the derived scope is never hidden. +- **The credential already reads these namespaces for mirroring.** Its `Role` in each one gains + `get`, `list` and `watch` on `commitrequests`, and `update` on `commitrequests/status`. With + `ReconcileAndDeleteFinished` it also gains `patch` and `delete` on `commitrequests`. +- **A request in a namespace that is lent but not admitted still gets `NotAllowed`** from the + GitTarget's provider, so a wrong grant is visible. +- **Overlapping slices are harmless**, because only the GitTarget's provider acts. + +It needs no new field: the provider's owner chooses which namespaces to lend in +`allowedGitTargets`, and the GitTarget's owner chooses which to admit in `allowedCommitRequests`. + +Costs: + +- **A request outside every lent namespace is seen by nobody and gets no status.** +- **The watches change when a provider grant changes.** Grants change rarely, and the mirror + already manages watches per namespace. +- **Watching only the namespaces the GitTarget mirrors would be narrower, and was not chosen.** It + would change with every WatchRule edit, and it would lose `NotAllowed` for namespaces that are + lent but not mirrored. + +#### Not chosen + +- **An explicit namespace list on the provider**, `commitRequests: {mode: Reconcile, + sourceNamespaces: [...]}`. It would be a third list of namespaces, next to `allowedGitTargets` + and `allowedCommitRequests`, and the three would drift apart. +- **One connection with impersonated identities**, the way an Argo CD AppProject uses + `destinationServiceAccounts`. `impersonate` is close to cluster-admin, and it concentrates power + in one credential, which is the opposite of what several identities are for. +- **Splitting the cluster from the identity, and riding the GitTarget's watch streams.** These are + in [Future work](#future-work). + +### cert-manager + +You were right: it does not go away. The audit ingress mints three things through cert-manager: + +- a root CA; +- its serving certificate; +- the kube-apiserver client certificate for mTLS (`audit-certificates.yaml`). + +These render only with `attribution.enabled`. Metrics TLS is off by default. So once admission is +removed: + +- **Default install** (no attribution): no certificate is rendered, and cert-manager is not + needed. +- **Attribution on**, which includes the managed product: cert-manager is needed, or Secrets you + bring yourself with `servers.audit.tls.certManager: false`. + +Two chart changes make this true: + +1. Gate the self-signed `Issuer` in `certificates.yaml` on some certificate being rendered, meaning + attribution with audit TLS, or metrics TLS. Today it renders whenever `certManager.enabled` is + true, which is the default, so an install without cert-manager's CRDs would fail on it. +2. Move [quickstart.md](../quickstart.md) step 1 into the "Want named users on your commits?" + section. + +## Future work + +### Future work: creating the CRD + +Not in this design. If it comes, these are the rules: + +- **Create it only when it is absent.** While either `Reconcile` value is set and discovery shows + that the cluster serves no `commitrequests.configbutler.ai`, the CRD is created from a copy built + into the operator binary. `AlreadyExists` counts as present. `NotInstalled` would then be + replaced by `Created` (True) and `CreationFailed` (False, with the API error and the missing + permission). +- **Never update it, and never delete it.** Not after an operator upgrade, and not on `Ignore` or + when the KubeProvider goes. Deleting a CRD deletes every CommitRequest on the cluster. + +Updating is ruled out for good, because it is a minefield: + +- **An update changes the rules for objects that are already stored.** A version that stops being + served, or validation that gets stricter, makes the next status write fail on an object the + update broke. +- **The CRD may have another owner**: Helm's `crds/`, Flux or Argo CD applying it from Git, + gitops-api's `workspaceCRDs`, or a kcp `APIExport`. Two writers fight, and a GitOps owner reverts + the operator's change. +- **An operator upgrade would change the API on every connected cluster at once**, with nobody + reviewing it. +- **The permission is broad.** `update` on CRDs is power over the cluster's API surface. + +Creating is different, because it replaces nothing. Its cost is the permission: `create` on +`customresourcedefinitions` is cluster-wide, on a credential that is otherwise read-mostly. Its +gain is one step fewer for every new source cluster or kcp workspace. In kcp, an `APIBinding` that +already provides `configbutler.ai` shows up in discovery, so nothing would be created. That is the +right outcome, but it would need a test. + +### Future work: other items + +- **A cluster ID** per KubeProvider, such as the `kube-system` namespace UID, or the logical cluster + name in kcp. It would close the known gap in + [Which path serves a request](#which-path-serves-a-request): the operator could tell that a + kubeconfig points back at the control cluster. +- **Split the cluster from the identity.** A new kind, for example `KubeCluster`, would hold the + endpoint, the audit route, the CRD state and the cluster ID. A KubeProvider would become one + identity on a cluster. + - It fixes `AuditRouteConflict` across namespaces, because every identity on a cluster should + read that cluster's one audit stream. + - It also stops N providers from reporting the same CRD state. + - It costs a new kind, a breaking split of KubeProvider, and a platform-team owner for the + cluster object. + - kcp gives every workspace its own logical cluster with one identity, so the managed product + does not need it. Build it when the first `AuditRouteConflict` between two identities shows up. +- **CommitRequests ride the GitTarget's own watch streams.** The request's author would come from + the same inline `attachAuthor` call that attaches an edit's author, and the source-cluster path's + own server would go away. The watch plane is the most delicate code in the project, so this is + too large for now. + +## What exists today + +| | `SaveRequest` (`configbutler.ai/v1alpha3`) | `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) | +| --- | --- | --- | +| Created on | the control cluster | the source cluster (kcp workspace, or any KubeProvider) | +| Author from | the `/validate-operator-types` admission webhook, written to Redis `author:v1:command:` | the source cluster's audit facts, same index as the edits | +| Author when that source is absent | none (`AuthorAttributed=False`). The default install is in this state: admission on, Redis empty | none, and refused `AuthorUnresolved` if the provider ever sent audit | +| Admitted by | `GitTarget.spec.allowedSaveRequests` (default: `from: Same`) | `GitTarget.spec.allowedRemoteSaveRequests` (default: none) and `KubeProvider.spec.remoteSaveRequests: Enabled` | +| Window spec | `attach` (`CurrentOrNext`/`Next`), `author` (`Own`/`SavedBy`/`TakeOver`), `attachTimeout` 2s, `idleTimeout`, `maxDuration` 2s | `attachTimeout` 5s, `idleTimeout`, `maxDuration` 2s | +| Extra spec | `whenNothingToCommit` (`Resolve`/`CommitEmpty`), `catchUp` | `catchUp` | +| Served by | a controller-runtime reconciler: `saverequest_controller.go`, `_finalize.go`, `_apply.go`, 1,468 lines | `internal/watch/remote_save_requests.go`, 702 lines | +| Same worker API | `git.AttachSaveRequest` | `git.AttachSaveRequest` | + +The decided design keeps both ways of picking a request up, under the left-hand column's name and +API group. It has one core with the right-hand column's identity rule, and one grant list. Both +end up in the same branch-worker code (`internal/git/save_request_attach*.go`, 1,212 lines), which +stays. + +## How it got here + +The design history, in the steps that matter here: + +1. **Audit with a 60s wait** (`c6220afc`, May). The `CommitRequest` author was read from its own + create event in the ordered audit stream. It flaked on `generateName` when an event was diverted + below the high-water mark (`91cc3172`). +2. **Admission** (`58dd37a8`, `96ec3906`, June). It was introduced for two reasons. Option C "kept + the wrong source … and therefore kept a wait", and "on a managed cluster where you cannot enable + an apiserver audit webhook, your commands are still attributed". +3. **The keyed fact index** (#211, July) and **watch-first ingestion** removed both reasons. + - A fact is now awaited from the index, bounded by `--author-attribution-grace` (3s by default, + `internal/watch/author_resolver.go:26`). + - That is the same pipeline, and the same wait, as the edits the save publishes. + - The 60s wait and the ordered-stream divert are gone. +4. **The race admission created** (#367, September). The request took the fast path (admission) and + its edit the slow one (audit join). Saves were "finalized before the write it exists to publish + reached the worker", so `attachTimeout` got a 2s default to cover the gap + ([wait-options](../design/commitrequest-save-wait-options.md), l.16-43, 97-114). +5. **"Works without audit" was never true on its own.** A named request on an audit-less cluster + meets windows that name nobody: `WindowMismatch`, "every time, permanently" + (wait-options l.277-281). It became true only on 2026-10-26, with `SavedBy`/`TakeOver` + (`cf3b97be`, `ee8946c5`), and only with an opt-in grant and an un-latched provider. +6. **`RemoteSaveRequest`** (`beddcdef` to `70d07d91`). It is authored from source audit. It was + built so that "a customer's only API surface is their kcp workspace" holds with no exceptions: + no tenant grant into the control cluster, and no `/editor/commit` on the save path + ([source-side-save-plan](../future/source-side-save-plan.md), l.46-59). + +In short: admission was the right answer to a pipeline that no longer exists. Every argument for it +since then has been about either takeover or gitops-api's current save path, and the source-side +plan already replaces that path. + +## Review findings + +F1 to F10 are complexity findings, not correctness bugs; the correctness review of these commits +happened in the three rounds recorded on the PR. F11 is a correctness bug, found while working out +the design. + +**F1. The default install pays for admission and gets nothing for saves.** The chart has +`servers.admission.enabled: true` while `queue.redis.addr` is `""`. The command-author store is +nil, so every `SaveRequest` reports `AuthorCaptureDisabled`. The webhook's only effect in the +default install is the WatchRule feedback (F6). + +**F2. cert-manager is a prerequisite of the default install only because of admission.** + +- The admission certificate is the only certificate a default install renders, and + [quickstart.md](../quickstart.md) step 1 says so: "The controller mounts an admission certificate + at startup, so cert-manager must be healthy before…". +- Attribution still needs cert-manager (see [cert-manager](#cert-manager)). +- [where-validation-lives](../spec/where-validation-lives.md) prices a webhook as "cert wiring in + the chart, a failure mode that can block tenant writes, and an extra moving part in every install + path". + +**F3. Two identity domains, joined only by name equality.** A control-cluster save attaches to a +window only when the admission `userInfo.username` equals the audit user on the source cluster. +That holds for gitops-api only because its server impersonates the Dex email +(source-side-save-plan l.42-48). For anyone else with two clusters, it is a coincidence. The design +removes it by refusing a control-cluster request whose GitTarget mirrors another cluster. + +**F4. Two servers for one job, already drifting.** + +- The reconciler has these, and the remote server lacks them: + - `save_requests_total`; + - the phase reasons (`WaitingForWorker` / `WaitingForWindow` / `CollectingWindow` / + `WaitingForPush`); + - `attach: Next`; + - `CommitEmpty`; + - the `AuthorAttributed` condition. +- The remote server has these, and the reconciler lacks them: + - `status.author`; + - `AuthorUnresolved`; + - a 5s default instead of 2s, because its timing model differs. +- `gitTargetUID` handling exists in both, written twice: `refuseOnceUnheld` / `readmit`. +- Every later fix has to be made twice, or the two drift further. The design keeps two ways of + picking a request up, but with one core behind them. + +**F5. Takeover is the largest piece of policy on the branch, and it serves one case.** + +- It takes three conditions, two enums of three values, the provider latch, a per-grant bound, + "max over admitting grants", and two commit trailers (`ee8946c5` + `cf3b97be`: +767/−171 lines). +- It exists only for a control-cluster save on a source cluster with no audit. +- Its name can come only from admission (source-side-save-plan l.260-262), so it is the one feature + that makes admission load-bearing. + +**F6. The WatchRule check on the same endpoint is feedback, not a gate.** + +- The writer refuses the configuration whether or not the webhook ran. +- The GitTarget already reports it as `MultipleSourceNamespaces` + (`internal/controller/gittarget_controller.go:91`). +- `failurePolicy: Ignore` means the webhook was never the guarantee. + +**F7. `/validate-all` is dead weight in e2e.** + +- It always allows, and matches every write on every resource with `timeoutSeconds: 1` + (`config/webhook/validating-webhook.yaml`). +- The chart does not ship it. +- Every API write in the e2e system under test makes an extra webhook round trip for nothing. + +**F8. No e2e proves a save's author round trip on the audit path.** + +- The kcp corner runs without audit (`test/e2e/setup/kcp/instance.yaml:11-14`), so + `RemoteSaveRequest` is only exercised on the unnamed-attach path + (`test/e2e/source_cluster_e2e_test.go:872`). +- The main suite runs with audit on, against the in-cluster provider. Once the control-cluster path + takes its author from audit instead of admission, `save_request_e2e_test.go` proves the round + trip with no change to where its requests are created. + +**F9. Two grant lists with opposite defaults on one GitTarget.** + +- `allowedSaveRequests` defaults to `from: Same`, with no `omitempty`, a Go mirror and an envtest. +- `allowedRemoteSaveRequests` defaults to none. +- A reader has to learn both, and why they differ. The design has one list and one default. + +**F10. Docs gap.** + +- [security-model.md](../security-model.md) does not list the admission endpoint or the + command-author Redis records as a trust boundary, and its Redis row says "Buffers decoded audit + events". +- The gap goes away with the endpoint. + +**F11. Two KubeProviders on one cluster fight over a request.** + +- `RemoteSaveRequestServer.admit` returns `NotAllowed`, "GitTarget … mirrors another cluster than + this one", when the request's GitTarget names a different KubeProvider + (`internal/watch/remote_save_requests.go:480`). +- `serve` writes that as a final `Ready=False`, and a finished `Ready` is never served again + (l.353). +- Two providers enabled on one cluster therefore race: for example the chart's `local` provider + next to a team's own `inCluster` one, or two kubeconfigs with different credentials. If the + provider that does not serve the GitTarget sees the request first, the request fails. +- It also waits for the author before it checks the GitTarget, so the wrong provider spends the + attribution grace first. +- The design fixes it, because the GitTarget's provider alone decides who serves + ([Which path serves a request](#which-path-serves-a-request)). + +## Options considered + +- **A: remove admission and takeover, keep both kinds.** Not chosen. Two kinds would keep F4 and + F9. +- **B: one kind on the source cluster only**, in the first version of this review named + `SaveRequest` in `source.configbutler.ai`. It was chosen at first and later revised. It made + CommitRequest the only operator kind that a single-cluster user reaches through a KubeProvider, + with a provider switch, a grant in source-cluster terms, a derived watch scope and CRD conditions + on the `local` provider. +- **B': one kind, two ways of picking it up, one core.** **Chosen.** A single-cluster user gets the + behavior of `main`, and other clusters get the special path. +- **C: no save kind at all.** Not chosen. It removes the user's message on their commit and the + `catchUp: AfterPush` signal built for gitops-api. + +## What goes, by numbers + +Line counts are `wc -l` at `70d07d91`. Rows marked "≈" are portions of shared files, estimated from +the functions and blocks involved. + +### Admission and takeover + +| Item | Non-test | Test | +| --- | ---: | ---: | +| `validate_operator_types_handler.go`, `admission_allow_handler.go`, `watchrule_source_namespace_admission.go` | 339 | 390 | +| `internal/queue/command_author_store.go` | 84 | 83 | +| `saverequest_attribution_agreement_test.go` | | 152 | +| `cmd/main.go`: flags, validation, server setup, store wiring (48 lines mention admission) | ≈110 | ≈40 | +| Takeover: API enums, grant field, `claimOn`, `claims`, trailers, `openWindow.claim` | ≈250 | ≈280 | +| Chart: `admission-server-certificate.yaml`, `validate-operator-types-webhook.yaml`, values, deployment, services, helpers | ≈150 | | +| `config/`: `webhook/` (124), admission `Certificate`, deployment, service and kustomization lines | ≈170 | | +| Docs: `spec/saverequest-admission-authorship.md` (68), sections in where-validation-lives, quickstart step 1, chart README rows | ≈150 | | +| **Subtotal** | **≈1,450** | **≈950** | + +### One kind, one core + +| Item | Non-test | Test | +| --- | ---: | ---: | +| The `source.configbutler.ai` group: `api/source/v1alpha1` (411) and its CRD (299), plus scheme, PROJECT and kustomize lines | ≈740 | | +| The control-cluster reconciler cut down to a front end over the shared core: 1,468 lines become about 300 | ≈1,170 | ≈1,200 | +| `requestSelector`, and the second grant list merged into one | ≈80 | ≈100 | +| RBAC rules, deepcopy, `hack/crdfields` rows | ≈80 | | +| Added back: F11's dispatch rule, the skipped await, the `NotAllowed` message for a GitTarget that mirrors another cluster | ≈−150 | ≈−250 | +| Added back: the revision check, both KubeProvider conditions, the derived watch scope | ≈−250 | ≈−350 | +| **Subtotal** | **≈1,670** | **≈700** | + +**Total: about 3,100 non-test lines (about 700 of them generated CRD and YAML) and about 1,650 +test lines, roughly 4,800 net.** + +- This is less than option B's 6,600, because the control-cluster path stays. That buys the + behavior of `main` for single-cluster users. +- The `v1alpha3` `CommitRequest` type and CRD stay, with the remote kind's spec and status. +- The rename of `b86dffbe` is undone for the kind. That is a large diff but adds no lines. +- `save_request_e2e_test.go` (620 lines) stays where it is. Only its kind name and its assertions + on the author change. + +### Kept: `attach: Next` and `whenNothingToCommit: CommitEmpty` + +Both stay in the design (decided 2026-10-27). They move into the shared core, so the source-cluster +path gains them, and the counts above are unchanged. The source-cluster server never had them, so +the move needs tests on that path: `Next` skipping an open window, and `CommitEmpty` making a +commit when there is nothing to save. + +### Settings + +| Kind | Removed | Added | +| --- | --- | --- | +| Operator flags | `--admission-webhook`, `--admission-webhook-bind-address`, `--admission-webhook-cert-path`, `--admission-webhook-cert-name`, `--admission-webhook-cert-key` | | +| Chart values | `servers.admission.{enabled, bindAddress, port, timeoutSeconds, tls.certManager, tls.certPath, tls.certName, tls.certKey, tls.secretNameOverride}` | | +| Install prerequisites | cert-manager for the default install (still needed with attribution) | | +| Kubernetes objects | `ValidatingWebhookConfiguration`, the admission `Certificate` and its Secret, Service port 9443, the `source.configbutler.ai` API group and its CRD | | +| Redis | `author:v1:command:*` | | +| KubeProvider | `spec.remoteSaveRequests` | `spec.commitRequests`, not with `inCluster`; condition `CommitRequestsServed`, printer column `Saves`, `status.commitRequests.namespaces`; in phase 2, condition `CommitRequestCRDUpToDate` | +| GitTarget | `spec.allowedSaveRequests`, `spec.allowedRemoteSaveRequests`, `unattributedWindows`, `requestSelector` | `spec.allowedCommitRequests`, whose grants set `consumerNamespaces` or `sourceNamespaces` | +| CommitRequest | `spec.window.author` | `gitTargetRef.namespace` (optional; required on a source cluster, where a request without it is not picked up), `catchUp` and `status.author` | +| CRD | | the schema revision line in the kind's description, and its hash test | +| Release assets | | `commitrequest-crd.yaml` | +| Conditions and reasons | `AuthorAttributed` (`AttributedFromAdmission`, `CommitterFallback`, `AuthorCaptureDisabled`), `TakeOverNotAllowed` | | +| Commit trailers | `Saved-By:`, `Author-Source:` | | + +Compared with `main`, which has `CommitRequest` and the admission webhook, a single-cluster user +sees only these changes: + +- the request may name a GitTarget in another namespace, which then admits it in + `allowedCommitRequests`; +- the author comes from audit, not admission; +- the webhook is gone, and with it the need for cert-manager when attribution is off. + +Saving on another cluster is new. + +## If takeover is needed again later + +Do not bring back a webhook server for it. Two lighter routes exist, and neither is proposed now: + +- **A `ValidatingAdmissionPolicy` on the source cluster** that requires + `object.spec.savedBy == request.userInfo.username`. + - It is in-tree CEL, GA since Kubernetes 1.30, with no server, certificate or Redis. + - kcp appears to support it per workspace (kcp PR #3743). That is unverified for the release you + run. + - It collides with the declined #5 principle, "authorship is established, never asserted" + ([attribution.md](../spec/attribution.md) l.39-67): the operator cannot prove the policy ran + when the request was created. That objection would have to be overruled knowingly. +- **A `MutatingAdmissionPolicy`** that stamps the creator onto the object. It is beta in 1.34 and + tracked for GA in 1.36, and the same objection applies. + +## What this frees time for: the kcp guide + +The pitch fits in one sentence: edit a data-only cluster, and every edit and every save lands in +Git under the name of the person who made it. + +**There today:** + +- a kcp install as kcp-operator CRs (`test/e2e/setup/kcp/`); +- a kubeconfig per workspace through the front-proxy (`test/e2e/kcp_workspace_test.go`); +- `kcp.io/cluster` stripped from Git; +- `attribution.auditRouteAnnotationKey`, for one shared audit stream. + +**Missing:** + +- the shard audit flags, and the annotation that routes each event to a workspace's KubeProvider; +- per-workspace OIDC (`WorkspaceAuthentication`), so audit names a person rather than an admin; +- installing `commitrequest-crd.yaml` into each workspace, which gitops-api's `workspaceCRDs` + already does in the managed product; +- the JSON-only caveat; +- an e2e with kcp audit on, which would also close F8 for the source-cluster path. + +The e2e instance leaves each of these out on purpose (`instance.yaml:11-14`), and the reference +deployment has them. + +**Order:** an e2e kcp corner with audit on comes first. A test then keeps the guide true, and it +is the first place a save's audit round trip is proven across clusters. + +## Proposed sequence + +Hard cuts, no migration. + +1. **gitops-api switches to the source-side save**: the browser creates a `CommitRequest` in the + workspace. This is the only ordering constraint. Until the cut lands, the kind it creates is + `RemoteSaveRequest`, so gitops-api can also wait and switch once, straight to the final name. +2. **One PR implements the design:** + - Delete the admission server, the command-author store, the takeover and the + `source.configbutler.ai` group. + - Rename the kind back to `CommitRequest`. + - Extract the shared core. The control-cluster reconciler and the source-cluster server become + front ends over it, with the dispatch rule that fixes F11. + - Add `allowedCommitRequests` with its two fields, and `commitRequests` with its CEL rule. + - In the core: acquire the fact stream for `commitrequests`, make status writes conditional on + the resourceVersion with a fresh decision on conflict, claim the request before attaching + (rule 3), and stamp `delete-after` for retention. + - Move `attach: Next` and `whenNothingToCommit: CommitEmpty` into the core, so both paths + have them. + - Scope each provider's watch to the namespaces it lends, with `status.commitRequests.namespaces` + and the `CommitRequestsServed` condition. + - Gate the chart `Issuer`, and move the quickstart's cert-manager step under attribution. + - Rewrite [saverequest-design.md](../spec/saverequest-design.md) to describe the one kind, and + add a sentence to [definitions.md](../definitions.md): CommitRequest is the one operator kind + that can also live on a source cluster. + - Publish the `commitrequest-crd.yaml` release asset, with the schema revision line and its hash + test. + - Tests, described in [Acceptance](#acceptance). +3. **Phase 2 of the CRD signal**, as its own PR: first verify that kcp serves per-workspace OpenAPI + v3, then add the `CommitRequestCRDUpToDate` condition, with a unit test per reason. +4. **The kcp audit e2e corner, then the guide.** + +## Acceptance + +The main PR is done when these pass, besides `task lint`, `task test` and `task test-e2e`: + +- **Overlapping observers**, as unit tests with fake clients: + - a control-cluster request seen by the local controller and by a kubeconfig watcher pointing at + the same cluster, with the namespace omitted and with it set; + - two kubeconfig providers on one source cluster; + - a GitTarget created during the attach wait, in both orders: when the owner's claim wins, the + non-owner cannot overwrite it; when the non-owner's `GitTargetNotFound` wins, the owner neither + attaches nor commits; + - a status conflict after the worker has produced an outcome: the outcome is re-applied, and the + save is not restarted; + - a restart while a request is `Saving`: the owner resumes it. +- **The existing source-cluster tests keep passing** through the extraction of the core, including + the kcp corner in `test/e2e/source_cluster_e2e_test.go`. +- **`attach: Next` and `CommitEmpty` on the source-cluster path**, which has never had them: `Next` + skips an open window and takes the next one, and `CommitEmpty` makes a commit when there is + nothing to save. +- **Delayed audit delivery**, as a unit test: the request's fact arrives late, but within the + grace, on a provider whose `AuditFactsReceived` is not yet True. The request is attributed. +- **Retention**: `delete-after` is stamped and honored on both paths. Removing the annotation keeps + the request. +- **The main e2e suite** asserts that a save's author comes from audit on the control-cluster + path. That is the round trip admission used to stand in for. + +**The kcp audit e2e is not a gate for removing admission.** The source-cluster path never used +admission, and removing admission changes only the control-cluster path, which the main suite +covers with audit on. The kcp audit corner proves the source-cluster round trip, and stays step 5. + +## Review round 2 (2026-10-27) + +An outside review of the previous revision raised five points: + +| Point | Verdict | Change | +| --- | --- | --- | +| A source watcher can refuse a valid control-cluster request | **Accepted.** The scenario needs a kubeconfig that points back at the control cluster. That is not an exotic case: it is how a tenant who is not listed in `--in-cluster-kube-providers` gets its own identity there | `GitTargetNamespaceRequired` is gone, and rule 1 applies: a path writes only what it knows to be true | +| Skipping the author wait before the first audit fact loses authors | **Accepted.** The shortcut was this design's own addition, and the code checks `sentAuditFacts` after the wait. The fix is to not add the shortcut | the wait runs whenever attribution is on | +| Several writers of `GitTargetNotFound` are unsafe | **Accepted, narrower than stated.** It needs two observers of one cluster and a GitTarget created during the attach wait. The cause exists, though: the writer overwrites without deciding again | rule 2: writes are conditional on the resourceVersion, and a conflict means deciding again | +| Adopting the remote TTL breaks `delete-after` retention | **Accepted** | the core keeps the annotation contract on both paths | +| The CRD compatibility protocol is large and unresolved | **Partly accepted.** The status writer does erase fields a newer schema added, and the fallback to reading the CRD cannot see an API that an `APIBinding` provides. Deferring the signal entirely was **not** accepted, because an outdated CRD is a signal you asked for | split into phase 1, discovery only, in the main PR, and phase 2, the revision, as its own PR behind a kcp check. A newer CRD is never served, so no preservation rules are needed | + +Also accepted: the core's responsibilities are now listed, including acquiring the fact stream, +which the control-cluster reconciler never did. + +**Rejected: the kcp audit e2e as a gate for removing admission.** See [Acceptance](#acceptance). + +## Review round 3 (2026-10-27) + +The same reviewer approved phase 1, and raised one invariant and four smaller points: + +| Point | Verdict | Change | +| --- | --- | --- | +| The owner must persist its claim before attaching, because today's `serve` continues after a failed write | **Accepted.** Checked: `write` only logs | rule 3, and both race orders in [Acceptance](#acceptance) | +| Deciding again after a later conflict must not restart a save, and ownership must survive a restart | **Accepted** | under rule 3 | +| The RBAC list omitted `patch` | **Accepted** | listed per mode | +| Not every condition message should say to apply the CRD | **Accepted** | a fix per reason | +| A missing revision means "compatibility unknown", not "from elsewhere" | **Accepted, and taken further.** Phase 1's release asset would otherwise ship without a revision, so every cluster set up in phase 1 would be unjudgeable in phase 2 | the revision line ships in phase 1; phase 2 reports `RevisionUnknown` and keeps serving | +| Keep `Next` and `CommitEmpty` unless their removal is settled | **Accepted**, and since settled: both stay | ported into the core | From 4bf9b3e85cac19fd0cb70ad9d6f06d4d0cda3427 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 09:25:11 +0000 Subject: [PATCH 23/33] refactor(api): CommitRequest is CommitRequest again Undo the SaveRequest rename of b86dffbe for the kind: its Go type and files, the commitrequests.configbutler.ai CRD and its RBAC, the --commit-request-ttl flag and controllerManager.commitRequestTTL value, gitopsreverser_commit_requests_total, and the commit_request values of message_source and timer_source. These are their names in 0.54.0, so the UPGRADING entry for the rename goes with it. The source-cluster kind keeps its RemoteSaveRequest name until it is folded into CommitRequest. Co-Authored-By: Claude Opus 5.5 --- .vale/styles/HouseStyle/Headings.yml | 2 +- README.md | 4 +- .../v1alpha1/remotesaverequest_types.go | 6 +- api/v1alpha3/access_grants.go | 52 +- api/v1alpha3/access_grants_defaults_test.go | 10 +- api/v1alpha3/argocdclaim_types.go | 2 +- ...equest_types.go => commitrequest_types.go} | 60 +- api/v1alpha3/examples_test.go | 4 +- api/v1alpha3/fluxclaim_types.go | 2 +- api/v1alpha3/gitprovider_types.go | 8 +- api/v1alpha3/gittarget_types.go | 18 +- api/v1alpha3/status_accessors.go | 2 +- api/v1alpha3/zz_generated.deepcopy.go | 302 +++++----- charts/gitops-reverser/README.md | 16 +- .../gitops-reverser/templates/deployment.yaml | 2 +- .../validate-operator-types-webhook.yaml | 8 +- charts/gitops-reverser/values.schema.json | 4 +- charts/gitops-reverser/values.yaml | 12 +- cmd/main.go | 28 +- .../bases/configbutler.ai_argocdclaims.yaml | 2 +- ...ml => configbutler.ai_commitrequests.yaml} | 28 +- .../crd/bases/configbutler.ai_fluxclaims.yaml | 2 +- .../crd/bases/configbutler.ai_gittargets.yaml | 214 +++---- config/crd/kustomization.yaml | 2 +- config/rbac/role.yaml | 12 +- config/samples/README.md | 2 +- .../{saverequest.yaml => commitrequest.yaml} | 4 +- config/webhook/validating-webhook.yaml | 10 +- docs/INDEX.md | 10 +- docs/README.md | 2 +- docs/TODO.md | 10 +- docs/UPGRADING.md | 23 +- docs/api-first-publication.md | 16 +- docs/architecture.md | 80 +-- docs/attribution-setup-guide.md | 2 +- docs/bi-directional.md | 4 +- docs/commit-messages.md | 8 +- docs/components.md | 6 +- docs/configuration.md | 82 +-- docs/definitions.md | 6 +- docs/demo/commit-window.md | 12 +- docs/design/applier-echoes.md | 4 +- docs/design/branch-worker-event-model.md | 8 +- .../design/commitrequest-save-wait-options.md | 22 +- docs/design/concurrent-author-windows.md | 4 +- ...e2e-coverage-gaps-and-improvements-plan.md | 2 +- docs/design/e2e-suite-trim.md | 2 +- docs/design/event-pipeline-overview.md | 2 +- docs/design/git-write-preflight.md | 2 +- .../gittarget-branch-worker-pending-writes.md | 4 +- docs/design/gittarget-parent-hardening.md | 2 +- docs/design/gittarget-state-of-affairs.md | 4 +- docs/design/reconcile-triggering.md | 6 +- .../crd-descriptions-in-kubectl-explain.md | 2 +- docs/facts/generated-name-support.md | 20 +- docs/facts/subresources.md | 14 +- docs/finished/redis-key-schema-v3.md | 4 +- ...fig-surface-for-a-structured-repository.md | 6 +- .../direction-and-configuration-surface.md | 4 +- docs/future/ha-gittarget-distribution-plan.md | 6 +- docs/future/source-side-save-plan.md | 48 +- docs/future/task-oriented-docs-and-website.md | 4 +- docs/interpreting-metrics.md | 42 +- docs/proud.md | 2 +- docs/rbac.md | 2 +- docs/review/20261025-crd-text-proposal.md | 2 +- .../20261027-save-request-simplification.md | 2 +- docs/spec/README.md | 4 +- docs/spec/attribution.md | 10 +- docs/spec/commit-window-refactor.md | 12 +- ... => commitrequest-admission-authorship.md} | 14 +- ...uest-design.md => commitrequest-design.md} | 12 +- docs/spec/current-manifest-support-review.md | 2 +- docs/spec/e2e-serial-registry.md | 2 +- docs/spec/e2e-test-design.md | 6 +- docs/spec/status-conditions-guide.md | 6 +- docs/spec/where-validation-lives.md | 4 +- hack/crdfields/fields.yaml | 12 +- internal/auditutil/identity_test.go | 30 +- ...dmission.go => commitrequest_admission.go} | 18 +- ...est.go => commitrequest_admission_test.go} | 28 +- .../access_grants_admission_test.go | 4 +- ...equest_apply.go => commitrequest_apply.go} | 30 +- ...ly_test.go => commitrequest_apply_test.go} | 68 +-- ...mmitrequest_attribution_agreement_test.go} | 26 +- ...troller.go => commitrequest_controller.go} | 552 +++++++++--------- ...st.go => commitrequest_controller_test.go} | 38 +- ... => commitrequest_controller_unit_test.go} | 530 ++++++++--------- ..._finalize.go => commitrequest_finalize.go} | 166 +++--- ..._test.go => commitrequest_kstatus_test.go} | 10 +- ...o => commitrequest_outcome_metric_test.go} | 124 ++-- ..._ttl_test.go => commitrequest_ttl_test.go} | 120 ++-- internal/controller/constants.go | 12 +- .../duration_fields_admission_test.go | 42 +- internal/controller/gitbranch_controller.go | 4 +- .../controller/gitbranch_controller_test.go | 2 +- .../controller/gittarget_publication_test.go | 8 +- .../grant_defaults_apiserver_test.go | 16 +- .../controller/resource_condition_metrics.go | 6 +- .../resource_condition_metrics_test.go | 4 +- internal/controller/status.go | 4 +- internal/controller/suite_test.go | 2 +- internal/git/applier_claim.go | 12 +- internal/git/applier_claim_test.go | 2 +- internal/git/apply_attempt_test.go | 4 +- internal/git/apply_converge_test.go | 48 +- internal/git/attribution_outcome_test.go | 36 +- internal/git/branch_worker.go | 132 ++--- internal/git/branch_worker_empty_repo_test.go | 2 +- internal/git/branch_worker_metrics_test.go | 14 +- .../git/branch_worker_queue_depth_test.go | 2 +- internal/git/commit.go | 4 +- internal/git/commit_executor.go | 50 +- internal/git/commit_executor_test.go | 6 +- ...est_attach.go => commit_request_attach.go} | 72 +-- ..._loop.go => commit_request_attach_loop.go} | 166 +++--- ..._test.go => commit_request_attach_test.go} | 46 +- ...y_test.go => commit_request_empty_test.go} | 4 +- ...est_owners.go => commit_request_owners.go} | 14 +- ...est.go => commit_request_withdraw_test.go} | 60 +- internal/git/commit_window_metrics_test.go | 2 +- internal/git/commit_window_timers_test.go | 8 +- internal/git/decided_write_test.go | 4 +- internal/git/dirty_worktree_recovery_test.go | 2 +- internal/git/intake_test.go | 8 +- internal/git/network_bound_test.go | 4 +- internal/git/open_window.go | 26 +- internal/git/parent_recovery_test.go | 10 +- internal/git/pending_writes.go | 16 +- internal/git/pending_writes_loop.go | 12 +- internal/git/render_fidelity_gate.go | 2 +- internal/git/render_fidelity_gate_test.go | 4 +- internal/git/replay_failure_test.go | 12 +- internal/git/replay_refusal_test.go | 6 +- internal/git/request_template_test.go | 14 +- internal/git/resync_push_test.go | 6 +- internal/git/retry.go | 2 +- internal/git/retry_test.go | 8 +- internal/git/save_apply.go | 40 +- internal/git/types.go | 30 +- internal/git/worker_manager.go | 20 +- internal/git/worker_repo_identity_test.go | 2 +- internal/git/write_gate.go | 6 +- internal/git/write_gate_test.go | 36 +- internal/telemetry/exporter.go | 24 +- internal/watch/event_router.go | 52 +- internal/watch/event_router_test.go | 76 +-- internal/watch/remote_save_requests.go | 26 +- internal/watch/remote_save_requests_test.go | 24 +- internal/watch/resource_policy.go | 2 +- internal/watch/target_watch.go | 2 +- .../validate_operator_types_handler.go | 10 +- .../validate_operator_types_handler_test.go | 18 +- test/e2e/Taskfile-e2e.yml | 2 +- test/e2e/bi_directional_echoes_e2e_test.go | 10 +- test/e2e/cluster/audit/policy.yaml | 4 +- ...e2e_test.go => commit_request_e2e_test.go} | 218 +++---- test/e2e/e2e_suite_test.go | 16 +- test/e2e/flux_applier_e2e_test.go | 18 +- test/e2e/helpers.go | 10 +- test/e2e/source_cluster_e2e_test.go | 32 +- 161 files changed, 2353 insertions(+), 2372 deletions(-) rename api/v1alpha3/{saverequest_types.go => commitrequest_types.go} (90%) rename config/crd/bases/{configbutler.ai_saverequests.yaml => configbutler.ai_commitrequests.yaml} (96%) rename config/samples/{saverequest.yaml => commitrequest.yaml} (94%) rename docs/spec/{saverequest-admission-authorship.md => commitrequest-admission-authorship.md} (87%) rename docs/spec/{saverequest-design.md => commitrequest-design.md} (96%) rename internal/authz/{saverequest_admission.go => commitrequest_admission.go} (61%) rename internal/authz/{saverequest_admission_test.go => commitrequest_admission_test.go} (70%) rename internal/controller/{saverequest_apply.go => commitrequest_apply.go} (77%) rename internal/controller/{saverequest_apply_test.go => commitrequest_apply_test.go} (59%) rename internal/controller/{saverequest_attribution_agreement_test.go => commitrequest_attribution_agreement_test.go} (86%) rename internal/controller/{saverequest_controller.go => commitrequest_controller.go} (58%) rename internal/controller/{saverequest_controller_test.go => commitrequest_controller_test.go} (82%) rename internal/controller/{saverequest_controller_unit_test.go => commitrequest_controller_unit_test.go} (70%) rename internal/controller/{saverequest_finalize.go => commitrequest_finalize.go} (67%) rename internal/controller/{saverequest_kstatus_test.go => commitrequest_kstatus_test.go} (92%) rename internal/controller/{saverequest_outcome_metric_test.go => commitrequest_outcome_metric_test.go} (74%) rename internal/controller/{saverequest_ttl_test.go => commitrequest_ttl_test.go} (51%) rename internal/git/{save_request_attach.go => commit_request_attach.go} (86%) rename internal/git/{save_request_attach_loop.go => commit_request_attach_loop.go} (74%) rename internal/git/{save_request_attach_test.go => commit_request_attach_test.go} (96%) rename internal/git/{save_request_empty_test.go => commit_request_empty_test.go} (99%) rename internal/git/{save_request_owners.go => commit_request_owners.go} (80%) rename internal/git/{save_request_withdraw_test.go => commit_request_withdraw_test.go} (83%) rename test/e2e/{save_request_e2e_test.go => commit_request_e2e_test.go} (73%) diff --git a/.vale/styles/HouseStyle/Headings.yml b/.vale/styles/HouseStyle/Headings.yml index e9967cbe..7279f21d 100644 --- a/.vale/styles/HouseStyle/Headings.yml +++ b/.vale/styles/HouseStyle/Headings.yml @@ -54,7 +54,7 @@ exceptions: - WatchRule - ClusterWatchRule - KubeProvider - - SaveRequest + - CommitRequest - RemoteSaveRequest - APIResourceCatalog - HelmRelease diff --git a/README.md b/README.md index c77e69e4..71582845 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ What sets it apart: a commit names the identity the API server authenticated, and no field lets anyone type a name in. It needs kube-apiserver flags, so managed control planes that hide them cannot use it. See [attribution](docs/spec/attribution.md#authorship-is-established-never-asserted). -- **A commit can say why.** Changes are grouped into commit windows, and a `SaveRequest` lets the +- **A commit can say why.** Changes are grouped into commit windows, and a `CommitRequest` lets the person making a change explain **why** it was necessary. See [commit windows](#commit-windows). ## How it works @@ -94,7 +94,7 @@ renders: the operator cannot tell authored resources from generated ones. See ### Commit windows Changes for the same target and author share a commit window, and each window becomes one commit. -When a window closes is yours to tune: its timers are set per `GitTarget`, and a `SaveRequest` can +When a window closes is yours to tune: its timers are set per `GitTarget`, and a `CommitRequest` can bring its own timers along with the commit message. ![Kubernetes resource changes flow through GitOps Reverser's commit window into Git](docs/images/commit-window.excalidraw.svg) diff --git a/api/source/v1alpha1/remotesaverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go index 159af3cd..266153f6 100644 --- a/api/source/v1alpha1/remotesaverequest_types.go +++ b/api/source/v1alpha1/remotesaverequest_types.go @@ -6,13 +6,13 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -// A RemoteSaveRequest is the source-cluster twin of a configbutler.ai SaveRequest, for the people who edit +// A RemoteSaveRequest is the source-cluster twin of a configbutler.ai CommitRequest, for the people who edit // a source cluster and cannot reach the cluster GitOps Reverser runs in. It is its own kind, in its own -// group, rather than a SaveRequest with another schema: its GitTarget is on another cluster than the +// group, rather than a CommitRequest with another schema: its GitTarget is on another cluster than the // request, its author comes from the source cluster's audit events rather than from admission, and it // is installed alone. docs/future/source-side-save-plan.md has the reasoning. // -// The spec is immutable, compared as SaveRequestSpec compares it: durations as durations, so a typed +// The spec is immutable, compared as CommitRequestSpec compares it: durations as durations, so a typed // client's re-spelling of "1m" as "1m0s" is not a change. // RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It diff --git a/api/v1alpha3/access_grants.go b/api/v1alpha3/access_grants.go index c66ee4d5..6cf7234c 100644 --- a/api/v1alpha3/access_grants.go +++ b/api/v1alpha3/access_grants.go @@ -219,16 +219,16 @@ type KubeProviderApplierClaimGrant struct { ApplierNamespaces []ScopeNamespace `json:"applierNamespaces"` } -// GitTargetSaveRequestGrant admits the SaveRequests of some namespaces to save -// a GitTarget's commit windows, and bounds what they may claim of a window -// whose writes name no author. -type GitTargetSaveRequestGrant struct { - // ConsumerNamespaces are the namespaces whose SaveRequests this grant +// GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to +// save a GitTarget's commit windows, and bounds what they may claim of a +// window whose writes name no author. +type GitTargetCommitRequestGrant struct { + // ConsumerNamespaces are the namespaces whose CommitRequests this grant // admits. from: "Same" is this GitTarget's namespace. // +required ConsumerNamespaces ConsumerNamespaces `json:"consumerNamespaces"` - // UnattributedWindows says how much an admitted SaveRequest may claim of a + // UnattributedWindows says how much an admitted CommitRequest may claim of a // commit window whose writes name no author, when it asks in // spec.window.author. It acts only while the GitTarget's KubeProvider has // never received an audit fact; on a cluster that has, a write naming no @@ -251,10 +251,10 @@ type GitTargetSaveRequestGrant struct { UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` } -// SaveRequestAdmission is what a GitTarget's allowedSaveRequests give a -// SaveRequest. +// CommitRequestAdmission is what a GitTarget's allowedCommitRequests give a +// CommitRequest. // +kubebuilder:object:generate=false -type SaveRequestAdmission struct { +type CommitRequestAdmission struct { // Admitted reports whether some grant admits the request's namespace. Admitted bool // UnattributedWindows is the most the admitting grants allow the request to @@ -262,15 +262,15 @@ type SaveRequestAdmission struct { UnattributedWindows UnattributedWindowPolicy } -// AdmitsSaveRequest reports what spec.allowedSaveRequests gives a SaveRequest in namespace, by the +// AdmitsCommitRequest reports what spec.allowedCommitRequests gives a CommitRequest in namespace, by the // labels that namespace carries in the control cluster. -func (t *GitTarget) AdmitsSaveRequest(namespace string, nsLabels map[string]string) (SaveRequestAdmission, error) { - grants := t.saveRequestGrants() +func (t *GitTarget) AdmitsCommitRequest(namespace string, nsLabels map[string]string) (CommitRequestAdmission, error) { + grants := t.commitRequestGrants() admitting, err := admitAll(consumersOf(grants), t.Namespace, namespace, nsLabels) if err != nil || len(admitting) == 0 { - return SaveRequestAdmission{}, err + return CommitRequestAdmission{}, err } - out := SaveRequestAdmission{Admitted: true, UnattributedWindows: UnattributedWindowsKeep} + out := CommitRequestAdmission{Admitted: true, UnattributedWindows: UnattributedWindowsKeep} for _, i := range admitting { policy := grants[i].UnattributedWindows if policy != "" && !out.UnattributedWindows.Allows(WindowAuthor(policy)) { @@ -280,9 +280,9 @@ func (t *GitTarget) AdmitsSaveRequest(namespace string, nsLabels map[string]stri return out, nil } -// DescribeAllowedSaveRequests names the namespaces spec.allowedSaveRequests admits. -func (t *GitTarget) DescribeAllowedSaveRequests() string { - return describeAdmitted(consumersOf(t.saveRequestGrants()), t.Namespace) +// DescribeAllowedCommitRequests names the namespaces spec.allowedCommitRequests admits. +func (t *GitTarget) DescribeAllowedCommitRequests() string { + return describeAdmitted(consumersOf(t.commitRequestGrants()), t.Namespace) } // SourceNamespace is a namespace of a source cluster, by exact name, or "*" @@ -369,10 +369,10 @@ func DefaultKubeProviderApplierClaimGrants() []KubeProviderApplierClaimGrant { }} } -// DefaultGitTargetSaveRequestGrants is GitTarget.spec.allowedSaveRequests when it is omitted: -// SaveRequests in the GitTarget's own namespace, claiming nothing of an unattributed window. -func DefaultGitTargetSaveRequestGrants() []GitTargetSaveRequestGrant { - return []GitTargetSaveRequestGrant{{ +// DefaultGitTargetCommitRequestGrants is GitTarget.spec.allowedCommitRequests when it is omitted: +// CommitRequests in the GitTarget's own namespace, claiming nothing of an unattributed window. +func DefaultGitTargetCommitRequestGrants() []GitTargetCommitRequestGrant { + return []GitTargetCommitRequestGrant{{ ConsumerNamespaces: ConsumerNamespaces{From: GrantFromSame}, UnattributedWindows: UnattributedWindowsKeep, }} @@ -401,13 +401,13 @@ func (p *KubeProvider) applierClaimGrants() []KubeProviderApplierClaimGrant { return orDefault(p.Spec.AllowedApplierClaims, DefaultKubeProviderApplierClaimGrants) } -// saveRequestGrants is spec.allowedSaveRequests as the API server serves it. -func (t *GitTarget) saveRequestGrants() []GitTargetSaveRequestGrant { - return orDefault(t.Spec.AllowedSaveRequests, DefaultGitTargetSaveRequestGrants) +// commitRequestGrants is spec.allowedCommitRequests as the API server serves it. +func (t *GitTarget) commitRequestGrants() []GitTargetCommitRequestGrant { + return orDefault(t.Spec.AllowedCommitRequests, DefaultGitTargetCommitRequestGrants) } -// consumers is the admitting half of a GitTargetSaveRequestGrant. -func (g *GitTargetSaveRequestGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } +// consumers is the admitting half of a GitTargetCommitRequestGrant. +func (g *GitTargetCommitRequestGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } // consumers is the admitting half of a GitProviderGitBranchGrant. func (g *GitProviderGitBranchGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } diff --git a/api/v1alpha3/access_grants_defaults_test.go b/api/v1alpha3/access_grants_defaults_test.go index 649f19c5..58332ce4 100644 --- a/api/v1alpha3/access_grants_defaults_test.go +++ b/api/v1alpha3/access_grants_defaults_test.go @@ -42,9 +42,9 @@ func TestGrantDefaultsMatchCRDs(t *testing.T) { assertCRDDefault(t, "configbutler.ai_kubeproviders.yaml", "allowedApplierClaims", DefaultKubeProviderApplierClaimGrants()) }) - t.Run("GitTarget allowedSaveRequests", func(t *testing.T) { - assertCRDDefault(t, "configbutler.ai_gittargets.yaml", "allowedSaveRequests", - DefaultGitTargetSaveRequestGrants()) + t.Run("GitTarget allowedCommitRequests", func(t *testing.T) { + assertCRDDefault(t, "configbutler.ai_gittargets.yaml", "allowedCommitRequests", + DefaultGitTargetCommitRequestGrants()) }) } @@ -77,11 +77,11 @@ func TestGrantLists_EmptySurvivesSerialization(t *testing.T) { } } - raw, err := json.Marshal(GitTargetSpec{AllowedSaveRequests: []GitTargetSaveRequestGrant{}}) + raw, err := json.Marshal(GitTargetSpec{AllowedCommitRequests: []GitTargetCommitRequestGrant{}}) require.NoError(t, err) var targetFields map[string]json.RawMessage require.NoError(t, json.Unmarshal(raw, &targetFields)) - assert.JSONEq(t, "[]", string(targetFields["allowedSaveRequests"]), "GitTarget spec.allowedSaveRequests") + assert.JSONEq(t, "[]", string(targetFields["allowedCommitRequests"]), "GitTarget spec.allowedCommitRequests") var decoded GitBranchSpec require.NoError(t, json.Unmarshal([]byte(`{"allowedGitTargets":[]}`), &decoded)) diff --git a/api/v1alpha3/argocdclaim_types.go b/api/v1alpha3/argocdclaim_types.go index 392e738e..eeedf011 100644 --- a/api/v1alpha3/argocdclaim_types.go +++ b/api/v1alpha3/argocdclaim_types.go @@ -186,7 +186,7 @@ type ArgoCDClaimSource struct { // - after objects were created through the API in its folders; // - once at startup, and when a GitTarget first names this ArgoCDClaim; // - once spec.catchUpAfter passed since a push of GitOps Reverser's own; -// - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. +// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type ArgoCDClaim struct { diff --git a/api/v1alpha3/saverequest_types.go b/api/v1alpha3/commitrequest_types.go similarity index 90% rename from api/v1alpha3/saverequest_types.go rename to api/v1alpha3/commitrequest_types.go index 9dfe91d6..4f7ae964 100644 --- a/api/v1alpha3/saverequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -8,7 +8,7 @@ import ( "k8s.io/apimachinery/pkg/types" ) -// SaveRequestSpec defines the desired state of SaveRequest. The spec is +// CommitRequestSpec defines the desired state of CommitRequest. The spec is // immutable after creation: a CEL validation rule rejects any update that // changes it, so a delayed audit event always acts on the spec the object was // created with. @@ -21,14 +21,14 @@ import ( // does not round-trip as one: a request created with "1m" reads into Go as a // time.Duration and serializes back as "1m0s", so every typed update — // including one that only touches metadata — was rejected as a spec change. -// SaveRequestWindow compares its durations as durations, which compares what +// CommitRequestWindow compares its durations as durations, which compares what // the fields mean instead of how they were spelled. -// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="SaveRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) == has(oldSelf.whenNothingToCommit) && (!has(self.whenNothingToCommit) || self.whenNothingToCommit == oldSelf.whenNothingToCommit) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="CommitRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.whenNothingToCommit) || self.whenNothingToCommit != 'CommitEmpty' || has(self.message)",message="whenNothingToCommit: CommitEmpty requires spec.message, because a message is all an empty commit records" -type SaveRequestSpec struct { +type CommitRequestSpec struct { // GitTargetRef names the GitTarget whose commit window this request // attaches to: in this namespace, or namespace names another. The - // GitTarget's spec.allowedSaveRequests must admit this namespace, or the + // GitTarget's spec.allowedCommitRequests must admit this namespace, or the // request fails with reason "NotAllowed". // +required // +kubebuilder:validation:XValidation:rule="self.name != ''",message="spec.gitTargetRef.name must not be empty" @@ -61,7 +61,7 @@ type SaveRequestSpec struct { // window within 2s and closes it 2s later. // +optional // +kubebuilder:default={attach: "CurrentOrNext", author: "Own", attachTimeout: "2s", maxDuration: "2s"} - Window *SaveRequestWindow `json:"window,omitempty"` + Window *CommitRequestWindow `json:"window,omitempty"` // WhenNothingToCommit decides what happens when the request ends with nothing // to commit: attachTimeout ran out with no window, or the window closed @@ -89,7 +89,7 @@ type SaveRequestSpec struct { CatchUp CatchUpPolicy `json:"catchUp,omitempty"` } -// CatchUpPolicy is when a SaveRequest's commit is applied by the GitTarget's +// CatchUpPolicy is when a CommitRequest's commit is applied by the GitTarget's // applier. type CatchUpPolicy string @@ -100,7 +100,7 @@ const ( CatchUpAfterPush CatchUpPolicy = "AfterPush" ) -// NothingToCommitAction is what a SaveRequest does when it ends with nothing +// NothingToCommitAction is what a CommitRequest does when it ends with nothing // to commit. type NothingToCommitAction string @@ -111,7 +111,7 @@ const ( NothingToCommitCommitEmpty NothingToCommitAction = "CommitEmpty" ) -// AttachPolicy selects the commit window a SaveRequest attaches to. +// AttachPolicy selects the commit window a CommitRequest attaches to. type AttachPolicy string const ( @@ -124,7 +124,7 @@ const ( AttachNext AttachPolicy = "Next" ) -// WindowAuthor is whose commit window a SaveRequest may attach to. +// WindowAuthor is whose commit window a CommitRequest may attach to. type WindowAuthor string const ( @@ -138,17 +138,17 @@ const ( WindowAuthorTakeOver WindowAuthor = "TakeOver" ) -// SaveRequestWindow is how a SaveRequest attaches to a commit window and +// CommitRequestWindow is how a CommitRequest attaches to a commit window and // when that window closes. A window is only ever opened by a write; a request // waits for one and never opens one. // // Every duration shares the shape GitTargetCommitSpec documents: a Go duration // string behind a pattern and a CEL bound. The immutability rule compares them -// as durations, for the reason SaveRequestSpec gives. -// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="SaveRequest spec is immutable after creation" +// as durations, for the reason CommitRequestSpec gives. +// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" // +kubebuilder:validation:XValidation:rule="self.attach != 'Next' || duration(self.attachTimeout) > duration('0s')",message="attach: Next needs a positive attachTimeout; with 0s it would close the author's window and give up in the same instant" -type SaveRequestWindow struct { +type CommitRequestWindow struct { // Attach selects the window. "CurrentOrNext" attaches to the author's open // window, or to the next one a write opens. "Next" closes the author's open // window under its own message first, and attaches to the next one; it @@ -163,7 +163,7 @@ type SaveRequestWindow struct { // to the creator's own window. "SavedBy" may also give its message to a // window whose writes name no author, naming no author. "TakeOver" may // also take such a window over, and the creator authors the commit. The - // unattributedWindows of a grant in the GitTarget's spec.allowedSaveRequests + // unattributedWindows of a grant in the GitTarget's spec.allowedCommitRequests // that admits this namespace must allow what is asked, or the request fails // with reason TakeOverNotAllowed. Defaults to "Own". // +optional @@ -203,7 +203,7 @@ type SaveRequestWindow struct { MaxDuration *metav1.Duration `json:"maxDuration,omitempty"` } -// SaveRequestStatus defines the observed state of SaveRequest. Progress +// CommitRequestStatus defines the observed state of CommitRequest. Progress // and outcome are reported entirely through conditions (kstatus-compatible), // so the object carries no lifecycle phase string: // @@ -225,12 +225,12 @@ type SaveRequestWindow struct { // - Pushed (domain): True once the commit is in the remote repository. // - Applied (domain): with spec.catchUp "AfterPush", whether the applier // caught up with the commit. It never affects Ready. -type SaveRequestStatus struct { +type CommitRequestStatus struct { // ObservedGeneration is the latest generation observed by the controller. // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty"` - // Conditions are the latest observations of this SaveRequest's state. + // Conditions are the latest observations of this CommitRequest's state. // Reconciling and Stalled follow kstatus. AuthorAttributed says whether the // request's creator, recorded at admission, names the commit author; False // is not a failure and does not affect Ready. @@ -262,7 +262,7 @@ type SaveRequestStatus struct { // +patchMergeKey=type Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` - // GitTargetUID is the UID of the GitTarget whose spec.allowedSaveRequests + // GitTargetUID is the UID of the GitTarget whose spec.allowedCommitRequests // admitted this request. A GitTarget with another UID, such as one created // after the request under the same name, must admit it again. // +optional @@ -297,42 +297,42 @@ type SaveRequestStatus struct { // +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` -// SaveRequest asks for a save: it attaches to its author's current or next +// CommitRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and // gives the commit its message. With spec.window.author, and where the // GitTarget allows it, it may also claim a window whose writes name no // author. Its spec is immutable, and status.commit reports the commit once // it is on the remote. -type SaveRequest struct { +type CommitRequest struct { metav1.TypeMeta `json:",inline"` // metadata is a standard object metadata // +optional metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` - // spec defines the desired state of SaveRequest. It is immutable after + // spec defines the desired state of CommitRequest. It is immutable after // creation. // +required - Spec SaveRequestSpec `json:"spec"` + Spec CommitRequestSpec `json:"spec"` - // status defines the observed state of SaveRequest. + // status defines the observed state of CommitRequest. // +optional - Status SaveRequestStatus `json:"status,omitempty,omitzero"` + Status CommitRequestStatus `json:"status,omitempty,omitzero"` } // +kubebuilder:object:root=true -// SaveRequestList contains a list of SaveRequest. -type SaveRequestList struct { +// CommitRequestList contains a list of CommitRequest. +type CommitRequestList struct { metav1.TypeMeta `json:",inline"` metav1.ListMeta `json:"metadata,omitempty"` - Items []SaveRequest `json:"items"` + Items []CommitRequest `json:"items"` } // GitTargetKey is the GitTarget spec.gitTargetRef names: in the request's namespace unless the ref // names another. -func (s *SaveRequest) GitTargetKey() types.NamespacedName { +func (s *CommitRequest) GitTargetKey() types.NamespacedName { namespace := s.Spec.GitTargetRef.Namespace if namespace == "" { namespace = s.Namespace @@ -341,5 +341,5 @@ func (s *SaveRequest) GitTargetKey() types.NamespacedName { } func init() { - SchemeBuilder.Register(&SaveRequest{}, &SaveRequestList{}) + SchemeBuilder.Register(&CommitRequest{}, &CommitRequestList{}) } diff --git a/api/v1alpha3/examples_test.go b/api/v1alpha3/examples_test.go index f0d94ee3..d1a3c23f 100644 --- a/api/v1alpha3/examples_test.go +++ b/api/v1alpha3/examples_test.go @@ -137,8 +137,8 @@ func newExampleObject(kind string) any { return &WatchRule{} case "ClusterWatchRule": return &ClusterWatchRule{} - case "SaveRequest": - return &SaveRequest{} + case "CommitRequest": + return &CommitRequest{} case ApplierClaimKindFlux: return &FluxClaim{} case ApplierClaimKindArgoCD: diff --git a/api/v1alpha3/fluxclaim_types.go b/api/v1alpha3/fluxclaim_types.go index 83930e73..e50554f2 100644 --- a/api/v1alpha3/fluxclaim_types.go +++ b/api/v1alpha3/fluxclaim_types.go @@ -192,7 +192,7 @@ type FluxClaimSource struct { // - after objects were created through the API in its folders; // - once at startup, and when a GitTarget first names this FluxClaim; // - once spec.catchUpAfter passed since a push of GitOps Reverser's own; -// - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. +// - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. // // GitTargets name it with spec.applierClaimRef. type FluxClaim struct { diff --git a/api/v1alpha3/gitprovider_types.go b/api/v1alpha3/gitprovider_types.go index 01d91771..2de1033a 100644 --- a/api/v1alpha3/gitprovider_types.go +++ b/api/v1alpha3/gitprovider_types.go @@ -224,21 +224,21 @@ type CommitMessageSpec struct { // can be empty, and Generation is 0 for a kind without a spec, so guard // them with {{with}}. // - // A SaveRequest's message replaces this template without changing the + // A CommitRequest's message replaces this template without changing the // author; spec.commit.message.requestTemplate can wrap that message. // +optional LiveTemplate string `json:"liveTemplate,omitempty"` // Rationale, kept out of the doc comment (and so out of the CRD schema) by the blank line - // below: a SaveRequest is created by whoever holds create on it in a namespace, so its + // below: a CommitRequest is created by whoever holds create on it in a namespace, so its // author supplies the message but must not set commit-message policy. Framing the message // rather than parsing it keeps that split — the operator owns the wording, the requester owns // the words, and nothing a requester writes is ever executed. Rejecting a template that never // renders RequestMessage is what keeps the split honest in the other direction: silently // discarding a requester's stated reason would be worse than having no template at all. - // RequestTemplate is the Go template that frames a SaveRequest's message. It - // is used only for a commit whose SaveRequest supplied a message, and gets + // RequestTemplate is the Go template that frames a CommitRequest's message. It + // is used only for a commit whose CommitRequest supplied a message, and gets // liveTemplate's fields plus RequestMessage, the message as written; template // syntax inside the message stays literal. Omitted, the message is committed // as written. A template that never renders RequestMessage is rejected, and diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 48ebeecb..435e46ba 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -186,23 +186,23 @@ type GitTargetSpec struct { // +optional Commit *GitTargetCommitSpec `json:"commit,omitempty"` - // AllowedSaveRequests says which SaveRequests may save this GitTarget's + // AllowedCommitRequests says which CommitRequests may save this GitTarget's // commit windows, by the namespace they are in, and how much each may // claim of a window whose writes name no author: // - // - Omitted: the SaveRequests in this GitTarget's namespace, as one grant + // - Omitted: the CommitRequests in this GitTarget's namespace, as one grant // with from: "Same" that claims nothing. - // - Empty list ([]): no SaveRequest. - // - A list of grants: the SaveRequests that at least one grant allows. + // - Empty list ([]): no CommitRequest. + // - A list of grants: the CommitRequests that at least one grant allows. // - // A list replaces the default: the SaveRequests in this namespace keep + // A list replaces the default: the CommitRequests in this namespace keep // access only if at least one grant allows them. A request no grant allows // fails with reason "NotAllowed". // +optional // +listType=atomic // +kubebuilder:validation:MaxItems=64 // +kubebuilder:default={{consumerNamespaces: {from: Same}, unattributedWindows: Keep}} - AllowedSaveRequests []GitTargetSaveRequestGrant `json:"allowedSaveRequests"` + AllowedCommitRequests []GitTargetCommitRequestGrant `json:"allowedCommitRequests"` // Consent lives on the GitTarget, not on the KubeProvider that serves the requests: the provider's // owner lends a cluster, and whether a save made on it may steer this folder's commits, with its @@ -266,7 +266,7 @@ type GitTargetCommitSpec struct { // Window sets when this GitTarget's commit windows close. A window opens on an // author's first write and collects that author's writes until idleTimeout // passes without one, or maxDuration after it opened, whichever comes first. A - // change applies to windows opened afterward. A SaveRequest attached to a + // change applies to windows opened afterward. A CommitRequest attached to a // window replaces both timers for that window. // +optional Window *CommitWindow `json:"window,omitempty"` @@ -277,7 +277,7 @@ type GitTargetCommitSpec struct { Message *CommitMessageSpec `json:"message,omitempty"` } -// UnattributedWindowPolicy is how much a SaveRequest may claim of a commit +// UnattributedWindowPolicy is how much a CommitRequest may claim of a commit // window whose writes name no author. type UnattributedWindowPolicy string @@ -290,7 +290,7 @@ const ( UnattributedWindowsTakeOver UnattributedWindowPolicy = "TakeOver" ) -// Allows reports whether a SaveRequest asking for author may attach as it asks under this policy: +// Allows reports whether a CommitRequest asking for author may attach as it asks under this policy: // "TakeOver" allows every ask, "SavedBy" allows "SavedBy" and "Own", and "Keep" (or empty) allows only // "Own". An unknown value allows only "Own". func (p UnattributedWindowPolicy) Allows(author WindowAuthor) bool { diff --git a/api/v1alpha3/status_accessors.go b/api/v1alpha3/status_accessors.go index 1480409f..e813db1a 100644 --- a/api/v1alpha3/status_accessors.go +++ b/api/v1alpha3/status_accessors.go @@ -16,7 +16,7 @@ import ( // accessors let the session do it once. See internal/controller's statusObject, which is the // interface these satisfy. // -// SaveRequest is deliberately not here: it does not use that helper (see the comment on +// CommitRequest is deliberately not here: it does not use that helper (see the comment on // requeueAfter for why), so an accessor on it would be a method nothing calls. // SetObservedGeneration records the generation this object's status describes. diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index 75d47d97..e22ff622 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -370,6 +370,138 @@ func (in *CommitMessageSpec) DeepCopy() *CommitMessageSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CommitRequest) DeepCopyInto(out *CommitRequest) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequest. +func (in *CommitRequest) DeepCopy() *CommitRequest { + if in == nil { + return nil + } + out := new(CommitRequest) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CommitRequest) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CommitRequestList) DeepCopyInto(out *CommitRequestList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CommitRequest, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestList. +func (in *CommitRequestList) DeepCopy() *CommitRequestList { + if in == nil { + return nil + } + out := new(CommitRequestList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CommitRequestList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CommitRequestSpec) DeepCopyInto(out *CommitRequestSpec) { + *out = *in + out.GitTargetRef = in.GitTargetRef + if in.Window != nil { + in, out := &in.Window, &out.Window + *out = new(CommitRequestWindow) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestSpec. +func (in *CommitRequestSpec) DeepCopy() *CommitRequestSpec { + if in == nil { + return nil + } + out := new(CommitRequestSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CommitRequestStatus) DeepCopyInto(out *CommitRequestStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestStatus. +func (in *CommitRequestStatus) DeepCopy() *CommitRequestStatus { + if in == nil { + return nil + } + out := new(CommitRequestStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CommitRequestWindow) DeepCopyInto(out *CommitRequestWindow) { + *out = *in + if in.AttachTimeout != nil { + in, out := &in.AttachTimeout, &out.AttachTimeout + *out = new(v1.Duration) + **out = **in + } + if in.IdleTimeout != nil { + in, out := &in.IdleTimeout, &out.IdleTimeout + *out = new(v1.Duration) + **out = **in + } + if in.MaxDuration != nil { + in, out := &in.MaxDuration, &out.MaxDuration + *out = new(v1.Duration) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CommitRequestWindow. +func (in *CommitRequestWindow) DeepCopy() *CommitRequestWindow { + if in == nil { + return nil + } + out := new(CommitRequestWindow) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CommitSigningSpec) DeepCopyInto(out *CommitSigningSpec) { *out = *in @@ -1066,6 +1198,22 @@ func (in *GitTarget) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GitTargetCommitRequestGrant) DeepCopyInto(out *GitTargetCommitRequestGrant) { + *out = *in + in.ConsumerNamespaces.DeepCopyInto(&out.ConsumerNamespaces) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetCommitRequestGrant. +func (in *GitTargetCommitRequestGrant) DeepCopy() *GitTargetCommitRequestGrant { + if in == nil { + return nil + } + out := new(GitTargetCommitRequestGrant) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetCommitSpec) DeepCopyInto(out *GitTargetCommitSpec) { *out = *in @@ -1213,22 +1361,6 @@ func (in *GitTargetRetentionStatus) DeepCopy() *GitTargetRetentionStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitTargetSaveRequestGrant) DeepCopyInto(out *GitTargetSaveRequestGrant) { - *out = *in - in.ConsumerNamespaces.DeepCopyInto(&out.ConsumerNamespaces) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetSaveRequestGrant. -func (in *GitTargetSaveRequestGrant) DeepCopy() *GitTargetSaveRequestGrant { - if in == nil { - return nil - } - out := new(GitTargetSaveRequestGrant) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = *in @@ -1264,9 +1396,9 @@ func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { *out = new(GitTargetCommitSpec) (*in).DeepCopyInto(*out) } - if in.AllowedSaveRequests != nil { - in, out := &in.AllowedSaveRequests, &out.AllowedSaveRequests - *out = make([]GitTargetSaveRequestGrant, len(*in)) + if in.AllowedCommitRequests != nil { + in, out := &in.AllowedCommitRequests, &out.AllowedCommitRequests + *out = make([]GitTargetCommitRequestGrant, len(*in)) for i := range *in { (*in)[i].DeepCopyInto(&(*out)[i]) } @@ -1624,138 +1756,6 @@ func (in *ResourceRule) DeepCopy() *ResourceRule { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequest) DeepCopyInto(out *SaveRequest) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequest. -func (in *SaveRequest) DeepCopy() *SaveRequest { - if in == nil { - return nil - } - out := new(SaveRequest) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *SaveRequest) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestList) DeepCopyInto(out *SaveRequestList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]SaveRequest, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestList. -func (in *SaveRequestList) DeepCopy() *SaveRequestList { - if in == nil { - return nil - } - out := new(SaveRequestList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *SaveRequestList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestSpec) DeepCopyInto(out *SaveRequestSpec) { - *out = *in - out.GitTargetRef = in.GitTargetRef - if in.Window != nil { - in, out := &in.Window, &out.Window - *out = new(SaveRequestWindow) - (*in).DeepCopyInto(*out) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestSpec. -func (in *SaveRequestSpec) DeepCopy() *SaveRequestSpec { - if in == nil { - return nil - } - out := new(SaveRequestSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestStatus) DeepCopyInto(out *SaveRequestStatus) { - *out = *in - if in.Conditions != nil { - in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestStatus. -func (in *SaveRequestStatus) DeepCopy() *SaveRequestStatus { - if in == nil { - return nil - } - out := new(SaveRequestStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SaveRequestWindow) DeepCopyInto(out *SaveRequestWindow) { - *out = *in - if in.AttachTimeout != nil { - in, out := &in.AttachTimeout, &out.AttachTimeout - *out = new(v1.Duration) - **out = **in - } - if in.IdleTimeout != nil { - in, out := &in.IdleTimeout, &out.IdleTimeout - *out = new(v1.Duration) - **out = **in - } - if in.MaxDuration != nil { - in, out := &in.MaxDuration, &out.MaxDuration - *out = new(v1.Duration) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SaveRequestWindow. -func (in *SaveRequestWindow) DeepCopy() *SaveRequestWindow { - if in == nil { - return nil - } - out := new(SaveRequestWindow) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *StreamsStatus) DeepCopyInto(out *StreamsStatus) { *out = *in diff --git a/charts/gitops-reverser/README.md b/charts/gitops-reverser/README.md index c5e6da11..c9c610ce 100644 --- a/charts/gitops-reverser/README.md +++ b/charts/gitops-reverser/README.md @@ -19,7 +19,7 @@ For the chart's optional starter `quickstart` block, see [`docs/configuration.md - ✅ **Git synchronization**: Push Kubernetes changes back to Git repositories - ✅ **Single-pod stability**: 1 replica by default while multi-pod support is in progress - ✅ **Automatic CRD installation**: KubeProvider, GitProvider, GitBranch, GitTarget, WatchRule, - ClusterWatchRule, SaveRequest, FluxClaim, and ArgoCDClaim CRDs installed automatically + ClusterWatchRule, CommitRequest, FluxClaim, and ArgoCDClaim CRDs installed automatically - ✅ **Watch-based ingestion**: object state comes from the Kubernetes watch API - ✅ **Optional audit attribution**: name commit authors from kube-apiserver audit events over HTTPS - ✅ **Prometheus metrics**: Built-in monitoring support @@ -30,7 +30,7 @@ For the chart's optional starter `quickstart` block, see [`docs/configuration.md - cert-manager (for TLS certificates) - **Valkey or Redis: optional but advised.** The default install runs without it in `configured-author` mode (watches cold-replay on restart). Add it to unlock warm-restart resume - cursors, SaveRequest author capture (the admission webhook), and attributed-author mode; enabling + cursors, CommitRequest author capture (the admission webhook), and attributed-author mode; enabling any of those requires a non-empty `queue.redis.addr`. - **Optional, attributed-author mode only:** kube-apiserver audit delivery, which adds mirrored-resource commit-author attribution. Without it the operator still mirrors state, authored by @@ -182,7 +182,7 @@ nodeSelector: | `controllerManager.insecureAllowGitHTTP` | Permit credentials with `http://` GitProvider URLs for trusted in-cluster development Git servers | `false` | | `controllerManager.branchWorkerQueueDepth` | Event queue depth per branch worker. A full queue refuses the write (`git_queue_refusals_total`) instead of stalling the watch path; the watch delivers it again from its cursor, and only a cursor that expires first loses that write's own commit (and, under `prune.mode: OnEvent`, a delete's removal). Costs queue depth × payload of pod memory, which `branchBufferMaxSize` does **not** cover | `1000` | | `controllerManager.branchBufferMaxSize` | Maximum in-memory event buffer per branch worker. Bounds the open commit window plus the writes retained for replay until a push succeeds. While the remote cannot be reached, retained writes that reach it close the worker to new writes, saves and resyncs until they are published; the watch delivers a refused event again | `8Mi` | -| `controllerManager.saveRequestTTL` | How long a `SaveRequest` is kept once it finishes. Written onto the request as the `configbutler.ai/delete-after` annotation; remove or edit that to keep one. `"0"` writes none | `48h` | +| `controllerManager.commitRequestTTL` | How long a `CommitRequest` is kept once it finishes. Written onto the request as the `configbutler.ai/delete-after` annotation; remove or edit that to keep one. `"0"` writes none | `48h` | | `git.refreshInterval` | How often an idle `GitTarget` re-proves where its branch is on the Git remote. `"0"` disables it | `10m` | | `applierAccess.flux` | Grant the operator `get`, `list` and `patch` on Flux `Kustomizations`, `get` and `patch` on `GitRepositories`, `list` on `HelmReleases` and `HelmCharts` (to find what else applies a source), and `create` on `SelfSubjectAccessReviews`, so it can hold the `Kustomization` a `FluxClaim` names suspended and resume it for each apply | `false` | | `applierAccess.argocd` | Grant the operator `get`, `update` and `patch` on Argo CD `Applications`, and `create` on `SelfSubjectAccessReviews`, so it can hold the automated sync of the `Application` an `ArgoCDClaim` names off and start its syncs | `false` | @@ -198,7 +198,7 @@ nodeSelector: | `quickstart.gitBranch.branch` | Branch the starter `GitBranch` writes. Must match `quickstart.gitProvider.branchPatterns` | `main` | | `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget`; set `.` only to deliberately target the repo root | `live-cluster` | | `quickstart.watchRule.rules` | Rules used by the starter `WatchRule` | `configmaps create/update/delete` | -| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record SaveRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | +| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record CommitRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | | `queue.redis.auth.existingSecret` | Name of a pre-created Secret holding the Redis password (only used when `queue.redis.addr` is set) | `""` | | `queue.redis.auth.existingSecretKey` | Key within the Secret that holds the password | `password` | | `queue.redis.auth.username` | Optional Redis ACL username | `""` | @@ -220,7 +220,7 @@ nodeSelector: | `kubeProvider.secretRef.key` | Key within that Secret. Empty reads `value` then `value.yaml` | `""` | | `kubeProvider.allowedGitTargets` | Rendered as the provider's `spec.allowedGitTargets`: which namespaces' `GitTarget`s may mirror through it, and what. Each grant has a `consumerNamespaces` block (`from`: `Same`, `Names`, `Selector`, `All`), and `sourceNamespaces`, `clusterResources` or both beside it. **Unset, the field is omitted and the CRD defaults it to the release namespace mirroring its own**; a list is the whole policy, and `[]` admits nobody, the release namespace included: a refused `GitTarget` names `spec.allowedGitTargets`. `quickstart.enabled` adds a grant for the quickstart namespace, and [`values-playground.yaml`](values-playground.yaml) grants every namespace its own. With an `inCluster` provider a grant bypasses live namespace RBAC through the operator's identity | `null` | | `kubeProvider.allowedApplierClaims` | Rendered as the provider's `spec.allowedApplierClaims`: which namespaces' `FluxClaim`s and `ArgoCDClaim`s may hold an applier through it (`consumerNamespaces`), and in which `applierNamespaces` it may live, such as `flux-system`. Admission to mirror is no admission to apply. Unset, the CRD defaults it to the release namespace holding appliers in its own; `[]` admits nobody | `null` | -| `servers.admission.enabled` | Install the validate-operator-types admission webhook that captures SaveRequest authors (a form of author attribution). Enabled by default; a no-op until `queue.redis.addr` is set | `true` | +| `servers.admission.enabled` | Install the validate-operator-types admission webhook that captures CommitRequest authors (a form of author attribution). Enabled by default; a no-op until `queue.redis.addr` is set | `true` | | `rbac.create` | Create the manager ClusterRole and its binding | `true` | | `rbac.watchTypes.mode` | Which types a `WatchRule` may read. `any` grants cluster-wide read on everything, including every Secret in the cluster. `selected` grants read on `rbac.watchTypes.selected` only, so the reverser cannot list or watch Secrets (it keeps `get` on named Secrets it is pointed at). See [`docs/rbac.md`](../../docs/rbac.md) | `any` | | `rbac.watchTypes.selected` | Types to grant when `mode: selected`, as `{apiGroups, resources}` entries (verbs are always `get,list,watch`). Required and non-empty in that mode; `namespaces`, `customresourcedefinitions` and `apiservices` come from the manager role and must not be restated | `[]` | @@ -287,7 +287,7 @@ This chart automatically manages the following CRDs: - **`gittargets.configbutler.ai`** - Branch/path and optional encryption configuration - **`watchrules.configbutler.ai`** - Namespaced watch rules - **`clusterwatchrules.configbutler.ai`** - Cluster-scoped watch rules -- **`saverequests.configbutler.ai`** - One-shot "save now" signals that finalize the open commit window +- **`commitrequests.configbutler.ai`** - One-shot "save now" signals that finalize the open commit window - **`fluxclaims.configbutler.ai`** - Applier claims holding a Flux `Kustomization` for bi-directional GitOps - **`argocdclaims.configbutler.ai`** - Applier claims holding an Argo CD `Application` for bi-directional GitOps @@ -309,7 +309,7 @@ kubectl apply -f https://github.com/ConfigButler/gitops-reverser/releases/downlo To manually remove CRDs after uninstallation: ```bash -kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai saverequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai +kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai commitrequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai ``` Delete the applier claims while the operator still runs, before this ([Uninstallation](#uninstallation)). @@ -400,7 +400,7 @@ helm uninstall gitops-reverser --namespace gitops-reverser kubectl delete namespace gitops-reverser # Delete CRDs (optional, but removes all custom resources) -kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai saverequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai +kubectl delete crd gitproviders.configbutler.ai gittargets.configbutler.ai watchrules.configbutler.ai clusterwatchrules.configbutler.ai commitrequests.configbutler.ai fluxclaims.configbutler.ai argocdclaims.configbutler.ai kubeproviders.configbutler.ai gitbranches.configbutler.ai gitreceivers.configbutler.ai ``` ## Troubleshooting diff --git a/charts/gitops-reverser/templates/deployment.yaml b/charts/gitops-reverser/templates/deployment.yaml index 63df3993..4f74ba4f 100644 --- a/charts/gitops-reverser/templates/deployment.yaml +++ b/charts/gitops-reverser/templates/deployment.yaml @@ -111,7 +111,7 @@ spec: - --branch-worker-queue-depth={{ int .Values.controllerManager.branchWorkerQueueDepth }} - --branch-buffer-max-size={{ .Values.controllerManager.branchBufferMaxSize }} - --git-refresh-interval={{ .Values.git.refreshInterval }} - - --save-request-ttl={{ .Values.controllerManager.saveRequestTTL }} + - --commit-request-ttl={{ .Values.controllerManager.commitRequestTTL }} {{- if .Values.servers.receiver.enabled }} - --receiver-bind-address={{ .Values.servers.receiver.bindAddress }} {{- end }} diff --git a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml b/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml index cfdf2fb0..3d382ebf 100644 --- a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml +++ b/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml @@ -2,10 +2,10 @@ --- # Two unrelated jobs on one endpoint, both narrow by construction. # -# It captures the authenticated submitter of our own command kinds (a SaveRequest today) +# It captures the authenticated submitter of our own command kinds (a CommitRequest today) # into the command-author Redis corner the controller reads back with no wait; a miss # leaves the request without a claimed actor (AuthorAttributed=False). See -# docs/spec/saverequest-admission-authorship.md. +# docs/spec/commitrequest-admission-authorship.md. # # It also rejects a WatchRule that would bring a second source namespace to a GitTarget # declaring spec.serializeNamespace: false. That rejection is FEEDBACK, not enforcement: @@ -32,7 +32,7 @@ webhooks: path: /validate-operator-types port: {{ .Values.servers.admission.port }} # Ignore, not Fail: a missed author capture must leave the request unnamed, never - # reject a user's SaveRequest, and the WatchRule check is feedback the write path + # reject a user's CommitRequest, and the WatchRule check is feedback the write path # already enforces on its own. failurePolicy: Ignore matchPolicy: Equivalent @@ -44,7 +44,7 @@ webhooks: operations: - CREATE resources: - - saverequests + - commitrequests scope: Namespaced # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it # was created, and that edit is the same mistake. diff --git a/charts/gitops-reverser/values.schema.json b/charts/gitops-reverser/values.schema.json index aa69f10b..4eb69f9d 100644 --- a/charts/gitops-reverser/values.schema.json +++ b/charts/gitops-reverser/values.schema.json @@ -118,9 +118,9 @@ "type": "string", "description": "Maximum in-memory event buffer per branch worker, as a Kubernetes resource quantity (e.g. 8Mi). Bounds the open commit window plus the writes retained for replay until a push succeeds." }, - "saveRequestTTL": { + "commitRequestTTL": { "type": "string", - "description": "How long a SaveRequest is kept once it finishes, as a duration (e.g. 48h). Written onto the request as the configbutler.ai/delete-after annotation when it finishes; \"0\" writes none, so requests are kept." + "description": "How long a CommitRequest is kept once it finishes, as a duration (e.g. 48h). Written onto the request as the configbutler.ai/delete-after annotation when it finishes; \"0\" writes none, so requests are kept." } } }, diff --git a/charts/gitops-reverser/values.yaml b/charts/gitops-reverser/values.yaml index 5c23b7e2..e255e890 100644 --- a/charts/gitops-reverser/values.yaml +++ b/charts/gitops-reverser/values.yaml @@ -97,12 +97,12 @@ controllerManager: # included: work that would exceed it pauses the worker's intake of new writes, saves and # resyncs until a push lands, and the watch delivers what was refused once intake reopens. branchBufferMaxSize: 8Mi - # How long a SaveRequest is kept once it finishes, as a duration. Every save leaves one object + # How long a CommitRequest is kept once it finishes, as a duration. Every save leaves one object # behind, so without this they accumulate for as long as the namespace lives. The controller # writes the deletion time onto the request as the annotation configbutler.ai/delete-after when # it finishes; remove or edit that annotation to keep one request or move its deletion. Set "0" # to write no annotation, so new requests are kept. - saveRequestTTL: 48h + commitRequestTTL: 48h # How the operator treats the Git remote. git: @@ -175,16 +175,16 @@ servers: secretNameOverride: "" # Validating admission server hosting the validate-operator-types webhook, which captures - # the submitter of each SaveRequest at admission and records it as the request's named actor + # the submitter of each CommitRequest at admission and records it as the request's named actor # (read back by the controller with no wait). Independent of `attribution` below: # attribution governs *mirrored-resource* authorship (from audit), this governs - # *command* authorship (from admission). Off → SaveRequests claim no actor and report + # *command* authorship (from admission). Off → CommitRequests claim no actor and report # AuthorAttributed=False. See - # docs/spec/saverequest-admission-authorship.md. + # docs/spec/commitrequest-admission-authorship.md. # # Command-author capture is Redis-backed: the captured author is written to Redis at admission and # read back at reconcile. Enabled by default so the webhook is always installed, but it is a no-op - # without Redis: the pod runs healthy and SaveRequests claim no actor + # without Redis: the pod runs healthy and CommitRequests claim no actor # (AuthorAttributed=False) until queue.redis.addr is set. admission: enabled: true diff --git a/cmd/main.go b/cmd/main.go index 3591a21a..89fae16e 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -337,7 +337,7 @@ func main() { Manager: watchMgr, Client: mgr.GetClient(), Finalizer: eventRouter, - TTL: cfg.saveRequestTTL, + TTL: cfg.commitRequestTTL, Log: ctrl.Log.WithName("remotesaverequests"), }), "unable to add the RemoteSaveRequest server") @@ -417,7 +417,7 @@ func main() { // lives in its own Redis corner (author:v1:command), independent of // --author-attribution (which governs mirrored-resource attribution). It is wired // whenever the admission server is on; the controller reads the captured submitter - // back with no wait (docs/spec/saverequest-admission-authorship.md). + // back with no wait (docs/spec/commitrequest-admission-authorship.md). // // AuthorLookup must be a nil interface — not a non-nil interface wrapping a nil // *CommandAuthorStore — when the webhook is off, so the controller's nil check @@ -433,19 +433,19 @@ func main() { "and recorded as the request's named actor") } else { setupLog.Info("validate-operator-types webhook enabled without Redis: command author capture is a " + - "no-op; SaveRequests claim no actor (AuthorAttributed=False). " + + "no-op; CommitRequests claim no actor (AuthorAttributed=False). " + "Set --redis-addr to capture command authors.") } } - if err := (&controller.SaveRequestReconciler{ + if err := (&controller.CommitRequestReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), APIReader: mgr.GetAPIReader(), Finalizer: eventRouter, AuthorLookup: commandAuthorLookup, - TTL: cfg.saveRequestTTL, + TTL: cfg.commitRequestTTL, }).SetupWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create controller", "controller", "SaveRequest") + setupLog.Error(err, "unable to create controller", "controller", "CommitRequest") os.Exit(1) } if cfg.admissionWebhookEnabled { @@ -505,7 +505,7 @@ type appConfig struct { attributionFactTTL time.Duration attributionGrace time.Duration gitRefreshInterval time.Duration - saveRequestTTL time.Duration + commitRequestTTL time.Duration attributionMaxFactsPerType int attributionMaxFacts int attributionCollectionWindow time.Duration @@ -596,7 +596,7 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { "run without Redis: watches cold-replay on restart instead of resuming, and attribution needs "+ "--author-attribution-transport=memory. Required by --author-attribution-transport=redis when "+ "attribution is on. --admission-webhook still runs without it, but command-author capture is a "+ - "no-op: SaveRequests claim no actor.") + "no-op: CommitRequests claim no actor.") fs.BoolVar(&cfg.authorAttribution, "author-attribution", true, "Name the real actor (human or service account) who caused each change as the Git commit author, "+ "resolved from matching audit facts; this runs the audit webhook ingress (default true). When "+ @@ -657,10 +657,10 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { "Bounded per-event wait for a matching audit fact to arrive before a watch event ships unresolved, "+ "authored \"unknown (attribution unresolved)\" (duration string; default 3s). Larger values "+ "raise attribution hit-rate at the cost of commit latency.") - fs.DurationVar(&cfg.saveRequestTTL, "save-request-ttl", controller.DefaultSaveRequestTTL, - "How long a SaveRequest is kept once it finishes (duration string; default 48h, 0 keeps "+ + fs.DurationVar(&cfg.commitRequestTTL, "commit-request-ttl", controller.DefaultCommitRequestTTL, + "How long a CommitRequest is kept once it finishes (duration string; default 48h, 0 keeps "+ "them). The deletion time is written onto the request as the "+ - controller.SaveRequestDeleteAfterAnnotation+" annotation when it finishes; remove or edit "+ + controller.CommitRequestDeleteAfterAnnotation+" annotation when it finishes; remove or edit "+ "that annotation to keep the request or move its deletion.") fs.DurationVar(&cfg.gitRefreshInterval, "git-refresh-interval", controller.DefaultGitRefreshInterval, "How often an IDLE GitTarget re-proves where its branch is on the remote (duration string; "+ @@ -769,8 +769,8 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { } // Rejected rather than read as "keep everything": a negative age has no meaning, and 0 already // says that. - if cfg.saveRequestTTL < 0 { - return appConfig{}, fmt.Errorf("save-request-ttl must be >= 0, got %s", cfg.saveRequestTTL) + if cfg.commitRequestTTL < 0 { + return appConfig{}, fmt.Errorf("commit-request-ttl must be >= 0, got %s", cfg.commitRequestTTL) } bufferQuantity, err := resource.ParseQuantity(branchBufferMaxSizeFlag) @@ -939,7 +939,7 @@ func validateAdmissionWebhookConfig(cfg appConfig) error { return nil } // No redis-addr requirement here: the admission webhook stays enabled without Redis and - // simply no-ops command-author capture (SaveRequests claim no actor). Redis is + // simply no-ops command-author capture (CommitRequests claim no actor). Redis is // wired only when configured; see the commandAuthorStore setup. if _, _, err := splitBindAddress(cfg.admissionWebhookBindAddress); err != nil { return fmt.Errorf("invalid admission-webhook-bind-address %q: %w", cfg.admissionWebhookBindAddress, err) diff --git a/config/crd/bases/configbutler.ai_argocdclaims.yaml b/config/crd/bases/configbutler.ai_argocdclaims.yaml index 9a538d4a..55d99bb1 100644 --- a/config/crd/bases/configbutler.ai_argocdclaims.yaml +++ b/config/crd/bases/configbutler.ai_argocdclaims.yaml @@ -68,7 +68,7 @@ spec: - after objects were created through the API in its folders; - once at startup, and when a GitTarget first names this ArgoCDClaim; - once spec.catchUpAfter passed since a push of GitOps Reverser's own; - - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. + - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: diff --git a/config/crd/bases/configbutler.ai_saverequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml similarity index 96% rename from config/crd/bases/configbutler.ai_saverequests.yaml rename to config/crd/bases/configbutler.ai_commitrequests.yaml index f4455d38..e33fa796 100644 --- a/config/crd/bases/configbutler.ai_saverequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -4,14 +4,14 @@ kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.23.0 - name: saverequests.configbutler.ai + name: commitrequests.configbutler.ai spec: group: configbutler.ai names: - kind: SaveRequest - listKind: SaveRequestList - plural: saverequests - singular: saverequest + kind: CommitRequest + listKind: CommitRequestList + plural: commitrequests + singular: commitrequest scope: Namespaced versions: - additionalPrinterColumns: @@ -54,7 +54,7 @@ spec: schema: openAPIV3Schema: description: |- - SaveRequest asks for a save: it attaches to its author's current or next + CommitRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and gives the commit its message. With spec.window.author, and where the GitTarget allows it, it may also claim a window whose writes name no @@ -80,7 +80,7 @@ spec: type: object spec: description: |- - spec defines the desired state of SaveRequest. It is immutable after + spec defines the desired state of CommitRequest. It is immutable after creation. properties: catchUp: @@ -102,7 +102,7 @@ spec: description: |- GitTargetRef names the GitTarget whose commit window this request attaches to: in this namespace, or namespace names another. The - GitTarget's spec.allowedSaveRequests must admit this namespace, or the + GitTarget's spec.allowedCommitRequests must admit this namespace, or the request fails with reason "NotAllowed". properties: name: @@ -196,7 +196,7 @@ spec: to the creator's own window. "SavedBy" may also give its message to a window whose writes name no author, naming no author. "TakeOver" may also take such a window over, and the creator authors the commit. The - unattributedWindows of a grant in the GitTarget's spec.allowedSaveRequests + unattributedWindows of a grant in the GitTarget's spec.allowedCommitRequests that admits this namespace must allow what is asked, or the request fails with reason TakeOverNotAllowed. Defaults to "Own". enum: @@ -227,7 +227,7 @@ spec: rule: duration(self) <= duration('5m') type: object x-kubernetes-validations: - - message: SaveRequest spec is immutable after creation + - message: CommitRequest spec is immutable after creation rule: self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) @@ -242,7 +242,7 @@ spec: - gitTargetRef type: object x-kubernetes-validations: - - message: SaveRequest spec is immutable after creation + - message: CommitRequest spec is immutable after creation rule: self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.whenNothingToCommit) @@ -255,7 +255,7 @@ spec: rule: '!has(self.whenNothingToCommit) || self.whenNothingToCommit != ''CommitEmpty'' || has(self.message)' status: - description: status defines the observed state of SaveRequest. + description: status defines the observed state of CommitRequest. properties: appliedRevision: description: |- @@ -274,7 +274,7 @@ spec: type: string conditions: description: |- - Conditions are the latest observations of this SaveRequest's state. + Conditions are the latest observations of this CommitRequest's state. Reconciling and Stalled follow kstatus. AuthorAttributed says whether the request's creator, recorded at admission, names the commit author; False is not a failure and does not affect Ready. @@ -359,7 +359,7 @@ spec: x-kubernetes-list-type: map gitTargetUID: description: |- - GitTargetUID is the UID of the GitTarget whose spec.allowedSaveRequests + GitTargetUID is the UID of the GitTarget whose spec.allowedCommitRequests admitted this request. A GitTarget with another UID, such as one created after the request under the same name, must admit it again. type: string diff --git a/config/crd/bases/configbutler.ai_fluxclaims.yaml b/config/crd/bases/configbutler.ai_fluxclaims.yaml index e9a650fe..385b5edb 100644 --- a/config/crd/bases/configbutler.ai_fluxclaims.yaml +++ b/config/crd/bases/configbutler.ai_fluxclaims.yaml @@ -68,7 +68,7 @@ spec: - after objects were created through the API in its folders; - once at startup, and when a GitTarget first names this FluxClaim; - once spec.catchUpAfter passed since a push of GitOps Reverser's own; - - after a SaveRequest with spec.catchUp "AfterPush" pushed a commit. + - after a CommitRequest with spec.catchUp "AfterPush" pushed a commit. GitTargets name it with spec.applierClaimRef. properties: diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index 02dff40b..454a072d 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -107,125 +107,33 @@ spec: spec: description: spec defines the desired state of GitTarget. properties: - allowedRemoteSaveRequests: - description: |- - AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster - spec.kubeProviderRef names, that may save this GitTarget's commit - windows. A request is admitted when one grant admits both its namespace - and its labels. Omitted or empty, none is. A request that GitTargets - mirror but none admits reports "NotAllowed". The KubeProvider must also - set spec.remoteSaveRequests "Enabled". - - An admitted request saves its creator's whole commit window on this - GitTarget, edits outside the request's namespace included. - items: - description: |- - GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some - namespaces of the source cluster to save a GitTarget's commit windows. - properties: - requestSelector: - description: |- - RequestSelector narrows this grant to the RemoteSaveRequests whose labels - match. Omitted, it admits every request from sourceNamespaces. Whoever - creates a request picks its labels, so this picks requests and proves - nothing about who made them. - properties: - matchExpressions: - description: matchExpressions is a list of label selector - requirements. The requirements are ANDed. - items: - description: |- - A label selector requirement is a selector that contains values, a key, and an operator that - relates the key and values. - properties: - key: - description: key is the label key that the selector - applies to. - type: string - operator: - description: |- - operator represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists and DoesNotExist. - type: string - values: - description: |- - values is an array of string values. If the operator is In or NotIn, - the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. This array is replaced during a strategic - merge patch. - items: - type: string - type: array - x-kubernetes-list-type: atomic - required: - - key - - operator - type: object - type: array - x-kubernetes-list-type: atomic - matchLabels: - additionalProperties: - type: string - description: |- - matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels - map is equivalent to an element of matchExpressions, whose key field is "key", the - operator is "In", and the values array contains only "value". The requirements are ANDed. - type: object - type: object - x-kubernetes-map-type: atomic - sourceNamespaces: - description: |- - SourceNamespaces are the namespaces, on the cluster the GitTarget's - spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: - exact names, or "*" for every namespace the GitTarget mirrors. - items: - description: |- - SourceNamespace is a namespace of a source cluster, by exact name, or "*" - for every namespace. - maxLength: 63 - type: string - x-kubernetes-validations: - - message: a source namespace is a namespace name, or '*' - alone; variables are refused, because a source cluster's - namespaces are not the control cluster's - rule: self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$') - maxItems: 64 - minItems: 1 - type: array - x-kubernetes-list-type: set - required: - - sourceNamespaces - type: object - maxItems: 16 - type: array - x-kubernetes-list-type: atomic - allowedSaveRequests: + allowedCommitRequests: default: - consumerNamespaces: from: Same unattributedWindows: Keep description: |- - AllowedSaveRequests says which SaveRequests may save this GitTarget's + AllowedCommitRequests says which CommitRequests may save this GitTarget's commit windows, by the namespace they are in, and how much each may claim of a window whose writes name no author: - - Omitted: the SaveRequests in this GitTarget's namespace, as one grant + - Omitted: the CommitRequests in this GitTarget's namespace, as one grant with from: "Same" that claims nothing. - - Empty list ([]): no SaveRequest. - - A list of grants: the SaveRequests that at least one grant allows. + - Empty list ([]): no CommitRequest. + - A list of grants: the CommitRequests that at least one grant allows. - A list replaces the default: the SaveRequests in this namespace keep + A list replaces the default: the CommitRequests in this namespace keep access only if at least one grant allows them. A request no grant allows fails with reason "NotAllowed". items: description: |- - GitTargetSaveRequestGrant admits the SaveRequests of some namespaces to save - a GitTarget's commit windows, and bounds what they may claim of a window - whose writes name no author. + GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to + save a GitTarget's commit windows, and bounds what they may claim of a + window whose writes name no author. properties: consumerNamespaces: description: |- - ConsumerNamespaces are the namespaces whose SaveRequests this grant + ConsumerNamespaces are the namespaces whose CommitRequests this grant admits. from: "Same" is this GitTarget's namespace. properties: from: @@ -338,7 +246,7 @@ spec: unattributedWindows: default: Keep description: |- - UnattributedWindows says how much an admitted SaveRequest may claim of a + UnattributedWindows says how much an admitted CommitRequest may claim of a commit window whose writes name no author, when it asks in spec.window.author. It acts only while the GitTarget's KubeProvider has never received an audit fact; on a cluster that has, a write naming no @@ -366,6 +274,98 @@ spec: maxItems: 64 type: array x-kubernetes-list-type: atomic + allowedRemoteSaveRequests: + description: |- + AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster + spec.kubeProviderRef names, that may save this GitTarget's commit + windows. A request is admitted when one grant admits both its namespace + and its labels. Omitted or empty, none is. A request that GitTargets + mirror but none admits reports "NotAllowed". The KubeProvider must also + set spec.remoteSaveRequests "Enabled". + + An admitted request saves its creator's whole commit window on this + GitTarget, edits outside the request's namespace included. + items: + description: |- + GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some + namespaces of the source cluster to save a GitTarget's commit windows. + properties: + requestSelector: + description: |- + RequestSelector narrows this grant to the RemoteSaveRequests whose labels + match. Omitted, it admits every request from sourceNamespaces. Whoever + creates a request picks its labels, so this picks requests and proves + nothing about who made them. + properties: + matchExpressions: + description: matchExpressions is a list of label selector + requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the selector + applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + sourceNamespaces: + description: |- + SourceNamespaces are the namespaces, on the cluster the GitTarget's + spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: + exact names, or "*" for every namespace the GitTarget mirrors. + items: + description: |- + SourceNamespace is a namespace of a source cluster, by exact name, or "*" + for every namespace. + maxLength: 63 + type: string + x-kubernetes-validations: + - message: a source namespace is a namespace name, or '*' + alone; variables are refused, because a source cluster's + namespaces are not the control cluster's + rule: self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$') + maxItems: 64 + minItems: 1 + type: array + x-kubernetes-list-type: set + required: + - sourceNamespaces + type: object + maxItems: 16 + type: array + x-kubernetes-list-type: atomic applierClaimRef: description: |- ApplierClaimRef names the FluxClaim or ArgoCDClaim, in this namespace, that @@ -428,7 +428,7 @@ spec: can be empty, and Generation is 0 for a kind without a spec, so guard them with {{with}}. - A SaveRequest's message replaces this template without changing the + A CommitRequest's message replaces this template without changing the author; spec.commit.message.requestTemplate can wrap that message. type: string reconcileTemplate: @@ -442,8 +442,8 @@ spec: type: string requestTemplate: description: |- - RequestTemplate is the Go template that frames a SaveRequest's message. It - is used only for a commit whose SaveRequest supplied a message, and gets + RequestTemplate is the Go template that frames a CommitRequest's message. It + is used only for a commit whose CommitRequest supplied a message, and gets liveTemplate's fields plus RequestMessage, the message as written; template syntax inside the message stays literal. Omitted, the message is committed as written. A template that never renders RequestMessage is rejected, and @@ -455,7 +455,7 @@ spec: Window sets when this GitTarget's commit windows close. A window opens on an author's first write and collects that author's writes until idleTimeout passes without one, or maxDuration after it opened, whichever comes first. A - change applies to windows opened afterward. A SaveRequest attached to a + change applies to windows opened afterward. A CommitRequest attached to a window replaces both timers for that window. properties: idleTimeout: diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 871c183f..d805d968 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -9,6 +9,6 @@ resources: - bases/configbutler.ai_gitreceivers.yaml - bases/configbutler.ai_gittargets.yaml - bases/configbutler.ai_kubeproviders.yaml - - bases/configbutler.ai_saverequests.yaml + - bases/configbutler.ai_commitrequests.yaml - bases/configbutler.ai_watchrules.yaml - bases/source.configbutler.ai_remotesaverequests.yaml diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 4b8a1e32..2e4f0972 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -69,13 +69,13 @@ rules: resources: - argocdclaims/status - clusterwatchrules/status + - commitrequests/status - fluxclaims/status - gitbranches/status - gitproviders/status - gitreceivers/status - gittargets/status - kubeproviders/status - - saverequests/status - watchrules/status verbs: - get @@ -99,21 +99,21 @@ rules: - apiGroups: - configbutler.ai resources: - - gitreceivers - - kubeproviders + - commitrequests verbs: + - delete - get - list + - patch - watch - apiGroups: - configbutler.ai resources: - - saverequests + - gitreceivers + - kubeproviders verbs: - - delete - get - list - - patch - watch - apiGroups: - source.configbutler.ai diff --git a/config/samples/README.md b/config/samples/README.md index 9df29b62..92a759d2 100644 --- a/config/samples/README.md +++ b/config/samples/README.md @@ -14,7 +14,7 @@ These samples are quick starting points for common GitOps Reverser setups. - `clusterwatchrule.yaml`: Minimal `ClusterWatchRule` for cluster-scoped resources. - `fluxclaim.yaml`, `argocdclaim.yaml`: An applier claim holding the Flux `Kustomization` or the Argo CD `Application` that applies `example-target`'s folder, for its `spec.applierClaimRef`. -- `saverequest.yaml`: Minimal `SaveRequest` — an on-demand "save" signal that finalizes a +- `commitrequest.yaml`: Minimal `CommitRequest` — an on-demand "save" signal that finalizes a `GitTarget`'s open commit window. It uses `metadata.generateName`, so apply it with `kubectl create -f`: `kubectl apply` refuses a generated name. diff --git a/config/samples/saverequest.yaml b/config/samples/commitrequest.yaml similarity index 94% rename from config/samples/saverequest.yaml rename to config/samples/commitrequest.yaml index 8561ae72..7b9ce546 100644 --- a/config/samples/saverequest.yaml +++ b/config/samples/commitrequest.yaml @@ -1,4 +1,4 @@ -# SaveRequest is a one-shot "save" signal. It attaches to its author's commit +# CommitRequest is a one-shot "save" signal. It attaches to its author's commit # window for the referenced GitTarget, gives that window its message, and # replaces the window's timers with its own. A window is only ever opened by a # write; a request waits for one and never opens one. The resulting commit is @@ -7,7 +7,7 @@ # Frontends typically use metadata.generateName so each "save" is a fresh # object. The referenced GitTarget must live in the same namespace. apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: generateName: save- namespace: default diff --git a/config/webhook/validating-webhook.yaml b/config/webhook/validating-webhook.yaml index 7cc46b49..37830cbc 100644 --- a/config/webhook/validating-webhook.yaml +++ b/config/webhook/validating-webhook.yaml @@ -17,10 +17,10 @@ metadata: # rejecting policy becomes product behavior, the chart must gain matching packaging # and this note must change. # 2. validate-operator-types.configbutler.ai — captures the submitter of our own command -# kinds (a SaveRequest today) at admission. This one IS product behavior and is +# kinds (a CommitRequest today) at admission. This one IS product behavior and is # packaged in the chart (charts/.../templates/validate-operator-types-webhook.yaml); the # entry here is the e2e SUT's copy. See -# docs/spec/saverequest-admission-authorship.md. +# docs/spec/commitrequest-admission-authorship.md. # # The cert-manager.io/inject-ca-from annotation injects the admission server CA bundle # for every webhook in this configuration. @@ -78,7 +78,7 @@ webhooks: path: /validate-operator-types port: 9443 # Ignore, not Fail: a missed author capture must degrade to the committer, never - # reject a user's SaveRequest. The handler also returns Allowed even when the Redis + # reject a user's CommitRequest. The handler also returns Allowed even when the Redis # write errors, so the command's success never depends on this webhook at all — # Ignore only covers the webhook being entirely unreachable. # @@ -91,7 +91,7 @@ webhooks: name: validate-operator-types.configbutler.ai # Narrow: our own command kinds (one rule per command kind), plus the one config kind # this endpoint validates. The submitter of a command is captured into the command-author - # Redis corner and read back by the SaveRequest controller with no wait; a WatchRule is + # Redis corner and read back by the CommitRequest controller with no wait; a WatchRule is # checked against the GitTarget it names and may be rejected. rules: - apiGroups: @@ -101,7 +101,7 @@ webhooks: operations: - CREATE resources: - - saverequests + - commitrequests scope: Namespaced # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it was # created, and that edit is the same mistake. diff --git a/docs/INDEX.md b/docs/INDEX.md index 7ee538c3..e0453767 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -102,8 +102,8 @@ misled. Full list in [`spec/README.md`](spec/README.md); the ones that carry a | [`type-followability.md`](spec/type-followability.md) | is a type followable, and if not, the one reason | | [`gitpath-foreign-content-stringency.md`](spec/gitpath-foreign-content-stringency.md) | refusing a path that shadows foreign content | | [`unsupported-folder-refusal-plan.md`](spec/unsupported-folder-refusal-plan.md) | `GitPathAccepted`, and refusing what we cannot own | -| [`saverequest-design.md`](spec/saverequest-design.md) | the SaveRequest window and its conditions | -| [`saverequest-admission-authorship.md`](spec/saverequest-admission-authorship.md) | how command submitters are captured and matched to commit windows | +| [`commitrequest-design.md`](spec/commitrequest-design.md) | the CommitRequest window and its conditions | +| [`commitrequest-admission-authorship.md`](spec/commitrequest-admission-authorship.md) | how command submitters are captured and matched to commit windows | | [`where-validation-lives.md`](spec/where-validation-lives.md) | schema → CEL → **the reconciler**; a webhook only for what exists solely at admission | | [`e2e-serial-registry.md`](spec/e2e-serial-registry.md) | which e2e specs must run Serial, and why | @@ -147,15 +147,15 @@ so you never have to read a proposal to find out it already shipped. | [`open-asks-priority.md`](design/open-asks-priority.md) | **the work queue.** Merges three overlapping backlogs — the downstream consumer asks, the API-surface block left unbuilt by the status and configuration-model review, and the config-surface proposal (B1–B6) — into one ordered queue under four stated tests, and says where we deliberately do **not** do what was asked. The standing caveat narrowed once the layout model reversed: a Tier 2 entry belongs to postponed [#294](https://github.com/ConfigButler/gitops-reverser/issues/294) only if it breaks a `GitTarget` field, and everything else is independently schedulable. Makes one design call against what was asked: **delete Option C sibling inference** rather than ship an off-switch for it, because it let a human's edit to the repository change operator behavior with nothing in status recording the move. That deletion has shipped, and "what the deletion taught" records what building it found. **F9 is Tier 1**: the only item whose answer is unknown rather than whose work is unscheduled, and it gates planning the enum work | | [`proving-the-metric-surface.md`](design/proving-the-metric-surface.md) | **design, partly built.** What identity an operator needs to locate a failure, and what an e2e check has to do to be worth trusting. Written after a metric value that was declared, switched on, surfaced as a condition reason and counted turned out to have no producer at all, and the check added to catch that class then skipped whenever the counter read zero — so deleting every recording call left the suite green. Two rules follow: an assertion must not take its trigger from the metric under test, and no enum value may be reachable only through an injected fake. Carries a cardinality policy that separates bounded configuration identity from per-request identity, a table of which instrument families should own `source_cluster` directly rather than by join, and the PromQL reasoning for `increase()` versus `max_over_time` (neither applies mechanically: one cannot see an event before the first sample, the other cannot reconstruct totals across a reset). Steps 1–4 are built; step 5, diagnostics export, the `e2e_run` scrape label and a pinned-seed regression are not | | [`placement-visibility-and-declared-defaults.md`](design/placement-visibility-and-declared-defaults.md) | **design, half built.** The three questions the inference deletion left, decided and then built in two goes. PR #291 shipped the deletion and none of the eight items queued behind it; the residue was filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) — **which shipped in 0.42.1 via [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) and is what reversed the layout model** — and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296), whose largest half then shipped in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326): `status.layout` became `status.placement`, and the ambiguous render root came with it as a refusal rather than a third counter outcome. The `declared` metric value has since split into `by_type`/`default` too, which was the one row with an external consumer because it changes a label a dashboard may select on. What still stands and is still unbuilt is the rest of the legibility set: the canonical path as a template constant, and `{kindLower}`, tracked on [#339](https://github.com/ConfigButler/gitops-reverser/issues/339) now that #296 is closed; the Event on the GitTarget was dropped. Its Question 2, a CRD default for `placement.default`, is **re-opened rather than superseded** and now lives as an open question in [`layout/model.md`](layout/model.md); the argument against it is legibility, since #319 removed the correctness objection | -| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **partly built**, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). One breaking wave on `GitTarget`, of which steps 2 to 7 have **shipped**: `spec.suspend`, `status.placement` and the reconcile-request annotation in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326), and B4's `commitWindow`/`commit.message` move plus the source-scope deletion in [#330](https://github.com/ConfigButler/gitops-reverser/pull/330). Organizing principle: **the folder is described on the GitTarget, the connection describes only the connection** — and this is where that became a struct boundary rather than a sentence, since grouping a field is free only in a release that is already breaking. `spec.mode` and `GitTarget.spec.interval` are both **dropped**, with re-open triggers, and `status.lastHandledReconcileAt` was dropped in the build for a stated reason. Step 8, the riders, is closed: the `SaveRequest` lifecycle hole shipped in [#400](https://github.com/ConfigButler/gitops-reverser/pull/400), the missing source-cluster provider message in [#258](https://github.com/ConfigButler/gitops-reverser/pull/258), and an asserted `SaveRequest.spec.author` is **declined** because authorship is established, never asserted ([#339](https://github.com/ConfigButler/gitops-reverser/issues/339)); #294 is closed. The reference collapse and the `scope` removal shipped in [#333](https://github.com/ConfigButler/gitops-reverser/pull/333); the `TooManyStreams` cap and the Event on a changed resolution were dropped. Also records that staying `v1alpha3` on loud rejections is a **one-consumer countdown** | +| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **partly built**, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). One breaking wave on `GitTarget`, of which steps 2 to 7 have **shipped**: `spec.suspend`, `status.placement` and the reconcile-request annotation in [#326](https://github.com/ConfigButler/gitops-reverser/pull/326), and B4's `commitWindow`/`commit.message` move plus the source-scope deletion in [#330](https://github.com/ConfigButler/gitops-reverser/pull/330). Organizing principle: **the folder is described on the GitTarget, the connection describes only the connection** — and this is where that became a struct boundary rather than a sentence, since grouping a field is free only in a release that is already breaking. `spec.mode` and `GitTarget.spec.interval` are both **dropped**, with re-open triggers, and `status.lastHandledReconcileAt` was dropped in the build for a stated reason. Step 8, the riders, is closed: the `CommitRequest` lifecycle hole shipped in [#400](https://github.com/ConfigButler/gitops-reverser/pull/400), the missing source-cluster provider message in [#258](https://github.com/ConfigButler/gitops-reverser/pull/258), and an asserted `CommitRequest.spec.author` is **declined** because authorship is established, never asserted ([#339](https://github.com/ConfigButler/gitops-reverser/issues/339)); #294 is closed. The reference collapse and the `scope` removal shipped in [#333](https://github.com/ConfigButler/gitops-reverser/pull/333); the `TooManyStreams` cap and the Event on a changed resolution were dropped. Also records that staying `v1alpha3` on loud rejections is a **one-consumer countdown** | | [`target-watch-plan.md`](design/target-watch-plan.md) | **partly built.** The diff is built and applied; removal semantics are not. The companion to [`watch-manager-ownership.md`](design/watch-manager-ownership.md): the ownership page says WHO applies a plan, this one says WHAT a plan is and what changing it may touch. A resource collection — group, resource, namespace, deliberately no served version — is the one identity the watch stream, the render-fidelity scope and the mark-and-sweep boundary all agree on, because a key that does not round-trip to the scope it sweeps under is the class of error that deletes user data. The plan is diffed into `keep`/`start`/`restart`/`stop` and applied per collection, so adding one WatchRule stops re-initializing every unrelated collection into a queue shared with other tenants; a `restart` is a served-version change, which is why the version is spec DATA rather than identity. Readiness and the fidelity revision are per scope, so a KEPT collection holds the result its own initialization produced rather than being asked to prove itself again over an unrelated edit. `stop` never touches files — removal is a Git-side sweep under the target's existing `spec.prune.mode`, not a watch-layer delete. "Cut at the producer" is the accepted consequence: nothing fences the queue, so a deselected collection may leave a short tail of writes, bounded by the queue and converged afterwards. Still open: the `stop` classification wants a settled `TypeRemoved` from `typeset` (see TODO), and removal on INTENT is undecided. | | [`collection-terminology-rename.md`](design/collection-terminology-rename.md) | **step 1, shipped.** Replaced cells with resource collections and removed per-rule `operations` from both rule kinds. Rules observe every object event; GitTarget prune mode controls removal. Records the rename census (760 Go identifiers, 265 documentation occurrences), the operation-filter inventory, API/Helm migration, compatibility boundaries, and test cases. | | [`watches-labels-simplification.md`](design/watches-labels-simplification.md) | **step 2, implemented.** After the collection rename and operation-filter removal, native per-rule `objectSelector` (#146). Kubernetes is authoritative: `Always` compares selected identities with all managed Git identities in the type/namespace scope, without comparing Git labels. Full support includes supported Kustomize layouts from the first release, plus label-exit attribution (now selector-agnostic, see the attribution spec) and ordered producer replacement. | | [`label-selection-follow-ups.md`](design/label-selection-follow-ups.md) | **design, open.** What the label-selection review rounds left open, with the evidence for each: real deletions in a selected collection wait the full 3s attribution grace (the watch cannot tell them from a label exit, measured), the resync fence that stops overlapping collections from coalescing (kept on purpose), same-second overlap ties, and one store rebuild per object coming back. Also lists what was fixed on the branch | | [`attribution-removal-wait-options.md`](design/attribution-removal-wait-options.md) | a removal now waits for evidence about the DELETION rather than accepting the object's last write, which stopped it naming whoever last edited the object as the author of a deletion they did not perform. Enumerates the eight situations a resolution can be in and shows the cost is concentrated in exactly one: a removal for which no delete fact will ever arrive (a graceful pod delete, a status-only removal, a type the audit policy skips) spends the whole grace to return the answer it had at t=0, measured at ~3.1s against ~70ms when evidence is present. Prices five options against that, and recommends a per-route watermark — stop waiting once the fact stream has demonstrably moved past this event — over a second timeout flag whose right value lives in the API server's config rather than ours. Open: the decision, and how common the case is outside the e2e suite | | [`concurrent-author-windows.md`](design/concurrent-author-windows.md) | **proposal, not built.** A branch worker holds one commit window at a time, so a second person's edit closes the first person's window and splits their change into two commits. Proposes a window per GitTarget and author, open side by side, with one rule that keeps every path's history in order: a write to a path another open window holds closes that window first (`path_conflict`). Branch-wide closes (foreign commit, apply, shutdown, buffer limit) close every window. Commit order becomes close order. A second save of one author stays refused (`SaveInProgress`). Open: what `buffer_limit` closes, windows of several GitTargets side by side, a cap on open windows, and a save cut short by a path conflict | -| [`commitrequest-save-wait-options.md`](design/commitrequest-save-wait-options.md) | **design, decided** — a five-phase plan, reported from a live cluster: a `SaveRequest` can expire before the write it exists to publish arrives, because the close delay defaulted to `0` while the write is still held head-of-line waiting for an audit fact, so the save resolves `Ready=True`/`NoWindowInGrace` and the edit commits seconds later under the target's `liveTemplate`. The decision is to repair the default (`spec.closeDelay`, now a duration string defaulting to `"2s"`, **shipped as phase 0**) and then let a request **name the write it is waiting for**, reusing the `(uid, resourceVersion)` identity the attribution join already keys on, including its settled `ExactCapable` rule. Records why the fact index cannot BE the wait (it sits upstream of the watch event, so a fact-keyed wait fires before the window opens) and what to take from it anyway — the register-then-check waiter discipline, and a best-effort retrospective oracle bounded by a TUNABLE TTL. Twelve situations, and the page is deliberate about the two it cannot fix cheaply: row 11 (a saturated queue is rejected in the WATCH path and never reaches the worker, so no worker-side wait can diagnose it) is an accepted, named gap rather than a promise, and row 10 is fixed only by a hook in the fidelity-drop branch that marks a wait terminally UNSATISFIABLE — satisfying before that gate would arm a finalize for a write that never commits. Carries a **preconditions chapter** for saves on today's build (audit policy first, then delay sizing, fidelity warm-up, save-button debounce, and assert on the commit rather than on `Ready`). `CommitterActivity` is deferred, not recommended: by its own arrival-not-existence rule it cannot satisfy the already-arrived case. Open: situation 12 (one actor, two saves, one window — blocks phase 2), whether the fact-index lookup earns its coupling, and how an unsatisfied wait is graded | -| [`commit-timing-surface.md`](design/commit-timing-surface.md) | **built.** One configuration surface for commit windows and saves, an in-place `v1alpha3` break with one tested upgrade procedure: only writes open a window; a `window` block (`idleTimeout`, `maxDuration`, target default `1m`) on both kinds, which a `SaveRequest` replaces whole for the window it attaches to; `attach: CurrentOrNext \| Next` with `attachTimeout`; opt-in `whenNothingToCommit: CommitEmpty` that keeps the cause in the reason. Precise clock and zero rules. Supersedes the save-wait timer model. Open: the target's `maxDuration` value | +| [`commitrequest-save-wait-options.md`](design/commitrequest-save-wait-options.md) | **design, decided** — a five-phase plan, reported from a live cluster: a `CommitRequest` can expire before the write it exists to publish arrives, because the close delay defaulted to `0` while the write is still held head-of-line waiting for an audit fact, so the save resolves `Ready=True`/`NoWindowInGrace` and the edit commits seconds later under the target's `liveTemplate`. The decision is to repair the default (`spec.closeDelay`, now a duration string defaulting to `"2s"`, **shipped as phase 0**) and then let a request **name the write it is waiting for**, reusing the `(uid, resourceVersion)` identity the attribution join already keys on, including its settled `ExactCapable` rule. Records why the fact index cannot BE the wait (it sits upstream of the watch event, so a fact-keyed wait fires before the window opens) and what to take from it anyway — the register-then-check waiter discipline, and a best-effort retrospective oracle bounded by a TUNABLE TTL. Twelve situations, and the page is deliberate about the two it cannot fix cheaply: row 11 (a saturated queue is rejected in the WATCH path and never reaches the worker, so no worker-side wait can diagnose it) is an accepted, named gap rather than a promise, and row 10 is fixed only by a hook in the fidelity-drop branch that marks a wait terminally UNSATISFIABLE — satisfying before that gate would arm a finalize for a write that never commits. Carries a **preconditions chapter** for saves on today's build (audit policy first, then delay sizing, fidelity warm-up, save-button debounce, and assert on the commit rather than on `Ready`). `CommitterActivity` is deferred, not recommended: by its own arrival-not-existence rule it cannot satisfy the already-arrived case. Open: situation 12 (one actor, two saves, one window — blocks phase 2), whether the fact-index lookup earns its coupling, and how an unsatisfied wait is graded | +| [`commit-timing-surface.md`](design/commit-timing-surface.md) | **built.** One configuration surface for commit windows and saves, an in-place `v1alpha3` break with one tested upgrade procedure: only writes open a window; a `window` block (`idleTimeout`, `maxDuration`, target default `1m`) on both kinds, which a `CommitRequest` replaces whole for the window it attaches to; `attach: CurrentOrNext \| Next` with `attachTimeout`; opt-in `whenNothingToCommit: CommitEmpty` that keeps the cause in the reason. Precise clock and zero rules. Supersedes the save-wait timer model. Open: the target's `maxDuration` value | | [`watch-and-catalog-architecture.md`](design/watch-and-catalog-architecture.md) | the target three-layer watch model — **needs a human call before building** | | [`metrics-observability-plan.md`](design/metrics-observability-plan.md) | **built.** the canonical metrics document, rewritten 2026-09 and shipped in the same wave. Reads the product as two ingestion paths — object state through watch, authorship through audit — joining at the resolver, and gives each stage one counter with a bounded `outcome` so the flow and its loss paths are drawable. Records what three review rounds corrected, including two reversals worth not re-litigating: the fact-loss counters must NOT be merged (an eviction counts a fact, a trim gap an occurrence, a decode error a whole batch), and a cumulative counter cannot replace a current count, which is why `status.streams` and `status.retainedDocuments` stayed. The instrument inventory lives in [`interpreting-metrics.md`](interpreting-metrics.md), not here. Open: the Grafana JSON and alert rules | | [`reconcile-triggering.md`](design/reconcile-triggering.md) | which controllers still fail to wake up | diff --git a/docs/README.md b/docs/README.md index e091c9ed..fddd72d9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,7 +11,7 @@ If you only want the supported product docs, start with the files below. - [`../README.md`](../README.md): product overview and supported workflows - [`quickstart.md`](quickstart.md): installation and your first Git commit -- [`demo/commit-window.md`](demo/commit-window.md): two resources and a `SaveRequest` +- [`demo/commit-window.md`](demo/commit-window.md): two resources and a `CommitRequest` - [`configuration.md`](configuration.md): core configuration objects, every setting, and how they fit together - [`commit-messages.md`](commit-messages.md): the three commit message templates and their context diff --git a/docs/TODO.md b/docs/TODO.md index d5777013..80b41eff 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -72,20 +72,20 @@ Their argument for each is in [gitops-api-open-asks.md](future/gitops-api-open-a - [ ] Two doc sentences: no condition detects an API server's 30s default `--audit-webhook-batch-max-wait` (say so in the `KubeProvider` docs), and a relist after a watch gap ships its changes as the configured author (say so in the operator docs). -- [ ] Optional: capture `SaveRequest` authors without Redis in single-pod mode, with an in-memory +- [ ] Optional: capture `CommitRequest` authors without Redis in single-pod mode, with an in-memory `CommandAuthorStore`. The only thing that still forces Redis on one replica. Low priority, and only useful until HA. -## Branch worker and SaveRequests +## Branch worker and CommitRequests -- [ ] `expireWaitingSaveRequests` ranges over a map, so empty commits for saves that expire in - the same pass land in random order; sort by `seq` (`internal/git/save_request_attach_loop.go`). +- [ ] `expireWaitingCommitRequests` ranges over a map, so empty commits for saves that expire in + the same pass land in random order; sort by `seq` (`internal/git/commit_request_attach_loop.go`). - [ ] `buildRequestRecordWrite` keeps only the username (`UserInfo{Username: pcr.author}`); carry the full `UserInfo` so a record commit has the same author identity as a window commit. - [ ] A record commit skips `requestTemplate`, though its doc comment says it is phrased like every other commit; fix the code or the comment. - [ ] Nits: a doubled doc comment on `commitWindowFor`; "close-delay" in the test names in - `internal/controller/saverequest_kstatus_test.go`. + `internal/controller/commitrequest_kstatus_test.go`. - [ ] **Parent observation, step 1**: keep parent availability and its timestamp after the write branch exists, from existing advertisements, with no extra connections. [gittarget-parent-observation.md](design/gittarget-parent-observation.md#implementation-and-validation). diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index a13b8829..75003d1e 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,27 +7,6 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. -## `CommitRequest` is `SaveRequest` - -`CommitRequest` is `SaveRequest`, in the same group and version (`configbutler.ai/v1alpha3`), with -the same spec and status. Nothing is migrated: a `CommitRequest` is a one-shot request, so let the -open ones finish, then upgrade and delete the old CRD, which Helm leaves installed: - -```bash -kubectl delete crd commitrequests.configbutler.ai -``` - -Everything named after the kind follows it: - -- **RBAC.** A Role that lets people `create` `commitrequests` must grant `saverequests` instead. -- **The TTL.** The flag `--commit-request-ttl` is `--save-request-ttl`, and the Helm value - `controllerManager.commitRequestTTL` is `controllerManager.saveRequestTTL`. -- **Metrics.** `gitopsreverser_commit_requests_total` is `gitopsreverser_save_requests_total`, with - the same labels. The `message_source` values `commit_request`, `commit_request_framed` and - `commit_request_fallback` of `git_commits_total` are `save_request`, `save_request_framed` and - `save_request_fallback`, and the `timer_source` value `commit_request` of - `git_commit_windows_total` is `save_request`. - ## A `FluxClaim` applies the tip a minute after GitOps Reverser's own pushes A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a @@ -35,7 +14,7 @@ A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none commit somebody else pushed. `spec.catchUpAfter` sets when that lag is caught up: once the applier has stayed behind the tip that long, an apply of the tip is asked for, one for a whole burst of saves. A `FluxClaim` defaults to `1m`; set `0s` to keep 0.54.0's behaviour. An `ArgoCDClaim` -defaults to `0s`. A `SaveRequest` that sets `catchUp: AfterPush` has the tip applied right after +defaults to `0s`. A `CommitRequest` that sets `catchUp: AfterPush` has the tip applied right after its commit is pushed, whatever the claim's interval, and reports the outcome in its `Applied` condition and `status.appliedRevision`. diff --git a/docs/api-first-publication.md b/docs/api-first-publication.md index 8d71d2a6..2ef82ddf 100644 --- a/docs/api-first-publication.md +++ b/docs/api-first-publication.md @@ -47,7 +47,7 @@ flowchart LR AUDIT["Optional audit facts
actor attribution"] -.-> WATCH GT["GitTarget
source, branch, path, commit window"] --> WATCH WATCH --> QUEUE["One FIFO per provider and branch"] - CR["SaveRequest
close a matching window"] --> QUEUE + CR["CommitRequest
close a matching window"] --> QUEUE QUEUE --> WINDOW["Open window
one author and one target"] QUEUE --> SNAP["Snapshot / atomic write"] WINDOW --> PENDING["Retained PendingWrite records"] @@ -68,10 +68,10 @@ flowchart LR | `BranchWorker` | Owns branch ordering, local repository operations, and publication | | Open window | Coalesces consecutive events from one author for one target | | `PendingWrite` | Retains captured content and resolved write metadata until publication | -| `SaveRequest` | Claims a matching window and reports its eventual outcome | +| `CommitRequest` | Claims a matching window and reports its eventual outcome | The watch stream supplies object state. Audit facts supply authorship when attribution is enabled. -Admission captures the submitter of a `SaveRequest`. These inputs have separate jobs; mirroring +Admission captures the submitter of a `CommitRequest`. These inputs have separate jobs; mirroring continues without actor attribution. See the [attribution contract](spec/attribution.md). The queue and retained writes are in memory. They are sufficient for replay during the worker's @@ -165,7 +165,7 @@ sequenceDiagram The idle timeout restarts on every matching edit, so continuous edits keep a window open until its `maxDuration` (`1m` by default, counted from the window opening) or another closing condition, such as an identity change or the byte threshold. A five-second idle timeout therefore does not promise -publication within five seconds of the first edit. A `SaveRequest` that attaches replaces both +publication within five seconds of the first edit. A `CommitRequest` that attaches replaces both timers with its own `idleTimeout` and `maxDuration`, both counted from the attach; an omitted `idleTimeout` leaves `maxDuration` alone to close the window. Its `attachTimeout` is separate: it runs from the worker's registration of the request and only bounds the wait for a window to attach @@ -282,7 +282,7 @@ are never held by the same worker. ## Story 3: save now, and know what reached Git -A `SaveRequest` attaches to a matching author-and-target window and closes it. It does not +A `CommitRequest` attaches to a matching author-and-target window and closes it. It does not bypass the push cooldown. Two settings with separate clocks set the timing: `attachTimeout` runs from the worker's registration of the request and bounds the wait for a window, and `maxDuration` runs from the attach and bounds the collection that follows. Both default to `2s` and accept `0s` @@ -290,7 +290,7 @@ to `5m`. A repeat attach restarts neither clock. ```mermaid sequenceDiagram - participant C as SaveRequest controller + participant C as CommitRequest controller participant W as Branch worker participant G as Git remote @@ -321,7 +321,7 @@ long a down remote takes. For a request the worker has not acted on within reports whatever the worker answers: failed when the worker cancelled it, or the real outcome when the worker resolved it first. A request reported failed is therefore never committed later. The worker keeps resolved outcomes for `15m`. -See the [request contract](spec/saverequest-design.md). +See the [request contract](spec/commitrequest-design.md). The two-second `attachTimeout` is not a guarantee that an earlier API edit has reached the worker. Attribution can wait up to three seconds per event, and queueing adds more. A request @@ -477,7 +477,7 @@ the earlier attribution and commit-window waits, cooldown, and recovery performe the cycle. It cannot by itself answer “how long from my API edit until Git accepted it?” A flat recovery counter also cannot prove progress: the worker may be retaining work while it -waits out its backoff. Check queue depth, failure logs, and `SaveRequest` outcomes alongside rates. +waits out its backoff. Check queue depth, failure logs, and `CommitRequest` outcomes alongside rates. A resync's fetch lands under `forced_recheck` whether or not the branch happened to hold retained writes, so a growing `publication` series means live publication is fetching again and nothing diff --git a/docs/architecture.md b/docs/architecture.md index 02cc8ed1..c6789ac7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -173,7 +173,7 @@ the Flux `Kustomization` applier and the Argo CD `Application` applier, its auto **Redis/Valkey is optional but advised.** The default configured-author mode runs without it: a plain `helm install` comes up healthy and watches initialize from current state on startup. When an endpoint is configured, Redis stores cursors for reconnecting running watches and the coordination records used by -SaveRequest author capture and HA. Attribution does not require it on its own: its facts travel on a +CommitRequest author capture and HA. Attribution does not require it on its own: its facts travel on a selectable transport, Redis Streams by default and an in-process ring with `--author-attribution-transport=memory`, which is refused with more than one replica. HA will require Redis as the shared store across replicas. @@ -240,7 +240,7 @@ optional and only add *who* did something: | **Discovery** (CRD/APIService) | *what types exist* | yes | the API surface: served GVRs, scope, preferred version, subresources; rules resolve against it | | **Watch** (per claimed `(GVR, scope)`) | *what changed* | yes | the object body, ordered per type; the only object-state source, with deletes reconciled via `sendInitialEvents` replay + mark-and-sweep | | **Audit webhook** (`/audit-webhook/`) | *who changed mirrored state* | no | a post-persist attribution fact, partitioned by source provider and joined to the watch event by resourceVersion ([Optional Attribution](#optional-attribution)) | -| **Validating admission webhook** (`/validate-operator-types`) | *who issued a command* | no | the submitter of a `SaveRequest`, captured at admission and keyed 1:1 by UID ([SaveRequest Finalize](#saverequest-finalize)) | +| **Validating admission webhook** (`/validate-operator-types`) | *who issued a command* | no | the submitter of a `CommitRequest`, captured at admission and keyed 1:1 by UID ([CommitRequest Finalize](#commitrequest-finalize)) | With both optional sources off, the product still mirrors state correctly, and every commit is authored by the committer. @@ -263,14 +263,14 @@ the write reaches etcd**: Audit avoids both: kube-apiserver posts a `ResponseComplete` event *after* the write persisted, carrying the resourceVersion that joins it to the watch event. -The one place admission *is* the right source is the `SaveRequest`, and only for its **command** meaning +The one place admission *is* the right source is the `CommitRequest`, and only for its **command** meaning (*who issued the save*) and not for mirroring the object. That holds because it is **our own type with a memory model we control**: the controller reconciles off its own watch, which only delivers persisted objects, so the author is captured at admission and read back 1:1 by UID: no resourceVersion join, no persistence proof from audit needed. (We still can't guarantee a create is never blocked, but it holds -across our edge cases.) If a `WatchRule` also selects SaveRequests, that *mirroring* runs through the +across our edge cases.) If a `WatchRule` also selects CommitRequests, that *mirroring* runs through the normal pipeline and is audit-attributed like any other resource. See -[SaveRequest Finalize](#saverequest-finalize). +[CommitRequest Finalize](#commitrequest-finalize). ### Proving it on new Kubernetes versions @@ -290,7 +290,7 @@ capture audit and admission with **no** resulting watch event. You configure GitOps Reverser through ten CRDs (group `configbutler.ai`, version `v1alpha3`). `WatchRule` and `ClusterWatchRule` choose which Kubernetes resources enter the pipeline. -`SaveRequest` can ask for the current window to be saved. `GitTarget` joins one source cluster to one +`CommitRequest` can ask for the current window to be saved. `GitTarget` joins one source cluster to one Git destination. `KubeProvider` supplies the source connection and authorization boundary; `GitProvider` supplies the repository, credentials, commit settings, and push policy, and a `GitBranch` holds one branch of that repository for every `GitTarget` that writes it. A @@ -304,7 +304,7 @@ graph LR subgraph CONTROL["Control plane: operator cluster"] WR[WatchRule] -->|gitTargetRef| GT[GitTarget] CWR[ClusterWatchRule] -->|gitTargetRef| GT - CR[SaveRequest] -->|gitTargetRef| GT + CR[CommitRequest] -->|gitTargetRef| GT GT -->|applierClaimRef| TR["FluxClaim /\nArgoCDClaim"] GT -->|kubeProviderRef| CP[KubeProvider] GT -->|gitBranchRef| GB[GitBranch] @@ -352,7 +352,7 @@ records the apply operation it started. |---|---|---| | `WatchRule` | namespaced | which NAMESPACED resources, in which source namespaces, route to a GitTarget | | `ClusterWatchRule` | cluster | which CLUSTER SCOPED resources route to a GitTarget | -| `SaveRequest` | namespaced | a one shot "save the open window now" signal | +| `CommitRequest` | namespaced | a one shot "save the open window now" signal | | `GitTarget` | namespaced | one materialization from a `KubeProvider` to `(GitBranch, path)` | | `KubeProvider` | namespaced | one source-cluster connection plus namespace access policy | | `GitBranch` | namespaced | one repository branch, held for every `GitTarget` that writes it | @@ -404,10 +404,10 @@ One `KubeProvider` grant answers two questions in different planes: own Kubernetes RBAC. The operator reads no `Namespace` objects in a source cluster for authorization. -### SaveRequest +### CommitRequest -- **Source**: [api/v1alpha3/saverequest_types.go](../api/v1alpha3/saverequest_types.go) -- **Controller**: [internal/controller/saverequest_controller.go](../internal/controller/saverequest_controller.go) +- **Source**: [api/v1alpha3/commitrequest_types.go](../api/v1alpha3/commitrequest_types.go) +- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) A one shot "save now" signal that finalizes the open commit window for a same namespace `GitTarget` instead of waiting for the silence timer. The **entire spec is immutable**. Key fields: @@ -431,7 +431,7 @@ instead of waiting for the silence timer. The **entire spec is immutable**. Key - `status.branch` / `status.commit`: set when the commit was pushed (`Pushed=True`). How attribution and finalization interact is described under -[SaveRequest Finalize](#saverequest-finalize). +[CommitRequest Finalize](#commitrequest-finalize). ### GitTarget @@ -704,7 +704,7 @@ Following the ConfigMap edit: [GitTargetEventStream](../internal/reconcile/git_target_event_stream.go), and the [BranchWorker](../internal/git/branch_worker.go) appends it to the open commit window for `(author, GitTarget)`. -5. **Commit + push.** When the window closes (5s of silence, a `SaveRequest`, or a buffer limit), the +5. **Commit + push.** When the window closes (5s of silence, a `CommitRequest`, or a buffer limit), the manifest aware writer patches `team-a/.../app-config.yaml` in place, commits as the resolved author, and pushes via [PushAtomic](../internal/git/git_atomic_push.go) (retrying with fetch/reset/replay if the remote moved). @@ -722,7 +722,7 @@ See [State Ingestion and Not Losing Deletes](#state-ingestion-and-not-losing-del ### Normal commit flow -Most writes are closed by the commit window timer. A `SaveRequest` is the explicit "save now" path: it +Most writes are closed by the commit window timer. A `CommitRequest` is the explicit "save now" path: it does not create new state, and it does not bypass watch. It only asks the branch worker to close a matching open window early. @@ -743,7 +743,7 @@ flowchart TD TIMER -->|silence timer| BASE TIMER -->|buffer limit or resync boundary| BASE - CR[SaveRequest persisted] --> ADMISSION[Admission submitter lookup
named or unnamed] + CR[CommitRequest persisted] --> ADMISSION[Admission submitter lookup
named or unnamed] ADMISSION --> ATTACH[Attach to matching open window] ATTACH -->|same author + GitTarget| BASE ATTACH -->|no matching window| BENIGN[Ready=True, Pushed=False] @@ -756,10 +756,10 @@ flowchart TD ``` The important boundary is the open window: it accepts only the same author and `GitTarget`. A -`SaveRequest` from another author never closes someone else's window; it resolves as a benign no-commit +`CommitRequest` from another author never closes someone else's window; it resolves as a benign no-commit instead. The one exception is opt-in, on a cluster that has never sent an audit fact: there no write can name its author, and a request may claim a window whose writes name none (step 2 of the -`SaveRequest` flow). +`CommitRequest` flow). ### Remote moved while we were writing @@ -784,7 +784,7 @@ flowchart TD CHECK -->|attempts exhausted or fetch fails| RETAIN[Keep pending writes for later retry] ``` -For a `SaveRequest`, success is reported only after the push reaches the remote. If the push had to +For a `CommitRequest`, success is reported only after the push reaches the remote. If the push had to replay on top of someone else's commit, `status.commit` is the refreshed post-replay commit, not the stale local commit from before the retry. @@ -818,7 +818,7 @@ flowchart TD A delivery is handled on the worker's own queue, so it is never judged while a push is in flight: the push has recorded its commit before the delivery is read. A rollback onto one of the worker's older commits is foreign, because the advertisement names a tip other than the last one observed. -A delivery also closes the open commit window, unless a `SaveRequest` holds it, so its +A delivery also closes the open commit window, unless a `CommitRequest` holds it, so its publication finds the move. A Git-side push usually reaches the worker through the flush itself: the push of the pending writes @@ -838,7 +838,7 @@ again is what stamps the owner labels or tracking annotation on the new object. worker's own leaves nothing pending. Three states still hold an apply back, because in them no flush can be made: a registered -`SaveRequest` (a save defines its own commit boundary, and the apply waits for it), a parent +`CommitRequest` (a save defines its own commit boundary, and the apply waits for it), a parent recovery, and a checkout a failed write left dirty. An apply also waits until every initial snapshot of the `GitTarget`s its applier covers is published (a `GitTarget` with no planned collection yet, or with a blocked watch, holds its applier the same way), and an empty tip (a @@ -1116,9 +1116,9 @@ queued behind it on the same `(GitTarget, GVR, scope)` goroutine. That cost, and is in [Watch Event Ordering](#watch-event-ordering) and [attribution-removal-wait-options.md](design/attribution-removal-wait-options.md). -The SaveRequest controller does **not** read this audit index. A SaveRequest's submitter is named by +The CommitRequest controller does **not** read this audit index. A CommitRequest's submitter is named by the `/validate-operator-types` validating admission webhook instead, captured synchronously at admission, -never joined from an audit fact (see [SaveRequest Finalize](#saverequest-finalize)). +never joined from an audit fact (see [CommitRequest Finalize](#commitrequest-finalize)). ### Author and committer identity in Git @@ -1194,7 +1194,7 @@ CommitDate: Mon Jun 30 12:00:05 2026 +0000 - **Signature construction**: [internal/git/commit.go](../internal/git/commit.go) (`authorName` / `authorEmail`) The operator never talks to the identity provider. It reads two fixed keys from the request's `user.extra` -map: the audit attribution path and the SaveRequest admission path resolve the **same** two keys: +map: the audit attribution path and the CommitRequest admission path resolve the **same** two keys: | `user.extra` key | OIDC claim it carries | Used for | |---|---|---| @@ -1513,7 +1513,7 @@ move a live worker; it replaces it, and the replacement starts with no clone, no observation. A `GitBranch` carries a finalizer. Deleting it releases the claim, then stops its worker, which -closes the worker's intake, pushes or refuses every pending write and settles every `SaveRequest` +closes the worker's intake, pushes or refuses every pending write and settles every `CommitRequest` it holds; only then does the finalizer go. Nothing deletes the branch on the remote. `GitBranch.status.remote` is a **branch** fact, and it is worth being exact about that, because the @@ -1529,7 +1529,7 @@ once a minute, and comes back when that floor lifts. ### Everything about a branch happens on that branch's queue Reading or writing a branch's repository state is the branch worker's job, and the worker does it on -one FIFO queue, on one goroutine. Live events, resyncs, `SaveRequest` attaches, periodic +one FIFO queue, on one goroutine. Live events, resyncs, `CommitRequest` attaches, periodic refreshes and push-receiver deliveries all enter the same queue and are served in order. Which writes are decided and which of them are published is therefore a fact one goroutine holds, which is what lets the worker flush before an apply request goes out. Nothing else opens a connection on a @@ -1554,9 +1554,9 @@ accumulate in one open commit window, which accepts only one `(author, GitTarget The window finalizes when its `idleTimeout` passes with no new matching event, its `maxDuration` runs out, the retained buffer reaches `--branch-buffer-max-size` (default `8Mi`), an `attach: Next` -`SaveRequest` closes it, a resync, an atomic batch, a push-receiver delivery or shutdown arrives, +`CommitRequest` closes it, a resync, an atomic batch, a push-receiver delivery or shutdown arrives, or an apply comes due for an applier on the branch. The timers -are the GitTarget's `spec.commit.window` until a `SaveRequest` attaches and replaces them. Successful local commits +are the GitTarget's `spec.commit.window` until a `CommitRequest` attaches and replaces them. Successful local commits are retained until a fixed push cooldown (`5s`) allows a push, which prevents remote push storms during bursts. The one exception is an apply's flush: the writes decided before the apply came due are pushed at once, cooldown or not, because the apply waits for them. @@ -1591,7 +1591,7 @@ the newer tip changed, and a write with no action left settles at once without a keeps the upserts that changed the tree and the documents its sweep removed: a replay plans each upsert onto the new tree and removes each document if it is still there and the prune mode still sweeps, and never sweeps again, so a document the newer tip added survives. A save is the exception: -a write carrying a `SaveRequest` keeps every object it captured, and the push decides its outcome. +a write carrying a `CommitRequest` keeps every object it captured, and the push decides its outcome. See [the pending-writes design](design/gittarget-branch-worker-pending-writes.md#a-write-carries-the-actions-decided-on-its-tree). A branch whose remote has been **deleted** takes the same path. The advertisement does not carry the @@ -1760,15 +1760,15 @@ unsafely: logged per-resource, counted in the resync summary as `placementSkippe *** -## SaveRequest finalize +## CommitRequest finalize -- **Controller**: [internal/controller/saverequest_controller.go](../internal/controller/saverequest_controller.go) +- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) - **Admission handler**: [internal/webhook/validate_operator_types_handler.go](../internal/webhook/validate_operator_types_handler.go) - **Admission author cache**: [internal/queue/command_author_store.go](../internal/queue/command_author_store.go) -A `SaveRequest` finalizes the open commit window for its GitTarget. The request author is resolved from +A `CommitRequest` finalizes the open commit window for its GitTarget. The request author is resolved from the `/validate-operator-types` validating admission webhook, which captures the authenticated submitter at admission (keyed by the object's UID) before the object is visible. The lookup is **present-or-never**: if the record is missing, the request is marked `AuthorAttributed=False`, claims no actor, and still attaches @@ -1776,17 +1776,17 @@ immediately. That is not a failure: 1. The controller stamps the in-progress conditions (`Reconciling=True`) and settles `AuthorAttributed` synchronously from the admission author cache. There is no audit wait on this path. -2. The controller eagerly **attaches** the request to the worker (`AttachSaveRequest`), which +2. The controller eagerly **attaches** the request to the worker (`AttachCommitRequest`), which waits for a window until `registration + attachTimeout`. The worker binds it to an open window only when the author state and GitTarget match, first come first served, and replaces the window's timers with the request's own, so time spent waiting never shortens the collection. A request that names its submitter may also claim a window on its `GitTarget` whose writes name no author, when it asks (`spec.window.author`), the grant admitting it allows it (`unattributedWindows` in the `GitTarget`'s - `spec.allowedSaveRequests`), and the + `spec.allowedCommitRequests`), and the `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`). With `SavedBy` the window takes the request's message, the commit names no author, and the message ends with a `Saved-By` trailer; with `TakeOver` the submitter authors the commit, and the message ends - with an `Author-Source: SaveRequest /` trailer. Once the provider has received a + with an `Author-Source: CommitRequest /` trailer. Once the provider has received a fact, a write that names no author means its fact went missing, and its window stays another's. A request that asks for more than its `GitTarget` allows fails at first sight with `TakeOverNotAllowed`. It **never finalizes @@ -1805,7 +1805,7 @@ immediately. That is not a failure: `AlreadyPresent` therefore means the remote confirmed there was nothing to add, not that a local plan found no diff. -The SaveRequest submitter is not recoverable from object state alone, so without an admission record the +The CommitRequest submitter is not recoverable from object state alone, so without an admission record the request cannot claim an actor. The final Git author remains the attached watch window's author: configured committer when attribution was disabled, or the explicit unresolved author when live attribution ran but could not resolve. There is no audit-fact join for the request itself: its submitter is captured at admission @@ -1842,14 +1842,14 @@ Controllers watch their dependencies so dependents reconcile quickly after spec at all (the installation's `inCluster` consent, and which namespace holds its audit route), and projects its readiness to dependent targets. A provider created, deleted or moved to another route wakes the others carrying that route, so an `AuditRouteConflict` clears without waiting for a requeue. -- `SaveRequestReconciler` runs with `MaxConcurrentReconciles=1` and attributes/attaches as above; its +- `CommitRequestReconciler` runs with `MaxConcurrentReconciles=1` and attributes/attaches as above; its optional `AuthorLookup` is the command-author cache populated by the `/validate-operator-types` webhook (wired whenever the admission webhook is enabled, independent of `--author-attribution`, and nil otherwise, so the request claims no actor). Dependency watches use narrow predicates to avoid status-only heartbeat churn: a `KubeProvider` also admits a `Ready` transition, and a `Namespace` admits only label changes. `GitProvider`, `GitBranch`, -`GitTarget`, and `SaveRequest` carry immutability constraints where a spec change would orphan a +`GitTarget`, and `CommitRequest` carry immutability constraints where a spec change would orphan a materialized subtree or invalidate an in-flight finalize. *** @@ -1880,7 +1880,7 @@ flowchart TD Hq -->|no| J[Configured-author: no fact streams or index; audit webhook skipped] I --> K[Setup + register Watch Manager] J --> K - K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + SaveRequest controllers; admission webhooks] + K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + CommitRequest controllers; admission webhooks] L --> M[Add cert watchers + health checks] M --> N[mgr.Start] ``` @@ -1918,7 +1918,7 @@ The families follow the architecture, which is why this section is a map rather |---|---|---| | `resource_condition` | the kstatus trio of every object a human declares | [Configuration state](interpreting-metrics.md#configuration-state) | | `watch_*` | ingest: events, sessions, replays, recoveries, resolved types, open streams | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push) | -| `git_*`, `save_requests_total` | the branch worker: documents, windows, commits, pushes, fetches, refusals, losses, the retained backlog | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push), [Tuning commit windows](interpreting-metrics.md#tuning-commit-windows) | +| `git_*`, `commit_requests_total` | the branch worker: documents, windows, commits, pushes, fetches, refusals, losses, the retained backlog | [Pipeline](interpreting-metrics.md#pipeline-watch-to-push), [Tuning commit windows](interpreting-metrics.md#tuning-commit-windows) | | `placement*` | where a new document lands, and the ones refused | [New-file placement](interpreting-metrics.md#new-file-placement) | | `apply_*`, `receiver_deliveries_total` | apply requests, the apply window and the push receiver | [Apply requests](interpreting-metrics.md#apply-requests-and-the-push-receiver) | | `audit_*`, `attribution_*` | the optional author lookup: the door, the fact pipeline, the join | [Audit attribution](interpreting-metrics.md#audit-attribution-optional) | @@ -2046,7 +2046,7 @@ Deeper dives live under [docs/design/](design/): transport seam, the in-process index, and what running without Redis costs. - [Watch event ordering under the attribution grace window](spec/watch-event-ordering-and-attribution-grace.md) - [Reconcile via watchlist mark and sweep](spec/reconcile-via-watchlist-mark-and-sweep.md) -- [SaveRequest design](spec/saverequest-design.md) +- [CommitRequest design](spec/commitrequest-design.md) **Types, discovery, status, and the Git write side:** diff --git a/docs/attribution-setup-guide.md b/docs/attribution-setup-guide.md index 72b74bd7..5e3ff512 100644 --- a/docs/attribution-setup-guide.md +++ b/docs/attribution-setup-guide.md @@ -27,7 +27,7 @@ queue: Redis/Valkey holds attribution facts and watch cursors. If its data is lost, recent facts are lost and watches cold-replay. For a disposable single-pod setup, `attribution.transport: memory` needs no Redis but loses facts whenever the reverser restarts. It also leaves Redis-backed features such -as `SaveRequest` author capture inactive unless you configure Redis separately. +as `CommitRequest` author capture inactive unless you configure Redis separately. The chart's `KubeProvider`, `local` in the release namespace, admits only the release namespace by default. Grant the namespaces whose `GitTarget`s may mirror through it in diff --git a/docs/bi-directional.md b/docs/bi-directional.md index 8565b08b..7fdafe03 100644 --- a/docs/bi-directional.md +++ b/docs/bi-directional.md @@ -152,7 +152,7 @@ stateDiagram-v2 ``` Three states still hold an apply back, because in them no flush can be made: a commit window a -[`SaveRequest`](configuration.md#saverequest) holds (a save defines its own commit boundary, and +[`CommitRequest`](configuration.md#commitrequest) holds (a save defines its own commit boundary, and the apply waits for it), a missing parent branch being recovered, and a checkout a failed write left dirty. In the last two the branch worker cannot say what is published. @@ -404,7 +404,7 @@ Committing and pushing takes on the order of a second at best, and the Kubernete great many changes in a second. Treat a live edit as a change you observe arriving, not a save that returns a confirmation. Two edits inside one commit window produce one commit holding the later value, so Git history records the states the worker captured, with gaps between them. When a -specific change has to be confirmed in Git, use a [`SaveRequest`](configuration.md#saverequest). +specific change has to be confirmed in Git, use a [`CommitRequest`](configuration.md#commitrequest). ## Keep controller bookkeeping out of Git diff --git a/docs/commit-messages.md b/docs/commit-messages.md index 55e89e5b..540c1ca3 100644 --- a/docs/commit-messages.md +++ b/docs/commit-messages.md @@ -15,8 +15,8 @@ message only on some branch passes validation and falls back at commit time. | Input, in precedence order | Message source | |---|---| -| Attached [save request](configuration.md#saverequest) **and** `requestTemplate` set | `requestTemplate`, with the request's message as `.RequestMessage` | -| Non-empty literal override, including an attached [save request](configuration.md#saverequest) | Exact supplied text | +| Attached [save request](configuration.md#commitrequest) **and** `requestTemplate` set | `requestTemplate`, with the request's message as `.RequestMessage` | +| Non-empty literal override, including an attached [save request](configuration.md#commitrequest) | Exact supplied text | | Live window of any size | `liveTemplate` | | Atomic snapshot or resync | `reconcileTemplate` | @@ -61,7 +61,7 @@ Git author header when attribution ran without resolving an actor. Messages neve ## Framing a save message -By default a [save request](configuration.md#saverequest)'s message **replaces** the template, so supplying one +By default a [save request](configuration.md#commitrequest)'s message **replaces** the template, so supplying one costs you the resource body `liveTemplate` would have produced: the commit says why, but no longer says what. `requestTemplate` composes the two. @@ -95,7 +95,7 @@ than scanning the template's text. reading a label some resources do not carry is caught when the target reconciles rather than at commit time. If one does fail to render in production, the request's message is committed verbatim instead of the -window being lost, and the commit is counted under `message_source="save_request_fallback"`. +window being lost, and the commit is counted under `message_source="commit_request_fallback"`. Alert on that rate: the commit itself succeeds and no condition moves, so it is the only signal. ## Kind, scope, and labels diff --git a/docs/components.md b/docs/components.md index fe2e2c10..3174d653 100644 --- a/docs/components.md +++ b/docs/components.md @@ -23,7 +23,7 @@ the target watches supply *state*, the branch worker owns *Git*, and audit only ```mermaid flowchart TB subgraph CFG["Config plane (operator's own cluster)"] - CRD["api/v1alpha3
GitTarget · WatchRule · ClusterWatchRule
GitProvider · GitBranch · KubeProvider · SaveRequest
FluxClaim · ArgoCDClaim"] + CRD["api/v1alpha3
GitTarget · WatchRule · ClusterWatchRule
GitProvider · GitBranch · KubeProvider · CommitRequest
FluxClaim · ArgoCDClaim"] CTRL["internal/controller
reconcilers"] ADM["internal/webhook
admission handlers"] AUTHZ["internal/authz
namespace admission"] @@ -78,7 +78,7 @@ The operator's own API surface and the reconcilers behind it. |---|---|---| | CRD types | [`api/v1alpha3/`](../api/v1alpha3/) | the ten kinds users apply | | Reconcilers | [`internal/controller/`](../internal/controller/) | one per kind, plus shared condition and status helpers | -| Admission handlers | [`internal/webhook/`](../internal/webhook/) | the always-allow observer and `validate-operator-types`, which captures a `SaveRequest` submitter | +| Admission handlers | [`internal/webhook/`](../internal/webhook/) | the always-allow observer and `validate-operator-types`, which captures a `CommitRequest` submitter | | Access grants | [`internal/authz/`](../internal/authz/) | one admission decision per act, read from the grant lists on the referenced object (`allowedGitBranches`, `allowedGitTargets`, `allowedApplierClaims`) with the scope that applies, in their own package because several call sites need the same answer | | Rule store | [`internal/rulestore/`](../internal/rulestore/) | compiled `WatchRule` and `ClusterWatchRule` cache the data plane reads | @@ -137,7 +137,7 @@ Optional, and deliberately unable to affect *what* is written. It only names the | Fact transport | [`internal/queue/fact_stream.go`](../internal/queue/fact_stream.go) | Redis Streams by default, an in-process ring with `--author-attribution-transport=memory` (which frees attribution from Redis, but not the watch resume cursors) | | Fact index and follower | [`internal/queue/fact_index.go`](../internal/queue/fact_index.go) | a bounded, TTL'd in-process index, followed per type while at least one watch needs it | | Author resolver | [`internal/watch/author_resolver.go`](../internal/watch/author_resolver.go) | joins a watch event to a fact within a bounded grace window | -| Command author store | [`internal/queue/command_author_store.go`](../internal/queue/command_author_store.go) | the `SaveRequest` submitter captured at admission | +| Command author store | [`internal/queue/command_author_store.go`](../internal/queue/command_author_store.go) | the `CommitRequest` submitter captured at admission | ### Support and tooling diff --git a/docs/configuration.md b/docs/configuration.md index 75b0a7c2..8dd671b3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -12,7 +12,7 @@ The short version: - `WatchRule` defines which namespaced resources should produce Git writes, and in which source namespaces - `ClusterWatchRule` does the same for cluster-scoped resources -- `SaveRequest` optionally saves: it closes its author's commit window on its own timers, with +- `CommitRequest` optionally saves: it closes its author's commit window on its own timers, with its own message if it has one - `FluxClaim` and `ArgoCDClaim` hold the Flux `Kustomization` or Argo CD `Application` that applies a folder back, so GitOps Reverser decides when it applies @@ -35,7 +35,7 @@ The usual flow is: 3. Create a `GitTarget` that points at the `GitBranch`, the `KubeProvider` to mirror from, and a folder in the repository. 4. Create one or more `WatchRule` or `ClusterWatchRule` objects that point at that target. -5. Create a `SaveRequest` only when you want a save: your writes in one commit, on the request's +5. Create a `CommitRequest` only when you want a save: your writes in one commit, on the request's timers rather than the target's, and optionally with your message. 6. Only when Flux or Argo CD also applies that same folder back to the cluster: hold its `Kustomization` or `Application` with an applier claim, name the applier claim on the `GitTarget`, and @@ -119,7 +119,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | -| `allowedSaveRequests` | `[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"Keep"}]` | The namespaces whose `SaveRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See [who may save](#who-may-save-specallowedsaverequests) | +| `allowedCommitRequests` | `[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"Keep"}]` | The namespaces whose `CommitRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See [who may save](#who-may-save-specallowedcommitrequests) | | `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and save its windows. See [`RemoteSaveRequest`](#remotesaverequest) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | @@ -174,11 +174,11 @@ The references are immutable: to hold another `Application`, delete the applier | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | | `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync stays behind the branch after a push of GitOps Reverser's own before a sync of the tip is asked for. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | -### `SaveRequest` (namespaced): save now +### `CommitRequest` (namespaced): save now | Field | Default | What it does | |---|---|---| -| `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedSaveRequests` must admit this one | +| `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedCommitRequests` must admit this one | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | | `window.author` | `Own` | Whose window to attach to: `Own`, or also claim a window whose writes name no author with `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | @@ -483,7 +483,7 @@ admits no new work under the revoked permission. Work it already held may comple - **A `GitBranch` no grant admits any more** gives up its branch the way a deleted one does, but keeps its finalizer. Its branch worker closes its intake, pushes or refuses each pending write and - settles every `SaveRequest`; an event that arrives afterwards is refused. The `GitBranch` reports + settles every `CommitRequest`; an event that arrives afterwards is refused. The `GitBranch` reports `Stalled=True` with the grant's reason. Nobody holds the branch until a waiting `GitBranch` takes it on its own reconcile, which checks its grant first, so a candidate revoked by the same edit never holds it. The new holder starts a worker of its own, with none of the old one's work. Writes @@ -554,7 +554,7 @@ kind: ClusterRole metadata: {name: gitops-reverser-tenant} rules: - apiGroups: [configbutler.ai] - resources: [gitbranches, gittargets, watchrules, fluxclaims, argocdclaims, saverequests] + resources: [gitbranches, gittargets, watchrules, fluxclaims, argocdclaims, commitrequests] verbs: [get, list, watch, create, update, patch, delete] --- apiVersion: v1 @@ -585,7 +585,7 @@ Role also needs whatever team A's own workloads require. | Identity | Manages | Cannot | |---|---|---| | The platform's reconciler | `GitProvider`, `KubeProvider` and their grants; Namespaces and their labels; tenant RBAC; platform `GitBranch`es such as `main` | | -| `team-a/tenant-reconciler` | `GitBranch`, `GitTarget`, `WatchRule`, applier claims and `SaveRequest`s in `team-a` | Create or edit a provider; label a Namespace; change RBAC | +| `team-a/tenant-reconciler` | `GitBranch`, `GitTarget`, `WatchRule`, applier claims and `CommitRequest`s in `team-a` | Create or edit a provider; label a Namespace; change RBAC | The last column is what makes the grants hold: a tenant cannot grant itself a label, and cannot create the second provider the limit describes. Flux's multi-tenancy lockdown @@ -1261,7 +1261,7 @@ at, and `Ready`; ### Deleting a `GitBranch` A `GitBranch` carries a finalizer. Deleting it stops new intake for every `GitTarget` on the -branch, pushes or refuses each pending write, gives each attached `SaveRequest` its outcome, and +branch, pushes or refuses each pending write, gives each attached `CommitRequest` its outcome, and only then removes the finalizer and lets another `GitBranch` hold the branch. **Nothing deletes the branch on the remote.** @@ -1422,7 +1422,7 @@ kubectl annotate gittarget -n \ The kstatus trio is also on Prometheus, so the same answer is alertable without a kubeconfig: `gitopsreverser_resource_condition{kind="GitTarget", type="Ready", status="False"} == 1` names every target that has not been accepted, and `reason` says which gate refused. The metric covers `Ready`, -`Reconciling` and `Stalled` for every object on this page except `SaveRequest`; the per-kind axis +`Reconciling` and `Stalled` for every object on this page except `CommitRequest`; the per-kind axis conditions above stay on the object, where `kubectl describe` reads them. See [interpreting-metrics.md](interpreting-metrics.md#configuration-state). @@ -1444,8 +1444,8 @@ spec: liveTemplate: "chore: sync {{.Count}} resource{{if ne .Count 1}}s{{end}}" ``` -Configure [automatic messages](commit-messages.md), submit a [save message](#saverequest), -and interpret its [conditions](#saverequest). +Configure [automatic messages](commit-messages.md), submit a [save message](#commitrequest), +and interpret its [conditions](#commitrequest). #### The commit window (`spec.commit.window`) @@ -1460,7 +1460,7 @@ timers, whichever comes first: `idleTimeout` may not exceed `maxDuration`. A window takes the target's timers when it opens, so editing them affects windows that open afterward, never one already running. A -[`SaveRequest`](#the-request-window-specwindow) that attaches to a window replaces both timers +[`CommitRequest`](#the-request-window-specwindow) that attaches to a window replaces both timers for that window. One branch worker holds one live window, bound to an author and target. Interleaved authors or @@ -1484,7 +1484,7 @@ ever update. Three templates decide how a commit is phrased: `liveTemplate` for a live window, `reconcileTemplate` for an atomic snapshot or resync, and `requestTemplate` for a window a -[save request](#saverequest) attached to. Each is a Go template over a documented context, +[save request](#commitrequest) attached to. Each is a Go template over a documented context, checked when the target reconciles rather than at commit time, and each has a built-in default, so a target that sets none still writes useful messages. @@ -1600,7 +1600,7 @@ commit seconds later. That is deliberate: a local commit that is never pushed wo operator's checkout and surface later, out of order, when you resume. Suspend is a valve on new work, not an undo. -A `SaveRequest` that ends on a suspended target fails with `FinalizeFailed` and the reason, rather +A `CommitRequest` that ends on a suspended target fails with `FinalizeFailed` and the reason, rather than resolving `Ready=True` with nothing written. The exception is `WindowMismatch`: a request that only saw another author's window still reports that. One whose commit was already made locally when you set `suspend` is pushed with it and resolves normally. @@ -1899,7 +1899,7 @@ spec: due, the branch worker closes the open commit window, publishes the writes decided up to that moment at once, and asks for the tip that publication produced. An edit decided after that is recorded in the apply window, so an apply that writes over it is withheld from Git and repaired. A -registered `SaveRequest` (a save sets its own commit boundary), a +registered `CommitRequest` (a save sets its own commit boundary), a parent branch being recovered, or a checkout a failed write left dirty holds the request back. No applier is asked for anything until every initial snapshot of every `GitTarget` naming its applier claim is published, and nothing is asked while the remote does not carry the branch. The rule is the same @@ -1910,7 +1910,7 @@ for a `Kustomization` and an `Application`. | `ForeignCommit` | The branch gained a commit GitOps Reverser did not push. The pending writes are replayed onto it and pushed first, so the request names that pushed commit | Every applier declared on the branch, once per applier | | `Refusal` | An eligible refused edit, with `spec.onRefusal: Reapply` | This target's applier | | `Repair` | An apply reverted an API edit made after the flush; Git kept the edit | That applier | -| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, once when a `GitTarget` first names the applier claim, once the applier claim's `catchUpAfter` has passed since a push of GitOps Reverser's own, and after a `SaveRequest` with [`catchUp: AfterPush`](#applying-a-save-at-once) pushed a commit | To adopt, that folder's applier; to converge, every applier declared on the branch at start, the newly named one, the one left behind, or the requesting `GitTarget`'s | +| `CatchUp` | To adopt: the API created an object in the folder, or a resync added an object to Git. To converge: once when the operator starts, once when a `GitTarget` first names the applier claim, once the applier claim's `catchUpAfter` has passed since a push of GitOps Reverser's own, and after a `CommitRequest` with [`catchUp: AfterPush`](#applying-a-save-at-once) pushed a commit | To adopt, that folder's applier; to converge, every applier declared on the branch at start, the newly named one, the one left behind, or the requesting `GitTarget`'s | An API edit asks for no apply, and neither does a push of GitOps Reverser's own, at once: the cluster already runs what they hold. Only the applier's own state is left behind, which `catchUpAfter` @@ -2963,16 +2963,16 @@ types in `clusterResources` do, and for those types its source credential's Kube cluster admin can create one, and no default grant, the chart's included, gives `clusterResources`. Use it sparingly. -## `SaveRequest` +## `CommitRequest` -`SaveRequest` is a one-shot "save" signal for a `GitTarget`. It does not create or change watch +`CommitRequest` is a one-shot "save" signal for a `GitTarget`. It does not create or change watch rules. It attaches to its author's commit window, gives that window its message, and decides when the window closes, instead of the target's timers. The important fields are: - `spec.gitTargetRef`: the target whose window to attach to, by `name`, in the request's own - namespace unless `namespace` names another; see [who may save](#who-may-save-specallowedsaverequests) + namespace unless `namespace` names another; see [who may save](#who-may-save-specallowedcommitrequests) - `spec.message`: optional literal commit message, preserved verbatim - `spec.window`: which window to attach to and when it closes; see [the request window](#the-request-window-specwindow) @@ -2983,7 +2983,7 @@ Example: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: save-now namespace: default @@ -3000,7 +3000,7 @@ spec: maxDuration: "2s" ``` -The entire spec is immutable. Create a new `SaveRequest` for each save attempt. +The entire spec is immutable. Create a new `CommitRequest` for each save attempt. A present message accepts 1–1024 Unicode characters, including newline. All other ASCII control characters, including tab, carriage return, and DEL, are rejected, as is whitespace-only text. @@ -3071,15 +3071,15 @@ attach to it: the outcome is `WindowMismatch` no matter how long the request wai that sends no audit events at all, see [saving where writes name no author](#saving-where-writes-name-no-author). -### Who may save: `spec.allowedSaveRequests` +### Who may save: `spec.allowedCommitRequests` -A `GitTarget` says which namespaces' `SaveRequest`s may save its windows in -`spec.allowedSaveRequests`, a grant list like the others (see [the grants](#the-grants)): +A `GitTarget` says which namespaces' `CommitRequest`s may save its windows in +`spec.allowedCommitRequests`, a grant list like the others (see [the grants](#the-grants)): -- **Omitted:** the `SaveRequest`s in the `GitTarget`'s own namespace, as one grant with +- **Omitted:** the `CommitRequest`s in the `GitTarget`'s own namespace, as one grant with `from: Same` that claims nothing of a window whose writes name no author. -- **`[]`:** no `SaveRequest`. -- **A list:** the `SaveRequest`s at least one grant admits. The list replaces the default, so the +- **`[]`:** no `CommitRequest`. +- **A list:** the `CommitRequest`s at least one grant admits. The list replaces the default, so the own namespace keeps access only if a grant admits it. ```yaml @@ -3089,7 +3089,7 @@ metadata: name: shared namespace: platform spec: - allowedSaveRequests: + allowedCommitRequests: - consumerNamespaces: {from: Same} - consumerNamespaces: from: Selector @@ -3110,7 +3110,7 @@ a window that names no actor. A request whose submitter was captured at admissio such a window, and its save ends `WindowMismatch`. Two settings let it claim one, and both default to claiming nothing: -- the `unattributedWindows` of the grant in the `GitTarget`'s `spec.allowedSaveRequests` that admits +- the `unattributedWindows` of the grant in the `GitTarget`'s `spec.allowedCommitRequests` that admits the request's namespace says how much it may claim: `Keep` (the default), `SavedBy`, or `TakeOver`; - the request's `spec.window.author` says what this save asks for: `Own` (the default), `SavedBy`, or `TakeOver`. @@ -3119,7 +3119,7 @@ claiming nothing: |---|---| | `Own` | nothing: it attaches only to its submitter's own window | | `SavedBy` | gives the window its message; the commit names no author, and its message ends with `Saved-By: ` | -| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: SaveRequest /; no audit record names who made these changes` | +| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: CommitRequest /; no audit record names who made these changes` | A grant's `TakeOver` allows both claims, and `SavedBy` allows only `SavedBy`. A request that asks for more than the grants admitting it allow fails when it is first seen, before it attaches: @@ -3144,7 +3144,7 @@ finalize and the push the request is still in flight, and the controller keeps r attach until it reads an outcome; the worker recognizes the re-send as the same request, so a request never resolves `NoWindow` while its commit waits out the push cooldown. Applying resources and a request together gives no ordering guarantee between controllers and watch streams. See the -[request contract](spec/saverequest-design.md). +[request contract](spec/commitrequest-design.md). ### When there is nothing to commit @@ -3176,7 +3176,7 @@ A crash between the push and the status write can record the message twice: the repeated request only while it runs. Progress and outcome are reported through kstatus-compatible **conditions** (no `phase` string). -`kubectl get saverequest` shows `Ready`, its `Reason` and the `Commit`; `-o wide` adds +`kubectl get commitrequest` shows `Ready`, its `Reason` and the `Commit`; `-o wide` adds `AuthorAttributed` and `Pushed`. Automation must stop on either `Ready=True` or `Stalled=True`; `kubectl wait --for=condition=Ready` alone keeps waiting on terminal failures. `Ready=True` includes successful no-commit outcomes. Require `Pushed=True` and `status.commit` for evidence that the @@ -3241,7 +3241,7 @@ deleted only after its `Applied` condition is settled. ### Finished requests are deleted after 48 hours -Every save leaves one `SaveRequest` behind. When a request finishes (`Ready=True` or +Every save leaves one `CommitRequest` behind. When a request finishes (`Ready=True` or `Stalled=True`), the controller writes the time it will be deleted onto it: ```yaml @@ -3259,7 +3259,7 @@ contract, so it is also how you change the outcome for one request: - Set it yourself when you create the request to choose the time up front. The controller never replaces a value that is already there. -The period comes from `--save-request-ttl` (Helm: `controllerManager.saveRequestTTL`, default +The period comes from `--commit-request-ttl` (Helm: `controllerManager.commitRequestTTL`, default `48h`). Changing it only affects requests that finish afterwards. `0` writes no annotation, so new requests are kept. A request that finished before this feature existed has no annotation and is never deleted. @@ -3269,7 +3269,7 @@ never deleted. A `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next to the edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It names one `GitTarget` on the cluster GitOps Reverser runs in, and saves its creator's commit window -there: the window gets its message and closes on its timers, as with a [`SaveRequest`](#saverequest). +there: the window gets its message and closes on its timers, as with a [`CommitRequest`](#commitrequest). ```yaml apiVersion: source.configbutler.ai/v1alpha1 @@ -3312,7 +3312,7 @@ credential needs there, in the `source.configbutler.ai` group: | Value | What GitOps Reverser does | The credential needs | | --- | --- | --- | | `Disabled` (default) | nothing | nothing | -| `Enabled` | serves the requests and writes their status, and deletes a finished request `--save-request-ttl` after it finished (`0` keeps it) | `get`, `list`, `watch` and `delete` on `remotesaverequests`; `update` on `remotesaverequests/status` | +| `Enabled` | serves the requests and writes their status, and deletes a finished request `--commit-request-ttl` after it finished (`0` keeps it) | `get`, `list`, `watch` and `delete` on `remotesaverequests`; `update` on `remotesaverequests/status` | ```yaml apiVersion: configbutler.ai/v1alpha3 @@ -3338,7 +3338,7 @@ An admitted `RemoteSaveRequest` therefore saves its creator's whole window, edit request's own namespace included: a grant in `allowedRemoteSaveRequests` authorizes exactly that. The request's namespace decides only whether a grant admits it. -As with a `SaveRequest`, a second request of the same author on the same `GitTarget`, made while the +As with a `CommitRequest`, a second request of the same author on the same `GitTarget`, made while the first still waits for a window or collects one, is refused at once with `SaveInProgress`: a window carries one message, and the two would race for it. @@ -3358,7 +3358,7 @@ loses the save's author without an error. `spec.window.attachTimeout` (default `5s`) counts from when GitOps Reverser picks the request up, which is after the request's own audit event arrived, so it only covers the spread between the request's event and its edits'. `maxDuration` (default `2s`) and `idleTimeout` close the attached -window as for a `SaveRequest`. +window as for a `CommitRequest`. ### Its status @@ -3378,7 +3378,7 @@ A request is served again after GitOps Reverser restarts. It keeps what its stat author found when it was first served, and an outcome already decided, in which case only its apply, if it asked for one, is followed. -A finished `RemoteSaveRequest` is deleted `--save-request-ttl` (default `48h`) after it finished; +A finished `RemoteSaveRequest` is deleted `--commit-request-ttl` (default `48h`) after it finished; `0` keeps it. ## Audit ingestion settings @@ -3585,7 +3585,7 @@ Move to hand-managed resources when you want: - more than one `GitTarget` - more than one watch rule - cluster-scoped watching with `ClusterWatchRule` -- ad hoc save requests with `SaveRequest` +- ad hoc save requests with `CommitRequest` - direct control over `GitProvider.spec.commit` - direct control over encryption settings diff --git a/docs/definitions.md b/docs/definitions.md index 2790f4fd..83b68675 100644 --- a/docs/definitions.md +++ b/docs/definitions.md @@ -176,10 +176,10 @@ the parent branch is not followed. **Commit window.** The interval a branch worker batches one author's writes to one target over before it commits. Only a write opens one; nothing else does. It closes on its `idleTimeout` of silence or its `maxDuration` after opening, whichever comes first, or earlier for a flush, unless a -`SaveRequest` holds it. Windows do not merge across authors, which is why attribution can be +`CommitRequest` holds it. Windows do not merge across authors, which is why attribution can be per-commit. -**Attach.** What a `SaveRequest`, or a `RemoteSaveRequest` made on a source cluster, does to a commit +**Attach.** What a `CommitRequest`, or a `RemoteSaveRequest` made on a source cluster, does to a commit window: it gives the window its message and replaces the window's timers with its own. A request attaches to at most one window and never opens one. One word for the act, in the API (`window.attach`, `attachTimeout`), the code and the docs: not @@ -199,7 +199,7 @@ a durable journal, which this is not. `status.remote`. **Commit.** A bare Git commit hash, wherever one appears in the API: `status.remote.commit`, -`status.remote.parent.commit`, `SaveRequest.status.commit`, `status.placement.resolvedAtCommit`. +`status.remote.parent.commit`, `CommitRequest.status.commit`, `status.placement.resolvedAtCommit`. Not "sha", which names an algorithm Git is in the middle of changing, and not "revision", which Flux uses for a polymorphic identifier that may be a tag, a chart version or a composite like `main@sha1:`. If this project ever needs that composite, `revision` is the word waiting for it. diff --git a/docs/demo/commit-window.md b/docs/demo/commit-window.md index 591fb9f5..9a0ac35c 100644 --- a/docs/demo/commit-window.md +++ b/docs/demo/commit-window.md @@ -1,6 +1,6 @@ # Commit-window example -Two Kubernetes resources become one Git commit: one created, one edited. A `SaveRequest` supplies +Two Kubernetes resources become one Git commit: one created, one edited. A `CommitRequest` supplies the message and its own timers, which replace the target's for that one window. You can save after the changes or before them; the example shows both. The request itself is outside the selected resource types, so saving never commits the save button. @@ -139,7 +139,7 @@ Save as `save-now.yaml`. Use `kubectl create` rather than `kubectl apply`: the r ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: generateName: save-hello- namespace: gitops-reverser-quickstart-demo @@ -179,7 +179,7 @@ The request can come first, and wait for the changes. Save as `save-long.yaml`: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: generateName: save-hello- namespace: gitops-reverser-quickstart-demo @@ -228,13 +228,13 @@ reaches the operator after the window closed goes into the next commit. The default configured-author setup without request author capture works for this example. If actor attribution and request author capture are enabled, submit the resources and the request as the same -Kubernetes user. See [SaveRequest](../configuration.md#saverequest) for author matching and +Kubernetes user. See [CommitRequest](../configuration.md#commitrequest) for author matching and outcomes. ## Check the result ```bash -kubectl get saverequests -n gitops-reverser-quickstart-demo -o wide +kubectl get commitrequests -n gitops-reverser-quickstart-demo -o wide ``` Look for `Pushed=True` and a `status.commit`. `Pushed` is a wide column; `Ready` and the `COMMIT` @@ -257,7 +257,7 @@ With `save-long.yaml` the first line is `feat(demo): save long running`. Either | Resource | Captured by this rule? | |---|---| | ServiceAccount, ConfigMap | Yes | -| SaveRequest, GitTarget, WatchRule | No | +| CommitRequest, GitTarget, WatchRule | No | | Services, Deployments, Pods, ReplicaSets | No | Without the request, the window closes on its own five seconds after the last change, and the diff --git a/docs/design/applier-echoes.md b/docs/design/applier-echoes.md index 7d86b66a..1ae72e9a 100644 --- a/docs/design/applier-echoes.md +++ b/docs/design/applier-echoes.md @@ -95,7 +95,7 @@ built and released together. 9. **A save is the strongest API intent, and is never trimmed.** A `CommitRequest` keeps every object it captured, and its outcome is settled by the push, as the - [CommitRequest contract](../spec/saverequest-design.md) says. Re-asserting a save over a + [CommitRequest contract](../spec/commitrequest-design.md) says. Re-asserting a save over a concurrent Git-side change of the same object is the "API wins" rule. ## The race @@ -679,7 +679,7 @@ The build departs from the plan above in twelve places, decided on 2026-10-20: startup baseline. - [`gittarget-branch-worker-pending-writes.md`](gittarget-branch-worker-pending-writes.md): decide, `advance`, materialize, and replay. -- [`../spec/saverequest-design.md`](../spec/saverequest-design.md): the save contract F2 keeps. +- [`../spec/commitrequest-design.md`](../spec/commitrequest-design.md): the save contract F2 keeps. - [`../spec/reconcile-via-watchlist-mark-and-sweep.md`](../spec/reconcile-via-watchlist-mark-and-sweep.md): the resync and its sweep. - [`internal/git/apply_window.go`](../../internal/git/apply_window.go): `withholdLateApply`. diff --git a/docs/design/branch-worker-event-model.md b/docs/design/branch-worker-event-model.md index 87adcfca..244a2af7 100644 --- a/docs/design/branch-worker-event-model.md +++ b/docs/design/branch-worker-event-model.md @@ -172,9 +172,9 @@ on the worker. For an attached save that committed locally, this left `WaitingForPush` with no time limit. After its safety window, `withdrawCommitRequest` in the -[controller](../../internal/controller/saverequest_controller.go) sends withdrawal and continues +[controller](../../internal/controller/commitrequest_controller.go) sends withdrawal and continues polling when the worker still holds the request. `handleWithdrawCommitRequest` in the -[attach loop](../../internal/git/save_request_attach_loop.go) leaves attached or committed +[attach loop](../../internal/git/commit_request_attach_loop.go) leaves attached or committed requests unchanged. Neither path retries the push. The API showed `Ready=False`, `Reconciling=True`, `Stalled=False`, and `Pushed=Unknown`, with reason `WaitingForPush`. @@ -315,7 +315,7 @@ Represent intent, decisions, and external work separately so recovery knows what | Category | Example | Meaning | |---|---|---| -| Command | `SaveRequested` | A caller asks to attach a save | +| Command | `CommitRequested` | A caller asks to attach a save | | Fact | `SaveAttached` | The worker bound that request to a particular window | | Effect intent | `PublishBatch` | A recorded decision that a push must still be attempted | | Effect result | `PublicationConfirmed` | Evidence that a particular attempt published its work | @@ -380,7 +380,7 @@ The current attach loop and controller already preserve several rules: Retain those rules in a durable lifecycle. One illustrative successful path is: ```text -SaveRequested(request UID, target UID, attribution, message, timing policy) +CommitRequested(request UID, target UID, attribution, message, timing policy) SaveRegistered(request UID, sequence, registration time, attach deadline) SaveAttached(request UID, window ID) WindowClosed(window ID, included inputs, reason) diff --git a/docs/design/commitrequest-save-wait-options.md b/docs/design/commitrequest-save-wait-options.md index bf005cde..73851628 100644 --- a/docs/design/commitrequest-save-wait-options.md +++ b/docs/design/commitrequest-save-wait-options.md @@ -89,7 +89,7 @@ smallest token the UI already has that can name the save it wants Git to show. Four mechanisms decide the outcome, and the failure is structural rather than a tuning miss. **The deadline is stamped from receipt.** `handleAttachCommitRequest` in -[`save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go) sets +[`commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go) sets `finalizeAt = time.Now().Add(closeDelay)` on first registration, and idempotent re-sends keep that first value. With the field omitted the deadline is the present instant, so the request is due on the next pass of the worker's event loop. @@ -105,7 +105,7 @@ audit in batches bounded by `--audit-webhook-batch-max-wait=1s`, so the floor on is bounded below by that batch plus the join. **Attach is author-bound.** `matchesWindow` in -[`save_request_attach.go`](../../internal/git/save_request_attach.go) admits a window only when +[`commit_request_attach.go`](../../internal/git/commit_request_attach.go) admits a window only when the author strings are equal, the two sides agree on whether an actor is named, and the `GitTarget` identity matches. @@ -331,10 +331,10 @@ importantly, the cluster-level default flag below is **blocked** on the pointer omitted field and an explicit `0` are the same value, so nothing downstream can tell them apart. The type is `v1alpha3` with one known consumer; this is the cheapest it will ever be. -- `api/v1alpha3/saverequest_types.go`, then `task generate` and `task manifests` -- nil handling at the controller seam (`saverequest_controller.go`, where the field is copied onto +- `api/v1alpha3/commitrequest_types.go`, then `task generate` and `task manifests` +- nil handling at the controller seam (`commitrequest_controller.go`, where the field is copied onto `AttachCommitRequest`) -- docs: `configuration.md`, `architecture.md`, `spec/saverequest-design.md`, an `UPGRADING.md` note +- docs: `configuration.md`, `architecture.md`, `spec/commitrequest-design.md`, an `UPGRADING.md` note **Half a day.** Fixes R2. Leaves rows 1, 4, 6 untouched, and moves row 3 from "always" to "when the cluster is slower than two seconds" — an improvement in reliability, not a change in kind. @@ -405,12 +405,12 @@ cluster never saw the write you named" (fact-index lookup, row 4 vs 5), "the tar accepting writes" (row 10), and "nothing was pending" (row 5). - `api/v1alpha3`, CRD, CEL cap -- `saverequest_controller.go` plumb; `saverequest_finalize.go` reasons and messages -- `pendingCommitRequest` satisfaction state and the two-timer model in `save_request_attach_loop.go` +- `commitrequest_controller.go` plumb; `commitrequest_finalize.go` reasons and messages +- `pendingCommitRequest` satisfaction state and the two-timer model in `commit_request_attach_loop.go` - `status.waitingFor` on the object while it waits (R9) -- docs: `spec/saverequest-design.md`, `spec/status-conditions-guide.md`, `interpreting-metrics.md`, +- docs: `spec/commitrequest-design.md`, `spec/status-conditions-guide.md`, `interpreting-metrics.md`, `configuration.md`, `architecture.md` -- unit tests plus a timing-sensitive e2e in `test/e2e/save_request_e2e_test.go` +- unit tests plus a timing-sensitive e2e in `test/e2e/commit_request_e2e_test.go` **Three to four days.** Fixes rows 1, 2, 3, 7, 9 outright, and row 10 via the hook above. Splits row 4 from row 5 only where a usable fact exists. Does **not** fix row 11. @@ -503,9 +503,9 @@ the participant's message on the commit. ## Related -- [`../spec/saverequest-design.md`](../spec/saverequest-design.md), the current contract. The +- [`../spec/commitrequest-design.md`](../spec/commitrequest-design.md), the current contract. The code depends on it, so each phase changes that page in the same commit. -- [`../spec/saverequest-admission-authorship.md`](../spec/saverequest-admission-authorship.md), +- [`../spec/commitrequest-admission-authorship.md`](../spec/commitrequest-admission-authorship.md), where the submitter comes from and why the lookup is present-or-never. - [`../spec/watch-event-ordering-and-attribution-grace.md`](../spec/watch-event-ordering-and-attribution-grace.md), the head-of-line wait this page proposes to wait behind rather than shorten. diff --git a/docs/design/concurrent-author-windows.md b/docs/design/concurrent-author-windows.md index ee48f3aa..0ebd89f7 100644 --- a/docs/design/concurrent-author-windows.md +++ b/docs/design/concurrent-author-windows.md @@ -23,7 +23,7 @@ sequenceDiagram Note over W: window 1 closes (identity_change), window 2 opens: Bob A->>W: edit cm/c Note over W: window 2 closes, window 3 opens: Alice - A->>W: SaveRequest "fix(shop): cart limit" + A->>W: CommitRequest "fix(shop): cart limit" Note over W: attaches to window 3 only ``` @@ -56,7 +56,7 @@ sequenceDiagram Note over W: Bob's window opens; Alice's stays open A->>W: edit cm/c Note over W: joins Alice's window - A->>W: SaveRequest + A->>W: CommitRequest Note over W: Alice's window: cm/a + cm/c, her message ``` diff --git a/docs/design/e2e-coverage-gaps-and-improvements-plan.md b/docs/design/e2e-coverage-gaps-and-improvements-plan.md index d3841793..9dade6bd 100644 --- a/docs/design/e2e-coverage-gaps-and-improvements-plan.md +++ b/docs/design/e2e-coverage-gaps-and-improvements-plan.md @@ -39,7 +39,7 @@ Reading it against the architecture surfaced three gaps, in increasing order of | Attribution from audit identity | [commit_author_attribution](../../test/e2e/commit_author_attribution_e2e_test.go), [impersonation](../../test/e2e/impersonation_test.go) | ✅ | | Commit signing (generated + BYOK) | [signing](../../test/e2e/signing_e2e_test.go) | ✅ | | Encryption / SOPS Secrets | [watchrule_configmap_secret](../../test/e2e/watchrule_configmap_secret_e2e_test.go), [flux_bi_directional](../../test/e2e/flux_bi_directional_e2e_test.go) | ✅ | -| Commit windows + CommitRequest | [commit_window_batching](../../test/e2e/commit_window_batching_e2e_test.go), [commit_request](../../test/e2e/save_request_e2e_test.go) | ✅ | +| Commit windows + CommitRequest | [commit_window_batching](../../test/e2e/commit_window_batching_e2e_test.go), [commit_request](../../test/e2e/commit_request_e2e_test.go) | ✅ | | Scale subresource → `spec.replicas` | [deployment_scale_subresource](../../test/e2e/deployment_scale_subresource_e2e_test.go) | ✅ | | Aggregated API server (LIST fallback) | [aggregated_apiserver](../../test/e2e/aggregated_apiserver_e2e_test.go) | ✅ | | GitTarget isolation / path overlap | [gittarget_isolation](../../test/e2e/gittarget_isolation_e2e_test.go), [gittarget_overlap](../../test/e2e/gittarget_overlap_e2e_test.go) | ✅ | diff --git a/docs/design/e2e-suite-trim.md b/docs/design/e2e-suite-trim.md index fed995a3..6fb6024e 100644 --- a/docs/design/e2e-suite-trim.md +++ b/docs/design/e2e-suite-trim.md @@ -24,7 +24,7 @@ can go; each costs a full rebuild. This removes a 14–18 minute leg from most P These matter more than the duplicates below. -1. **[`save_request_e2e_test.go`](../../test/e2e/save_request_e2e_test.go)**: a comment says the +1. **[`commit_request_e2e_test.go`](../../test/e2e/commit_request_e2e_test.go)**: a comment says the generateName spec "is skipped in configured-author mode", but it has no `Skip`. Add it, or fix the comment. 2. **[`signing_e2e_test.go`](../../test/e2e/signing_e2e_test.go)**: its own comment says the diff --git a/docs/design/event-pipeline-overview.md b/docs/design/event-pipeline-overview.md index dc0c7551..501a9719 100644 --- a/docs/design/event-pipeline-overview.md +++ b/docs/design/event-pipeline-overview.md @@ -571,5 +571,5 @@ remains: | Retry deadline | [`internal/git/retry.go`](../../internal/git/retry.go) | | Publication report and backlog gauges | [`internal/git/publication.go`](../../internal/git/publication.go) | | Resyncs | [`internal/git/resync_flush.go`](../../internal/git/resync_flush.go) | -| Saves | [`internal/git/save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go) | +| Saves | [`internal/git/commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go) | | Missing parent probe | [`internal/git/parent_recovery.go`](../../internal/git/parent_recovery.go) | diff --git a/docs/design/git-write-preflight.md b/docs/design/git-write-preflight.md index b5aa4890..033a0f16 100644 --- a/docs/design/git-write-preflight.md +++ b/docs/design/git-write-preflight.md @@ -879,7 +879,7 @@ point where an author can most easily respond. [render-verification]: ../../internal/manifestanalyzer/render_verify.go [namespace-admission]: ../../internal/webhook/watchrule_source_namespace_admission.go [oracle-tests]: ../../internal/git/kustomize_oracle_test.go -[commitrequest]: ../spec/saverequest-design.md +[commitrequest]: ../spec/commitrequest-design.md [placement-tests]: ../../internal/git/placement_test.go [consent]: support-boundary/admission-consent.md [validation-placement]: ../spec/where-validation-lives.md diff --git a/docs/design/gittarget-branch-worker-pending-writes.md b/docs/design/gittarget-branch-worker-pending-writes.md index d132ec80..fe7ff683 100644 --- a/docs/design/gittarget-branch-worker-pending-writes.md +++ b/docs/design/gittarget-branch-worker-pending-writes.md @@ -492,7 +492,7 @@ Remaining limits: Updated the event model's "Next implementation" and "Problems" sections, the findings in the state-of-affairs page, the paragraph after the gate table in -[`saverequest-design.md`](../spec/saverequest-design.md#when-the-target-may-not-be-written), +[`commitrequest-design.md`](../spec/commitrequest-design.md#when-the-target-may-not-be-written), and [`architecture.md`](../architecture.md), which gained a section on a remote that cannot be reached. The effective-point comment in `write_gate.go` already said what was built and is unchanged. `UPGRADING.md` gained its entries with each step. The PR body carries the argument; this @@ -574,7 +574,7 @@ decided from is used once: the replay, under `prune.mode: Always` too. - **A save keeps every object.** A write carrying a `CommitRequest` is never trimmed: it keeps each object it captured, unchanged ones included, and the push decides its outcome (`AlreadyPresent` - or `Committed`), as the [CommitRequest contract](../spec/saverequest-design.md) says. + or `Committed`), as the [CommitRequest contract](../spec/commitrequest-design.md) says. Pinned by [`applier_echoes_test.go`](../../internal/git/applier_echoes_test.go): an unchanged object is no action, a resync replay applies its decided actions, a replayed resync obeys a tightened prune diff --git a/docs/design/gittarget-parent-hardening.md b/docs/design/gittarget-parent-hardening.md index c5b25bf9..aa4827bb 100644 --- a/docs/design/gittarget-parent-hardening.md +++ b/docs/design/gittarget-parent-hardening.md @@ -87,7 +87,7 @@ older API-wave plan as part of any of these changes. Recovery can hold work for minutes, so a request must not be failed on a controller clock while the worker still holds it. That is already fixed on this branch: the controller waits for any request the worker holds (`CollectingWindow`, `WaitingForPush`), and fails an unheld one only after -withdrawing it from the worker (`docs/spec/saverequest-design.md`). The §4.4 tests must still +withdrawing it from the worker (`docs/spec/commitrequest-design.md`). The §4.4 tests must still include a request retained through a recovery longer than `attachTimeout + maxDuration + 120s`, resolving `Committed`. diff --git a/docs/design/gittarget-state-of-affairs.md b/docs/design/gittarget-state-of-affairs.md index dddbbe88..e4e4a984 100644 --- a/docs/design/gittarget-state-of-affairs.md +++ b/docs/design/gittarget-state-of-affairs.md @@ -30,7 +30,7 @@ Restarting the Pod loses in-memory decisions and save receipts. Redis persistenc | Every call to a Git server is bounded; a lost push reply is settled by evidence | [`network_bound.go`](../../internal/git/network_bound.go) | `network_bound_test.go` | | One publication report feeds every `GitTarget` on the branch, its held saves, and the backlog metrics | [`publication.go`](../../internal/git/publication.go), [`gittarget_publication.go`](../../internal/controller/gittarget_publication.go) | `publication_test.go`, `gittarget_publication_test.go` | | Replay refusal settles only the affected entry | `pending_writes_loop.go`, `branch_worker.go` | [`replay_refusal_test.go`](../../internal/git/replay_refusal_test.go) | -| Save outcomes follow remote publication; the controller cannot time out a held save | [`save_request_attach_loop.go`](../../internal/git/save_request_attach_loop.go), [`saverequest_controller.go`](../../internal/controller/saverequest_controller.go) | Held-request retry and controller safety-bound tests | +| Save outcomes follow remote publication; the controller cannot time out a held save | [`commit_request_attach_loop.go`](../../internal/git/commit_request_attach_loop.go), [`commitrequest_controller.go`](../../internal/controller/commitrequest_controller.go) | Held-request retry and controller safety-bound tests | | Parent selection and compare-and-swap remain publication guards | [`git_atomic_push.go`](../../internal/git/git_atomic_push.go) | Parent-change tests and `TestGitRoundTripLedger` | | New watch streams start with a fresh replay; reconnects resume a cursor only after the stream's own replay completed | [`target_watch.go`](../../internal/watch/target_watch.go) | `runTargetWatch`, `TestRunTargetWatch_ResumesOnlyAfterItsOwnReplayCompleted` | | A live event changes the dedup baseline and the cursor only once the worker accepted it | [`target_watch.go`](../../internal/watch/target_watch.go) | `refused_admission_test.go`, `desired_state_change_filter_test.go` | @@ -144,7 +144,7 @@ and persistence and HA ([HA plan](../future/ha-gittarget-distribution-plan.md)). | `gittarget-branch-worker-pending-writes.md` | The steps as built, the pause/resume contract, and the review findings | | [`branch-worker-event-model.md`](branch-worker-event-model.md) | Transition semantics and the later boundary for reconstructing execution | | [`push-cooldown.md`](push-cooldown.md) | Healthy publication cadence; the success cooldown remains unchanged | -| [`saverequest-design.md`](../spec/saverequest-design.md) | Current save contract, including retained versus unaccepted requests | +| [`commitrequest-design.md`](../spec/commitrequest-design.md) | Current save contract, including retained versus unaccepted requests | | `ha-gittarget-distribution-plan.md` | Future durable acceptance, publication evidence, storage budgets, and HA | Parent observation, empty-repository bootstrap policy, configuration freshness, and refusal state diff --git a/docs/design/reconcile-triggering.md b/docs/design/reconcile-triggering.md index 11ae7204..17a89254 100644 --- a/docs/design/reconcile-triggering.md +++ b/docs/design/reconcile-triggering.md @@ -58,7 +58,7 @@ and out-of-band age-key updates ride the 5-minute reconcile instead. Sources: `gitprovider_controller.go`, `gittarget_controller.go`, `watchrule_controller.go`, `clusterwatchrule_controller.go`, -`saverequest_controller.go`. +`commitrequest_controller.go`. ### 1.3 Two structural consequences @@ -143,9 +143,9 @@ not reconciled until an unrelated event happens to fire. (Full treatment, incl. subtree-hash detection and Flux/Argo coordination, in the sibling doc.) **A4. CommitRequest outcomes are polled, not pushed.** The worker records a terminal -outcome (`recordCommitRequestOutcome`, `internal/git/save_request_attach_loop.go:47`) +outcome (`recordCommitRequestOutcome`, `internal/git/commit_request_attach_loop.go:47`) and the controller reads it (`LookupCommitRequestOutcome`) — but by **requeuing -every 2 s** (`saverequest_controller.go:195`). The capture half of the recipe +every 2 s** (`commitrequest_controller.go:195`). The capture half of the recipe already exists; only the push edge is missing. A nudge would replace the poll. ### Gap B — control-plane status does not propagate to dependents diff --git a/docs/facts/crd-descriptions-in-kubectl-explain.md b/docs/facts/crd-descriptions-in-kubectl-explain.md index 2a6cd8aa..2254eba2 100644 --- a/docs/facts/crd-descriptions-in-kubectl-explain.md +++ b/docs/facts/crd-descriptions-in-kubectl-explain.md @@ -75,7 +75,7 @@ of the generated CRD and point `crdDir` at the copy. These came out of the same review. Each one is behavior that a shorter or rearranged description can easily misstate. -- **A SaveRequest's message can be wrapped.** The message is never evaluated as a template. +- **A CommitRequest's message can be wrapped.** The message is never evaluated as a template. When the GitTarget sets `spec.commit.message.requestTemplate`, that template frames the message as `RequestMessage`; without one, the message is committed as written. See [`commit_executor.go`](../../internal/git/commit_executor.go). diff --git a/docs/facts/generated-name-support.md b/docs/facts/generated-name-support.md index 3c7cb339..e75c723d 100644 --- a/docs/facts/generated-name-support.md +++ b/docs/facts/generated-name-support.md @@ -31,12 +31,12 @@ A client that wants the server to allocate the name sends a collection `POST` with `metadata.generateName` and without `metadata.name`: ```text -POST /apis/configbutler.ai/v1alpha3/namespaces/voter-production/saverequests +POST /apis/configbutler.ai/v1alpha3/namespaces/voter-production/commitrequests ``` ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: namespace: voter-production generateName: coffee-save- @@ -50,7 +50,7 @@ The successful API response contains the final server-allocated identity: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: namespace: voter-production generateName: coffee-save- @@ -94,16 +94,16 @@ Example audit event shape: "level": "RequestResponse", "stage": "ResponseComplete", "verb": "create", - "requestURI": "/apis/configbutler.ai/v1alpha3/namespaces/voter-production/saverequests", + "requestURI": "/apis/configbutler.ai/v1alpha3/namespaces/voter-production/commitrequests", "objectRef": { "apiGroup": "configbutler.ai", "apiVersion": "v1alpha1", - "resource": "saverequests", + "resource": "commitrequests", "namespace": "voter-production" }, "requestObject": { "apiVersion": "configbutler.ai/v1alpha3", - "kind": "SaveRequest", + "kind": "CommitRequest", "metadata": { "namespace": "voter-production", "generateName": "coffee-save-" @@ -111,7 +111,7 @@ Example audit event shape: }, "responseObject": { "apiVersion": "configbutler.ai/v1alpha3", - "kind": "SaveRequest", + "kind": "CommitRequest", "metadata": { "namespace": "voter-production", "generateName": "coffee-save-", @@ -139,14 +139,14 @@ This keeps the URL-level audit reference authoritative when it is complete, but still handles server-named objects whose final identity only appears in the response body. -## `SaveRequest` example +## `CommitRequest` example -`SaveRequest` objects are intentionally suitable for generated names because +`CommitRequest` objects are intentionally suitable for generated names because each object is a one-shot save signal: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: namespace: voter-production generateName: save- diff --git a/docs/facts/subresources.md b/docs/facts/subresources.md index 17acfcbd..079eb357 100644 --- a/docs/facts/subresources.md +++ b/docs/facts/subresources.md @@ -188,7 +188,7 @@ marker on the type: ``` Every `configbutler.ai` CRD enables `/status`: `GitProvider`, `GitTarget`, -`WatchRule`, `ClusterWatchRule`, and `SaveRequest`. A real `GitTarget`: +`WatchRule`, `ClusterWatchRule`, and `CommitRequest`. A real `GitTarget`: ```yaml apiVersion: configbutler.ai/v1alpha3 @@ -263,7 +263,7 @@ status: ``` No `configbutler.ai` resource enables `/scale`: `GitProvider`, `GitTarget`, -`WatchRule`, `ClusterWatchRule`, and `SaveRequest` are not scalable workloads, +`WatchRule`, `ClusterWatchRule`, and `CommitRequest` are not scalable workloads, so a replica count has no meaning for them. ## Audit logs and the `/scale` subresource @@ -432,14 +432,14 @@ For an imperative operation, Kubernetes API design has two options: a custom subresource (requires an aggregated API server), or a **request resource** — a normal CRD that represents "please do X", carrying its outcome in `status`. -`configbutler.ai` uses the request-resource pattern with **`SaveRequest`**. +`configbutler.ai` uses the request-resource pattern with **`CommitRequest`**. Instead of a hypothetical `POST /gittargets/{name}/commit` subresource, creating -a `SaveRequest` object finalizes the open commit window for the referenced +a `CommitRequest` object finalizes the open commit window for the referenced `GitTarget` and reports the result back in status: ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: save-now namespace: default @@ -475,7 +475,7 @@ action: the request itself is a durable, inspectable object. | Durable declarative intent | CRD | | Controller reports observed state | CRD + `/status` | | HPA / `kubectl scale` compatibility | CRD + `/scale` | -| One-off declarative job/request | Request CRD (e.g. `SaveRequest`) | +| One-off declarative job/request | Request CRD (e.g. `CommitRequest`) | | Derived synchronous or streaming output | Aggregated API subresource | | Proxy / tunnel / console / session | Aggregated API subresource | | Imperative action | Request CRD, or aggregated API subresource if it must be synchronous/streaming | @@ -501,5 +501,5 @@ The pattern Kubernetes follows: `configbutler.ai` stays entirely within CRDs: every type is served by the main kube-apiserver, every type uses `/status`, none use `/scale`, and imperative -actions are modeled as request resources (`SaveRequest`) rather than custom +actions are modeled as request resources (`CommitRequest`) rather than custom subresources. diff --git a/docs/finished/redis-key-schema-v3.md b/docs/finished/redis-key-schema-v3.md index 1d741f66..ae2ad576 100644 --- a/docs/finished/redis-key-schema-v3.md +++ b/docs/finished/redis-key-schema-v3.md @@ -5,7 +5,7 @@ > Status: **IMPLEMENTED** — 2026-06-29. The `v2 → v3` key-format bump landed for the live Redis families: > the audit author index, the watch-resume cursor, and the new command-author **store** (storage > mechanics only — the CommitRequest admission/controller rewrite in -> [saverequest-admission-authorship.md](../spec/saverequest-admission-authorship.md) is a separate round and +> [commitrequest-admission-authorship.md](../spec/commitrequest-admission-authorship.md) is a separate round and > is **not** wired up yet). Scope: (1) replace the opaque `attr` namespace with a top-level `author` > domain; (2) re-lay audit-sourced resource author facts as a readable, **object-grouped** hierarchy > (`:object::` + `:last`, plus a type-scoped `:rv:` escape hatch), carrying @@ -24,7 +24,7 @@ > `internal/queue/attribution_index.go`, > [`internal/queue/redis_store.go`](../../internal/queue/redis_store.go), > [`internal/watch/author_resolver.go`](../../internal/watch/author_resolver.go), -> [CommitRequest authorship from admission](../spec/saverequest-admission-authorship.md), +> [CommitRequest authorship from admission](../spec/commitrequest-admission-authorship.md), > [attribution](../spec/attribution.md), > [watch event ordering & attribution grace](../spec/watch-event-ordering-and-attribution-grace.md), > [watch-first ingestion architecture](watch-first-ingestion-architecture.md). diff --git a/docs/future/config-surface-for-a-structured-repository.md b/docs/future/config-surface-for-a-structured-repository.md index 55465fc2..f04c15e8 100644 --- a/docs/future/config-surface-for-a-structured-repository.md +++ b/docs/future/config-surface-for-a-structured-repository.md @@ -4,7 +4,7 @@ > Date: 2026-07-24 > Absorbs and extends the API-surface block left unbuilt by the status and configuration-model > review — `spec.suspend`, `spec.interval` and the reconcile-request annotation, the -> `SaveRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in +> `CommitRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in > [`gittarget-api-wave.md`](../design/gittarget-api-wave.md). ## The one-sentence finding @@ -239,7 +239,7 @@ Still open from the companion review, all API-surface, all cheaper now than afte - **F6** — `spec.suspend` on GitTarget / WatchRule / ClusterWatchRule / GitProvider; `spec.interval` on GitProvider at minimum; jitter the requeue; `reconcile.configbutler.ai/requestedAt` + `status.lastHandledReconcileAt`. -- **F10** — SaveRequest lifecycle: `ttlSecondsAfterFinished` or an ownerRef, plus the +- **F10** — CommitRequest lifecycle: `ttlSecondsAfterFinished` or an ownerRef, plus the `delete` verb. Today they accumulate in etcd forever and the controller cannot reap them. - **F9** — verify the stored `scope: Namespaced` status-write path on the minimum supported Kubernetes version (one envtest). @@ -398,7 +398,7 @@ still `v1alpha3` and while there are no users. 3. **F6** — `spec.suspend` on the four reconciled kinds; `spec.interval` on GitProvider; requeue jitter; `reconcile.configbutler.ai/requestedAt` + `status.lastHandledReconcileAt`. 4. **B3** — `spec.placement.mode: Infer|Declared|Strict`. -5. **F10** — SaveRequest TTL/ownerRef + the `delete` verb. +5. **F10** — CommitRequest TTL/ownerRef + the `delete` verb. ### Wave 2 — legibility, non-breaking, ships any time after diff --git a/docs/future/direction-and-configuration-surface.md b/docs/future/direction-and-configuration-surface.md index 88b6b243..878f99c7 100644 --- a/docs/future/direction-and-configuration-surface.md +++ b/docs/future/direction-and-configuration-surface.md @@ -40,7 +40,7 @@ to make and cheap to hold. compatibility frontier. - **The hard prerequisites already shipped or are designed.** The config-plane split (inline `GitTarget.spec.kubeConfig`) points a target at a remote or ephemeral intent - cluster. `SaveRequest` carries authorship and intent. The branch-per-experiment flow + cluster. `CommitRequest` carries authorship and intent. The branch-per-experiment flow is designed in [`idea-application-editing.md`](idea-application-editing.md). - **The layout model is a direction-B feature wearing direction-A clothes.** `kind: Kustomize` with `create: true` bootstraps structure into an empty repository; @@ -295,7 +295,7 @@ environment its own dry run, status, and commit batching. | #295 placement bugs | fix now; direction-agnostic correctness | before the wave | | #296 visibility | metric split, `{kindLower}`, template constant now; `status.layout` rides the wave | split | | #293 layout model | the centerpiece; more valuable under direction B than its own text claims | the wave's core | -| #294 API wave | core four (layout, mode, suspend/interval, commitWindow move) gate nothing else; SaveRequest items ride along without gating | the wave | +| #294 API wave | core four (layout, mode, suspend/interval, commitWindow move) gate nothing else; CommitRequest items ride along without gating | the wave | | #146 per-rule objectSelector | yes; in direction B it is how an intent cluster marks what belongs in the artifact | any time | | #148 leader election | yes, scoped to active-passive; decoupled from full HA distribution | any time | | #210 conventional PR titles | implement the ~15-line workflow check or close; it is a process wish, not a bug | close out | diff --git a/docs/future/ha-gittarget-distribution-plan.md b/docs/future/ha-gittarget-distribution-plan.md index b96fb861..2ee6c009 100644 --- a/docs/future/ha-gittarget-distribution-plan.md +++ b/docs/future/ha-gittarget-distribution-plan.md @@ -72,7 +72,7 @@ The repository has useful foundations, but it is not HA today. content under the write/prune policy, but cannot recover lost intermediate events or accepted save decisions. Persisted cursors alone do not establish lossless failover. - BranchWorker keeps open commit windows, local commits, unpushed writes, and - SaveRequest outcomes in memory. Its local clone is disposable. + CommitRequest outcomes in memory. Its local clone is disposable. - The pending-writes plan provides ordered pending writes, one materializer, one retry schedule, isolated replay refusals, and an intake budget: during a retry, admission closes on everything accepted and not yet published, queued payloads and empty records included, and producers offer @@ -223,7 +223,7 @@ accepted the push requires publication recovery before repeating an operation or Status projection progresses independently after completion. Retain checkpoints, payload references, pending effects, and receipts needed by every unfinished step. Acknowledging delivery -or deleting a `SaveRequest` must not remove the only deduplication evidence while commands can +or deleting a `CommitRequest` must not remove the only deduplication evidence while commands can still be redelivered. Receipt collection needs a stated projection and redelivery boundary. The local clone remains disposable. Recovery reconstructs the recorded workflow, uses the @@ -418,7 +418,7 @@ durations rather than relying on undocumented defaults. - On startup, claim pending records, rebuild workers from remote Git, then establish watches from their durable cursors or fresh replay. - Enforce journal ownership epochs and reject stale writers at the durable transition boundary. -- Project recorded `SaveRequest` outcomes into Kubernetes status by UID. Recover the terminal +- Project recorded `CommitRequest` outcomes into Kubernetes status by UID. Recover the terminal receipt and any pending projection after failover, without deciding the save outcome again. Exit criterion: failover and supported rolling upgrades preserve the single-active contract, diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index 77991d80..6ca3f687 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -1,6 +1,6 @@ # Plan: saving from the source cluster -A `SaveRequest` saves the writes in its author's open commit window. It lives in the control +A `CommitRequest` saves the writes in its author's open commit window. It lives in the control cluster, where the operator runs, while the edits it saves happen on the source cluster. This plan adds a second way to save: a small request created **on the source cluster**, next to the edits, attributed from that cluster's audit stream like the edits are. @@ -10,10 +10,10 @@ opt-in, for a source cluster that cannot send audit events at all. There it is t put its author's name on the commit. Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The two request kinds are a pair: -`SaveRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and +`CommitRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/remote_save_requests.go`) on the source cluster. Each names its GitTarget, which admits it in a grant list: -`spec.allowedSaveRequests` (by control-cluster namespace; omitted, its own namespace) and +`spec.allowedCommitRequests` (by control-cluster namespace; omitted, its own namespace) and `spec.allowedRemoteSaveRequests` (by source namespace and labels; omitted, none). A KubeProvider also serves its cluster's remote requests only with `spec.remoteSaveRequests: Enabled`, and the GitTarget must mirror that cluster. The takeover is opt-in, as described below, bounded per grant. @@ -28,7 +28,7 @@ implementation: see [After the API review](#after-the-api-review). 1. **Build the source-side save first.** It gives the right author wherever the source cluster has audit, and an honestly unresolved one where it does not. -2. **It is a new kind, defined by GitOps Reverser.** A `SaveRequest` with another schema, or a +2. **It is a new kind, defined by GitOps Reverser.** A `CommitRequest` with another schema, or a kind gitops-api defines and translates, were both considered. See [Why a new kind](#why-a-new-kind). 3. **The takeover is the second step.** Some people cannot configure their API server's audit, so @@ -39,16 +39,16 @@ implementation: see [After the API review](#after-the-api-review). ## Why: two identity domains -Today the save and the edits are attributed in two places: the `SaveRequest`'s author comes from +Today the save and the edits are attributed in two places: the `CommitRequest`'s author comes from the control cluster's admission webhook, the edits' authors from the source cluster's audit events. They match only when one person has the same username in both. -gitops-api makes them match. Its server impersonates the human when it creates the `SaveRequest`, +gitops-api makes them match. Its server impersonates the human when it creates the `CommitRequest`, so the admission webhook records the same Dex email that kcp's audit records for the edits, and the ordinary author rule attaches. gitops-api does not need a takeover. What it pays for that is elsewhere: -- **A grant into the control cluster.** Every tenant's identity may `create saverequests` in its +- **A grant into the control cluster.** Every tenant's identity may `create commitrequests` in its `tenant-` namespace there. It is the one place a tenant reaches outside its kcp workspace. - **A server on the save path.** The browser cannot reach that namespace, so every save goes through @@ -86,21 +86,21 @@ author can read it. person who made the edits. - **On a source with no audit**, the save names nobody, like the edits. A request that names no actor may attach to a window that names none, which is today's rule - (`TestPendingSaveRequest_UnresolvedRequestAttachesToUnresolvedWindow`). The save gets its message + (`TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow`). The save gets its message and its timers, and the author in Git stays unresolved. The window still holds everyone's unattributed writes, but nobody is named as their author. ### Why a new kind -gitops-api could define its own kind and translate it into a `SaveRequest`, but it cannot learn who +gitops-api could define its own kind and translate it into a `CommitRequest`, but it cannot learn who created the object: watching the workspace shows the object and nothing about its creator. Only the audit stream says that, and GitOps Reverser is what reads it. A translation would also spend part of the attach wait on an extra hop, and add a second status to keep in step. -Reusing `SaveRequest` with a different schema would be worse than a separate kind: +Reusing `CommitRequest` with a different schema would be worse than a separate kind: - **Another shape.** It names a GitTarget on another cluster than the one it is created on. -- **Another trust model.** Its author comes from audit; a `SaveRequest`'s comes from admission. +- **Another trust model.** Its author comes from audit; a `CommitRequest`'s comes from admission. - **One name for two things** invites the confusion that `tenant-` already costs gitops-api. - **Installed alone.** A workspace gets one small CRD instead of the whole `configbutler.ai` set, so nobody is offered GitTargets that nothing reads. gitops-api installs CRDs into workspaces through its @@ -121,10 +121,10 @@ only". See [After the API review](#after-the-api-review). On a source with audit, a request is attributed only once its audit fact has joined its watch event, and the edits it saves wait in the same audit pipeline (kcp batches audit with a `batchMaxWait` of 1s). The request's wait for a window starts when the branch worker registers it, which is after the -request itself was attributed. That is how a `SaveRequest`'s `attachTimeout` is anchored today -(`handleAttachSaveRequest` in `internal/git/save_request_attach_loop.go`). The timeout then covers +request itself was attributed. That is how a `CommitRequest`'s `attachTimeout` is anchored today +(`handleAttachCommitRequest` in `internal/git/commit_request_attach_loop.go`). The timeout then covers only the spread between the request's stream and the edits' streams. Its -default can still be set for a source with audit, rather than copied from `SaveRequest`'s `2s`. +default can still be set for a source with audit, rather than copied from `CommitRequest`'s `2s`. ### An unresolved save on a source that has audit @@ -197,10 +197,10 @@ do. Only the GitTarget's owner knows which it is, so the GitTarget decides. ### What the GitTarget allows An enum, because there are two levels of trust worth offering: `unattributedWindows`, on each grant -in the GitTarget's `spec.allowedSaveRequests`. A request claims only what the grant admitting its +in the GitTarget's `spec.allowedCommitRequests`. A request claims only what the grant admitting its namespace allows; admitted by several, it may claim the most any of them allows. (The first version was one field, `spec.commit.unattributedWindows`, for requests in the GitTarget's own namespace; -it moved into the grants when a `SaveRequest` could name a GitTarget in another namespace.) +it moved into the grants when a `CommitRequest` could name a GitTarget in another namespace.) - **`"Keep"`** (the default): a save attaches only to its author's window, as today. On a cluster with no audit, a save from the control cluster finds no window of its own. @@ -217,7 +217,7 @@ it moved into the grants when a `SaveRequest` could name a GitTarget in another message ends with a trailer that says the name was taken on trust: ```text - Author-Source: SaveRequest team-a/save-1; no audit record names who made these changes + Author-Source: CommitRequest team-a/save-1; no audit record names who made these changes ``` A grant on the KubeProvider was considered and set aside: the provider's owner usually does not know @@ -259,7 +259,7 @@ The two cover different clusters: A source-side save cannot offer `"TakeOver"`: on a cluster with no audit, nothing records who created it either, so it has no name to give. The name has to come from the control cluster's admission -webhook, which is why a takeover is always made by a control-cluster `SaveRequest`. +webhook, which is why a takeover is always made by a control-cluster `CommitRequest`. ## What happens to `cf3b97be` @@ -268,7 +268,7 @@ writes name no author, authoring the commit as the submitter, and the `Author-So ways forward before the branch merges: - **Turn it into the opt-in here.** Add `spec.commit.unattributedWindows` on the GitTarget and - `spec.window.author` on the `SaveRequest`, the controller's check and refusal, `"SavedBy"`, and + `spec.window.author` on the `CommitRequest`, the controller's check and refusal, `"SavedBy"`, and tests for each row above. The source-cluster e2e's save spec then sets both. - **Take it out here**, and build the takeover in its own PR after the source-side save. @@ -276,8 +276,8 @@ ways forward before the branch merges: 1. The source-side kind's name and API group. Proposed: `RemoteSaveRequest` in `source.configbutler.ai/v1alpha1`. "Save" is the word its user means and the one the project - already uses for what a request asks; `SimpleSaveRequest` was considered and set aside, since - it is a different kind with another trust model, not a simpler `SaveRequest`. Its spec carries + already uses for what a request asks; `SimpleCommitRequest` was considered and set aside, since + it is a different kind with another trust model, not a simpler `CommitRequest`. Its spec carries no Git terms (a message, a wait, a catch-up), and its status still reports each GitTarget's commit, so a save stays traceable. Its own group makes "never mirror it" and "install it alone" one check each. @@ -285,7 +285,7 @@ ways forward before the branch merges: 3. The reason name for an unresolved save on a source that has audit (`AuthorUnresolved` above). 4. Whether the operator should hold a second save on a GitTarget until the first has resolved. 5. Names for the takeover: `spec.commit.unattributedWindows` on the GitTarget and - `spec.window.author` on the `SaveRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. + `spec.window.author` on the `CommitRequest`, each `"Keep"`/`"Own"`, `"SavedBy"`, `"TakeOver"`. 6. `cf3b97be`: turn it into the opt-in on this branch, or take it out and build the takeover later? ## After the API review @@ -319,8 +319,8 @@ also removed a cold-start failure: with no named target, discovery that had not keeps the author found when it was first served (its status writes moved its resourceVersion, so its audit fact no longer matches), and keeps an outcome already decided, following only its apply. -A `SaveRequest` gained a cross-namespace `spec.gitTargetRef`, and with it -`GitTarget.spec.allowedSaveRequests`: a grant list like the others, defaulting to one `from: Same` +A `CommitRequest` gained a cross-namespace `spec.gitTargetRef`, and with it +`GitTarget.spec.allowedCommitRequests`: a grant list like the others, defaulting to one `from: Same` grant, whose grants also carry the `unattributedWindows` bound. The GitTarget that admitted a request is recorded by UID (`status.gitTargetUID`), so a GitTarget created after the request under the same name must admit it before anything is attached. diff --git a/docs/future/task-oriented-docs-and-website.md b/docs/future/task-oriented-docs-and-website.md index ce54feb5..ca295f63 100644 --- a/docs/future/task-oriented-docs-and-website.md +++ b/docs/future/task-oriented-docs-and-website.md @@ -63,7 +63,7 @@ need them, and say when each capability is required or optional. - **Make a change through Kubernetes.** For an end user: create, update, or delete a selected resource; follow the change into Git; understand timing and refusal outcomes. - **Group changes and explain why.** For an end user: collect related edits, attach intent through - a `SaveRequest`, and verify the resulting commit or no-commit outcome. Explain author matching + a `CommitRequest`, and verify the resulting commit or no-commit outcome. Explain author matching and window timing. - **Work alongside other people on the same branch.** For collaborating teams: explain how authors and targets share a branch, what splits a commit window, and what happens when edits overlap. @@ -205,7 +205,7 @@ provider-specific claim when writing it. Link this guide from normal changes and grouped changes. Keep "who made the change" and "why the change was needed" as distinct tasks. Explain the separate admission-based submitter capture used -by `SaveRequest` where it affects matching, so readers do not assume audit setup controls both. +by `CommitRequest` where it affects matching, so readers do not assume audit setup controls both. ## Guide structure diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index a3a33796..ef3c0842 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -117,9 +117,9 @@ red is a panel nobody reads. Object names are labels here because these objects are configuration, bounded by how many someone wrote, as are the `GitTarget`, `GitBranch`, applier claim and `GitReceiver` names other families carry. The resources being mirrored are data, unbounded, and never a label -([metrics-observability-plan.md](design/metrics-observability-plan.md) §7.1). `SaveRequest` is +([metrics-observability-plan.md](design/metrics-observability-plan.md) §7.1). `CommitRequest` is excluded on the same reasoning: one is created per save, so it is data-shaped, and its state is -read from `save_requests_total` in aggregate and from the object's own conditions individually. +read from `commit_requests_total` in aggregate and from the object's own conditions individually. **Which configuration objects are not Ready?** The panel to start from: @@ -206,7 +206,7 @@ boundary, the commit, the push. Background: | `git_queue_refusals_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `kind` | Work refused at enqueue: the queue was full, intake was paused through an outage, or the worker was stopping. `kind` is `write` / `attach` / `resync` / `refresh`. The producer keeps a refused item and offers it again: the watch delivers a write again from its cursor, the controller re-sends a save, a resync is gathered again, and the next reconcile asks for a refresh. What it can cost, when its watch cursor expires before the write comes back, is the write's own commit and, under `prune.mode: OnEvent`, a deleted object's removal from Git. | | `git_commit_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | A live commit window that died between routing and pushing; a resync's failure is `git_resync_failures_total`. `reason` is `refused` (a Git path a human must fix) / `error`. Every increment is a window's events lost until the next resync. An unreachable remote does not count: a decided window waits for the publication retry instead. | | `git_writes_dropped_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | A live event the branch worker dropped because its `GitTarget`'s render fidelity was not established. `reason` is `render_fidelity_diverged`: a rendered value does not match the live object, and every write is dropped until a fresh replay measures the folder clean. Or it is `render_fidelity_pending`: a collection of the `GitTarget` has not finished its replay, which closes every collection of that target until it does. A live event that meets a pending gate is held, not dropped, and committed with its author once the gate opens; it is counted here only when the hold gives up, 10 minutes after the first one was held or once the branch holds more than its retained-byte budget, and the worker then asks for a fresh snapshot, which commits the current state as the operator (a deletion comes back only under prune mode `Always`). Nothing replays a dropped write, so each one is an edit missing from Git until the object changes again or its collection replays. The worker logs the first drop under each gate status at Info, naming the collections the gate waits for, and each hold, release and give-up at Info. | -| `git_commit_windows_total` | counter | `gittarget_namespace`, `gittarget_name`, `close_reason`, `timer_source` | One per closed commit window, whether or not it then produced a commit. `close_reason` is `idle_timeout` / `max_duration` / `attach_next` / `identity_change` / `buffer_limit` / `resync_before_apply` / `foreign_commit` (a Git host said the branch may have moved, so the window is published now) / `apply_due` (an apply came due for an applier on the branch, so the window's edits are published before the apply is sent) / `shutdown`; `timer_source` is `target` or `save_request`. See [tuning commit windows](#tuning-commit-windows). | +| `git_commit_windows_total` | counter | `gittarget_namespace`, `gittarget_name`, `close_reason`, `timer_source` | One per closed commit window, whether or not it then produced a commit. `close_reason` is `idle_timeout` / `max_duration` / `attach_next` / `identity_change` / `buffer_limit` / `resync_before_apply` / `foreign_commit` (a Git host said the branch may have moved, so the window is published now) / `apply_due` (an apply came due for an applier on the branch, so the window's edits are published before the apply is sent) / `shutdown`; `timer_source` is `target` or `commit_request`. See [tuning commit windows](#tuning-commit-windows). | | `git_commit_window_duration_seconds` | histogram | `gittarget_namespace`, `gittarget_name` | How long each window collected: from the write that opened it to the start of its finalize. Its count equals `git_commit_windows_total` summed over `close_reason` and `timer_source`. | | `git_queue_depth` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | Pending + in-flight + committed-but-unpushed. Read at scrape time. | | `git_retained_bytes` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | What the worker holds until a push lands, in the units of the retained-byte budget: the open window, the retained writes and waiting saves, each with a 1 KiB per-item charge. Serialized bytes, not heap. | @@ -216,7 +216,7 @@ boundary, the commit, the push. Background: | `git_next_retry_timestamp_seconds` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | When the next attempt at the pending writes is due, as a Unix timestamp. No sample while no retry is scheduled. | | `git_materialization_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | Attempts to commit decided writes that stopped before any push: `reason` is `unreachable` or `parent_unavailable`. Nothing is lost; the writes wait for the retry. A failure `git_pushes_total` cannot see, because no push cycle started. | | `git_branch_targets` | gauge | `gitbranch_namespace`, `gitbranch_name`, `gittarget_namespace`, `gittarget_name`, `source_cluster` | Always 1. The **join** between the GitTarget-labeled half of the pipeline and the GitBranch-labeled half: the `GitTarget`'s `spec.gitBranchRef`. Published per configured GitTarget, whether or not its GitBranch exists or its worker runs. | -| `save_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `SaveRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `save_in_progress` (refused: an earlier save of the same author on the target was in progress) / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | +| `commit_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `CommitRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `save_in_progress` (refused: an earlier save of the same author on the target was in progress) / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | | `placements_total` | counter | `source`, `disposition`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new document at a resolved path. | | `placement_refusals_total` | counter | `reason`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new resource the writer declined. Every increment is a resource **absent** from the mirror. | | `placement_kustomization_entries_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | `added` / `no_change` / `failed`. | @@ -533,9 +533,9 @@ their writes into one commit batch, so the worker is the honest attribution unit `user`, `serviceaccount`, `committer`, or `unresolved`; reconcile/resync commits and configured-author mode use `committer`. -`message_source` is `live`, `reconcile`, or one of three request values: `save_request` (a save -request's message committed verbatim, no `requestTemplate` configured), `save_request_framed` -(`requestTemplate` rendered), and `save_request_fallback` (`requestTemplate` failed to render, so +`message_source` is `live`, `reconcile`, or one of three request values: `commit_request` (a save +request's message committed verbatim, no `requestTemplate` configured), `commit_request_framed` +(`requestTemplate` rendered), and `commit_request_fallback` (`requestTemplate` failed to render, so the message was committed verbatim instead). The fallback is the one to alert on. It is a **successful** commit (nothing is refused and no @@ -544,7 +544,7 @@ looks exactly like one that was never configured: ```promql sum by (gitbranch_namespace, gitbranch_name) ( - rate(gitopsreverser_git_commits_total{message_source="save_request_fallback"}[15m]) + rate(gitopsreverser_git_commits_total{message_source="commit_request_fallback"}[15m]) ) > 0 ``` @@ -562,15 +562,15 @@ Prometheus pod scrape with `honor_labels=false` renames a bare `namespace` label selector would silently match nothing. The same reasoning applies to `watch_recovery_total` and `git_queue_depth`. -`message_source` says where each commit's message came from: `save_request` is text a -`SaveRequest` supplied and that was used verbatim, `live` is a live window rendered through the +`message_source` says where each commit's message came from: `commit_request` is text a +`CommitRequest` supplied and that was used verbatim, `live` is a live window rendered through the target's `liveTemplate`, and `reconcile` is a snapshot or resync rendered through `reconcileTemplate`. It is read from the same decision that renders the message, so it cannot report a source the renderer did not use. -`save_request` counts commits that USED a request-supplied message, **not** SaveRequests. A +`commit_request` counts commits that USED a request-supplied message, **not** CommitRequests. A request that omits `spec.message` takes the target's `liveTemplate`, so it counts as `live`, and a -low `save_request` share does not by itself mean requests are failing. +low `commit_request` share does not by itself mean requests are failing. **How much of the history is people naming their own changes?** @@ -578,7 +578,7 @@ low `save_request` share does not by itself mean requests are failing. sum by (message_source) (rate(gitopsreverser_git_commits_total[15m])) ``` -A `save_request` share that falls to zero after a rollout **may** mean save requests stopped +A `commit_request` share that falls to zero after a rollout **may** mean save requests stopped reaching an open window, but omitted messages, saves that were no-ops, and pushes that never landed all read the same way here, because none of them produce a commit carrying a request message. Confirm against the requests themselves before changing the window: a request that @@ -586,10 +586,10 @@ attached reports `Ready=True`, one that produced a pushed commit also reports `P `status.commit`, and one that gave up reports `Stalled=True`. Only if those show requests resolving without their message reaching a commit is the request's `window` worth tuning. -`save_requests_total` answers the same question in aggregate, which per-object conditions cannot: +`commit_requests_total` answers the same question in aggregate, which per-object conditions cannot: ```promql -sum by (outcome) (rate(gitopsreverser_save_requests_total[15m])) +sum by (outcome) (rate(gitopsreverser_commit_requests_total[15m])) ``` **Which target's saves are failing?** The counter carries the GitTarget, so the fleet view above is @@ -597,7 +597,7 @@ an aggregation rather than the only view available: ```promql sum by (gittarget_namespace, gittarget_name, outcome) ( - rate(gitopsreverser_save_requests_total[15m]) + rate(gitopsreverser_commit_requests_total[15m]) ) ``` @@ -618,13 +618,13 @@ seen during its wait, so a foreign window that closes before `attachTimeout` run `no_window` as "users are being refused" over-reports; reading only `committed` under-reports. **What one increment means.** One terminal DECISION, recorded outside the status-write retry loop. -That is not one per `SaveRequest`: a terminal status that never persisted is re-decided when the +That is not one per `CommitRequest`: a terminal status that never persisted is re-decided when the request is redelivered, and a restart re-reconciles anything non-terminal. Read rates and ratios from it rather than exact request counts. ### Tuning commit windows -A GitTarget's `commit.window` timers decide when its changes become a commit, and a `SaveRequest` +A GitTarget's `commit.window` timers decide when its changes become a commit, and a `CommitRequest` can bring its own. Two metrics show what those timers actually do. **Which timer ends the windows?** Every closed window is counted once, with what closed it and whose @@ -657,7 +657,7 @@ picture toward `max_duration`. stopping. - `apply_due` closes the window because an apply came due for an applier on the branch: its edits are published at once, ahead of the push cooldown, so the apply includes them. A window a - `SaveRequest` holds is never closed this way; the apply waits for the save. Its share follows the + `CommitRequest` holds is never closed this way; the apply waits for the save. Its share follows the apply traffic (Git-side pushes, API creates, repairs), and no timer changes it. `foreign_commit` is the same early close for a push a Git host reported. @@ -1421,7 +1421,7 @@ sum by (outcome) (rate(gitopsreverser_secret_encryptions_total[5m])) | `max by (gitbranch_namespace, gitbranch_name) (gitopsreverser_apply_appliers_held) > 0`, `for: 15m` | An applier is held behind the initial-snapshot barrier: a `GitTarget` naming its claim watches nothing, or its folder is missing. | | `sum by (applier_kind, claim_namespace, claim_name, reason) (rate(gitopsreverser_apply_requests_total{outcome="failed"}[15m])) > 0`, `for: 30m` | Apply requests keep failing for the named applier claim; the worker retries at its backoff, and the cluster runs older content than the branch meanwhile. | | `sum by (receiver_namespace, receiver_name) (increase(gitopsreverser_receiver_deliveries_total{outcome="rejected"}[15m])) > 0` | A Git host is calling a `GitReceiver` with the wrong secret, so foreign pushes are found only by the periodic refresh. | -| `sum by (gitbranch_namespace, gitbranch_name) (rate(gitopsreverser_git_commits_total{message_source="save_request_fallback"}[15m])) > 0` | A `requestTemplate` stopped rendering: saves still commit, with the message verbatim. | +| `sum by (gitbranch_namespace, gitbranch_name) (rate(gitopsreverser_git_commits_total{message_source="commit_request_fallback"}[15m])) > 0` | A `requestTemplate` stopped rendering: saves still commit, with the message verbatim. | | `gitopsreverser_resource_condition{type="Ready", status="False"} == 1`, `for: 15m` | A declared object has not been accepted. The `== 1` is not optional: each condition publishes one series per status, so the selector alone matches the two zeroes too. | | `gitopsreverser_resource_condition{type="Stalled", status="True"} == 1` | Permanently wedged: `Stalled` is the kstatus "nothing will retry" signal, so this needs a human and will not clear on its own. `reason` names the gate. | | any `gitopsreverser_*` series with `otel_metric_overflow="true"` | A metric family exceeded its per-instrument cardinality cap, so the labels identifying those series are gone and every query over that family is under-reporting. Raise the cap or cut a label. | @@ -1442,7 +1442,7 @@ Listed so a missing panel is never read as a healthy zero. The plan for them is - **A relevance-filter breakdown beyond the ingest census.** `watch_events_total{outcome}` names where an event stopped; it does not attribute an `unchanged` to sanitization versus a genuine no-op diff. -- **SaveRequest timing.** `save_requests_total` counts how saves end; how long a save waited +- **CommitRequest timing.** `commit_requests_total` counts how saves end; how long a save waited to attach (including the waits that ran out), how long it took end to end, and whether a `no_window` save was recorded in an empty commit are not emitted yet. - **The reference dashboard and alert rules.** The queries in this document are the specification diff --git a/docs/proud.md b/docs/proud.md index 2bc8b73a..c49fc1be 100644 --- a/docs/proud.md +++ b/docs/proud.md @@ -61,7 +61,7 @@ per process, so you can see exactly what ran concurrently and where the bottlene cluster is **reused** between runs via `.stamps`, not torn down. And when things go wrong: - **Ctrl-C, or a setup failure, preserves the entire live state** so you can poke around. -- A **failed spec auto-dumps diagnostics** — events, `SaveRequest`s, controller logs, and a pod +- A **failed spec auto-dumps diagnostics** — events, `CommitRequest`s, controller logs, and a pod describe — into the log and the Allure attachment. - The spec's **Git checkout survives** under `.stamps/repos/`, with a working remote and credentials, so you can `git log`, `git pull`, or `git push` the reversed history of a past run. diff --git a/docs/rbac.md b/docs/rbac.md index 4f2a4709..2daae367 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -99,7 +99,7 @@ the same rules for the cluster the operator runs in, for an `inCluster` provider Whoever may create a `RemoteSaveRequest` in a namespace may save their own commit window on any `GitTarget` whose `spec.allowedRemoteSaveRequests` admits it. That grant, not the provider's switch, is the `GitTarget` owner's consent. On the cluster GitOps Reverser runs in, whoever may create a -`SaveRequest` in a namespace may save on every `GitTarget` whose `spec.allowedSaveRequests` admits +`CommitRequest` in a namespace may save on every `GitTarget` whose `spec.allowedCommitRequests` admits that namespace. ## The operator does not read Secrets wholesale diff --git a/docs/review/20261025-crd-text-proposal.md b/docs/review/20261025-crd-text-proposal.md index 67c93cdb..e1854687 100644 --- a/docs/review/20261025-crd-text-proposal.md +++ b/docs/review/20261025-crd-text-proposal.md @@ -108,7 +108,7 @@ the links track their default branches. ### 1. Explain the message override accurately -Location: [CommitRequest.spec.message](../../api/v1alpha3/saverequest_types.go). +Location: [CommitRequest.spec.message](../../api/v1alpha3/commitrequest_types.go). ```text Message supplies the requested commit message. Omitted, the GitTarget's diff --git a/docs/review/20261027-save-request-simplification.md b/docs/review/20261027-save-request-simplification.md index c959bd57..c0f5a8b7 100644 --- a/docs/review/20261027-save-request-simplification.md +++ b/docs/review/20261027-save-request-simplification.md @@ -864,7 +864,7 @@ Hard cuts, no migration. - Scope each provider's watch to the namespaces it lends, with `status.commitRequests.namespaces` and the `CommitRequestsServed` condition. - Gate the chart `Issuer`, and move the quickstart's cert-manager step under attribution. - - Rewrite [saverequest-design.md](../spec/saverequest-design.md) to describe the one kind, and + - Rewrite [commitrequest-design.md](../spec/commitrequest-design.md) to describe the one kind, and add a sentence to [definitions.md](../definitions.md): CommitRequest is the one operator kind that can also live on a source cluster. - Publish the `commitrequest-crd.yaml` release asset, with the schema revision line and its hash diff --git a/docs/spec/README.md b/docs/spec/README.md index 6e68086c..b1fc6b72 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -46,8 +46,8 @@ three lifecycle classes and was unreadable spread across them. The rows below st | [`sops-single-file-no-multidoc.md`](sops-single-file-no-multidoc.md) | one encrypted file is one document | | [`scale-subresource-audit-rehydration.md`](scale-subresource-audit-rehydration.md) | history: the `/scale` field-patch design; a scale is now mirrored from the watch, and the field-patch writer is what remains | | [`commit-window-refactor.md`](commit-window-refactor.md) | one grouped commit = one (author, GitTarget) | -| [`saverequest-design.md`](saverequest-design.md) | how a request binds to a same-actor commit window and reports its outcome | -| [`saverequest-admission-authorship.md`](saverequest-admission-authorship.md) | a request's submitter comes from admission, present-or-never, apart from audit facts | +| [`commitrequest-design.md`](commitrequest-design.md) | how a request binds to a same-actor commit window and reports its outcome | +| [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md) | a request's submitter comes from admission, present-or-never, apart from audit facts | | [`status-conditions-guide.md`](status-conditions-guide.md) | the conditions and reasons every kind reports | | [`gitpath-foreign-content-stringency.md`](gitpath-foreign-content-stringency.md) | a `GitTarget` owns its subtree and refuses foreign content; `.gittargetignore` | | [`unsupported-folder-refusal-plan.md`](unsupported-folder-refusal-plan.md) | history of the `GitPathAccepted` refusal of unsupported folder content | diff --git a/docs/spec/attribution.md b/docs/spec/attribution.md index 6aab537f..9b2b27a4 100644 --- a/docs/spec/attribution.md +++ b/docs/spec/attribution.md @@ -18,7 +18,7 @@ > [what attribution gives up, on purpose](#what-attribution-gives-up-on-purpose). > > Related: [`watch-event-ordering-and-attribution-grace.md`](watch-event-ordering-and-attribution-grace.md), -> [`saverequest-admission-authorship.md`](saverequest-admission-authorship.md), +> [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md), > [`../attribution-setup-guide.md`](../attribution-setup-guide.md), > [`../interpreting-metrics.md`](../interpreting-metrics.md). @@ -40,7 +40,7 @@ When no usable fact is found the commit ships with the explicit unresolved autho A commit names a person only when the Kubernetes API server **authenticated** that person for the change: the user in the audit record of a mirrored edit, or the submitter in the admission request -of a `SaveRequest`. Every other commit carries one of two identities the operator decides, never +of a `CommitRequest`. Every other commit carries one of two identities the operator decides, never one a submitter supplies: - **Attribution is off** (the chart default): the configured Git identity. @@ -49,13 +49,13 @@ one a submitter supplies: No field in any spec can name an author, and none ever will. -That is why `SaveRequest` has no `spec.author`, although one was asked for (consumer ask #5, with a +That is why `CommitRequest` has no `spec.author`, although one was asked for (consumer ask #5, with a `SubjectAccessReview` guard): - **A name in a spec is only a claim.** Anyone who can create the object can write any name. A permission check narrows who may write it, but what it records is still only what the submitter said. -- **The claim outlives the check.** A `SaveRequest` applied from Git or copied between clusters +- **The claim outlives the check.** A `CommitRequest` applied from Git or copied between clusters carries its name to wherever it lands, while the submitter who was checked is someone else. - **It would make the rule above a matter of configuration.** "A wrong author is worse than no author" only holds if nothing can put a name into the history that the API server did not @@ -64,7 +64,7 @@ That is why `SaveRequest` has no `spec.author`, although one was asked for (cons To commit under your own name, authenticate as yourself. Use your identity provider (OIDC, for example) so that the API server knows who you are. A request that cannot be attributed claims no actor (`AuthorAttributed=False`); it never falls back to a name it was given. See -[`saverequest-admission-authorship.md`](saverequest-admission-authorship.md). +[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). ## 1. Deletion is attributed at intent time diff --git a/docs/spec/commit-window-refactor.md b/docs/spec/commit-window-refactor.md index fc15a8f8..a7bf0ae9 100644 --- a/docs/spec/commit-window-refactor.md +++ b/docs/spec/commit-window-refactor.md @@ -94,7 +94,7 @@ flowchart TD cooldowns, and retry scheduling. It retains `[]PendingWrite` until a push succeeds. Local commit creation does not clear pending work. A write whose first commit changes nothing in Git is the exception: it settles at once and is neither -retained nor pushed. A save (a write carrying a `SaveRequest`) is always +retained nor pushed. A save (a write carrying a `CommitRequest`) is always retained, and the push decides its outcome. Per-event writes are processed as a stream. The branch worker keeps one open @@ -106,7 +106,7 @@ closes the window so its edits are published before the apply request is sent ([the flush](../design/apply-trigger.md#flush-then-apply), `close_reason="apply_due"`). A push notice closes it so its writes are published, and replayed onto the move, at once. Neither applies while a -`SaveRequest` holds the window: no flush is made while a request is registered, +`CommitRequest` holds the window: no flush is made while a request is registered, and a push notice leaves an attached window to its save. A timer of `0s` closes the window in the same step that collected the write. Repeated writes to the same Git path inside the open window are last-write-wins while preserving first-seen @@ -134,7 +134,7 @@ between: - `PendingWriteAtomic` represents one caller-defined atomic write. - `PendingWriteResync` represents one GitTarget's snapshot, applied by mark-and-sweep. -- `PendingWriteRequestRecord` records a `SaveRequest`'s message in an empty +- `PendingWriteRequestRecord` records a `CommitRequest`'s message in an empty commit when no window reached the request and it asked for `whenNothingToCommit: CommitEmpty`. @@ -174,9 +174,9 @@ planner-time regrouping pass. Message precedence is a non-empty literal override, then `commit.message.liveTemplate` for live windows or `commit.message.reconcileTemplate` for atomic snapshots and resyncs. Every live window -uses the same context, including singleton and `0s` windows. A `SaveRequest`'s message is the +uses the same context, including singleton and `0s` windows. A `CommitRequest`'s message is the literal override, and `commit.message.requestTemplate` can frame it (see -[`saverequest-design.md`](saverequest-design.md)). Literal text never changes authorship. +[`commitrequest-design.md`](commitrequest-design.md)). Literal text never changes authorship. Resolved templates and literal overrides travel with pending writes through retries and conflict replay, so editing a target does not change an already finalized write's message configuration. @@ -229,7 +229,7 @@ OPEN WINDOW rather than once per worker, and snapshotted when the window opens: editing a target never moves a deadline already running. That is affordable because a window is already bound to exactly one `GitTarget` by construction: it finalizes the moment the target changes. An unreadable target, or one declaring -no window, takes the `5s` and `1m` defaults. A `SaveRequest` that attaches +no window, takes the `5s` and `1m` defaults. A `CommitRequest` that attaches replaces both timers for that window. The byte cap is an operator startup setting, `--branch-buffer-max-size` diff --git a/docs/spec/saverequest-admission-authorship.md b/docs/spec/commitrequest-admission-authorship.md similarity index 87% rename from docs/spec/saverequest-admission-authorship.md rename to docs/spec/commitrequest-admission-authorship.md index 1341da2a..9818f09f 100644 --- a/docs/spec/saverequest-admission-authorship.md +++ b/docs/spec/commitrequest-admission-authorship.md @@ -1,25 +1,25 @@ -# SaveRequest authorship from admission +# CommitRequest authorship from admission > **spec** — current behaviour. The code depends on this document; change one, change the other. > Index: [`../INDEX.md`](../INDEX.md) -A `SaveRequest` is a command to close an existing GitTarget commit window. Its submitter is captured +A `CommitRequest` is a command to close an existing GitTarget commit window. Its submitter is captured by the `/validate-operator-types` validating admission webhook, while mirrored-resource attribution is separately derived from kube-apiserver audit facts. The two paths answer different questions and must not be conflated. ## Contract -| Concern | SaveRequest submitter | Mirrored-resource author | +| Concern | CommitRequest submitter | Mirrored-resource author | |---|---|---| | Source | validating admission request `userInfo` | post-persist audit fact joined to watch event | -| Key | SaveRequest UID | source provider, GVR, UID, and resourceVersion | +| Key | CommitRequest UID | source provider, GVR, UID, and resourceVersion | | Timing | written before the object persists | waits up to `--author-attribution-grace` after the watch event | | Miss | request claims no actor | watch commit has the explicit unresolved author | The admission handler records the authenticated submitter in the Redis command-author store and always allows the request. It is deliberately best-effort: a Redis failure or an unavailable webhook must not -reject a user's command. The controller reads the record when the persisted `SaveRequest` is first +reject a user's command. The controller reads the record when the persisted `CommitRequest` is first reconciled. The result is **present-or-never**; waiting cannot create a record that admission did not write. ## Attach and final Git author @@ -63,6 +63,6 @@ An installation may therefore have either, both, or neither source of author inf Related live contracts: -- [architecture: SaveRequest Finalize](../architecture.md#saverequest-finalize) -- [configuration: SaveRequest conditions](../configuration.md#when-there-is-nothing-to-commit) +- [architecture: CommitRequest Finalize](../architecture.md#commitrequest-finalize) +- [configuration: CommitRequest conditions](../configuration.md#when-there-is-nothing-to-commit) - [status conditions guide](status-conditions-guide.md) diff --git a/docs/spec/saverequest-design.md b/docs/spec/commitrequest-design.md similarity index 96% rename from docs/spec/saverequest-design.md rename to docs/spec/commitrequest-design.md index e073d648..46896136 100644 --- a/docs/spec/saverequest-design.md +++ b/docs/spec/commitrequest-design.md @@ -1,11 +1,11 @@ -# SaveRequest window finalization +# CommitRequest window finalization > **spec**: current behavior, reviewed at `c12f7970` on 2026-10-20. > The code depends on this document; change one, change the other. > Index: [`../INDEX.md`](../INDEX.md) -A `SaveRequest` is a one-shot “save now” command for one `GitTarget`. It does not mirror the -`SaveRequest` object. Instead, it can attach a message to a matching open commit window and asks the +A `CommitRequest` is a one-shot “save now” command for one `GitTarget`. It does not mirror the +`CommitRequest` object. Instead, it can attach a message to a matching open commit window and asks the worker to close that window after the requested collect delay. ## Request and window contract @@ -75,7 +75,7 @@ pushed commit is required; `Ready=True` also includes successful no-commit outco ## Authorship -The validating admission webhook captures a command submitter before the `SaveRequest` persists. The +The validating admission webhook captures a command submitter before the `CommitRequest` persists. The controller performs one best-effort, present-or-never lookup: | Admission result | `AuthorAttributed` | Request claim | @@ -91,14 +91,14 @@ that: - a resolved live attribution fact uses the authenticated actor; and - a live attribution miss uses `unknown (attribution unresolved) `. -See [SaveRequest admission authorship](saverequest-admission-authorship.md) for capture provenance and +See [CommitRequest admission authorship](commitrequest-admission-authorship.md) for capture provenance and [architecture: author identity](../architecture.md#author-and-committer-identity-in-git) for the three Git author states. ## Lifecycle and outcomes The controller stamps its conditions on first reconcile, immediately attaches the request, and polls the -worker. There is no audit wait for a `SaveRequest`; a delayed admission record cannot arrive after the +worker. There is no audit wait for a `CommitRequest`; a delayed admission record cannot arrive after the object is visible. | Outcome | Conditions | diff --git a/docs/spec/current-manifest-support-review.md b/docs/spec/current-manifest-support-review.md index eab430b2..f5d8b4d7 100644 --- a/docs/spec/current-manifest-support-review.md +++ b/docs/spec/current-manifest-support-review.md @@ -1147,7 +1147,7 @@ core logic. > > Running it against `config/samples` already surfaced a real constraint: > `manifestedit` derives manifest identity from a concrete `metadata.name`, so a -> `generateName`-only object (`saverequest.yaml`) is classified non-KRM. That +> `generateName`-only object (`commitrequest.yaml`) is classified non-KRM. That > is correct given today's identity rules and is exactly the kind of gap the > analyzer is meant to make visible. diff --git a/docs/spec/e2e-serial-registry.md b/docs/spec/e2e-serial-registry.md index 76a697f4..c10f5bf6 100644 --- a/docs/spec/e2e-serial-registry.md +++ b/docs/spec/e2e-serial-registry.md @@ -71,7 +71,7 @@ commit it asserts on. The two `audit-consumer`-labelled containers (`Commit Window Batching` ([commit_window_batching_e2e_test.go](../../test/e2e/commit_window_batching_e2e_test.go)), -`Save Request` ([save_request_e2e_test.go](../../test/e2e/save_request_e2e_test.go))) +`Save Request` ([commit_request_e2e_test.go](../../test/e2e/commit_request_e2e_test.go))) were also de-serialized. Their real serial cause was **not** the shared audit *pipeline* — namespace-scoped WatchRules already route each spec's events only to its own GitTarget — but a shared *repo*: both borrowed one Gitea repo from a diff --git a/docs/spec/e2e-test-design.md b/docs/spec/e2e-test-design.md index 6479c1a4..2e9b4aab 100644 --- a/docs/spec/e2e-test-design.md +++ b/docs/spec/e2e-test-design.md @@ -213,7 +213,7 @@ These namespaces are typically named: - `-test-manager-watchrule` - `-test-signing` -- `-test-save-request` +- `-test-commit-request` - and so on Those namespaces are created in `BeforeAll` and removed in `AfterAll` for the suite. @@ -263,7 +263,7 @@ Each repo-using e2e file owns its own repo, for example: | `watchrule_configmap_secret_e2e_test.go` (one of the `manager` files) | `testNamespaceFor("manager-watchrule")` | `e2e-manager-watchrule-` | | `signing_e2e_test.go` | `testNamespaceFor("signing")` | `e2e-signing-` | | `commit_window_batching_e2e_test.go` | `testNamespaceFor("commit-window-batching")` | `e2e-commit-window-` | -| `save_request_e2e_test.go` | `testNamespaceFor("save-request")`, and `save-request-bundle` for the bundle describe | `e2e-save-request-`, `e2e-save-request-bundle-` | +| `commit_request_e2e_test.go` | `testNamespaceFor("commit-request")`, and `commit-request-bundle` for the bundle describe | `e2e-commit-request-`, `e2e-commit-request-bundle-` | | `commit_author_attribution_e2e_test.go` | `testNamespaceFor("commit-author")` | `e2e-commit-author-` | | `quickstart_framework_e2e_test.go` | `testNamespaceFor("quickstart-framework")` | `e2e-quickstart-framework-` | | `flux_bi_directional_e2e_test.go` | `testNamespaceFor("bi-directional")` | `e2e-bi-directional-` | @@ -384,7 +384,7 @@ The e2e package includes, among others: | `controller_basics_e2e_test.go` and the other `manager` files | `manager` | Main reconciliation and GitOps behavior | | `signing_e2e_test.go` | `signing` | Commit signing behavior | | `commit_window_batching_e2e_test.go` | `commit-window-batching`, `audit-consumer` | Commit-window batching through the audit pipeline | -| `save_request_e2e_test.go` | `save-request`, `audit-consumer` | SaveRequest finalization through the audit pipeline | +| `commit_request_e2e_test.go` | `commit-request`, `audit-consumer` | CommitRequest finalization through the audit pipeline | | `commit_author_attribution_e2e_test.go` | `manager` | OIDC claim to Git author attribution | | `image_refresh_test.go` | `image-refresh` | Build/load/restart invalidation chain | | `quickstart_framework_e2e_test.go` | `quickstart-framework` | Installer-level quickstart flow | diff --git a/docs/spec/status-conditions-guide.md b/docs/spec/status-conditions-guide.md index 5d9d44f8..938df9c5 100644 --- a/docs/spec/status-conditions-guide.md +++ b/docs/spec/status-conditions-guide.md @@ -134,7 +134,7 @@ never reconciles one key twice at once), and nothing else re-enqueues the object `For()` of every controller using `reconcileStatus` filters status-only updates (`GenerationChangedPredicate`, or a variant of it that also admits a reconcile request). So the loss is recorded, and the reconcile comes back after `RequeueWriteLostInterval` (100ms) instead of on its -normal cadence, and recomputes from fresh data. `SaveRequest` does not use this helper: its +normal cadence, and recomputes from fresh data. `CommitRequest` does not use this helper: its `For()` has no such filter, and it retries its terminal status write in place. ### Reason vocabulary @@ -326,9 +326,9 @@ A `GitTarget` naming an applier claim gains `ApplierClaimReady`, a gate on its o claim (`ApplierClaimNotFound`) or one that is not `Ready` yet keeps it `Reconciling`. -### SaveRequest (one-shot) +### CommitRequest (one-shot) -SaveRequest runs once to a terminal outcome. Best-practice 1 above would suggest a `Succeeded` summary, +CommitRequest runs once to a terminal outcome. Best-practice 1 above would suggest a `Succeeded` summary, but it deliberately keeps `Ready` so every CRD in this project shares one summary type and the kstatus trio; the bounded "did the work actually happen" signal lives on the `Pushed` domain condition instead. diff --git a/docs/spec/where-validation-lives.md b/docs/spec/where-validation-lives.md index ee64ad1a..c003c44a 100644 --- a/docs/spec/where-validation-lives.md +++ b/docs/spec/where-validation-lives.md @@ -52,10 +52,10 @@ being refused at admission. [`/validate-operator-types`](../../internal/webhook/validate_operator_types_handler.go#L27) is **not a validation gate**. It captures the *submitter's identity* on a command kind -(`SaveRequest`) so a commit can be authored by a real Kubernetes user, and for a command it +(`CommitRequest`) so a commit can be authored by a real Kubernetes user, and for a command it **always allows**. Identity is the textbook case for admission: it exists in the `AdmissionRequest` and on no persisted field, so no reconciler could recover it later. See -[`saverequest-admission-authorship.md`](saverequest-admission-authorship.md). +[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). The same endpoint carries one rejection, and it is feedback, not a gate: a `WatchRule` that would bring a second source namespace to a `GitTarget` declaring `spec.serializeNamespace: false` is diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index e423b579..fdc970ed 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -158,11 +158,11 @@ kinds: - path: commit.message default: the built-in templates doc: How commits are phrased. See [commit messages](commit-messages.md) - - path: allowedSaveRequests + - path: allowedCommitRequests doc: >- - The namespaces whose `SaveRequest`s may save this target's windows, and how much each may + The namespaces whose `CommitRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See - [who may save](#who-may-save-specallowedsaverequests) + [who may save](#who-may-save-specallowedcommitrequests) - path: allowedRemoteSaveRequests default: none admitted doc: >- @@ -301,13 +301,13 @@ kinds: the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` - - kind: SaveRequest - title: "`SaveRequest` (namespaced): save now" + - kind: CommitRequest + title: "`CommitRequest` (namespaced): save now" fields: - path: gitTargetRef doc: >- The target whose commit window the request attaches to: in this namespace, or `namespace` - names another, whose `allowedSaveRequests` must admit this one + names another, whose `allowedCommitRequests` must admit this one - path: message default: the target's templates doc: Commit message, committed verbatim unless `requestTemplate` frames it diff --git a/internal/auditutil/identity_test.go b/internal/auditutil/identity_test.go index 9d0d937f..c0362eda 100644 --- a/internal/auditutil/identity_test.go +++ b/internal/auditutil/identity_test.go @@ -15,11 +15,11 @@ import ( itypes "github.com/ConfigButler/gitops-reverser/internal/types" ) -func rawSaveRequestBody(t *testing.T, namespace, name, uid string) *runtime.Unknown { +func rawCommitRequestBody(t *testing.T, namespace, name, uid string) *runtime.Unknown { t.Helper() obj := &unstructured.Unstructured{} obj.SetAPIVersion("configbutler.ai/v1alpha3") - obj.SetKind("SaveRequest") + obj.SetKind("CommitRequest") if namespace != "" { obj.SetNamespace(namespace) } @@ -38,13 +38,13 @@ func TestIdentityFromAuditEvent_ObjectRefWins(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", Name: "save-1", UID: "uid-objref", }, } - ev.ResponseObject = rawSaveRequestBody(t, "team-a", "save-from-body", "uid-from-body") + ev.ResponseObject = rawCommitRequestBody(t, "team-a", "save-from-body", "uid-from-body") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -59,12 +59,12 @@ func TestIdentityFromAuditEvent_BackfillsNameFromResponseObject(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", Name: "", }, } - ev.ResponseObject = rawSaveRequestBody(t, "team-a", "save-generated-abcde", "uid-resp") + ev.ResponseObject = rawCommitRequestBody(t, "team-a", "save-generated-abcde", "uid-resp") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -79,10 +79,10 @@ func TestIdentityFromAuditEvent_BackfillsNamespaceFromBody(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", }, } - ev.ResponseObject = rawSaveRequestBody(t, "team-from-body", "save-x", "uid-x") + ev.ResponseObject = rawCommitRequestBody(t, "team-from-body", "save-x", "uid-x") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -96,13 +96,13 @@ func TestIdentityFromAuditEvent_FallsBackToRequestObject_NonDelete(t *testing.T) ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", }, } // Only RequestObject is populated; for non-delete the helper prefers // ResponseObject but must fall back to RequestObject when it is absent. - ev.RequestObject = rawSaveRequestBody(t, "team-a", "save-from-request", "uid-req") + ev.RequestObject = rawCommitRequestBody(t, "team-a", "save-from-request", "uid-req") got := IdentityFromAuditEvent(ev, itypes.OperationCreate) @@ -116,12 +116,12 @@ func TestIdentityFromAuditEvent_DeletePrefersRequestObject(t *testing.T) { ev := auditv1.Event{ Verb: "delete", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", }, } - ev.RequestObject = rawSaveRequestBody(t, "team-a", "from-request", "uid-req") - ev.ResponseObject = rawSaveRequestBody(t, "team-a", "from-response", "uid-resp") + ev.RequestObject = rawCommitRequestBody(t, "team-a", "from-request", "uid-req") + ev.ResponseObject = rawCommitRequestBody(t, "team-a", "from-response", "uid-resp") got := IdentityFromAuditEvent(ev, itypes.OperationDelete) @@ -134,7 +134,7 @@ func TestIdentityFromAuditEvent_NoBodyKeepsObjectRef(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", Name: "save-1", }, @@ -161,7 +161,7 @@ func TestIdentityFromAuditEvent_IgnoresMalformedBody(t *testing.T) { ev := auditv1.Event{ Verb: "create", ObjectRef: &auditv1.ObjectReference{ - Resource: "saverequests", + Resource: "commitrequests", Namespace: "team-a", }, } diff --git a/internal/authz/saverequest_admission.go b/internal/authz/commitrequest_admission.go similarity index 61% rename from internal/authz/saverequest_admission.go rename to internal/authz/commitrequest_admission.go index 6d38627c..4d9ba182 100644 --- a/internal/authz/saverequest_admission.go +++ b/internal/authz/commitrequest_admission.go @@ -10,25 +10,25 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// SaveRequestAdmitted reports whether request may save the commit windows of target, the GitTarget -// its spec.gitTargetRef names: a grant in target's spec.allowedSaveRequests admits the request's +// CommitRequestAdmitted reports whether request may save the commit windows of target, the GitTarget +// its spec.gitTargetRef names: a grant in target's spec.allowedCommitRequests admits the request's // namespace, by the labels that namespace carries. The admission also says how much the request may // claim of a window whose writes name no author. A non-NotFound read of the Namespace is returned as // err, so the caller retries instead of refusing a request on a transient failure. -func SaveRequestAdmitted( +func CommitRequestAdmitted( ctx context.Context, reader client.Reader, - request *configv1alpha3.SaveRequest, + request *configv1alpha3.CommitRequest, target *configv1alpha3.GitTarget, -) (Decision, configv1alpha3.SaveRequestAdmission, error) { +) (Decision, configv1alpha3.CommitRequestAdmission, error) { nsLabels, err := namespaceLabels(ctx, reader, request.Namespace) if err != nil { - return Decision{}, configv1alpha3.SaveRequestAdmission{}, err + return Decision{}, configv1alpha3.CommitRequestAdmission{}, err } - admission, selErr := target.AdmitsSaveRequest(request.Namespace, nsLabels) + admission, selErr := target.AdmitsCommitRequest(request.Namespace, nsLabels) if !admission.Admitted { - return notAdmitted("GitTarget", client.ObjectKeyFromObject(target), target.DescribeAllowedSaveRequests(), - "spec.allowedSaveRequests", request.Namespace, selErr), admission, nil + return notAdmitted("GitTarget", client.ObjectKeyFromObject(target), target.DescribeAllowedCommitRequests(), + "spec.allowedCommitRequests", request.Namespace, selErr), admission, nil } return Decision{Allowed: true}, admission, nil } diff --git a/internal/authz/saverequest_admission_test.go b/internal/authz/commitrequest_admission_test.go similarity index 70% rename from internal/authz/saverequest_admission_test.go rename to internal/authz/commitrequest_admission_test.go index 9d95c09d..193cf4c6 100644 --- a/internal/authz/saverequest_admission_test.go +++ b/internal/authz/commitrequest_admission_test.go @@ -18,21 +18,21 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// A GitTarget admits a SaveRequest by the namespace it is in and that namespace's labels, and gives it +// A GitTarget admits a CommitRequest by the namespace it is in and that namespace's labels, and gives it // the most any admitting grant allows of a window whose writes name no author. -func TestSaveRequestAdmitted(t *testing.T) { +func TestCommitRequestAdmitted(t *testing.T) { tenant := &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{ Name: testNS, Labels: map[string]string{"platform.example.com/tenant": "true"}, }} - request := &configv1alpha3.SaveRequest{ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: testNS}} - target := func(grants ...configv1alpha3.GitTargetSaveRequestGrant) *configv1alpha3.GitTarget { + request := &configv1alpha3.CommitRequest{ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: testNS}} + target := func(grants ...configv1alpha3.GitTargetCommitRequestGrant) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "shared", Namespace: platformNS}, - Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: grants}, + Spec: configv1alpha3.GitTargetSpec{AllowedCommitRequests: grants}, } } - bySelector := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetSaveRequestGrant { - return configv1alpha3.GitTargetSaveRequestGrant{ + bySelector := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetCommitRequestGrant { + return configv1alpha3.GitTargetCommitRequestGrant{ ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{ From: configv1alpha3.GrantFromSelector, Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"platform.example.com/tenant": "true"}}, @@ -40,27 +40,27 @@ func TestSaveRequestAdmitted(t *testing.T) { UnattributedWindows: policy, } } - all := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetSaveRequestGrant { - return configv1alpha3.GitTargetSaveRequestGrant{ + all := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetCommitRequestGrant { + return configv1alpha3.GitTargetCommitRequestGrant{ ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, UnattributedWindows: policy, } } reader := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithObjects(tenant).Build() - decision, _, err := SaveRequestAdmitted(context.Background(), reader, request, target()) + decision, _, err := CommitRequestAdmitted(context.Background(), reader, request, target()) require.NoError(t, err) assert.False(t, decision.Allowed, "the default admits only the GitTarget's own namespace") assert.Equal(t, ReasonNamespaceNotAuthorized, decision.Reason) - assert.Contains(t, decision.Message, "spec.allowedSaveRequests") + assert.Contains(t, decision.Message, "spec.allowedCommitRequests") - decision, admission, err := SaveRequestAdmitted(context.Background(), reader, request, + decision, admission, err := CommitRequestAdmitted(context.Background(), reader, request, target(bySelector(configv1alpha3.UnattributedWindowsSavedBy))) require.NoError(t, err) assert.True(t, decision.Allowed, "the namespace's labels match") assert.Equal(t, configv1alpha3.UnattributedWindowsSavedBy, admission.UnattributedWindows) - _, admission, err = SaveRequestAdmitted(context.Background(), reader, request, + _, admission, err = CommitRequestAdmitted(context.Background(), reader, request, target(all(configv1alpha3.UnattributedWindowsKeep), bySelector(configv1alpha3.UnattributedWindowsTakeOver))) require.NoError(t, err) assert.Equal(t, configv1alpha3.UnattributedWindowsTakeOver, admission.UnattributedWindows, @@ -71,6 +71,6 @@ func TestSaveRequestAdmitted(t *testing.T) { return errors.New("apiserver unavailable") }, }).Build() - _, _, err = SaveRequestAdmitted(context.Background(), failing, request, target(all(""))) + _, _, err = CommitRequestAdmitted(context.Background(), failing, request, target(all(""))) require.Error(t, err, "a namespace that cannot be read is retried, never refused") } diff --git a/internal/controller/access_grants_admission_test.go b/internal/controller/access_grants_admission_test.go index dc981ca1..cdf85963 100644 --- a/internal/controller/access_grants_admission_test.go +++ b/internal/controller/access_grants_admission_test.go @@ -159,11 +159,11 @@ var _ = Describe("Access grants at admission", func() { {"KubeProvider allowedApplierClaims", func(g map[string]any) *unstructured.Unstructured { return kubeProvider("allowedApplierClaims", g) }, map[string]any{"applierNamespaces": []any{"flux-system"}}}, - {"GitTarget allowedSaveRequests", + {"GitTarget allowedCommitRequests", func(g map[string]any) *unstructured.Unstructured { return object("GitTarget", map[string]any{ "gitBranchRef": map[string]any{"name": "b"}, "path": "apps", - "kubeProviderRef": map[string]any{"name": "k"}, "allowedSaveRequests": []any{g}, + "kubeProviderRef": map[string]any{"name": "k"}, "allowedCommitRequests": []any{g}, }) }, map[string]any{"unattributedWindows": "SavedBy"}}, diff --git a/internal/controller/saverequest_apply.go b/internal/controller/commitrequest_apply.go similarity index 77% rename from internal/controller/saverequest_apply.go rename to internal/controller/commitrequest_apply.go index 254035c6..deed6f43 100644 --- a/internal/controller/saverequest_apply.go +++ b/internal/controller/commitrequest_apply.go @@ -17,7 +17,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -// A SaveRequest with catchUp: AfterPush keeps being reconciled once it is Ready, to +// A CommitRequest with catchUp: AfterPush keeps being reconciled once it is Ready, to // report in its Applied condition how the apply it asked for went (git.SaveApply). Ready is the // save's alone and never changes here. @@ -32,33 +32,33 @@ const applyLostAfter = time.Minute // followsApply reports whether a finished request still reports its apply: it asked for one, its // save ended Ready, and Applied is not settled yet. -func followsApply(cr *configbutleraiv1alpha3.SaveRequest) bool { +func followsApply(cr *configbutleraiv1alpha3.CommitRequest) bool { return cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush && conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) && !applyReported(cr) } // applyReported reports whether Applied is settled: True, False, or Unknown because it was lost. -func applyReported(cr *configbutleraiv1alpha3.SaveRequest) bool { +func applyReported(cr *configbutleraiv1alpha3.CommitRequest) bool { c := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeApplied) return c != nil && (c.Status != metav1.ConditionUnknown || c.Reason == crReasonApplyLost) } // followApply reads where the request's apply stands, writes it to Applied, and comes back until it // is settled; then the request expires as any finished one does. -func (r *SaveRequestReconciler) followApply( +func (r *CommitRequestReconciler) followApply( ctx context.Context, - cr *configbutleraiv1alpha3.SaveRequest, + cr *configbutleraiv1alpha3.CommitRequest, ) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("SaveRequestReconciler") - attach := git.AttachSaveRequest{ + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") + attach := git.AttachCommitRequest{ Namespace: cr.Namespace, Name: cr.Name, UID: string(cr.UID), GitTargetName: cr.GitTargetKey().Name, GitTargetNamespace: cr.GitTargetKey().Namespace, } - apply, known, err := r.Finalizer.LookupSaveRequestApply(ctx, attach) + apply, known, err := r.Finalizer.LookupCommitRequestApply(ctx, attach) if err != nil { - log.V(1).Info("Cannot read the SaveRequest's apply yet; will retry", + log.V(1).Info("Cannot read the CommitRequest's apply yet; will retry", "name", client.ObjectKeyFromObject(cr), "err", err.Error()) - return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil } if !known { apply = unknownApply(cr) @@ -70,18 +70,18 @@ func (r *SaveRequestReconciler) followApply( // A conflict reads the request again; anything else retries the same way. return ctrl.Result{}, client.IgnoreNotFound(err) } - log.Info("SaveRequest apply reported", "name", client.ObjectKeyFromObject(cr), + log.Info("CommitRequest apply reported", "name", client.ObjectKeyFromObject(cr), "reason", apply.Reason, "appliedRevision", cr.Status.AppliedRevision, "message", apply.Message) } if applyReported(cr) { return r.expireFinished(ctx, cr) } - return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil } // unknownApply is the apply of a request its worker knows nothing of: still being taken on, shortly // after Ready, and lost after applyLostAfter. -func unknownApply(cr *configbutleraiv1alpha3.SaveRequest) git.SaveApply { +func unknownApply(cr *configbutleraiv1alpha3.CommitRequest) git.SaveApply { if ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady); ready != nil && time.Since(ready.LastTransitionTime.Time) < applyLostAfter { return git.SaveApply{Reason: git.SaveApplyApplying, Message: "waiting for the branch worker to take the apply"} @@ -91,7 +91,7 @@ func unknownApply(cr *configbutleraiv1alpha3.SaveRequest) git.SaveApply { } // setAppliedCondition writes apply to Applied and status.appliedRevision. -func setAppliedCondition(cr *configbutleraiv1alpha3.SaveRequest, apply git.SaveApply) { +func setAppliedCondition(cr *configbutleraiv1alpha3.CommitRequest, apply git.SaveApply) { status := metav1.ConditionUnknown switch { case apply.Applied: @@ -100,5 +100,5 @@ func setAppliedCondition(cr *configbutleraiv1alpha3.SaveRequest, apply git.SaveA case apply.Done: status = metav1.ConditionFalse } - setSaveRequestCondition(cr, ConditionTypeApplied, status, apply.Reason, apply.Message) + setCommitRequestCondition(cr, ConditionTypeApplied, status, apply.Reason, apply.Message) } diff --git a/internal/controller/saverequest_apply_test.go b/internal/controller/commitrequest_apply_test.go similarity index 59% rename from internal/controller/saverequest_apply_test.go rename to internal/controller/commitrequest_apply_test.go index 348b9749..b8adbd8c 100644 --- a/internal/controller/saverequest_apply_test.go +++ b/internal/controller/commitrequest_apply_test.go @@ -16,26 +16,26 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -func catchingUp(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { +func catchingUp(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { cr.Spec.CatchUp = configv1alpha3.CatchUpAfterPush return cr } -// TestSaveRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with +// TestCommitRequestReconcile_ReportsTheApplyItAskedFor: once the save is Ready, a request with // catchUp: AfterPush is reconciled on, and Applied follows the worker's report until it // is settled. Ready stays the save's. -func TestSaveRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { - c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) +func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") assert.Zero(t, f.applyLookups, "the save resolves first") f.apply, f.applyKnown = git.SaveApply{Reason: git.SaveApplyRetrying, Message: "the apply did not complete"}, true - res := reconcileSaveRequest(t, r, "save-1") + res := reconcileCommitRequest(t, r, "save-1") assert.Positive(t, res.RequeueAfter, "followed until it is settled") - got := fetchSaveRequest(t, c, "save-1") + got := fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyRetrying) requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) @@ -45,20 +45,20 @@ func TestSaveRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { Reason: git.SaveApplyApplied, AppliedRevision: "main@sha1:c0ffee", } - reconcileSaveRequest(t, r, "save-1") - got = fetchSaveRequest(t, c, "save-1") + reconcileCommitRequest(t, r, "save-1") + got = fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionTrue, git.SaveApplyApplied) assert.Equal(t, "main@sha1:c0ffee", got.Status.AppliedRevision) lookups := f.applyLookups - reconcileSaveRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") assert.Equal(t, lookups, f.applyLookups, "settled: not looked up again") } -// TestSaveRequestReconcile_ASettledApplyWithoutAnApply: a request that pushed nothing, or whose +// TestCommitRequestReconcile_ASettledApplyWithoutAnApply: a request that pushed nothing, or whose // GitTarget names no applier, is Applied=True: no apply was needed, and the cluster already holds what // it saved. One whose commit the branch lost is Applied=False. None touches Ready. -func TestSaveRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { +func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { for _, tc := range []struct { apply git.SaveApply want metav1.ConditionStatus @@ -68,14 +68,14 @@ func TestSaveRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { {git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyNoApplier}, metav1.ConditionTrue}, } { t.Run(tc.apply.Reason, func(t *testing.T) { - c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() f.apply, f.applyKnown = tc.apply, true - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-1") - reconcileSaveRequest(t, r, "save-1") - got := fetchSaveRequest(t, c, "save-1") + reconcileCommitRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, tc.want, tc.apply.Reason) requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Empty(t, got.Status.AppliedRevision) @@ -83,36 +83,36 @@ func TestSaveRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { } } -// TestSaveRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply +// TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost: a worker that knows nothing of the apply // shortly after Ready is still taking it on; long after, it restarted, and Applied says it is lost. -func TestSaveRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { - c := newSaveRequestClient(t, nil, catchingUp(newSaveRequest("save-1"))) +func TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { + c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-1") + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileCommitRequest(t, r, "save-1") - reconcileSaveRequest(t, r, "save-1") - got := fetchSaveRequest(t, c, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, git.SaveApplyApplying) ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) ready.LastTransitionTime = metav1.NewTime(time.Now().Add(-2 * applyLostAfter)) require.NoError(t, c.Status().Update(context.Background(), &got)) - reconcileSaveRequest(t, r, "save-1") - got = fetchSaveRequest(t, c, "save-1") + reconcileCommitRequest(t, r, "save-1") + got = fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeApplied, metav1.ConditionUnknown, crReasonApplyLost) assert.False(t, followsApply(&got), "lost is settled") } -// TestSaveRequestReconcile_NoCatchUpReportsNoApply: the default asks no apply, and has no Applied. -func TestSaveRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { - c := newSaveRequestClient(t, nil, newSaveRequest("save-1")) +// TestCommitRequestReconcile_NoCatchUpReportsNoApply: the default asks no apply, and has no Applied. +func TestCommitRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save-1")) f := committedFinalizer() - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-1") - reconcileSaveRequest(t, r, "save-1") - got := fetchSaveRequest(t, c, "save-1") + reconcileCommitRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") + got := fetchCommitRequest(t, c, "save-1") assert.Nil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeApplied)) assert.Zero(t, f.applyLookups) } diff --git a/internal/controller/saverequest_attribution_agreement_test.go b/internal/controller/commitrequest_attribution_agreement_test.go similarity index 86% rename from internal/controller/saverequest_attribution_agreement_test.go rename to internal/controller/commitrequest_attribution_agreement_test.go index 066c779f..80b44bf5 100644 --- a/internal/controller/saverequest_attribution_agreement_test.go +++ b/internal/controller/commitrequest_attribution_agreement_test.go @@ -56,7 +56,7 @@ func windowOutcomeAttributionMissed(t *testing.T) git.AttributionOutcome { return outcome } -// TestSaveRequest_OutcomesAgree is the cross-subsystem half of the P0 regression test. +// TestCommitRequest_OutcomesAgree is the cross-subsystem half of the P0 regression test. // // The gap that let the blocker through was that every existing test set both sides of the // comparison to the same literal, so nothing ever checked that the two INDEPENDENT producers @@ -66,12 +66,12 @@ func windowOutcomeAttributionMissed(t *testing.T) git.AttributionOutcome { // // The default deployment is the first row: --admission-webhook defaults to false, so the // controller emits attributionNotAttempted, while --author-attribution=false leaves the window's -// outcome at the zero value. When those two were different strings, no SaveRequest could +// outcome at the zero value. When those two were different strings, no CommitRequest could // attach to any window and the user's commit message was silently dropped. -func TestSaveRequest_OutcomesAgree(t *testing.T) { +func TestCommitRequest_OutcomesAgree(t *testing.T) { tests := []struct { name string - requestSide saveRequestAttribution + requestSide commitRequestAttribution windowSide git.AttributionOutcome wantNamesActor bool }{ @@ -102,7 +102,7 @@ func TestSaveRequest_OutcomesAgree(t *testing.T) { requestOutcome := tc.requestSide.gitOutcome() assert.Equal(t, tc.wantNamesActor, requestOutcome.NamesActor()) assert.Equal(t, requestOutcome.NamesActor(), tc.windowSide.NamesActor(), - "the SaveRequest side produced %q and the window side produced %q; they must "+ + "the CommitRequest side produced %q and the window side produced %q; they must "+ "agree on whether an actor was named, or the request cannot attach and the "+ "user's commit message is silently dropped", requestOutcome, tc.windowSide) @@ -116,17 +116,17 @@ func TestAttributionOutcome_OffLeavesZeroOutcome(t *testing.T) { assert.Equal(t, git.AttributionNotAttempted, windowOutcomeAttributionOff(), "configured-author mode must leave the event's outcome at AttributionNotAttempted") assert.Equal(t, git.AttributionNotAttempted, attributionNotAttempted.gitOutcome(), - "a SaveRequest with command-author capture off must carry AttributionNotAttempted") + "a CommitRequest with command-author capture off must carry AttributionNotAttempted") assert.Equal(t, git.AttributionUnresolved, windowOutcomeAttributionMissed(t), "attribution that ran and found nothing must be AttributionUnresolved, not the zero value") } -// TestSaveRequestAttribution_GitOutcome keeps every controller outcome's projection explicit, +// TestCommitRequestAttribution_GitOutcome keeps every controller outcome's projection explicit, // including the defensive zero value. -func TestSaveRequestAttribution_GitOutcome(t *testing.T) { +func TestCommitRequestAttribution_GitOutcome(t *testing.T) { tests := []struct { name string - input saveRequestAttribution + input commitRequestAttribution want git.AttributionOutcome }{ {name: "unset is not attempted", input: attributionUnset, want: git.AttributionNotAttempted}, @@ -142,11 +142,11 @@ func TestSaveRequestAttribution_GitOutcome(t *testing.T) { } } -// TestSaveRequestAttribution_UnsetStatusIsCaptureDisabled keeps an unset decision visible and safe. -func TestSaveRequestAttribution_UnsetStatusIsCaptureDisabled(t *testing.T) { - var request configv1alpha3.SaveRequest +// TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled keeps an unset decision visible and safe. +func TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled(t *testing.T) { + var request configv1alpha3.CommitRequest - setSaveRequestAttributed(&request, attributionUnset) + setCommitRequestAttributed(&request, attributionUnset) requireCondition(t, request, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) } diff --git a/internal/controller/saverequest_controller.go b/internal/controller/commitrequest_controller.go similarity index 58% rename from internal/controller/saverequest_controller.go rename to internal/controller/commitrequest_controller.go index ada33e74..cf622506 100644 --- a/internal/controller/saverequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -24,9 +24,9 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/queue" ) -// SaveRequestFinalizer is the EventRouter seam the reconciler drives, using the -// attach-then-poll protocol (docs/spec/saverequest-design.md): -// ServiceSaveRequest registers the attach idempotently on the GitTarget's branch +// CommitRequestFinalizer is the EventRouter seam the reconciler drives, using the +// attach-then-poll protocol (docs/spec/commitrequest-design.md): +// ServiceCommitRequest registers the attach idempotently on the GitTarget's branch // worker (attach it to the author's window with its message and timers) // and returns the request's current outcome — resolved=false means keep polling. // watch.EventRouter satisfies it without adaptation. @@ -34,21 +34,21 @@ import ( // There is no watermark barrier: the interactive case is covered by the human gap between the // edit and the save, UC2 by the request's attach wait. The worker anchors that wait at its own // registration and runs the window's timers, so the controller never holds the finalize itself. -type SaveRequestFinalizer interface { - ServiceSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) - // WithdrawSaveRequest cancels a request the worker has not acted on and reports the worker's - // answer: resolved with git.ErrSaveRequestWithdrawn when it is cancelled, the real outcome +type CommitRequestFinalizer interface { + ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + // WithdrawCommitRequest cancels a request the worker has not acted on and reports the worker's + // answer: resolved with git.ErrCommitRequestWithdrawn when it is cancelled, the real outcome // when the worker got there first, or resolved=false with the worker's phase while it decides. - WithdrawSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) - // LookupSaveRequestApply reports where the apply a request with catchUp: + WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + // LookupCommitRequestApply reports where the apply a request with catchUp: // AfterPush asked for stands, and false while the worker knows nothing of it. - LookupSaveRequestApply(ctx context.Context, attach git.AttachSaveRequest) (git.SaveApply, bool, error) + LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) } -// CommandAuthorLookup resolves the author of a SaveRequest from the submitter +// CommandAuthorLookup resolves the author of a CommitRequest from the submitter // captured at admission by the validate-operator-types webhook, keyed by the persisted // object's UID. *queue.CommandAuthorStore satisfies it without adaptation. The lookup -// is present-or-never (docs/spec/saverequest-admission-authorship.md): a miss is +// is present-or-never (docs/spec/commitrequest-admission-authorship.md): a miss is // immediate and final — the webhook is not configured (or a best-effort write missed) — // and the request claims no actor with no wait. type CommandAuthorLookup interface { @@ -65,30 +65,30 @@ const windowMismatchMessage = "the open commit window belongs to a different aut // resolveTimeoutMessage explains the fail-closed resolve bound: the worker never acted on the // request within the safety window, so it was withdrawn rather than polled forever. -const resolveTimeoutMessage = "the SaveRequest finalize did not resolve within the safety window" +const resolveTimeoutMessage = "the CommitRequest finalize did not resolve within the safety window" // noWorkerTimeoutMessage is resolveTimeoutMessage for a GitTarget that never started a branch // worker, the one cause the controller can name without asking anybody. -const noWorkerTimeoutMessage = "the SaveRequest's GitTarget did not start a branch worker within the safety window" +const noWorkerTimeoutMessage = "the CommitRequest's GitTarget did not start a branch worker within the safety window" const ( - // saveRequestPollInterval is the requeue cadence while polling the worker + // commitRequestPollInterval is the requeue cadence while polling the worker // for the attached request's outcome (attach-then-poll). - saveRequestPollInterval = 2 * time.Second + commitRequestPollInterval = 2 * time.Second - // saveRequestPushAllowance is what the safety bound allows on top of a request's own + // commitRequestPushAllowance is what the safety bound allows on top of a request's own // timers, for the push cooldown and its retries. Authorship is settled synchronously at first // sight, so no attribution wait belongs here. - saveRequestPushAllowance = 120 * time.Second + commitRequestPushAllowance = 120 * time.Second - // The request window defaults mirror the +kubebuilder:default markers on SaveRequestWindow. + // The request window defaults mirror the +kubebuilder:default markers on CommitRequestWindow. // The API server fills them in, so these apply only to a client that bypasses defaulting; // resolving to the same values keeps the two paths from disagreeing. defaultAttachTimeout = 2 * time.Second defaultMaxDuration = 2 * time.Second ) -// requestWindow is a SaveRequest's window, resolved to the values the worker applies. +// requestWindow is a CommitRequest's window, resolved to the values the worker applies. type requestWindow struct { attach configbutleraiv1alpha3.AttachPolicy attachTimeout time.Duration @@ -98,7 +98,7 @@ type requestWindow struct { // resolveRequestWindow reads spec.window, falling back field by field to the schema defaults. An // explicit "0s" stays zero, and an omitted idleTimeout stays nil (no idle close). -func resolveRequestWindow(spec configbutleraiv1alpha3.SaveRequestSpec) requestWindow { +func resolveRequestWindow(spec configbutleraiv1alpha3.CommitRequestSpec) requestWindow { resolved := requestWindow{ attach: configbutleraiv1alpha3.AttachCurrentOrNext, attachTimeout: defaultAttachTimeout, @@ -130,10 +130,10 @@ func resolveRequestWindow(spec configbutleraiv1alpha3.SaveRequestSpec) requestWi // only the worker can say how that ends, however long a down remote or a recovery takes. Past the // bound, an unheld request is withdrawn, and fails only once the worker agrees. func (w requestWindow) resolveTimeout() time.Duration { - return w.attachTimeout + w.maxDuration + saveRequestPushAllowance + return w.attachTimeout + w.maxDuration + commitRequestPushAllowance } -// SaveRequestReconciler deliberately does NOT use reconcileStatus, which every other controller +// CommitRequestReconciler deliberately does NOT use reconcileStatus, which every other controller // here shares. The difference is not stylistic, so it is worth saying once: // // - Its For() carries no GenerationChangedPredicate, so a status write re-enqueues the object. @@ -150,9 +150,9 @@ func (w requestWindow) resolveTimeout() time.Duration { // one, but a re-run is no longer the only thing standing between a flushed window and a // NoOpenWindow that never happened.) // -// SaveRequestReconciler drives a SaveRequest through its state machine -// (docs/spec/saverequest-design.md and -// docs/spec/saverequest-admission-authorship.md): +// CommitRequestReconciler drives a CommitRequest through its state machine +// (docs/spec/commitrequest-design.md and +// docs/spec/commitrequest-admission-authorship.md): // // 1. ATTRIBUTE — a single synchronous read of the submitter captured at admission // (present-or-never). A hit names that submitter as the author @@ -166,7 +166,7 @@ func (w requestWindow) resolveTimeout() time.Duration { // registration, so there is no controller-side delay. While it polls, the controller // reports the phase the worker reports. No window before the attach deadline resolves // NoWindow; a window belonging to someone else stays open for its own author. -type SaveRequestReconciler struct { +type CommitRequestReconciler struct { client.Client Scheme *runtime.Scheme @@ -181,219 +181,219 @@ type SaveRequestReconciler struct { // AuthorLookup is nil (the validate-operator-types webhook is disabled), requests // claim no actor immediately, with AuthorAttributed=False. The attached window // determines the eventual Git author. - Finalizer SaveRequestFinalizer + Finalizer CommitRequestFinalizer AuthorLookup CommandAuthorLookup // TTL is how long a request is kept once it finishes. It is written onto the request as - // SaveRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. + // CommitRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. TTL time.Duration } -// SaveRequestDeleteAfterAnnotation carries the RFC 3339 time after which the controller deletes a -// finished SaveRequest. The controller writes it once, when the request finishes, and from then +// CommitRequestDeleteAfterAnnotation carries the RFC 3339 time after which the controller deletes a +// finished CommitRequest. The controller writes it once, when the request finishes, and from then // on the annotation alone decides: remove it to keep the request, or edit it to move the deletion. // A request that finished before this annotation existed carries none and is never deleted. -const SaveRequestDeleteAfterAnnotation = "configbutler.ai/delete-after" +const CommitRequestDeleteAfterAnnotation = "configbutler.ai/delete-after" -// +kubebuilder:rbac:groups=configbutler.ai,resources=saverequests,verbs=get;list;watch;patch;delete -// +kubebuilder:rbac:groups=configbutler.ai,resources=saverequests/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests,verbs=get;list;watch;patch;delete +// +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests/status,verbs=get;update;patch -// Reconcile advances one SaveRequest through attribute → attach + poll → -// terminal status. With MaxConcurrentReconciles=1 concurrent SaveRequests are +// Reconcile advances one CommitRequest through attribute → attach + poll → +// terminal status. With MaxConcurrentReconciles=1 concurrent CommitRequests are // serialized by construction, and the worker keys attaches by request identity so // re-sends across poll requeues are idempotent. -func (r *SaveRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("SaveRequestReconciler") +func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - saveRequest, done, err := r.loadActionableSaveRequest(ctx, req) + commitRequest, done, err := r.loadActionableCommitRequest(ctx, req) if err != nil { return ctrl.Result{}, err } if done { - if saveRequest != nil { - if followsApply(saveRequest) { - return r.followApply(ctx, saveRequest) + if commitRequest != nil { + if followsApply(commitRequest) { + return r.followApply(ctx, commitRequest) } - return r.expireFinished(ctx, saveRequest) + return r.expireFinished(ctx, commitRequest) } return ctrl.Result{}, nil } if r.Finalizer == nil { - log.V(1).Info("SaveRequest finalize disabled: no Finalizer configured", + log.V(1).Info("CommitRequest finalize disabled: no Finalizer configured", "name", req.NamespacedName) return ctrl.Result{}, nil } - if handled, err := r.refusePrunedGitTargetRef(ctx, saveRequest); handled || err != nil { + if handled, err := r.refusePrunedGitTargetRef(ctx, commitRequest); handled || err != nil { return ctrl.Result{}, err } // 1. ATTRIBUTE: settle the request's actor synchronously (present-or-never). // A hit names the admission submitter; a miss claims no actor. Either way the // decision is final — there is no wait and no requeue for the author. - author, attribution := r.attributeAuthor(ctx, saveRequest) + author, attribution := r.attributeAuthor(ctx, commitRequest) - claim, admitted, refusal, err := r.admitAtFirstSight(ctx, log, saveRequest, attribution) - if err != nil || admitted == saveRequestRefused { + claim, admitted, refusal, err := r.admitAtFirstSight(ctx, log, commitRequest, attribution) + if err != nil || admitted == commitRequestRefused { return ctrl.Result{}, err } - if err := r.stampFirstSightConditions(ctx, saveRequest, attribution); err != nil { + if err := r.stampFirstSightConditions(ctx, commitRequest, attribution); err != nil { return ctrl.Result{}, err } // The schema rejects an invalid literal message at admission, so a rejection here names an // object the schema could not have seen: one stored before the rule landed. It is terminal - // rather than retried, because SaveRequest.spec is immutable and no apply can repair it. - if err := git.ValidateLiteralCommitMessage(saveRequest.Spec.Message); err != nil { - // The fallback identity: this fails before ServiceSaveRequest, so nothing has + // rather than retried, because CommitRequest.spec is immutable and no apply can repair it. + if err := git.ValidateLiteralCommitMessage(commitRequest.Spec.Message); err != nil { + // The fallback identity: this fails before ServiceCommitRequest, so nothing has // confirmed the named GitTarget exists. - r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, - saveRequestTarget{}) + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, attribution, + commitRequestTarget{}) return ctrl.Result{}, nil } // 2. ATTACH + POLL: register the attach idempotently the instant we attribute // (no controller-side delay — the worker anchors the wait at its registration) // and poll the outcome. - window := resolveRequestWindow(saveRequest.Spec) - attach := git.AttachSaveRequest{ - Namespace: saveRequest.Namespace, - Name: saveRequest.Name, - UID: string(saveRequest.UID), + window := resolveRequestWindow(commitRequest.Spec) + attach := git.AttachCommitRequest{ + Namespace: commitRequest.Namespace, + Name: commitRequest.Name, + UID: string(commitRequest.UID), Author: author.Author, Attribution: attribution.gitOutcome(), - GitTargetName: saveRequest.GitTargetKey().Name, - GitTargetNamespace: saveRequest.GitTargetKey().Namespace, - GitTargetUID: saveRequest.Status.GitTargetUID, - Message: saveRequest.Spec.Message, + GitTargetName: commitRequest.GitTargetKey().Name, + GitTargetNamespace: commitRequest.GitTargetKey().Namespace, + GitTargetUID: commitRequest.Status.GitTargetUID, + Message: commitRequest.Spec.Message, Attach: window.attach, AttachTimeout: window.attachTimeout, IdleTimeout: window.idleTimeout, MaxDuration: window.maxDuration, - CommitEmpty: saveRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: saveRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, Unattributed: claim, } - if admitted == saveRequestReplaced { - return r.refuseOnceUnheld(ctx, log, req, saveRequest, attribution, attach, refusal) + if admitted == commitRequestReplaced { + return r.refuseOnceUnheld(ctx, log, req, commitRequest, attribution, attach, refusal) } - return r.sendSaveRequest(ctx, log, req, saveRequest, attribution, attach, - time.Since(saveRequest.CreationTimestamp.Time) >= window.resolveTimeout(), - admitted == saveRequestTargetMissing) + return r.sendCommitRequest(ctx, log, req, commitRequest, attribution, attach, + time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout(), + admitted == commitRequestTargetMissing) } -// sendSaveRequest sends the attach, or the withdraw once the request is past its bound, or, while +// sendCommitRequest sends the attach, or the withdraw once the request is past its bound, or, while // its GitTarget does not exist and never admitted it, neither: nothing is attached before a GitTarget // admitted the request, so one created later is asked first, on a later poll. -func (r *SaveRequestReconciler) sendSaveRequest( +func (r *CommitRequestReconciler) sendCommitRequest( ctx context.Context, log logr.Logger, req ctrl.Request, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - attach git.AttachSaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + attach git.AttachCommitRequest, pastTheBound, targetMissing bool, ) (ctrl.Result, error) { switch { case pastTheBound: // Past the bound the controller only withdraws: an attach sent now would sit behind the // withdraw on the worker's FIFO and could register the request after it was cancelled. - return r.withdrawSaveRequest(ctx, log, req, saveRequest, attribution, attach) + return r.withdrawCommitRequest(ctx, log, req, commitRequest, attribution, attach) case targetMissing: - return r.pollSaveRequest(ctx, saveRequest, attribution, git.PhaseWaitingForWorker, "") + return r.pollCommitRequest(ctx, commitRequest, attribution, git.PhaseWaitingForWorker, "") } - return r.attachSaveRequest(ctx, log, req, saveRequest, attribution, attach) + return r.attachCommitRequest(ctx, log, req, commitRequest, attribution, attach) } -// saveRequestAdmission is how admitAtFirstSight left a request. -type saveRequestAdmission int +// commitRequestAdmission is how admitAtFirstSight left a request. +type commitRequestAdmission int const ( - // saveRequestAdmitted is a request its GitTarget admitted, now or before. - saveRequestAdmitted saveRequestAdmission = iota - // saveRequestTargetMissing is a request whose GitTarget does not exist and never admitted it. - saveRequestTargetMissing - // saveRequestRefused is a request now failed. - saveRequestRefused - // saveRequestReplaced is a request a GitTarget refuses that replaced the one that admitted it. An + // commitRequestAdmitted is a request its GitTarget admitted, now or before. + commitRequestAdmitted commitRequestAdmission = iota + // commitRequestTargetMissing is a request whose GitTarget does not exist and never admitted it. + commitRequestTargetMissing + // commitRequestRefused is a request now failed. + commitRequestRefused + // commitRequestReplaced is a request a GitTarget refuses that replaced the one that admitted it. An // attach may have been sent before, so it is refused only once no worker holds it. - saveRequestReplaced + commitRequestReplaced ) -// admitAtFirstSight runs admitSaveRequest, records the GitTarget that admitted the request, and fails +// admitAtFirstSight runs admitCommitRequest, records the GitTarget that admitted the request, and fails // a request it refuses. A refusal is final unless that same GitTarget, by UID, admitted the request // before: then its grants were lowered after an attach may have been sent, and the worker, which may // hold the request, keeps what it was first sent. A GitTarget that did not exist when the request was // first seen admitted nothing, so its refusal stands. One that was recreated since admitted nothing -// either, but the GitTarget it replaced did: its refusal is saveRequestReplaced, returned with the +// either, but the GitTarget it replaced did: its refusal is commitRequestReplaced, returned with the // refusal, and waits for the worker that may hold the request. err is a read or write to retry. -func (r *SaveRequestReconciler) admitAtFirstSight( +func (r *CommitRequestReconciler) admitAtFirstSight( ctx context.Context, log logr.Logger, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, -) (git.UnattributedClaim, saveRequestAdmission, error, error) { - claim, targetUID, err := r.admitSaveRequest(ctx, saveRequest) - recorded := saveRequest.Status.GitTargetUID + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, +) (git.UnattributedClaim, commitRequestAdmission, error, error) { + claim, targetUID, err := r.admitCommitRequest(ctx, commitRequest) + recorded := commitRequest.Status.GitTargetUID switch { case err == nil && targetUID == "": if recorded != "" { // Its GitTarget is gone since it admitted the request: the attach path reports that. - return claim, saveRequestAdmitted, nil, nil + return claim, commitRequestAdmitted, nil, nil } - return claim, saveRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, saveRequest, attribution) + return claim, commitRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, commitRequest, attribution) case err == nil: if recorded == targetUID { - return claim, saveRequestAdmitted, nil, nil + return claim, commitRequestAdmitted, nil, nil } // Recorded before any attach is sent for this GitTarget, so a later refusal can tell the two // apart. At first sight the in-progress stamp that follows writes it. - saveRequest.Status.GitTargetUID = targetUID - if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) == nil { - return claim, saveRequestAdmitted, nil, nil + commitRequest.Status.GitTargetUID = targetUID + if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) == nil { + return claim, commitRequestAdmitted, nil, nil } - return claim, saveRequestAdmitted, nil, r.Status().Update(ctx, saveRequest) - case !errors.Is(err, errSaveRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed): - return claim, saveRequestAdmitted, nil, err + return claim, commitRequestAdmitted, nil, r.Status().Update(ctx, commitRequest) + case !errors.Is(err, errCommitRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed): + return claim, commitRequestAdmitted, nil, err case recorded == targetUID: - return git.UnattributedClaimNone, saveRequestAdmitted, nil, nil + return git.UnattributedClaimNone, commitRequestAdmitted, nil, nil case recorded != "": - return git.UnattributedClaimNone, saveRequestReplaced, err, nil + return git.UnattributedClaimNone, commitRequestReplaced, err, nil } - r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, err, attribution, - resolvedSaveRequestTarget(saveRequest)) - return claim, saveRequestRefused, nil, nil + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, attribution, + resolvedCommitRequestTarget(commitRequest)) + return claim, commitRequestRefused, nil, nil } // refuseOnceUnheld fails a request that a replacement of its GitTarget refuses. The GitTarget it // replaced admitted the request, so an attach may have been sent, and the worker holding it may have // committed it. That worker answers first: its outcome is reported, and a request it holds is waited // for. The refusal is written only once the request is withdrawn, or nothing held it. -func (r *SaveRequestReconciler) refuseOnceUnheld( +func (r *CommitRequestReconciler) refuseOnceUnheld( ctx context.Context, log logr.Logger, req ctrl.Request, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - attach git.AttachSaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + attach git.AttachCommitRequest, refusal error, ) (ctrl.Result, error) { - result, resolved, withdrawErr := r.Finalizer.WithdrawSaveRequest(ctx, attach) + result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) if withdrawErr != nil || !resolved { if withdrawErr != nil { - log.V(1).Info("SaveRequest withdraw not yet serviceable; will retry", + log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", "name", req.NamespacedName, "err", withdrawErr.Error()) } - return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) } finalizeErr := result.Err - if errors.Is(finalizeErr, git.ErrSaveRequestWithdrawn) { + if errors.Is(finalizeErr, git.ErrCommitRequestWithdrawn) { result, finalizeErr = git.FinalizeResult{}, refusal } - r.writeTerminalStatus(ctx, log, saveRequest, result, finalizeErr, attribution, - resolvedSaveRequestTarget(saveRequest)) + r.writeTerminalStatus(ctx, log, commitRequest, result, finalizeErr, attribution, + resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } @@ -401,15 +401,15 @@ func (r *SaveRequestReconciler) refuseOnceUnheld( // than the grants admitting it allow. It is terminal: the spec is immutable. var errTakeOverNotAllowed = errors.New("spec.window.author asks for more than the GitTarget allows") -// errSaveRequestNotAllowed is a request whose GitTarget's spec.allowedSaveRequests admits no grant for +// errCommitRequestNotAllowed is a request whose GitTarget's spec.allowedCommitRequests admits no grant for // its namespace. It is terminal: the spec is immutable, so a later grant reaches only a new request. -var errSaveRequestNotAllowed = errors.New("the GitTarget does not admit SaveRequests from this namespace") +var errCommitRequestNotAllowed = errors.New("the GitTarget does not admit CommitRequests from this namespace") -// admitSaveRequest decides whether the request may save its GitTarget's windows, and what it may claim -// of a window whose writes name no author (git.AttachSaveRequest.Unattributed). +// admitCommitRequest decides whether the request may save its GitTarget's windows, and what it may claim +// of a window whose writes name no author (git.AttachCommitRequest.Unattributed). // -// A grant in the GitTarget's spec.allowedSaveRequests must admit the request's namespace, or the -// request is errSaveRequestNotAllowed. It then claims what spec.window.author asks for, and only while +// A grant in the GitTarget's spec.allowedCommitRequests must admit the request's namespace, or the +// request is errCommitRequestNotAllowed. It then claims what spec.window.author asks for, and only while // the GitTarget's KubeProvider has never received an audit fact: its AuditFactsReceived condition is // not True, so no write watched on that cluster can name its author. On a cluster that has sent one, a // write naming no author means its fact went missing, and nothing is claimed. A request that asks for @@ -417,37 +417,37 @@ var errSaveRequestNotAllowed = errors.New("the GitTarget does not admit SaveRequ // // A GitTarget that does not exist admits nothing yet, and is reported with an empty UID. Any other // read error is returned as is, so the request is retried rather than refused. -func (r *SaveRequestReconciler) admitSaveRequest( +func (r *CommitRequestReconciler) admitCommitRequest( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, ) (git.UnattributedClaim, string, error) { var target configbutleraiv1alpha3.GitTarget - if err := r.Get(ctx, saveRequest.GitTargetKey(), &target); err != nil { + if err := r.Get(ctx, commitRequest.GitTargetKey(), &target); err != nil { if apierrors.IsNotFound(err) { return git.UnattributedClaimNone, "", nil } - return git.UnattributedClaimNone, "", fmt.Errorf("read GitTarget %s: %w", saveRequest.GitTargetKey(), err) + return git.UnattributedClaimNone, "", fmt.Errorf("read GitTarget %s: %w", commitRequest.GitTargetKey(), err) } uid := string(target.UID) - claim, err := r.claimOn(ctx, saveRequest, &target) + claim, err := r.claimOn(ctx, commitRequest, &target) return claim, uid, err } -// claimOn is admitSaveRequest for a GitTarget that exists. -func (r *SaveRequestReconciler) claimOn( +// claimOn is admitCommitRequest for a GitTarget that exists. +func (r *CommitRequestReconciler) claimOn( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, target *configbutleraiv1alpha3.GitTarget, ) (git.UnattributedClaim, error) { - decision, admission, err := authz.SaveRequestAdmitted(ctx, r.Client, saveRequest, target) + decision, admission, err := authz.CommitRequestAdmitted(ctx, r.Client, commitRequest, target) if err != nil { return git.UnattributedClaimNone, err } if !decision.Allowed { - return git.UnattributedClaimNone, fmt.Errorf("%w: %s", errSaveRequestNotAllowed, decision.Message) + return git.UnattributedClaimNone, fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message) } ask := configbutleraiv1alpha3.WindowAuthorOwn - if window := saveRequest.Spec.Window; window != nil && window.Author != "" { + if window := commitRequest.Spec.Window; window != nil && window.Author != "" { ask = window.Author } if ask == configbutleraiv1alpha3.WindowAuthorOwn { @@ -456,7 +456,7 @@ func (r *SaveRequestReconciler) claimOn( if !admission.UnattributedWindows.Allows(ask) { return git.UnattributedClaimNone, fmt.Errorf("%w: spec.window.author is %q, and the grants of GitTarget %s "+ "that admit namespace %q allow unattributedWindows %q", errTakeOverNotAllowed, ask, - saveRequest.GitTargetKey(), saveRequest.Namespace, admission.UnattributedWindows) + commitRequest.GitTargetKey(), commitRequest.Namespace, admission.UnattributedWindows) } key := target.KubeProviderKey() if key.Name == "" { @@ -475,70 +475,70 @@ func (r *SaveRequestReconciler) claimOn( return git.UnattributedClaimSavedBy, nil } -// attachSaveRequest sends the attach and reports the outcome once the worker has one. Until then +// attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then // it records the worker's phase and polls; a service error is a not-yet-serviceable worker, so it // polls on the same terms. -func (r *SaveRequestReconciler) attachSaveRequest( +func (r *CommitRequestReconciler) attachCommitRequest( ctx context.Context, log logr.Logger, req ctrl.Request, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - attach git.AttachSaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + attach git.AttachCommitRequest, ) (ctrl.Result, error) { - result, resolved, serviceErr := r.Finalizer.ServiceSaveRequest(ctx, attach) + result, resolved, serviceErr := r.Finalizer.ServiceCommitRequest(ctx, attach) if serviceErr != nil || !resolved { if serviceErr != nil { - log.V(1).Info("SaveRequest attach not yet serviceable; will retry", + log.V(1).Info("CommitRequest attach not yet serviceable; will retry", "name", req.NamespacedName, "err", serviceErr.Error()) } - return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) } if result.Err != nil { - log.Error(result.Err, "SaveRequest finalize failed", - "gitTarget", saveRequest.GitTargetKey().String(), "name", req.NamespacedName) + log.Error(result.Err, "CommitRequest finalize failed", + "gitTarget", commitRequest.GitTargetKey().String(), "name", req.NamespacedName) } - r.writeTerminalStatus(ctx, log, saveRequest, result, result.Err, attribution, - resolvedSaveRequestTarget(saveRequest)) + r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, + resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } -// withdrawSaveRequest handles a request past its safety window. A request the worker holds keeps +// withdrawCommitRequest handles a request past its safety window. A request the worker holds keeps // polling with no bound of its own: only the worker can say how it ends. Any other request is // withdrawn, and fails only on the worker's answer, so a request reported failed can never be // committed afterwards. A withdraw that cannot be serviced yet polls on the same terms. -func (r *SaveRequestReconciler) withdrawSaveRequest( +func (r *CommitRequestReconciler) withdrawCommitRequest( ctx context.Context, log logr.Logger, req ctrl.Request, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - attach git.AttachSaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + attach git.AttachCommitRequest, ) (ctrl.Result, error) { - result, resolved, withdrawErr := r.Finalizer.WithdrawSaveRequest(ctx, attach) + result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) if withdrawErr != nil || !resolved { if withdrawErr != nil { - log.V(1).Info("SaveRequest withdraw not yet serviceable; will retry", + log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", "name", req.NamespacedName, "err", withdrawErr.Error()) } // Not answered yet, or the worker holds the request: keep polling. - return r.pollSaveRequest(ctx, saveRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) } - if !errors.Is(result.Err, git.ErrSaveRequestWithdrawn) { + if !errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { // The worker resolved it on its own before the withdraw: report that outcome, as the // attach path would have. - r.writeTerminalStatus(ctx, log, saveRequest, result, result.Err, attribution, - resolvedSaveRequestTarget(saveRequest)) + r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, + resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } // Withdrawn. A worker's answer, or a GitTarget that was read, carries the branch, and only // then is the request's identity known. - target := saveRequestTarget{} + target := commitRequestTarget{} if result.Branch != "" { - target = resolvedSaveRequestTarget(saveRequest) + target = resolvedCommitRequestTarget(commitRequest) } message := resolveTimeoutMessage @@ -548,60 +548,60 @@ func (r *SaveRequestReconciler) withdrawSaveRequest( case result.Phase == git.PhaseWaitingForWorker: message = noWorkerTimeoutMessage } - log.Info("SaveRequest was not acted on within the safety window; withdrawn and failed closed", + log.Info("CommitRequest was not acted on within the safety window; withdrawn and failed closed", "name", req.NamespacedName, "reason", message) - r.writeTerminalStatus(ctx, log, saveRequest, git.FinalizeResult{}, errors.New(message), attribution, target) + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, errors.New(message), attribution, target) return ctrl.Result{}, nil } -// pollSaveRequest reports where the worker says the request stands and polls again. -func (r *SaveRequestReconciler) pollSaveRequest( +// pollCommitRequest reports where the worker says the request stands and polls again. +func (r *CommitRequestReconciler) pollCommitRequest( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - phase git.SaveRequestPhase, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + phase git.CommitRequestPhase, held string, ) (ctrl.Result, error) { - if err := r.recordWorkerPhase(ctx, saveRequest, attribution, phase, held); err != nil { + if err := r.recordWorkerPhase(ctx, commitRequest, attribution, phase, held); err != nil { return ctrl.Result{}, err } - return ctrl.Result{RequeueAfter: saveRequestPollInterval}, nil + return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil } // stampFirstSightConditions stamps the still-running conditions the first time a request is seen, // so the object reports its progress (kstatus InProgress) and AuthorAttributed is settled // immediately. A disabled controller returns before this, so it never stamps. -func (r *SaveRequestReconciler) stampFirstSightConditions( +func (r *CommitRequestReconciler) stampFirstSightConditions( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, ) error { - if findCondition(saveRequest.Status.Conditions, ConditionTypeReady) != nil { + if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) != nil { return nil } - markSaveRequestProgressing(saveRequest, attribution, "", "") - if err := r.Status().Update(ctx, saveRequest); err != nil { + markCommitRequestProgressing(commitRequest, attribution, "", "") + if err := r.Status().Update(ctx, commitRequest); err != nil { return err } - logf.FromContext(ctx).V(1).Info("Stamped SaveRequest in-progress conditions", - "name", client.ObjectKeyFromObject(saveRequest)) + logf.FromContext(ctx).V(1).Info("Stamped CommitRequest in-progress conditions", + "name", client.ObjectKeyFromObject(commitRequest)) return nil } -// refusePrunedGitTargetRef fails a SaveRequest whose spec names no GitTarget, which can only be +// refusePrunedGitTargetRef fails a CommitRequest whose spec names no GitTarget, which can only be // one thing: an object stored before spec.targetRef was renamed to spec.gitTargetRef, whose old // value the apiserver stopped serving the moment the new CRDs landed. Admission refuses an empty // name on every path, so nothing else can produce it. // // It is terminal rather than retried, and it has to be, because the object cannot be repaired: -// SaveRequest.spec is wholly immutable, so no apply can put the name back. Retrying would spend +// CommitRequest.spec is wholly immutable, so no apply can put the name back. Retrying would spend // the controller's attention on an object that will never resolve, and would report a transient // "get GitTarget" error for a permanent condition. The message names the only fix there is. -func (r *SaveRequestReconciler) refusePrunedGitTargetRef( +func (r *CommitRequestReconciler) refusePrunedGitTargetRef( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, ) (bool, error) { - if saveRequest.Spec.GitTargetRef.Name != "" { + if commitRequest.Spec.GitTargetRef.Name != "" { return false, nil } @@ -610,18 +610,18 @@ func (r *SaveRequestReconciler) refusePrunedGitTargetRef( // state that must not be missing from the counter. Note the status write below is returned for // requeue, so a failing write re-decides and increments again: see the instrument's doc // comment on what one increment means. - recordSaveRequestOutcome(ctx, crOutcomeFailed, saveRequestTarget{}) - failSaveRequest(saveRequest, crReasonGitTargetRefPruned, + recordCommitRequestOutcome(ctx, crOutcomeFailed, commitRequestTarget{}) + failCommitRequest(commitRequest, crReasonGitTargetRefPruned, "spec.gitTargetRef is empty: this request was created before spec.targetRef was renamed, "+ - "and its value was pruned by the upgrade. A SaveRequest spec is immutable, so this "+ + "and its value was pruned by the upgrade. A CommitRequest spec is immutable, so this "+ "one cannot be repaired — delete it and create a new one.") - if err := r.Status().Update(ctx, saveRequest); err != nil { + if err := r.Status().Update(ctx, commitRequest); err != nil { return true, err } - r.stampDeleteAfter(ctx, saveRequest) + r.stampDeleteAfter(ctx, commitRequest) logf.FromContext(ctx).Info( - "SaveRequest names no GitTarget: created before the gitTargetRef rename", - "name", client.ObjectKeyFromObject(saveRequest)) + "CommitRequest names no GitTarget: created before the gitTargetRef rename", + "name", client.ObjectKeyFromObject(commitRequest)) return true, nil } @@ -634,25 +634,25 @@ func (r *SaveRequestReconciler) refusePrunedGitTargetRef( // The lookup result is logged at Info: it is the counterpart to the admission handler's // "recorded command author" line, so a hit/miss pair makes the whole capture→read path // legible (the first thing to check when a request unexpectedly claims no actor). -func (r *SaveRequestReconciler) attributeAuthor( +func (r *CommitRequestReconciler) attributeAuthor( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, -) (queue.CommandAuthor, saveRequestAttribution) { - log := logf.FromContext(ctx).WithName("SaveRequestReconciler") + commitRequest *configbutleraiv1alpha3.CommitRequest, +) (queue.CommandAuthor, commitRequestAttribution) { + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") if r.AuthorLookup == nil { log.Info("command-author lookup disabled (validate-operator-types webhook off); request claims no actor", - "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID) + "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) // Capture is off, so nothing was attempted — distinct from a capture that ran and // found no record, which is what attributionCommitter now means. return queue.CommandAuthor{}, attributionNotAttempted } - if author, ok := r.AuthorLookup.LookupCommandAuthor(ctx, saveRequest.UID); ok { + if author, ok := r.AuthorLookup.LookupCommandAuthor(ctx, commitRequest.UID); ok { log.Info("command author resolved from admission record", - "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID, "author", author.Author) + "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID, "author", author.Author) return author, attributionFromAdmission } log.Info("no admission command-author record found; request claims no actor", - "name", client.ObjectKeyFromObject(saveRequest), "uid", saveRequest.UID) + "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) return queue.CommandAuthor{}, attributionCommitter } @@ -660,23 +660,23 @@ func (r *SaveRequestReconciler) attributeAuthor( // change, so polling does not re-write status every interval. An empty phase — the worker has not // registered the request yet — keeps whatever is shown, so the controller never reports a phase // the worker has not confirmed. -func (r *SaveRequestReconciler) recordWorkerPhase( +func (r *CommitRequestReconciler) recordWorkerPhase( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, - attribution saveRequestAttribution, - phase git.SaveRequestPhase, + commitRequest *configbutleraiv1alpha3.CommitRequest, + attribution commitRequestAttribution, + phase git.CommitRequestPhase, held string, ) error { - if progressRecorded(saveRequest, phase, held) { + if progressRecorded(commitRequest, phase, held) { return nil } - markSaveRequestProgressing(saveRequest, attribution, phase, held) - return r.Status().Update(ctx, saveRequest) + markCommitRequestProgressing(commitRequest, attribution, phase, held) + return r.Status().Update(ctx, commitRequest) } // progressRecorded reports whether the request's status already says this phase, and why it is // held, so a poll writes only what changed. An empty phase says nothing new. -func progressRecorded(cr *configbutleraiv1alpha3.SaveRequest, phase git.SaveRequestPhase, held string) bool { +func progressRecorded(cr *configbutleraiv1alpha3.CommitRequest, phase git.CommitRequestPhase, held string) bool { current := findCondition(cr.Status.Conditions, ConditionTypeReconciling) if current == nil { return false @@ -688,35 +688,35 @@ func progressRecorded(cr *configbutleraiv1alpha3.SaveRequest, phase git.SaveRequ return current.Reason == reason && current.Message == message } -// loadActionableSaveRequest fetches the SaveRequest and short-circuits +// loadActionableCommitRequest fetches the CommitRequest and short-circuits // everything that must not reach the finalize: a deleted object, a terminal // outcome, and — via an uncached re-read — a stale cache echo of our own // terminal write from a previous invocation (work past this point must happen -// at most once per SaveRequest). done=true means the finalize has nothing +// at most once per CommitRequest). done=true means the finalize has nothing // further to do; the request is returned with it when it is terminal, so its // expiry can be decided, and nil when it is gone. -func (r *SaveRequestReconciler) loadActionableSaveRequest( +func (r *CommitRequestReconciler) loadActionableCommitRequest( ctx context.Context, req ctrl.Request, -) (*configbutleraiv1alpha3.SaveRequest, bool, error) { - var saveRequest configbutleraiv1alpha3.SaveRequest - if err := r.Get(ctx, req.NamespacedName, &saveRequest); err != nil { +) (*configbutleraiv1alpha3.CommitRequest, bool, error) { + var commitRequest configbutleraiv1alpha3.CommitRequest + if err := r.Get(ctx, req.NamespacedName, &commitRequest); err != nil { return nil, true, client.IgnoreNotFound(err) } - if saveRequestIsTerminal(&saveRequest) { - return &saveRequest, true, nil + if commitRequestIsTerminal(&commitRequest) { + return &commitRequest, true, nil } if r.APIReader != nil { - if err := r.APIReader.Get(ctx, req.NamespacedName, &saveRequest); err != nil { + if err := r.APIReader.Get(ctx, req.NamespacedName, &commitRequest); err != nil { return nil, true, client.IgnoreNotFound(err) } - if saveRequestIsTerminal(&saveRequest) { - return &saveRequest, true, nil + if commitRequestIsTerminal(&commitRequest) { + return &commitRequest, true, nil } } - return &saveRequest, false, nil + return &commitRequest, false, nil } // expireFinished deletes a finished request once the time in its delete-after annotation has @@ -728,20 +728,20 @@ func (r *SaveRequestReconciler) loadActionableSaveRequest( // that races it fails the delete with a conflict, and the requeue reads the edit. The edit itself // needs no extra wiring: this controller has no generation filter, so a metadata change reconciles // the request again. -func (r *SaveRequestReconciler) expireFinished( +func (r *CommitRequestReconciler) expireFinished( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, ) (ctrl.Result, error) { - value, ok := saveRequest.Annotations[SaveRequestDeleteAfterAnnotation] - if !ok || !saveRequest.DeletionTimestamp.IsZero() { + value, ok := commitRequest.Annotations[CommitRequestDeleteAfterAnnotation] + if !ok || !commitRequest.DeletionTimestamp.IsZero() { return ctrl.Result{}, nil } - log := logf.FromContext(ctx).WithName("SaveRequestReconciler") + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") deleteAfter, parsed := parseDeleteAfter(value) if !parsed { // Kept rather than guessed at: an annotation a person mistyped is not consent to delete. - log.Info("SaveRequest delete-after annotation is not an RFC 3339 time; keeping the request", - "name", client.ObjectKeyFromObject(saveRequest), "value", value) + log.Info("CommitRequest delete-after annotation is not an RFC 3339 time; keeping the request", + "name", client.ObjectKeyFromObject(commitRequest), "value", value) return ctrl.Result{}, nil } @@ -749,13 +749,13 @@ func (r *SaveRequestReconciler) expireFinished( return ctrl.Result{RequeueAfter: remaining}, nil } - uid, resourceVersion := saveRequest.UID, saveRequest.ResourceVersion - err := r.Delete(ctx, saveRequest, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) + uid, resourceVersion := commitRequest.UID, commitRequest.ResourceVersion + err := r.Delete(ctx, commitRequest, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) if err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } - log.Info("Deleted finished SaveRequest past its delete-after time", - "name", client.ObjectKeyFromObject(saveRequest), "deleteAfter", value) + log.Info("Deleted finished CommitRequest past its delete-after time", + "name", client.ObjectKeyFromObject(commitRequest), "deleteAfter", value) return ctrl.Result{}, nil } @@ -776,74 +776,74 @@ func parseDeleteAfter(value string) (time.Time, bool) { // // A write that still fails is logged, not requeued: the request is then simply kept, the safe // direction, and a requeue would re-enter the finalize this controller must run at most once. -func (r *SaveRequestReconciler) stampDeleteAfter( +func (r *CommitRequestReconciler) stampDeleteAfter( ctx context.Context, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, ) { if r.TTL <= 0 { return } - log := logf.FromContext(ctx).WithName("SaveRequestReconciler") + log := logf.FromContext(ctx).WithName("CommitRequestReconciler") reader := r.APIReader if reader == nil { reader = r.Client } deleteAfter := time.Now().Add(r.TTL).UTC().Format(time.RFC3339) - current := saveRequest - for attempt := 1; attempt <= saveRequestStatusUpdateAttempts; attempt++ { - if _, set := current.Annotations[SaveRequestDeleteAfterAnnotation]; set { + current := commitRequest + for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { + if _, set := current.Annotations[CommitRequestDeleteAfterAnnotation]; set { return } patch := client.MergeFromWithOptions(current.DeepCopy(), client.MergeFromWithOptimisticLock{}) if current.Annotations == nil { current.Annotations = map[string]string{} } - current.Annotations[SaveRequestDeleteAfterAnnotation] = deleteAfter + current.Annotations[CommitRequestDeleteAfterAnnotation] = deleteAfter err := r.Patch(ctx, current, patch) if err == nil || apierrors.IsNotFound(err) { return } if !apierrors.IsConflict(err) { - log.Error(err, "Failed to stamp delete-after on a finished SaveRequest; it will be kept", - "name", client.ObjectKeyFromObject(saveRequest)) + log.Error(err, "Failed to stamp delete-after on a finished CommitRequest; it will be kept", + "name", client.ObjectKeyFromObject(commitRequest)) return } - var fresh configbutleraiv1alpha3.SaveRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(saveRequest), &fresh); getErr != nil { + var fresh configbutleraiv1alpha3.CommitRequest + if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { if !apierrors.IsNotFound(getErr) { - log.Error(getErr, "Failed to re-read SaveRequest to stamp delete-after; it will be kept", - "name", client.ObjectKeyFromObject(saveRequest)) + log.Error(getErr, "Failed to re-read CommitRequest to stamp delete-after; it will be kept", + "name", client.ObjectKeyFromObject(commitRequest)) } return } - if fresh.UID != saveRequest.UID { + if fresh.UID != commitRequest.UID { return } current = &fresh } log.Error(nil, "Gave up stamping delete-after after repeated conflicts; the request will be kept", - "name", client.ObjectKeyFromObject(saveRequest)) + "name", client.ObjectKeyFromObject(commitRequest)) } -// saveRequestStatusUpdateAttempts bounds the terminal-status conflict retry. -const saveRequestStatusUpdateAttempts = 3 +// commitRequestStatusUpdateAttempts bounds the terminal-status conflict retry. +const commitRequestStatusUpdateAttempts = 3 -// writeTerminalStatus records the finalize outcome on the SaveRequest, +// writeTerminalStatus records the finalize outcome on the CommitRequest, // retrying on optimistic-concurrency conflicts. Like the audit-consumer path // it replaces, a permanently failing write is logged and given up on rather // than returned for requeue: re-running the reconcile would re-finalize an // already-flushed window and mis-report the outcome as NoOpenWindow. -func (r *SaveRequestReconciler) writeTerminalStatus( +func (r *CommitRequestReconciler) writeTerminalStatus( ctx context.Context, log logr.Logger, - saveRequest *configbutleraiv1alpha3.SaveRequest, + commitRequest *configbutleraiv1alpha3.CommitRequest, result git.FinalizeResult, finalizeErr error, - attribution saveRequestAttribution, - target saveRequestTarget, + attribution commitRequestAttribution, + target commitRequestTarget, ) { - expectedUID := saveRequest.UID + expectedUID := commitRequest.UID reader := r.APIReader if reader == nil { reader = r.Client @@ -852,10 +852,10 @@ func (r *SaveRequestReconciler) writeTerminalStatus( // Once, here, and deliberately not inside the loop below: result and finalizeErr are inputs // that do not change across attempts, so this is the point the outcome is settled. Recording // per attempt would over-count every request that hit a conflict. - recordSaveRequestOutcome(ctx, saveRequestOutcome(result, finalizeErr), target) + recordCommitRequestOutcome(ctx, commitRequestOutcome(result, finalizeErr), target) - current := saveRequest - for attempt := 1; attempt <= saveRequestStatusUpdateAttempts; attempt++ { + current := commitRequest + for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { applyFinalizeResultToStatus(current, result, finalizeErr, attribution) err := r.Status().Update(ctx, current) @@ -865,14 +865,14 @@ func (r *SaveRequestReconciler) writeTerminalStatus( if finalizeErr != nil { finalizeError = finalizeErr.Error() } - readyReason, readyMessage := saveRequestReadyReason(current) - log.Info("SaveRequest finalized", + readyReason, readyMessage := commitRequestReadyReason(current) + log.Info("CommitRequest finalized", "name", client.ObjectKeyFromObject(current), - "ready", saveRequestConditionStatus(current, ConditionTypeReady), + "ready", commitRequestConditionStatus(current, ConditionTypeReady), "reason", readyReason, "message", readyMessage, - "authorAttributed", saveRequestConditionStatus(current, ConditionTypeAuthorAttributed), - "pushed", saveRequestConditionStatus(current, ConditionTypePushed), + "authorAttributed", commitRequestConditionStatus(current, ConditionTypeAuthorAttributed), + "pushed", commitRequestConditionStatus(current, ConditionTypePushed), "branch", current.Status.Branch, "commit", current.Status.Commit, "outcome", result.Outcome, @@ -882,42 +882,42 @@ func (r *SaveRequestReconciler) writeTerminalStatus( return } if !apierrors.IsConflict(err) { - log.Error(err, "Failed to write SaveRequest status") + log.Error(err, "Failed to write CommitRequest status") return } - log.V(1).Info("Conflict writing SaveRequest status; retrying", "attempt", attempt) - var fresh configbutleraiv1alpha3.SaveRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(saveRequest), &fresh); getErr != nil { + log.V(1).Info("Conflict writing CommitRequest status; retrying", "attempt", attempt) + var fresh configbutleraiv1alpha3.CommitRequest + if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { if apierrors.IsNotFound(getErr) { - log.Info("SaveRequest deleted before status could be written; skipping") + log.Info("CommitRequest deleted before status could be written; skipping") return } - log.Error(getErr, "Failed to re-read SaveRequest for status update") + log.Error(getErr, "Failed to re-read CommitRequest for status update") return } // Never stamp the outcome onto a different incarnation or over a // terminal outcome another writer got in first. if fresh.UID != expectedUID { - log.Info("SaveRequest UID changed before status could be written; skipping", + log.Info("CommitRequest UID changed before status could be written; skipping", "expectedUID", expectedUID, "objectUID", fresh.UID) return } - if saveRequestIsTerminal(&fresh) { + if commitRequestIsTerminal(&fresh) { return } current = &fresh } - log.Error(nil, "Gave up writing SaveRequest status after repeated conflicts") + log.Error(nil, "Gave up writing CommitRequest status after repeated conflicts") } // SetupWithManager sets up the controller with the Manager. // MaxConcurrentReconciles is pinned to 1 on purpose: the single worker IS the -// multi-SaveRequest ordering design — concurrent SaveRequests for the same +// multi-CommitRequest ordering design — concurrent CommitRequests for the same // GitTarget are serialized exactly as a dedicated finalize-coordinator // goroutine would serialize them, without the extra moving parts (see -// docs/spec/saverequest-design.md). +// docs/spec/commitrequest-design.md). // // Restart recovery is best-effort by design: the // message is durable in spec.message, so on restart any non-terminal request is @@ -927,10 +927,10 @@ func (r *SaveRequestReconciler) writeTerminalStatus( // in-memory outcome is gone, the re-driven attach finds the work already mirrored, // and it resolves Rejected/AlreadyPresent. We do not build a durable record to // close that. -func (r *SaveRequestReconciler) SetupWithManager(mgr ctrl.Manager) error { +func (r *CommitRequestReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&configbutleraiv1alpha3.SaveRequest{}). + For(&configbutleraiv1alpha3.CommitRequest{}). WithOptions(controller.Options{MaxConcurrentReconciles: 1}). - Named("saverequest"). + Named("commitrequest"). Complete(r) } diff --git a/internal/controller/saverequest_controller_test.go b/internal/controller/commitrequest_controller_test.go similarity index 82% rename from internal/controller/saverequest_controller_test.go rename to internal/controller/commitrequest_controller_test.go index e7ffe49b..041a4624 100644 --- a/internal/controller/saverequest_controller_test.go +++ b/internal/controller/commitrequest_controller_test.go @@ -15,32 +15,32 @@ import ( configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -var _ = Describe("SaveRequest controller", func() { +var _ = Describe("CommitRequest controller", func() { const namespace = "default" // The suite registers the reconciler in attributed-author mode (a non-nil AuthorLookup // that never resolves) with a Finalizer that never resolves, so these specs cover // the in-progress stamp and the terminal short-circuit only; the full // attribute → attach → terminal flow and the configured-author path are covered by - // the unit tests in saverequest_controller_unit_test.go. - It("stamps a freshly created SaveRequest with in-progress conditions", func() { - saveRequest := &configbutleraiv1alpha3.SaveRequest{ + // the unit tests in commitrequest_controller_unit_test.go. + It("stamps a freshly created CommitRequest with in-progress conditions", func() { + commitRequest := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "save-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{ Name: "team-a-config", }, Message: "increase checkout API memory", }, } - Expect(k8sClient.Create(ctx, saveRequest)).To(Succeed()) - key := client.ObjectKeyFromObject(saveRequest) + Expect(k8sClient.Create(ctx, commitRequest)).To(Succeed()) + key := client.ObjectKeyFromObject(commitRequest) Eventually(func(g Gomega) { - var fetched configbutleraiv1alpha3.SaveRequest + var fetched configbutleraiv1alpha3.CommitRequest g.Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) reconciling := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReconciling) g.Expect(reconciling).NotTo(BeNil()) @@ -52,23 +52,23 @@ var _ = Describe("SaveRequest controller", func() { }) It("does not overwrite a terminal outcome that is already recorded", func() { - saveRequest := &configbutleraiv1alpha3.SaveRequest{ + commitRequest := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "save-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{ Name: "team-a-config", }, }, } - Expect(k8sClient.Create(ctx, saveRequest)).To(Succeed()) - key := client.ObjectKeyFromObject(saveRequest) + Expect(k8sClient.Create(ctx, commitRequest)).To(Succeed()) + key := client.ObjectKeyFromObject(commitRequest) // Wait for the initial in-progress stamp. Eventually(func(g Gomega) { - var fetched configbutleraiv1alpha3.SaveRequest + var fetched configbutleraiv1alpha3.CommitRequest g.Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) ready := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) @@ -76,7 +76,7 @@ var _ = Describe("SaveRequest controller", func() { }, 10*time.Second, 200*time.Millisecond).Should(Succeed()) // Simulate a finalize having recorded the terminal outcome (Ready=True). - var fetched configbutleraiv1alpha3.SaveRequest + var fetched configbutleraiv1alpha3.CommitRequest Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) apimeta.SetStatusCondition(&fetched.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: crReasonCommitted, Message: "committed", @@ -90,7 +90,7 @@ var _ = Describe("SaveRequest controller", func() { // The controller must leave the terminal outcome intact. Consistently(func(g Gomega) { - var checked configbutleraiv1alpha3.SaveRequest + var checked configbutleraiv1alpha3.CommitRequest g.Expect(k8sClient.Get(ctx, key, &checked)).To(Succeed()) ready := apimeta.FindStatusCondition(checked.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) @@ -104,16 +104,16 @@ var _ = Describe("SaveRequest controller", func() { // rejected only when the CRD is installed, and a rule that compiles but matches nothing simply // never rejects anything, so it is pinned against a real API server rather than only through the // controller's Go validator. -var _ = Describe("SaveRequest message schema", func() { +var _ = Describe("CommitRequest message schema", func() { const namespace = "default" - newRequest := func(message string) *configbutleraiv1alpha3.SaveRequest { - return &configbutleraiv1alpha3.SaveRequest{ + newRequest := func(message string) *configbutleraiv1alpha3.CommitRequest { + return &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ GenerateName: "message-schema-", Namespace: namespace, }, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: "team-a-config"}, Message: message, }, diff --git a/internal/controller/saverequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go similarity index 70% rename from internal/controller/saverequest_controller_unit_test.go rename to internal/controller/commitrequest_controller_unit_test.go index 16bd59a7..ccdd2a07 100644 --- a/internal/controller/saverequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -36,7 +36,7 @@ type fakeFinalizer struct { resolved bool err error - calls []git.AttachSaveRequest + calls []git.AttachCommitRequest // The withdraw reply. Left unset, the withdraw answers like the router does before it cancels // anything: with the worker's outcome, or its phase while unresolved. withdrawPending makes an @@ -45,7 +45,7 @@ type fakeFinalizer struct { withdrawResolved bool withdrawErr error withdrawPending bool - withdrawCalls []git.AttachSaveRequest + withdrawCalls []git.AttachCommitRequest // The apply a request with catchUp: AfterPush asked for, and the lookups made. apply git.SaveApply @@ -54,15 +54,15 @@ type fakeFinalizer struct { applyLookups int } -func (f *fakeFinalizer) LookupSaveRequestApply( - _ context.Context, _ git.AttachSaveRequest, +func (f *fakeFinalizer) LookupCommitRequestApply( + _ context.Context, _ git.AttachCommitRequest, ) (git.SaveApply, bool, error) { f.applyLookups++ return f.apply, f.applyKnown, f.applyErr } -func (f *fakeFinalizer) WithdrawSaveRequest( - _ context.Context, attach git.AttachSaveRequest, +func (f *fakeFinalizer) WithdrawCommitRequest( + _ context.Context, attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { f.withdrawCalls = append(f.withdrawCalls, attach) if f.withdrawPending || f.withdrawResolved || f.withdrawErr != nil || f.withdrawResult.Phase != "" { @@ -71,8 +71,8 @@ func (f *fakeFinalizer) WithdrawSaveRequest( return f.result, f.resolved, f.err } -func (f *fakeFinalizer) ServiceSaveRequest( - _ context.Context, attach git.AttachSaveRequest, +func (f *fakeFinalizer) ServiceCommitRequest( + _ context.Context, attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { f.calls = append(f.calls, attach) return f.result, f.resolved, f.err @@ -98,17 +98,17 @@ func attributedAlice() *fakeAuthorLookup { return &fakeAuthorLookup{author: queue.CommandAuthor{Author: "alice"}, found: true} } -// newSaveRequest builds a request created just now, inside its safety window, as the API server +// newCommitRequest builds a request created just now, inside its safety window, as the API server // would stamp it. pastTheBound ages one beyond it. -func newSaveRequest(name string) *configv1alpha3.SaveRequest { - return &configv1alpha3.SaveRequest{ +func newCommitRequest(name string) *configv1alpha3.CommitRequest { + return &configv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ Name: name, Namespace: "default", UID: types.UID("uid-" + name), CreationTimestamp: metav1.Now(), }, - Spec: configv1alpha3.SaveRequestSpec{ + Spec: configv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: "team-a-config"}, Message: "save: " + name, }, @@ -116,13 +116,13 @@ func newSaveRequest(name string) *configv1alpha3.SaveRequest { } // pastTheBound ages a request beyond its safety window, where the controller only withdraws. -func pastTheBound(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { +func pastTheBound(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(cr.Spec).resolveTimeout())) return cr } -// withReadyCommitted stamps a terminal Committed SaveRequest (Ready=True). -func withReadyCommitted(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { +// withReadyCommitted stamps a terminal Committed CommitRequest (Ready=True). +func withReadyCommitted(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionTrue, Reason: crReasonCommitted, Message: "committed", @@ -132,7 +132,7 @@ func withReadyCommitted(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequ // withInProgress stamps the still-running conditions a post-restart redelivery // would already carry (Ready=False, Reconciling=True), which is non-terminal. -func withInProgress(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest { +func withInProgress(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ Type: ConditionTypeReady, Status: metav1.ConditionFalse, Reason: string(git.PhaseWaitingForWindow), Message: "in progress", @@ -147,7 +147,7 @@ func withInProgress(cr *configv1alpha3.SaveRequest) *configv1alpha3.SaveRequest return cr } -func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...client.Object) client.WithWatch { +func newCommitRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...client.Object) client.WithWatch { t.Helper() objects = withGitTargets(objects) scheme := runtime.NewScheme() @@ -156,7 +156,7 @@ func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...clien builder := fake.NewClientBuilder(). WithScheme(scheme). WithObjects(objects...). - WithStatusSubresource(&configv1alpha3.SaveRequest{}) + WithStatusSubresource(&configv1alpha3.CommitRequest{}) if fns != nil { builder = builder.WithInterceptorFuncs(*fns) } @@ -164,7 +164,7 @@ func newSaveRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...clien } // withGitTargets gives each GitTarget a UID, as the API server would, and, when objects hold no -// GitTarget at all, adds the one each SaveRequest names: a request is only attached once a GitTarget +// GitTarget at all, adds the one each CommitRequest names: a request is only attached once a GitTarget // admitted it. A test that passes GitTargets of its own gets only those. func withGitTargets(objects []client.Object) []client.Object { present := map[client.ObjectKey]bool{} @@ -180,7 +180,7 @@ func withGitTargets(objects []client.Object) []client.Object { return objects } for _, obj := range objects { - request, ok := obj.(*configv1alpha3.SaveRequest) + request, ok := obj.(*configv1alpha3.CommitRequest) if !ok || request.Spec.GitTargetRef.Name == "" || present[request.GitTargetKey()] { continue } @@ -193,7 +193,7 @@ func withGitTargets(objects []client.Object) []client.Object { return objects } -func reconcileSaveRequest(t *testing.T, r *SaveRequestReconciler, name string) ctrl.Result { +func reconcileCommitRequest(t *testing.T, r *CommitRequestReconciler, name string) ctrl.Result { t.Helper() res, err := r.Reconcile(context.Background(), ctrl.Request{ NamespacedName: client.ObjectKey{Namespace: "default", Name: name}, @@ -202,9 +202,9 @@ func reconcileSaveRequest(t *testing.T, r *SaveRequestReconciler, name string) c return res } -func fetchSaveRequest(t *testing.T, c client.Client, name string) configv1alpha3.SaveRequest { +func fetchCommitRequest(t *testing.T, c client.Client, name string) configv1alpha3.CommitRequest { t.Helper() - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest require.NoError(t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: name}, &cr)) return cr } @@ -213,7 +213,7 @@ func fetchSaveRequest(t *testing.T, c client.Client, name string) configv1alpha3 // returns it for further assertions. func requireCondition( t *testing.T, - cr configv1alpha3.SaveRequest, + cr configv1alpha3.CommitRequest, condType string, status metav1.ConditionStatus, reason string, @@ -228,18 +228,18 @@ func requireCondition( return *c } -func TestSaveRequestReconcile_Committed(t *testing.T) { - cr := newSaveRequest("save-1") - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_Committed(t *testing.T) { + cr := newCommitRequest("save-1") + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-1") + reconcileCommitRequest(t, r, "save-1") - got := fetchSaveRequest(t, c, "save-1") + got := fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) @@ -259,18 +259,18 @@ func TestSaveRequestReconcile_Committed(t *testing.T) { assert.Equal(t, "uid-save-1", call.UID) } -func TestSaveRequestReconcile_NoOpenWindow(t *testing.T) { - cr := newSaveRequest("save-now") - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_NoOpenWindow(t *testing.T) { + cr := newCommitRequest("save-now") + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") - got := fetchSaveRequest(t, c, "save-now") + got := fetchCommitRequest(t, c, "save-now") // A benign no-commit is Ready=True (serviced) with the specific reason, Pushed=False. requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonNoWindow) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonNoWindow) @@ -280,7 +280,7 @@ func TestSaveRequestReconcile_NoOpenWindow(t *testing.T) { // whenNothingToCommit: CommitEmpty reaches the worker, and a recorded empty commit reports its // commit and Pushed=True while the Ready reason keeps the cause. -func TestSaveRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *testing.T) { +func TestCommitRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *testing.T) { for _, tc := range []struct { outcome git.FinalizeOutcome reason string @@ -289,20 +289,20 @@ func TestSaveRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *te {git.FinalizeAlreadyPresent, crReasonAlreadyPresent}, } { t.Run(tc.reason, func(t *testing.T) { - cr := newSaveRequest("save-empty") + cr := newCommitRequest("save-empty") cr.Spec.WhenNothingToCommit = configv1alpha3.NothingToCommitCommitEmpty - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: tc.outcome, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-empty") + reconcileCommitRequest(t, r, "save-empty") require.Len(t, f.calls, 1) assert.True(t, f.calls[0].CommitEmpty, "the choice must reach the worker") - got := fetchSaveRequest(t, c, "save-empty") + got := fetchCommitRequest(t, c, "save-empty") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, tc.reason) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeStalled, metav1.ConditionFalse, "") @@ -314,19 +314,19 @@ func TestSaveRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t *te } // The author-bound refusal: an open window belonging to someone else is left -// open and the SaveRequest reports the WindowMismatch reason (no commit). -func TestSaveRequestReconcile_WindowMismatchIsExplained(t *testing.T) { - cr := newSaveRequest("save-foreign") - c := newSaveRequestClient(t, nil, cr) +// open and the CommitRequest reports the WindowMismatch reason (no commit). +func TestCommitRequestReconcile_WindowMismatchIsExplained(t *testing.T) { + cr := newCommitRequest("save-foreign") + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-foreign") + reconcileCommitRequest(t, r, "save-foreign") - got := fetchSaveRequest(t, c, "save-foreign") + got := fetchCommitRequest(t, c, "save-foreign") ready := requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonWindowMismatch) assert.Equal(t, windowMismatchMessage, ready.Message) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonWindowMismatch) @@ -335,65 +335,65 @@ func TestSaveRequestReconcile_WindowMismatchIsExplained(t *testing.T) { // A matching window that finalized with no diff (loop prevention) reports the // AlreadyPresent reason — never left hanging. -func TestSaveRequestReconcile_AlreadyPresentRejected(t *testing.T) { - cr := newSaveRequest("save-noop") - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_AlreadyPresentRejected(t *testing.T) { + cr := newCommitRequest("save-noop") + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-noop") + reconcileCommitRequest(t, r, "save-noop") - got := fetchSaveRequest(t, c, "save-noop") + got := fetchCommitRequest(t, c, "save-noop") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonAlreadyPresent) requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, crReasonAlreadyPresent) assert.Empty(t, got.Status.Commit) } // A resolved outcome that carries an error becomes a Failed (Stalled=True) request. -func TestSaveRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { - cr := newSaveRequest("save-fail") - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { + cr := newCommitRequest("save-fail") + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Err: errors.New("commit failed: unreachable remote")}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-fail") + reconcileCommitRequest(t, r, "save-fail") - got := fetchSaveRequest(t, c, "save-fail") + got := fetchCommitRequest(t, c, "save-fail") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonFinalizeFailed) stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Contains(t, stalled.Message, "unreachable remote") requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, "") } -// TestSaveRequestReconcile_LookupMissClaimsNoActor verifies that a lookup miss claims no actor -// immediately — present-or-never, no wait. A freshly created SaveRequest attaches at once with a +// TestCommitRequestReconcile_LookupMissClaimsNoActor verifies that a lookup miss claims no actor +// immediately — present-or-never, no wait. A freshly created CommitRequest attaches at once with a // blank author and records AuthorAttributed=False (CommitterFallback). The matching window determines // the Git author. -func TestSaveRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { - cr := newSaveRequest("save-fresh") +func TestCommitRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { + cr := newCommitRequest("save-fresh") cr.CreationTimestamp = metav1.Now() // fresh: the old path would have requeued here - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Branch: "main"}, resolved: true, } lookup := &fakeAuthorLookup{found: false} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} - res := reconcileSaveRequest(t, r, "save-fresh") + res := reconcileCommitRequest(t, r, "save-fresh") assert.Zero(t, res.RequeueAfter, "a miss is final; the request must not poll for an author") assert.Equal(t, 1, lookup.calls, "the author is looked up exactly once, synchronously") require.Len(t, f.calls, 1, "the attach is sent immediately on a miss") assert.Empty(t, f.calls[0].Author, "a miss attaches with a blank, unnamed actor") - got := fetchSaveRequest(t, c, "save-fresh") + got := fetchCommitRequest(t, c, "save-fresh") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback) @@ -403,15 +403,15 @@ func TestSaveRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { // not zeros. The API server normally fills the block in, so nil arrives here only from a client // that bypassed defaulting — and a zero there would be the old defect: the request expires before // the write it exists to publish can open a window. -func TestSaveRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { - cr := newSaveRequest("save-default") +func TestCommitRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { + cr := newCommitRequest("save-default") cr.CreationTimestamp = metav1.Now() require.Nil(t, cr.Spec.Window, "the fixture must leave the block unset") - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-default") + reconcileCommitRequest(t, r, "save-default") require.Len(t, f.calls, 1) assert.Equal(t, configv1alpha3.AttachCurrentOrNext, f.calls[0].Attach) @@ -422,19 +422,19 @@ func TestSaveRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { // The pointer types exist so that an explicit zero survives defaulting, and each field keeps its // own meaning: a zero attachTimeout is not a zero maxDuration. -func TestSaveRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing.T) { - cr := newSaveRequest("save-now") +func TestCommitRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing.T) { + cr := newCommitRequest("save-now") cr.CreationTimestamp = metav1.Now() - cr.Spec.Window = &configv1alpha3.SaveRequestWindow{ + cr.Spec.Window = &configv1alpha3.CommitRequestWindow{ AttachTimeout: &metav1.Duration{}, IdleTimeout: &metav1.Duration{}, MaxDuration: &metav1.Duration{Duration: 30 * time.Second}, } - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") require.Len(t, f.calls, 1) assert.Zero(t, f.calls[0].AttachTimeout, "an explicit \"0s\" must not be re-read as an omitted field") @@ -446,78 +446,78 @@ func TestSaveRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing.T // The window is the worker's job: the controller does not hold the finalize itself. While the // worker has not resolved the attach, the controller polls and reports the phase the WORKER // reports, never one it inferred from having sent the attach. -func TestSaveRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) { - cr := newSaveRequest("save-linger") +func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) { + cr := newCommitRequest("save-linger") cr.CreationTimestamp = metav1.Now() - cr.Spec.Window = &configv1alpha3.SaveRequestWindow{MaxDuration: &metav1.Duration{Duration: 30 * time.Second}} - c := newSaveRequestClient(t, nil, cr) + cr.Spec.Window = &configv1alpha3.CommitRequestWindow{MaxDuration: &metav1.Duration{Duration: 30 * time.Second}} + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileSaveRequest(t, r, "save-linger") + res := reconcileCommitRequest(t, r, "save-linger") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "an unresolved attach must be polled, not held by a controller-side delay") require.Len(t, f.calls, 1, "the attach is sent the instant the author is known") assert.Equal(t, 30*time.Second, f.calls[0].MaxDuration, "the window is passed to the worker, not consumed here") - got := fetchSaveRequest(t, c, "save-linger") + got := fetchCommitRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, saveRequestIsTerminal(&got), "an unresolved request is not terminal") + assert.False(t, commitRequestIsTerminal(&got), "an unresolved request is not terminal") - for _, phase := range []git.SaveRequestPhase{ + for _, phase := range []git.CommitRequestPhase{ git.PhaseWaitingForWindow, git.PhaseCollectingWindow, git.PhaseWaitingForPush, } { f.result = git.FinalizeResult{Phase: phase} - reconcileSaveRequest(t, r, "save-linger") - got = fetchSaveRequest(t, c, "save-linger") + reconcileCommitRequest(t, r, "save-linger") + got = fetchCommitRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, string(phase)) requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, string(phase)) } // A phase the worker has not (yet) reported keeps what is shown rather than guessing. f.result = git.FinalizeResult{} - reconcileSaveRequest(t, r, "save-linger") - got = fetchSaveRequest(t, c, "save-linger") + reconcileCommitRequest(t, r, "save-linger") + got = fetchCommitRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, string(git.PhaseWaitingForPush)) } // A transient service error (e.g. the GitTarget momentarily unreadable) keeps the // request polling within the safety window rather than failing it outright. -func TestSaveRequestReconcile_ServiceErrorPolls(t *testing.T) { - cr := newSaveRequest("save-transient") +func TestCommitRequestReconcile_ServiceErrorPolls(t *testing.T) { + cr := newCommitRequest("save-transient") cr.CreationTimestamp = metav1.Now() - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{err: errors.New("get GitTarget: not found")} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileSaveRequest(t, r, "save-transient") + res := reconcileCommitRequest(t, r, "save-transient") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) - got := fetchSaveRequest(t, c, "save-transient") + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) + got := fetchCommitRequest(t, c, "save-transient") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, saveRequestIsTerminal(&got)) + assert.False(t, commitRequestIsTerminal(&got)) } // Past the safety window a request the worker never acted on is withdrawn, and fails closed once // the worker confirms the withdraw. -func TestSaveRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { - cr := pastTheBound(newSaveRequest("save-stuck")) - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { + cr := pastTheBound(newCommitRequest("save-stuck")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWindow}, - withdrawResult: git.FinalizeResult{Branch: "main", Err: git.ErrSaveRequestWithdrawn}, + withdrawResult: git.FinalizeResult{Branch: "main", Err: git.ErrCommitRequestWithdrawn}, withdrawResolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-stuck") + reconcileCommitRequest(t, r, "save-stuck") require.Len(t, f.withdrawCalls, 1, "the request is withdrawn before it is failed") assert.Empty(t, f.calls, "past the bound no attach is sent: it would queue behind the withdraw") - got := fetchSaveRequest(t, c, "save-stuck") + got := fetchCommitRequest(t, c, "save-stuck") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonFinalizeFailed) stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Equal(t, resolveTimeoutMessage, stalled.Message) @@ -525,27 +525,27 @@ func TestSaveRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { // A request the worker holds is never failed on the controller's clock: a down remote or a parent // branch recovery can keep it in WaitingForPush far past the bound, and the worker still commits it. -func TestSaveRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { - for _, phase := range []git.SaveRequestPhase{git.PhaseCollectingWindow, git.PhaseWaitingForPush} { +func TestCommitRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { + for _, phase := range []git.CommitRequestPhase{git.PhaseCollectingWindow, git.PhaseWaitingForPush} { t.Run(string(phase), func(t *testing.T) { - cr := pastTheBound(newSaveRequest("save-held")) - c := newSaveRequestClient(t, nil, cr) + cr := pastTheBound(newCommitRequest("save-held")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{result: git.FinalizeResult{Phase: phase}} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileSaveRequest(t, r, "save-held") + res := reconcileCommitRequest(t, r, "save-held") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) assert.Empty(t, f.calls, "past the bound no attach is sent") assert.Len(t, f.withdrawCalls, 1, "the withdraw finds the request held and changes nothing") - got := fetchSaveRequest(t, c, "save-held") - assert.False(t, saveRequestIsTerminal(&got)) + got := fetchCommitRequest(t, c, "save-held") + assert.False(t, commitRequestIsTerminal(&got)) // The push lands much later: the request resolves Committed, not FinalizeFailed. f.result = git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"} f.resolved = true - reconcileSaveRequest(t, r, "save-held") - got = fetchSaveRequest(t, c, "save-held") + reconcileCommitRequest(t, r, "save-held") + got = fetchCommitRequest(t, c, "save-held") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) }) } @@ -553,115 +553,115 @@ func TestSaveRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { // Until the worker answers the withdraw the request keeps polling; and when the worker turns out to // hold it after all (it attached between the poll and the withdraw), nothing fails. -func TestSaveRequestReconcile_AnUnansweredWithdrawKeepsPolling(t *testing.T) { +func TestCommitRequestReconcile_AnUnansweredWithdrawKeepsPolling(t *testing.T) { for name, reply := range map[string]git.FinalizeResult{ "not handled yet": {}, "the worker now holds it": {Phase: git.PhaseCollectingWindow}, } { t.Run(name, func(t *testing.T) { - cr := pastTheBound(newSaveRequest("save-race")) - c := newSaveRequestClient(t, nil, cr) + cr := pastTheBound(newCommitRequest("save-race")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWindow}, withdrawResult: reply, withdrawPending: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - res := reconcileSaveRequest(t, r, "save-race") + res := reconcileCommitRequest(t, r, "save-race") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter) - got := fetchSaveRequest(t, c, "save-race") - assert.False(t, saveRequestIsTerminal(&got)) + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) + got := fetchCommitRequest(t, c, "save-race") + assert.False(t, commitRequestIsTerminal(&got)) }) } } // A worker that resolved the request on its own just before the withdraw wins: its real outcome is // what the request reports. -func TestSaveRequestReconcile_AWithdrawThatLostTheRaceReportsTheRealOutcome(t *testing.T) { - cr := pastTheBound(newSaveRequest("save-late")) - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_AWithdrawThatLostTheRaceReportsTheRealOutcome(t *testing.T) { + cr := pastTheBound(newCommitRequest("save-late")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ withdrawResult: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, withdrawResolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-late") + reconcileCommitRequest(t, r, "save-late") - got := fetchSaveRequest(t, c, "save-late") + got := fetchCommitRequest(t, c, "save-late") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Equal(t, "abc", got.Status.Commit) } // A GitTarget that never started a branch worker is named as the cause. -func TestSaveRequestReconcile_NoWorkerIsNamedWhenItFailsClosed(t *testing.T) { - cr := pastTheBound(newSaveRequest("save-no-worker")) - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_NoWorkerIsNamedWhenItFailsClosed(t *testing.T) { + cr := pastTheBound(newCommitRequest("save-no-worker")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Phase: git.PhaseWaitingForWorker}, withdrawResult: git.FinalizeResult{ - Branch: "main", Phase: git.PhaseWaitingForWorker, Err: git.ErrSaveRequestWithdrawn, + Branch: "main", Phase: git.PhaseWaitingForWorker, Err: git.ErrCommitRequestWithdrawn, }, withdrawResolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-no-worker") + reconcileCommitRequest(t, r, "save-no-worker") - got := fetchSaveRequest(t, c, "save-no-worker") + got := fetchCommitRequest(t, c, "save-no-worker") stalled := requireCondition(t, got, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Equal(t, noWorkerTimeoutMessage, stalled.Message) } -func TestSaveRequestReconcile_TerminalShortCircuits(t *testing.T) { - cr := withReadyCommitted(newSaveRequest("save-done")) - c := newSaveRequestClient(t, nil, cr) +func TestCommitRequestReconcile_TerminalShortCircuits(t *testing.T) { + cr := withReadyCommitted(newCommitRequest("save-done")) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{} lookup := attributedAlice() - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} - reconcileSaveRequest(t, r, "save-done") + reconcileCommitRequest(t, r, "save-done") assert.Zero(t, lookup.calls) - assert.Empty(t, f.calls, "a terminal SaveRequest must never re-attach") + assert.Empty(t, f.calls, "a terminal CommitRequest must never re-attach") } // A reconcile triggered by a stale cache echo (the cached object still says // in-progress while the live object is already terminal) must not re-run the // finalize: the uncached APIReader read is the guard. -func TestSaveRequestReconcile_StaleCacheEchoDoesNotRefinalize(t *testing.T) { - stale := withInProgress(newSaveRequest("save-echo")) - cached := newSaveRequestClient(t, nil, stale) - terminal := withReadyCommitted(newSaveRequest("save-echo")) - live := newSaveRequestClient(t, nil, terminal) +func TestCommitRequestReconcile_StaleCacheEchoDoesNotRefinalize(t *testing.T) { + stale := withInProgress(newCommitRequest("save-echo")) + cached := newCommitRequestClient(t, nil, stale) + terminal := withReadyCommitted(newCommitRequest("save-echo")) + live := newCommitRequestClient(t, nil, terminal) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: cached, APIReader: live, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: cached, APIReader: live, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-echo") + reconcileCommitRequest(t, r, "save-echo") assert.Empty(t, f.calls) } -// Webhook-disabled mode (no AuthorLookup) never waits: a freshly created SaveRequest +// Webhook-disabled mode (no AuthorLookup) never waits: a freshly created CommitRequest // attaches immediately with a blank author and records AuthorAttributed=False // (AuthorCaptureDisabled). The matching window determines the Git author. -func TestSaveRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T) { - cr := newSaveRequest("save-configured-author") +func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T) { + cr := newCommitRequest("save-configured-author") cr.CreationTimestamp = metav1.Now() // fresh: a waiting path would requeue instead of commit - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil - res := reconcileSaveRequest(t, r, "save-configured-author") + res := reconcileCommitRequest(t, r, "save-configured-author") assert.Zero(t, res.RequeueAfter, "webhook-disabled mode must not requeue waiting for an author") - got := fetchSaveRequest(t, c, "save-configured-author") + got := fetchCommitRequest(t, c, "save-configured-author") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) @@ -673,55 +673,55 @@ func TestSaveRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T // Webhook-disabled mode settles AuthorAttributed=False (AuthorCaptureDisabled) immediately // and never parks the request in a "waiting for the author" state: even while the // attach is still being polled, the request reports progress. -func TestSaveRequestReconcile_ConfiguredAuthorAttributedImmediately(t *testing.T) { - cr := newSaveRequest("save-committer-poll") +func TestCommitRequestReconcile_ConfiguredAuthorAttributedImmediately(t *testing.T) { + cr := newCommitRequest("save-committer-poll") cr.CreationTimestamp = metav1.Now() - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil - res := reconcileSaveRequest(t, r, "save-committer-poll") + res := reconcileCommitRequest(t, r, "save-committer-poll") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, "an unresolved attach is polled") + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "an unresolved attach is polled") require.Len(t, f.calls, 1) assert.Empty(t, f.calls[0].Author) - got := fetchSaveRequest(t, c, "save-committer-poll") + got := fetchCommitRequest(t, c, "save-committer-poll") requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, saveRequestIsTerminal(&got), "in progress, not terminal") + assert.False(t, commitRequestIsTerminal(&got), "in progress, not terminal") } // With no Finalizer wired the controller is inert: it neither attaches, stamps any // condition, nor drives the request to a terminal outcome. (Production always wires // the Finalizer; this is the disabled guard at the top of Reconcile.) -func TestSaveRequestReconcile_NilFinalizerIsInert(t *testing.T) { - cr := newSaveRequest("save-no-finalizer") - c := newSaveRequestClient(t, nil, cr) - r := &SaveRequestReconciler{Client: c, APIReader: c} // Finalizer nil, AuthorLookup nil +func TestCommitRequestReconcile_NilFinalizerIsInert(t *testing.T) { + cr := newCommitRequest("save-no-finalizer") + c := newCommitRequestClient(t, nil, cr) + r := &CommitRequestReconciler{Client: c, APIReader: c} // Finalizer nil, AuthorLookup nil - res := reconcileSaveRequest(t, r, "save-no-finalizer") + res := reconcileCommitRequest(t, r, "save-no-finalizer") assert.Zero(t, res.RequeueAfter) - got := fetchSaveRequest(t, c, "save-no-finalizer") + got := fetchCommitRequest(t, c, "save-no-finalizer") assert.Empty(t, got.Status.Conditions, "a disabled controller stamps nothing") - assert.False(t, saveRequestIsTerminal(&got)) + assert.False(t, commitRequestIsTerminal(&got)) } -func TestSaveRequestReconcile_ObjectDeletedIsBenign(t *testing.T) { - c := newSaveRequestClient(t, nil) +func TestCommitRequestReconcile_ObjectDeletedIsBenign(t *testing.T) { + c := newCommitRequestClient(t, nil) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "gone") + reconcileCommitRequest(t, r, "gone") assert.Empty(t, f.calls) } -func TestSaveRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { +func TestCommitRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { // Already in-progress: this models a post-restart redelivery, so the only // status write is the terminal one — the write the conflict hits. - cr := withInProgress(newSaveRequest("save-retry")) + cr := withInProgress(newCommitRequest("save-retry")) conflicts := 1 fns := interceptor.Funcs{ @@ -735,32 +735,32 @@ func TestSaveRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { if conflicts > 0 { conflicts-- return apierrors.NewConflict( - schema.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}, + schema.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}, obj.GetName(), errors.New("simulated")) } return c.SubResource(subResource).Update(ctx, obj, opts...) }, } - c := newSaveRequestClient(t, &fns, cr) + c := newCommitRequestClient(t, &fns, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "ddd111", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-retry") + reconcileCommitRequest(t, r, "save-retry") - got := fetchSaveRequest(t, c, "save-retry") + got := fetchCommitRequest(t, c, "save-retry") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) assert.Equal(t, "ddd111", got.Status.Commit) require.Len(t, f.calls, 1, "the conflict retry must re-write status, not re-attach") } -// --- pure helpers (formerly internal/queue/save_request_test.go) --- +// --- pure helpers (formerly internal/queue/commit_request_test.go) --- func TestApplyFinalizeResultToStatus(t *testing.T) { t.Run("committed", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus( &cr, git.FinalizeResult{ @@ -781,7 +781,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("no window in grace is a benign ready (no admission author)", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, nil, attributionCommitter) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonNoWindow) @@ -791,7 +791,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("window mismatch surfaces the reason", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, nil, attributionFromAdmission) ready := requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonWindowMismatch) @@ -799,7 +799,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("already present is a benign ready", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, nil, attributionFromAdmission) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonAlreadyPresent) @@ -807,7 +807,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("finalize error stalls", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc"}, errors.New("boom"), attributionFromAdmission) @@ -817,7 +817,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) t.Run("unknown outcome stalls", func(t *testing.T) { - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{}, nil, attributionFromAdmission) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionFalse, crReasonUnexpectedOutcome) stalled := requireCondition(t, cr, ConditionTypeStalled, metav1.ConditionTrue, crReasonUnexpectedOutcome) @@ -825,30 +825,30 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { }) } -func TestSaveRequestIsTerminal(t *testing.T) { - assert.False(t, saveRequestIsTerminal(newSaveRequest("empty"))) - assert.False(t, saveRequestIsTerminal(withInProgress(newSaveRequest("in-progress")))) - assert.True(t, saveRequestIsTerminal(withReadyCommitted(newSaveRequest("committed")))) +func TestCommitRequestIsTerminal(t *testing.T) { + assert.False(t, commitRequestIsTerminal(newCommitRequest("empty"))) + assert.False(t, commitRequestIsTerminal(withInProgress(newCommitRequest("in-progress")))) + assert.True(t, commitRequestIsTerminal(withReadyCommitted(newCommitRequest("committed")))) - stalled := newSaveRequest("failed") + stalled := newCommitRequest("failed") apimeta.SetStatusCondition(&stalled.Status.Conditions, metav1.Condition{ Type: ConditionTypeStalled, Status: metav1.ConditionTrue, Reason: crReasonFinalizeFailed, Message: "boom", }) - assert.True(t, saveRequestIsTerminal(stalled)) + assert.True(t, commitRequestIsTerminal(stalled)) } -// A SaveRequest stored before spec.targetRef became spec.gitTargetRef serves an EMPTY name after +// A CommitRequest stored before spec.targetRef became spec.gitTargetRef serves an EMPTY name after // the upgrade prunes it, and its spec is immutable, so no apply can put the name back. The // reconciler must say that once and stop, rather than retry a "get GitTarget" that names the empty // string and can never succeed. -func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { - request := &configv1alpha3.SaveRequest{ +func TestCommitRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { + request := &configv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ Name: "stored-before-the-rename", Namespace: "team-a", UID: "cr-uid", }, - Spec: configv1alpha3.SaveRequestSpec{ + Spec: configv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: ""}, }, } @@ -862,7 +862,7 @@ func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { Build() finalizer := &fakeFinalizer{} - reconciler := &SaveRequestReconciler{Client: k8sClient, Scheme: scheme, Finalizer: finalizer} + reconciler := &CommitRequestReconciler{Client: k8sClient, Scheme: scheme, Finalizer: finalizer} result, err := reconciler.Reconcile(context.Background(), ctrl.Request{ NamespacedName: types.NamespacedName{Name: request.Name, Namespace: request.Namespace}, @@ -872,10 +872,10 @@ func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { assert.Zero(t, result.RequeueAfter, "it must not be retried: no apply can ever fix it") assert.Empty(t, finalizer.calls, "the router must never be asked for GitTarget \"\"") - var got configv1alpha3.SaveRequest + var got configv1alpha3.CommitRequest require.NoError(t, k8sClient.Get(context.Background(), types.NamespacedName{Name: request.Name, Namespace: request.Namespace}, &got)) - assert.True(t, saveRequestIsTerminal(&got), "the request must reach a terminal condition") + assert.True(t, commitRequestIsTerminal(&got), "the request must reach a terminal condition") stalled := findCondition(got.Status.Conditions, ConditionTypeStalled) require.NotNil(t, stalled) assert.Equal(t, metav1.ConditionTrue, stalled.Status) @@ -884,7 +884,7 @@ func TestSaveRequestReconciler_PrunedGitTargetRefIsTerminal(t *testing.T) { "the message must name the only repair there is, because the spec is immutable") } -func TestSaveRequestReconciler_LiteralValidation(t *testing.T) { +func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { for _, tc := range []struct { name, message string valid bool @@ -896,18 +896,18 @@ func TestSaveRequestReconciler_LiteralValidation(t *testing.T) { {"control", "bad\tmessage", false}, } { t.Run(tc.name, func(t *testing.T) { - cr := newSaveRequest(tc.name) + cr := newCommitRequest(tc.name) cr.Spec.Message = tc.message - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) finalizer := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, Finalizer: finalizer} - reconcileSaveRequest(t, r, cr.Name) + r := &CommitRequestReconciler{Client: c, Finalizer: finalizer} + reconcileCommitRequest(t, r, cr.Name) if tc.valid { require.Len(t, finalizer.calls, 1) assert.Equal(t, tc.message, finalizer.calls[0].Message) } else { require.Empty(t, finalizer.calls) - var got configv1alpha3.SaveRequest + var got configv1alpha3.CommitRequest require.NoError(t, c.Get(context.Background(), client.ObjectKeyFromObject(cr), &got)) assert.True(t, conditionIsTrue(got.Status.Conditions, ConditionTypeStalled)) } @@ -918,7 +918,7 @@ func TestSaveRequestReconciler_LiteralValidation(t *testing.T) { // A request claims a window whose writes name no author only when it asked, its GitTarget allows // what it asked, and the GitTarget's KubeProvider has never received an audit fact. A request that // asks for more than its GitTarget allows fails at once, before anything is attached. -func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { +func TestCommitRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { target := func(policy configv1alpha3.UnattributedWindowPolicy) *configv1alpha3.GitTarget { gt := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, @@ -927,7 +927,7 @@ func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaud }, } if policy != "" { - gt.Spec.AllowedSaveRequests = []configv1alpha3.GitTargetSaveRequestGrant{{ + gt.Spec.AllowedCommitRequests = []configv1alpha3.GitTargetCommitRequestGrant{{ ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromSame}, UnattributedWindows: policy, }} @@ -1013,17 +1013,17 @@ func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaud } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - cr := newSaveRequest("save") - cr.Spec.Window = &configv1alpha3.SaveRequestWindow{Author: tc.ask} - c := newSaveRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) + cr := newCommitRequest("save") + cr.Spec.Window = &configv1alpha3.CommitRequestWindow{Author: tc.ask} + c := newCommitRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") if tc.refused { assert.Empty(t, f.calls, "a refused request is never attached") - var got configv1alpha3.SaveRequest + var got configv1alpha3.CommitRequest require.NoError( t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got), @@ -1041,43 +1041,43 @@ func TestSaveRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaud } } -// A SaveRequest may name a GitTarget in another namespace, which saves for it only when a grant in its -// spec.allowedSaveRequests admits the request's namespace. Omitted, the list admits the GitTarget's own +// A CommitRequest may name a GitTarget in another namespace, which saves for it only when a grant in its +// spec.allowedCommitRequests admits the request's namespace. Omitted, the list admits the GitTarget's own // namespace alone; a refused request fails at once, before anything is attached. -func TestSaveRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) { - platformTarget := func(grants []configv1alpha3.GitTargetSaveRequestGrant) *configv1alpha3.GitTarget { +func TestCommitRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) { + platformTarget := func(grants []configv1alpha3.GitTargetCommitRequestGrant) *configv1alpha3.GitTarget { return &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "shared", Namespace: "platform"}, - Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: grants}, + Spec: configv1alpha3.GitTargetSpec{AllowedCommitRequests: grants}, } } - fromNames := func(names ...configv1alpha3.GrantNamespace) []configv1alpha3.GitTargetSaveRequestGrant { - return []configv1alpha3.GitTargetSaveRequestGrant{{ + fromNames := func(names ...configv1alpha3.GrantNamespace) []configv1alpha3.GitTargetCommitRequestGrant { + return []configv1alpha3.GitTargetCommitRequestGrant{{ ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromNames, Names: names}, }} } cases := []struct { name string - grants []configv1alpha3.GitTargetSaveRequestGrant + grants []configv1alpha3.GitTargetCommitRequestGrant admitted bool }{ {"the default admits only the GitTarget's own namespace", nil, false}, {"a grant for another namespace", fromNames("team-b"), false}, {"a grant for the request's namespace", fromNames("default"), true}, - {"every namespace", []configv1alpha3.GitTargetSaveRequestGrant{{ + {"every namespace", []configv1alpha3.GitTargetCommitRequestGrant{{ ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, }}, true}, - {"an empty list admits nobody", []configv1alpha3.GitTargetSaveRequestGrant{}, false}, + {"an empty list admits nobody", []configv1alpha3.GitTargetCommitRequestGrant{}, false}, } for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { - cr := newSaveRequest("save") + cr := newCommitRequest("save") cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "shared", Namespace: "platform"} - c := newSaveRequestClient(t, nil, cr, platformTarget(tc.grants)) + c := newCommitRequestClient(t, nil, cr, platformTarget(tc.grants)) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") if tc.admitted { require.NotEmpty(t, f.calls, "the request must be attached") @@ -1086,13 +1086,13 @@ func TestSaveRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) return } assert.Empty(t, f.calls, "a refused request is never attached") - var got configv1alpha3.SaveRequest + var got configv1alpha3.CommitRequest require.NoError(t, c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got)) ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) require.NotNil(t, ready) assert.Equal(t, metav1.ConditionFalse, ready.Status) assert.Equal(t, crReasonNotAllowed, ready.Reason) - assert.Contains(t, ready.Message, "spec.allowedSaveRequests") + assert.Contains(t, ready.Message, "spec.allowedCommitRequests") assert.True(t, apimeta.IsStatusConditionTrue(got.Status.Conditions, ConditionTypeStalled)) }) } @@ -1100,30 +1100,30 @@ func TestSaveRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T) // A request whose GitTarget does not exist is not attached: it waits, and once a GitTarget by that // name appears it must admit the request first. Its earlier in-progress conditions grant nothing. -func TestSaveRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testing.T) { - cr := newSaveRequest("save") +func TestCommitRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testing.T) { + cr := newCommitRequest("save") cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "later", Namespace: "platform"} - c := newSaveRequestClient(t, nil, cr, &configv1alpha3.GitTarget{ + c := newCommitRequestClient(t, nil, cr, &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "unrelated", Namespace: "platform"}, }) f := &fakeFinalizer{resolved: true, result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - res := reconcileSaveRequest(t, r, "save") + res := reconcileCommitRequest(t, r, "save") assert.Empty(t, f.calls, "nothing is attached before a GitTarget admitted the request") assert.Positive(t, res.RequeueAfter, "it waits for the GitTarget") - got := fetchSaveRequest(t, c, "save") + got := fetchCommitRequest(t, c, "save") require.NotNil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady), "it reports progress") assert.Empty(t, got.Status.GitTargetUID) require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "later", Namespace: "platform", UID: "uid-later"}, - Spec: configv1alpha3.GitTargetSpec{AllowedSaveRequests: []configv1alpha3.GitTargetSaveRequestGrant{}}, + Spec: configv1alpha3.GitTargetSpec{AllowedCommitRequests: []configv1alpha3.GitTargetCommitRequestGrant{}}, })) - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") assert.Empty(t, f.calls, "the GitTarget that appeared admits nobody") - got = fetchSaveRequest(t, c, "save") + got = fetchCommitRequest(t, c, "save") ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) require.NotNil(t, ready) assert.Equal(t, metav1.ConditionFalse, ready.Status) @@ -1132,26 +1132,26 @@ func TestSaveRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testing. // The GitTarget that admitted a request is recorded by UID, and a refusal by that same GitTarget // afterwards, once its grants were lowered, does not fail a request an attach may already have reached. -func TestSaveRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { - cr := newSaveRequest("save") +func TestCommitRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { + cr := newCommitRequest("save") target := &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}} - c := newSaveRequestClient(t, nil, cr, target) + c := newCommitRequestClient(t, nil, cr, target) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") require.Len(t, f.calls, 1) assert.Equal(t, string(target.UID), f.calls[0].GitTargetUID, "the attach is bound to the admitting GitTarget") - got := fetchSaveRequest(t, c, "save") + got := fetchCommitRequest(t, c, "save") assert.Equal(t, string(target.UID), got.Status.GitTargetUID) require.NoError(t, c.Get(context.Background(), client.ObjectKeyFromObject(target), target)) - target.Spec.AllowedSaveRequests = []configv1alpha3.GitTargetSaveRequestGrant{} + target.Spec.AllowedCommitRequests = []configv1alpha3.GitTargetCommitRequestGrant{} require.NoError(t, c.Update(context.Background(), target)) - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") assert.Len(t, f.calls, 2, "the attach is sent again, as before the grant was lowered") - got = fetchSaveRequest(t, c, "save") + got = fetchCommitRequest(t, c, "save") ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) require.NotNil(t, ready) assert.NotEqual(t, crReasonNotAllowed, ready.Reason) @@ -1160,7 +1160,7 @@ func TestSaveRequestAttach_ALoweredGrantKeepsARequestItAdmitted(t *testing.T) { // A GitTarget recreated with grants that refuse a request does not fail it while the worker the // GitTarget it replaced sent it to may hold it: that worker answers first. A save it committed is // reported as committed; one nothing held is refused; one it holds is waited for. -func TestSaveRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T) { +func TestCommitRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T) { tests := []struct { name string withdraw git.FinalizeResult @@ -1177,7 +1177,7 @@ func TestSaveRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T) }, { name: "nothing held it", - withdraw: git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, + withdraw: git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, resolved: true, wantReason: crReasonNotAllowed, }, @@ -1188,30 +1188,32 @@ func TestSaveRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T) } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - cr := newSaveRequest("save") + cr := newCommitRequest("save") target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default", UID: "uid-first"}, } - c := newSaveRequestClient(t, nil, cr, target) + c := newCommitRequestClient(t, nil, cr, target) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileSaveRequest(t, r, "save") + reconcileCommitRequest(t, r, "save") require.Len(t, f.calls, 1) require.NoError(t, c.Delete(context.Background(), target)) - refusing := configv1alpha3.GitTargetSpec{AllowedSaveRequests: []configv1alpha3.GitTargetSaveRequestGrant{}} + refusing := configv1alpha3.GitTargetSpec{ + AllowedCommitRequests: []configv1alpha3.GitTargetCommitRequestGrant{}, + } require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default", UID: "uid-second"}, Spec: refusing, })) f.withdrawResult, f.withdrawResolved, f.withdrawPending = tc.withdraw, tc.resolved, true - res := reconcileSaveRequest(t, r, "save") + res := reconcileCommitRequest(t, r, "save") assert.Len(t, f.calls, 1, "nothing is attached through the replacement") require.Len(t, f.withdrawCalls, 1) assert.Equal(t, "uid-first", f.withdrawCalls[0].GitTargetUID, "the worker is asked about the first one") - got := fetchSaveRequest(t, c, "save") + got := fetchCommitRequest(t, c, "save") assert.Equal(t, "uid-first", got.Status.GitTargetUID) ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) require.NotNil(t, ready) diff --git a/internal/controller/saverequest_finalize.go b/internal/controller/commitrequest_finalize.go similarity index 67% rename from internal/controller/saverequest_finalize.go rename to internal/controller/commitrequest_finalize.go index 880089b1..6c03f362 100644 --- a/internal/controller/saverequest_finalize.go +++ b/internal/controller/commitrequest_finalize.go @@ -16,7 +16,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/telemetry" ) -// SaveRequest condition reasons (CamelCase tokens surfaced on status.conditions). +// CommitRequest condition reasons (CamelCase tokens surfaced on status.conditions). const ( crReasonCommitted = "Committed" crReasonNoWindow = "NoWindow" @@ -33,7 +33,7 @@ const ( crReasonSaveInProgress = "SaveInProgress" ) -// The bounded `outcome` values on gitopsreverser_save_requests_total. +// The bounded `outcome` values on gitopsreverser_commit_requests_total. // // They are derived from git.FinalizeOutcome rather than from the condition reasons above: the // FinalizeResult is what DECIDES, and the reasons include non-terminal and attribution-only values @@ -48,13 +48,13 @@ const ( crOutcomeFailed = "failed" ) -// saveRequestOutcome maps one finalize result to its counter value. +// commitRequestOutcome maps one finalize result to its counter value. // // A finalize error outranks the outcome because FinalizeResult.Outcome is only set when Err is nil, // and an unrecognized outcome counts as failed for the same reason applyFinalizeResultToStatus // fails it: an empty or unknown outcome with no error is a bug, and a bug that increments nothing // is a bug nobody sees. -func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { +func commitRequestOutcome(result git.FinalizeResult, finalizeErr error) string { if finalizeErr != nil { return crOutcomeFailed } @@ -74,48 +74,48 @@ func saveRequestOutcome(result git.FinalizeResult, finalizeErr error) string { } } -// saveRequestTarget is the GitTarget identity one terminal outcome is counted under. +// commitRequestTarget is the GitTarget identity one terminal outcome is counted under. // // The zero value is the documented FALLBACK for a request whose target could not be resolved, and // it publishes empty labels rather than the requested name. That is deliberate: spec.gitTargetRef // is whatever the client wrote, so labelling an unresolved reference would let anyone with create -// rights on saverequests mint an unbounded number of series by naming targets that do not exist. +// rights on commitrequests mint an unbounded number of series by naming targets that do not exist. // The requested reference is still on the object's conditions and in the logs, which is where an // unbounded string belongs. -type saveRequestTarget struct { +type commitRequestTarget struct { namespace string name string } -// resolvedSaveRequestTarget names the GitTarget a request was serviced against. +// resolvedCommitRequestTarget names the GitTarget a request was serviced against. // // It is only correct to call once the workflow has RESOLVED that target, which -// EventRouter.ServiceSaveRequest does by getting the GitTarget and returning an error when it is +// EventRouter.ServiceCommitRequest does by getting the GitTarget and returning an error when it is // absent. So a nil service error is the proof that this identity exists, and no extra API lookup is // needed to bound the label — the plan's rule that metric recording must not block terminal status // on another failing call. // -// The key is SaveRequest.GitTargetKey, the GitTarget the attach names, which is the key -// ServiceSaveRequest gets. -func resolvedSaveRequestTarget(cr *configv1alpha3.SaveRequest) saveRequestTarget { +// The key is CommitRequest.GitTargetKey, the GitTarget the attach names, which is the key +// ServiceCommitRequest gets. +func resolvedCommitRequestTarget(cr *configv1alpha3.CommitRequest) commitRequestTarget { key := cr.GitTargetKey() - return saveRequestTarget{namespace: key.Namespace, name: key.Name} + return commitRequestTarget{namespace: key.Namespace, name: key.Name} } -// recordSaveRequestOutcome increments the terminal-outcome counter once. +// recordCommitRequestOutcome increments the terminal-outcome counter once. // // It is called at the point the outcome is SETTLED, never from inside applyFinalizeResultToStatus: -// that runs inside writeTerminalStatus's conflict-retry loop, up to saveRequestStatusUpdateAttempts +// that runs inside writeTerminalStatus's conflict-retry loop, up to commitRequestStatusUpdateAttempts // times for one request, so recording there would over-report exactly the contended case. The // counter measures what the pipeline decided; whether the terminal status then reached the API // server is a separate failure, already logged. // // See the instrument's doc comment for what this does and does not promise across reconciles. -func recordSaveRequestOutcome(ctx context.Context, outcome string, target saveRequestTarget) { - if telemetry.SaveRequestsTotal == nil { +func recordCommitRequestOutcome(ctx context.Context, outcome string, target commitRequestTarget) { + if telemetry.CommitRequestsTotal == nil { return } - telemetry.SaveRequestsTotal.Add(ctx, 1, metric.WithAttributes( + telemetry.CommitRequestsTotal.Add(ctx, 1, metric.WithAttributes( attribute.String("outcome", outcome), attribute.String("gittarget_namespace", target.namespace), attribute.String("gittarget_name", target.name), @@ -132,18 +132,18 @@ const noWindowMessage = "no matching commit window was there to attach to before const alreadyPresentMessage = "the change already matches the remote, so no commit was made" // notStalledMessage is the boilerplate message for a False Stalled condition. -const notStalledMessage = "the SaveRequest is not stalled" +const notStalledMessage = "the CommitRequest is not stalled" -// saveRequestAttribution is the settled author decision threaded from the +// commitRequestAttribution is the settled author decision threaded from the // attribute step into the terminal status write so the AuthorAttributed condition can // record whether the commit author was named. It is binary and always settled at // first sight (present-or-never, §2) — there is no pending or timed-out state. -type saveRequestAttribution int +type commitRequestAttribution int const ( // attributionUnset is the defensive zero value. It must never report a resolved // actor when a caller neglected to initialize the attribution decision. - attributionUnset saveRequestAttribution = iota + attributionUnset commitRequestAttribution = iota // attributionFromAdmission means the submitter was captured at admission by the // validate-operator-types webhook and named as the commit author. attributionFromAdmission @@ -159,7 +159,7 @@ const ( // gitOutcome projects the controller's attribution state onto the git layer's outcome, which // is what the worker matches windows on. Keeping the projection here — rather than passing the // controller's own enum down — means the git package never learns about admission records. -func (a saveRequestAttribution) gitOutcome() git.AttributionOutcome { +func (a commitRequestAttribution) gitOutcome() git.AttributionOutcome { switch a { case attributionUnset: return git.AttributionNotAttempted @@ -174,10 +174,10 @@ func (a saveRequestAttribution) gitOutcome() git.AttributionOutcome { } } -// setSaveRequestCondition upserts a condition keyed by type, stamping it with +// setCommitRequestCondition upserts a condition keyed by type, stamping it with // the request's generation so kstatus can tell a current status from a stale one. -func setSaveRequestCondition( - cr *configv1alpha3.SaveRequest, +func setCommitRequestCondition( + cr *configv1alpha3.CommitRequest, conditionType string, status metav1.ConditionStatus, reason, message string, @@ -200,7 +200,7 @@ const ( // progressFor maps a worker-reported phase to its progress reason and message. held is why a // request waiting for its push has not reached the remote yet: the branch worker's publication // report, in the words its GitTarget uses. -func progressFor(phase git.SaveRequestPhase, held string) (string, string) { +func progressFor(phase git.CommitRequestPhase, held string) (string, string) { switch phase { case git.PhaseWaitingForWorker: return string(phase), waitingForWorkerMessage @@ -218,54 +218,54 @@ func progressFor(phase git.SaveRequestPhase, held string) (string, string) { } } -// setSaveRequestProgress stamps the four progress conditions (Ready=False, +// setCommitRequestProgress stamps the four progress conditions (Ready=False, // Reconciling=True, Stalled=False, Pushed=Unknown) with one unifying reason, so a // single wait reads consistently across them. -func setSaveRequestProgress(cr *configv1alpha3.SaveRequest, reason, message string) { - setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) - setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionTrue, reason, message) - setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionUnknown, reason, pushPendingMessage) +func setCommitRequestProgress(cr *configv1alpha3.CommitRequest, reason, message string) { + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionTrue, reason, message) + setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionUnknown, reason, pushPendingMessage) } -// markSaveRequestProgressing records the still-running state for a worker-reported phase +// markCommitRequestProgressing records the still-running state for a worker-reported phase // (Reconciling=True). An empty phase is the generic Progressing: sent, not yet confirmed. -func markSaveRequestProgressing( - cr *configv1alpha3.SaveRequest, - attribution saveRequestAttribution, - phase git.SaveRequestPhase, +func markCommitRequestProgressing( + cr *configv1alpha3.CommitRequest, + attribution commitRequestAttribution, + phase git.CommitRequestPhase, held string, ) { cr.Status.ObservedGeneration = cr.Generation - setSaveRequestAttributed(cr, attribution) + setCommitRequestAttributed(cr, attribution) reason, message := progressFor(phase, held) - setSaveRequestProgress(cr, reason, message) + setCommitRequestProgress(cr, reason, message) } -// setSaveRequestAttributed records the settled, binary author decision on the +// setCommitRequestAttributed records the settled, binary author decision on the // AuthorAttributed condition. A False reason distinguishes an absent record // (CommitterFallback) from capture disabled (AuthorCaptureDisabled). Neither is a failure or // affects Ready; the request claims no actor. -func setSaveRequestAttributed(cr *configv1alpha3.SaveRequest, attribution saveRequestAttribution) { +func setCommitRequestAttributed(cr *configv1alpha3.CommitRequest, attribution commitRequestAttribution) { switch attribution { case attributionUnset: - setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled, "command-author capture state was unset; the request can attach only to a window with no named actor") case attributionFromAdmission: - setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, + setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission, "the submitter was captured at admission and named as the commit author") case attributionCommitter: // Capture RAN and found nothing. Distinct from attributionNotAttempted below, whose // message this used to carry — reporting "the webhook is not configured" for a // webhook that is configured and simply missed sends operators to the wrong place. - setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback, "no admission author record was found for this request; it can attach only to a "+ "window with no named actor") case attributionNotAttempted: - setSaveRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, + setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled, "command-author capture is disabled (the validate-operator-types webhook is not "+ "configured); the request can attach only to a window with no named actor") @@ -273,28 +273,28 @@ func setSaveRequestAttributed(cr *configv1alpha3.SaveRequest, attribution saveRe } // applyFinalizeResultToStatus maps a FinalizeResult (or a finalize error) and the -// settled attribution onto a SaveRequest's terminal conditions. Ready is the +// settled attribution onto a CommitRequest's terminal conditions. Ready is the // summary; Reconciling/Stalled carry kstatus; Attributed records the author // decision; Pushed records whether a commit reached the remote. func applyFinalizeResultToStatus( - cr *configv1alpha3.SaveRequest, + cr *configv1alpha3.CommitRequest, result git.FinalizeResult, finalizeErr error, - attribution saveRequestAttribution, + attribution commitRequestAttribution, ) { cr.Status.ObservedGeneration = cr.Generation cr.Status.Branch = result.Branch - setSaveRequestAttributed(cr, attribution) + setCommitRequestAttributed(cr, attribution) if finalizeErr != nil { reason := crReasonFinalizeFailed switch { case errors.Is(finalizeErr, errTakeOverNotAllowed): reason = crReasonTakeOverNotAllowed - case errors.Is(finalizeErr, errSaveRequestNotAllowed): + case errors.Is(finalizeErr, errCommitRequestNotAllowed): reason = crReasonNotAllowed } - failSaveRequest(cr, reason, finalizeErr.Error()) + failCommitRequest(cr, reason, finalizeErr.Error()) return } @@ -302,29 +302,29 @@ func applyFinalizeResultToStatus( case git.FinalizeCommitted: cr.Status.Commit = result.Commit const committedMsg = "the open commit window was closed, committed, and pushed" - setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, crReasonCommitted, committedMsg) - setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, crReasonCommitted, notStalledMessage) - setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, + setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, crReasonCommitted, committedMsg) + setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, crReasonCommitted, notStalledMessage) + setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, "the commit was pushed to the remote repository") - setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted, committedMsg) + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted, committedMsg) case git.FinalizeNoOpenWindow: // Benign: the attach timeout ran out with nothing pending to save. resolveNothingToCommit(cr, result, crReasonNoWindow, noWindowMessage) case git.FinalizeWindowMismatch: // The author-bound refusal: deliberately not a failure — the foreign // window stays open for its own author — but the reason is surfaced. - rejectSaveRequest(cr, crReasonWindowMismatch, windowMismatchMessage) + rejectCommitRequest(cr, crReasonWindowMismatch, windowMismatchMessage) case git.FinalizeAlreadyPresent: // The change already matches the remote, so the commit was dropped. resolveNothingToCommit(cr, result, crReasonAlreadyPresent, alreadyPresentMessage) case git.FinalizeSaveInProgress: // Refused, not failed by the worker: the spec is immutable, so the caller saves again with a // new request once the earlier one resolved. - failSaveRequest(cr, crReasonSaveInProgress, saveInProgressMessage) + failCommitRequest(cr, crReasonSaveInProgress, saveInProgressMessage) default: // An empty or unknown outcome with no error is a bug, not a benign // rejection; fail it so it is not silently hidden. - failSaveRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) + failCommitRequest(cr, crReasonUnexpectedOutcome, "unexpected finalize outcome: "+string(result.Outcome)) } } @@ -335,58 +335,58 @@ const recordedEmptyMessage = "; the message was recorded in an empty commit" // resolveNothingToCommit records an outcome with nothing to commit. Without a commit it is a benign // rejection. With one, whenNothingToCommit: CommitEmpty recorded the message: the reason keeps the // cause, and status.commit and Pushed=True say the empty commit reached the remote. -func resolveNothingToCommit(cr *configv1alpha3.SaveRequest, result git.FinalizeResult, reason, message string) { +func resolveNothingToCommit(cr *configv1alpha3.CommitRequest, result git.FinalizeResult, reason, message string) { if result.Commit == "" { - rejectSaveRequest(cr, reason, message) + rejectCommitRequest(cr, reason, message) return } cr.Status.Commit = result.Commit message += recordedEmptyMessage - setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, + setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded, "the empty commit was pushed to the remote repository") - setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) } -// rejectSaveRequest records a benign terminal outcome that produced no commit: +// rejectCommitRequest records a benign terminal outcome that produced no commit: // Ready=True (the request was serviced correctly, with the specific reason), // Pushed=False, and Stalled=False — kstatus Current, not Failed. -func rejectSaveRequest(cr *configv1alpha3.SaveRequest, reason, message string) { - setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) - setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, message) - setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) +func rejectCommitRequest(cr *configv1alpha3.CommitRequest, reason, message string) { + setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) + setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionTrue, reason, message) } -// failSaveRequest records a hard terminal failure: Ready=False, Pushed=False, and +// failCommitRequest records a hard terminal failure: Ready=False, Pushed=False, and // Stalled=True — kstatus Failed, a human-fixable block. -func failSaveRequest(cr *configv1alpha3.SaveRequest, reason, message string) { - setSaveRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) - setSaveRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, "no commit was pushed") - setSaveRequestCondition(cr, ConditionTypeStalled, metav1.ConditionTrue, reason, message) - setSaveRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) +func failCommitRequest(cr *configv1alpha3.CommitRequest, reason, message string) { + setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionFalse, reason, "no commit was pushed") + setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionTrue, reason, message) + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) } -// saveRequestIsTerminal reports whether the request has reached a terminal +// commitRequestIsTerminal reports whether the request has reached a terminal // outcome: Ready=True (committed or benignly rejected) or Stalled=True (failed). -func saveRequestIsTerminal(cr *configv1alpha3.SaveRequest) bool { +func commitRequestIsTerminal(cr *configv1alpha3.CommitRequest) bool { return conditionIsTrue(cr.Status.Conditions, ConditionTypeReady) || conditionIsTrue(cr.Status.Conditions, ConditionTypeStalled) } -// saveRequestConditionStatus returns the status of one condition as a loggable +// commitRequestConditionStatus returns the status of one condition as a loggable // string, or "" when the condition is absent. -func saveRequestConditionStatus(cr *configv1alpha3.SaveRequest, conditionType string) string { +func commitRequestConditionStatus(cr *configv1alpha3.CommitRequest, conditionType string) string { if c := apimeta.FindStatusCondition(cr.Status.Conditions, conditionType); c != nil { return string(c.Status) } return "" } -// saveRequestReadyReason returns the Ready condition's reason and message for +// commitRequestReadyReason returns the Ready condition's reason and message for // logging (both empty when the condition is absent). -func saveRequestReadyReason(cr *configv1alpha3.SaveRequest) (string, string) { +func commitRequestReadyReason(cr *configv1alpha3.CommitRequest) (string, string) { if c := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady); c != nil { return c.Reason, c.Message } diff --git a/internal/controller/saverequest_kstatus_test.go b/internal/controller/commitrequest_kstatus_test.go similarity index 92% rename from internal/controller/saverequest_kstatus_test.go rename to internal/controller/commitrequest_kstatus_test.go index 8ab71e84..c84221dc 100644 --- a/internal/controller/saverequest_kstatus_test.go +++ b/internal/controller/commitrequest_kstatus_test.go @@ -13,10 +13,10 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/git" ) -// TestSaveRequestKstatusContract pins the SaveRequest condition set to the +// TestCommitRequestKstatusContract pins the CommitRequest condition set to the // kstatus the GitOps tooling derives from it: Reconciling=True → InProgress, // Stalled=True → Failed, otherwise Current (with observedGeneration current). -func TestSaveRequestKstatusContract(t *testing.T) { +func TestCommitRequestKstatusContract(t *testing.T) { tests := []struct { name string conds []map[string]interface{} @@ -96,7 +96,7 @@ func TestSaveRequestKstatusContract(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - result, err := kstatus.Compute(saveRequestStatusObject(tt.conds)) + result, err := kstatus.Compute(commitRequestStatusObject(tt.conds)) require.NoError(t, err) assert.Equal(t, tt.wantStatus, result.Status) if tt.wantMsg != "" { @@ -106,10 +106,10 @@ func TestSaveRequestKstatusContract(t *testing.T) { } } -func saveRequestStatusObject(conditions []map[string]interface{}) *unstructured.Unstructured { +func commitRequestStatusObject(conditions []map[string]interface{}) *unstructured.Unstructured { return &unstructured.Unstructured{Object: map[string]interface{}{ "apiVersion": "configbutler.ai/v1alpha3", - "kind": "SaveRequest", + "kind": "CommitRequest", "metadata": map[string]interface{}{ "name": "save-a", "namespace": "default", diff --git a/internal/controller/saverequest_outcome_metric_test.go b/internal/controller/commitrequest_outcome_metric_test.go similarity index 74% rename from internal/controller/saverequest_outcome_metric_test.go rename to internal/controller/commitrequest_outcome_metric_test.go index ff6fc277..d03dc32f 100644 --- a/internal/controller/saverequest_outcome_metric_test.go +++ b/internal/controller/commitrequest_outcome_metric_test.go @@ -29,12 +29,12 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/watch" ) -const saveRequestsMetric = "gitopsreverser_save_requests_total" +const commitRequestsMetric = "gitopsreverser_commit_requests_total" // outcomeCount reads one bounded outcome's counter value, and reports whether the series exists. func outcomeCount(t *testing.T, reader *sdkmetric.ManualReader, outcome string) (int64, bool) { t.Helper() - return telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{"outcome": outcome}) + return telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{"outcome": outcome}) } // outcomeCountForTarget reads one outcome's counter for one GitTarget identity. The empty pair is @@ -45,7 +45,7 @@ func outcomeCountForTarget( outcome, namespace, name string, ) (int64, bool) { t.Helper() - return telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{ + return telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{ "outcome": outcome, "gittarget_namespace": namespace, "gittarget_name": name, @@ -54,20 +54,20 @@ func outcomeCountForTarget( // The reason the label was added: two targets' saves must be tellable apart, so an operator can // alert on one tenant's target rather than on the fleet. -func TestSaveRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { +func TestCommitRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) for _, target := range []string{"team-a-config", "team-b-config"} { - cr := newSaveRequest("save-" + target) + cr := newCommitRequest("save-" + target) cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target} - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-"+target) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileCommitRequest(t, r, "save-"+target) } for _, target := range []string{"team-a-config", "team-b-config"} { @@ -79,19 +79,19 @@ func TestSaveRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { // Cardinality is bounded by configuration rather than by saves: repeated saves against one target // reuse one series. That is the property that makes the label affordable. -func TestSaveRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { +func TestCommitRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) for i := range 3 { name := fmt.Sprintf("save-repeat-%d", i) - c := newSaveRequestClient(t, nil, newSaveRequest(name)) + c := newCommitRequestClient(t, nil, newCommitRequest(name)) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, name) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + reconcileCommitRequest(t, r, name) } value, ok := outcomeCountForTarget(t, reader, crOutcomeCommitted, "default", "team-a-config") @@ -101,33 +101,33 @@ func TestSaveRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { // The bound on the label. spec.gitTargetRef is whatever the client wrote, so a request naming a // target that does not exist must NOT mint a series under that name: anyone with create rights on -// saverequests could otherwise grow the metric without limit. The requested reference stays on +// commitrequests could otherwise grow the metric without limit. The requested reference stays on // the conditions and in the logs, which is where an unbounded string belongs. -func TestSaveRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { +func TestCommitRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - cr := newSaveRequest("save-bogus") + cr := newCommitRequest("save-bogus") cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "no-such-target"} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(cr.Spec).resolveTimeout())) - c := newSaveRequestClient(t, nil, cr) - // A service error is what EventRouter.ServiceSaveRequest returns when the GitTarget Get fails, + c := newCommitRequestClient(t, nil, cr) + // A service error is what EventRouter.ServiceCommitRequest returns when the GitTarget Get fails, // and a missing GitTarget has no worker to hold the request, so the withdraw resolves at once. f := &fakeFinalizer{ err: errors.New("get GitTarget default/no-such-target: not found"), - withdrawResult: git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, + withdrawResult: git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, withdrawResolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-bogus") + reconcileCommitRequest(t, r, "save-bogus") value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "", "") require.True(t, ok, "an unresolved target must still be counted, under the fallback identity") assert.Equal(t, int64(1), value) - _, minted := telemetry.CollectInt64Sum(reader, saveRequestsMetric, map[string]string{ + _, minted := telemetry.CollectInt64Sum(reader, commitRequestsMetric, map[string]string{ "gittarget_name": "no-such-target", }) assert.False(t, minted, "a name that resolved to nothing must never become a label value") @@ -136,7 +136,7 @@ func TestSaveRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { // The mapping is derived from FinalizeResult rather than from the condition reasons, so pin it // directly. The default arm matters most: an unmapped outcome must count as failed, because a bug // that increments nothing is a bug nobody sees. -func TestSaveRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { +func TestCommitRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { tests := []struct { name string result git.FinalizeResult @@ -159,14 +159,14 @@ func TestSaveRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - assert.Equal(t, tc.want, saveRequestOutcome(tc.result, tc.err)) + assert.Equal(t, tc.want, commitRequestOutcome(tc.result, tc.err)) }) } } // Every terminal outcome must reach the counter through a real reconcile, not only through the // mapping function. A value with no raise site is a label nobody can alert on. -func TestSaveRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) { +func TestCommitRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) { tests := []struct { name string result git.FinalizeResult @@ -199,11 +199,11 @@ func TestSaveRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) reader, err := telemetry.InitTestExporter() require.NoError(t, err) - c := newSaveRequestClient(t, nil, newSaveRequest("save-outcome")) + c := newCommitRequestClient(t, nil, newCommitRequest("save-outcome")) f := &fakeFinalizer{result: tc.result, err: tc.err, resolved: true} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-outcome") + reconcileCommitRequest(t, r, "save-outcome") value, ok := outcomeCount(t, reader, tc.want) require.True(t, ok, "outcome %q published no series", tc.want) @@ -215,7 +215,7 @@ func TestSaveRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing.T) // The trap the recording site exists to avoid. applyFinalizeResultToStatus is called once per // attempt inside writeTerminalStatus's conflict-retry loop, so recording there would over-report // exactly the contended request. Two conflicts, one increment. -func TestSaveRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { +func TestCommitRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -231,20 +231,20 @@ func TestSaveRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { if conflicts > 0 { conflicts-- return apierrors.NewConflict( - schema.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}, + schema.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}, obj.GetName(), errors.New("simulated")) } return c.SubResource(subResource).Update(ctx, obj, opts...) }, } - c := newSaveRequestClient(t, &fns, withInProgress(newSaveRequest("save-conflict"))) + c := newCommitRequestClient(t, &fns, withInProgress(newCommitRequest("save-conflict"))) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-conflict") + reconcileCommitRequest(t, r, "save-conflict") value, ok := outcomeCount(t, reader, crOutcomeCommitted) require.True(t, ok) @@ -254,8 +254,8 @@ func TestSaveRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { // The documented limit, asserted as behavior rather than left to be discovered. A status write that // fails permanently is not retried into a terminal state, so the next reconcile re-decides and // increments again. This is why the instrument promises one increment per terminal DECISION -// ATTEMPT and not one per SaveRequest; deduplicating would need a durable marker on the object. -func TestSaveRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing.T) { +// ATTEMPT and not one per CommitRequest; deduplicating would need a durable marker on the object. +func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -270,15 +270,15 @@ func TestSaveRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing return apierrors.NewInternalError(errors.New("status write is down")) }, } - c := newSaveRequestClient(t, &fns, withInProgress(newSaveRequest("save-lost"))) + c := newCommitRequestClient(t, &fns, withInProgress(newCommitRequest("save-lost"))) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-lost") - reconcileSaveRequest(t, r, "save-lost") + reconcileCommitRequest(t, r, "save-lost") + reconcileCommitRequest(t, r, "save-lost") value, ok := outcomeCount(t, reader, crOutcomeCommitted) require.True(t, ok) @@ -289,24 +289,24 @@ func TestSaveRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testing // refusePrunedGitTargetRef reaches a terminal state without passing through writeTerminalStatus. // It is rare and migration-only, which is exactly the kind of terminal state that must not be // missing from the counter. -func TestSaveRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { +func TestCommitRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - pruned := &configv1alpha3.SaveRequest{ + pruned := &configv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{Name: "save-pruned", Namespace: "default", UID: types.UID("uid-save-pruned")}, - Spec: configv1alpha3.SaveRequestSpec{ + Spec: configv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: ""}, Message: "save: pruned", }, } - c := newSaveRequestClient(t, nil, pruned) + c := newCommitRequestClient(t, nil, pruned) f := &fakeFinalizer{} - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-pruned") + reconcileCommitRequest(t, r, "save-pruned") - got := fetchSaveRequest(t, c, "save-pruned") + got := fetchCommitRequest(t, c, "save-pruned") requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, crReasonGitTargetRefPruned) value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "", "") @@ -315,41 +315,41 @@ func TestSaveRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { require.Empty(t, f.calls, "a request naming no GitTarget must never reach the finalizer") } -// A recreated SaveRequest is a new incarnation of the same save, and each one is its own +// A recreated CommitRequest is a new incarnation of the same save, and each one is its own // decision. Counting per incarnation is what makes a rate over the counter meaningful. -func TestSaveRequestMetric_RecreatedRequestCountsPerIncarnation(t *testing.T) { +func TestCommitRequestMetric_RecreatedRequestCountsPerIncarnation(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) - c := newSaveRequestClient(t, nil, newSaveRequest("save-again")) + c := newCommitRequestClient(t, nil, newCommitRequest("save-again")) f := &fakeFinalizer{ result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} - reconcileSaveRequest(t, r, "save-again") + reconcileCommitRequest(t, r, "save-again") // Delete and recreate under a fresh UID, as a second save from the same editor would. - first := fetchSaveRequest(t, c, "save-again") + first := fetchCommitRequest(t, c, "save-again") require.NoError(t, c.Delete(context.Background(), &first)) - recreated := newSaveRequest("save-again") + recreated := newCommitRequest("save-again") recreated.UID = types.UID("uid-save-again-2") require.NoError(t, c.Create(context.Background(), recreated)) - reconcileSaveRequest(t, r, "save-again") + reconcileCommitRequest(t, r, "save-again") value, ok := outcomeCount(t, reader, crOutcomeWindowMismatch) require.True(t, ok) assert.Equal(t, int64(2), value) } -// TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels runs the real EventRouter against a +// TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels runs the real EventRouter against a // GitTarget that exists but whose branch worker never starts. The request waits in // WaitingForWorker, then fails closed at the safety bound, and that failure must be counted under // its GitTarget: the target DID resolve, so the empty fallback pair would hide a real tenant's // failed save from every per-target dashboard and alert. -func TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { +func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { reader, err := telemetry.InitTestExporter() require.NoError(t, err) @@ -360,28 +360,28 @@ func TestSaveRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { GitBranchRef: meta.NamespacedObjectReference{Name: gitBranchNameForTest("team-a-provider", "main")}, }, } - fresh := newSaveRequest("save-fresh") + fresh := newCommitRequest("save-fresh") fresh.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target.Name} fresh.CreationTimestamp = metav1.Now() - stale := newSaveRequest("save-stale") + stale := newCommitRequest("save-stale") stale.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: target.Name} // Older than the resolve timeout, so the poll gives up on this pass rather than requeueing. stale.CreationTimestamp = metav1.NewTime(time.Now().Add(-2 * resolveRequestWindow(stale.Spec).resolveTimeout())) - c := newSaveRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) + c := newCommitRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, itypes.SensitiveResourcePolicy{}) router := watch.NewEventRouter(workers, nil, c, logr.Discard()) - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} // Inside the safety window: pending, and the status says why. - res := reconcileSaveRequest(t, r, fresh.Name) + res := reconcileCommitRequest(t, r, fresh.Name) assert.Positive(t, res.RequeueAfter, "a missing worker is polled, not resolved") reconciling := apimeta.FindStatusCondition( - fetchSaveRequest(t, c, fresh.Name).Status.Conditions, ConditionTypeReconciling) + fetchCommitRequest(t, c, fresh.Name).Status.Conditions, ConditionTypeReconciling) require.NotNil(t, reconciling) assert.Equal(t, string(git.PhaseWaitingForWorker), reconciling.Reason) // Past it: failed, under the GitTarget that resolved. - reconcileSaveRequest(t, r, stale.Name) + reconcileCommitRequest(t, r, stale.Name) value, ok := outcomeCountForTarget(t, reader, crOutcomeFailed, "default", target.Name) require.True(t, ok, "the failure is counted under the GitTarget it resolved to") diff --git a/internal/controller/saverequest_ttl_test.go b/internal/controller/commitrequest_ttl_test.go similarity index 51% rename from internal/controller/saverequest_ttl_test.go rename to internal/controller/commitrequest_ttl_test.go index b06d139b..5f84ba02 100644 --- a/internal/controller/saverequest_ttl_test.go +++ b/internal/controller/commitrequest_ttl_test.go @@ -19,15 +19,15 @@ import ( ) // finishedWithDeleteAfter is a Committed request carrying the given delete-after value. -func finishedWithDeleteAfter(name, deleteAfter string) *configv1alpha3.SaveRequest { - cr := withReadyCommitted(newSaveRequest(name)) - cr.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: deleteAfter} +func finishedWithDeleteAfter(name, deleteAfter string) *configv1alpha3.CommitRequest { + cr := withReadyCommitted(newCommitRequest(name)) + cr.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: deleteAfter} return cr } -func saveRequestExists(t *testing.T, c client.Client, name string) bool { +func commitRequestExists(t *testing.T, c client.Client, name string) bool { t.Helper() - var cr configv1alpha3.SaveRequest + var cr configv1alpha3.CommitRequest err := c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: name}, &cr) if apierrors.IsNotFound(err) { return false @@ -45,146 +45,146 @@ func committedFinalizer() *fakeFinalizer { // Finishing is what writes the deletion time, so it is on the object for anyone to read or change // from the moment the outcome is. -func TestSaveRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { - c := newSaveRequestClient(t, nil, newSaveRequest("save-now")) - r := &SaveRequestReconciler{ +func TestCommitRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) + r := &CommitRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: 48 * time.Hour, } - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") - got := fetchSaveRequest(t, c, "save-now") - value, ok := got.Annotations[SaveRequestDeleteAfterAnnotation] + got := fetchCommitRequest(t, c, "save-now") + value, ok := got.Annotations[CommitRequestDeleteAfterAnnotation] require.True(t, ok, "a finished request carries its deletion time") deleteAfter, err := time.Parse(time.RFC3339, value) require.NoError(t, err) assert.WithinDuration(t, time.Now().Add(48*time.Hour), deleteAfter, time.Minute) } -func TestSaveRequestTTL_ZeroStampsNothing(t *testing.T) { - c := newSaveRequestClient(t, nil, newSaveRequest("save-now")) - r := &SaveRequestReconciler{ +func TestCommitRequestTTL_ZeroStampsNothing(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) + r := &CommitRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), } - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") - assert.NotContains(t, fetchSaveRequest(t, c, "save-now").Annotations, SaveRequestDeleteAfterAnnotation) + assert.NotContains(t, fetchCommitRequest(t, c, "save-now").Annotations, CommitRequestDeleteAfterAnnotation) } // A request that finished before the annotation existed, or one someone removed it from, is kept: // only the transition to terminal stamps, never a later look at a terminal request. -func TestSaveRequestTTL_AFinishedRequestWithoutTheAnnotationIsKept(t *testing.T) { - c := newSaveRequestClient(t, nil, withReadyCommitted(newSaveRequest("save-before"))) - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}, TTL: time.Nanosecond} +func TestCommitRequestTTL_AFinishedRequestWithoutTheAnnotationIsKept(t *testing.T) { + c := newCommitRequestClient(t, nil, withReadyCommitted(newCommitRequest("save-before"))) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}, TTL: time.Nanosecond} - res := reconcileSaveRequest(t, r, "save-before") + res := reconcileCommitRequest(t, r, "save-before") assert.Zero(t, res.RequeueAfter) - got := fetchSaveRequest(t, c, "save-before") - assert.NotContains(t, got.Annotations, SaveRequestDeleteAfterAnnotation, "it is not stamped after the fact") + got := fetchCommitRequest(t, c, "save-before") + assert.NotContains(t, got.Annotations, CommitRequestDeleteAfterAnnotation, "it is not stamped after the fact") } -func TestSaveRequestTTL_DeletesOnceTheTimeHasPassed(t *testing.T) { +func TestCommitRequestTTL_DeletesOnceTheTimeHasPassed(t *testing.T) { past := time.Now().Add(-time.Minute).UTC().Format(time.RFC3339) - c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-old", past)) - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} + c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-old", past)) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileSaveRequest(t, r, "save-old") + res := reconcileCommitRequest(t, r, "save-old") assert.Zero(t, res.RequeueAfter) - assert.False(t, saveRequestExists(t, c, "save-old"), + assert.False(t, commitRequestExists(t, c, "save-old"), "the annotation alone decides; a TTL of zero now does not reach back to it") } // The request comes back when its time is up rather than on a poll, so a namespace full of recent // saves costs one requeue each and nothing in between. -func TestSaveRequestTTL_ComesBackWhenTheTimeIsUp(t *testing.T) { +func TestCommitRequestTTL_ComesBackWhenTheTimeIsUp(t *testing.T) { future := time.Now().Add(47 * time.Hour).UTC().Format(time.RFC3339) - c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-recent", future)) - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} + c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-recent", future)) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileSaveRequest(t, r, "save-recent") + res := reconcileCommitRequest(t, r, "save-recent") - assert.True(t, saveRequestExists(t, c, "save-recent")) + assert.True(t, commitRequestExists(t, c, "save-recent")) assert.InDelta(t, (47 * time.Hour).Seconds(), res.RequeueAfter.Seconds(), 60) } -func TestSaveRequestTTL_AnUnreadableTimeKeepsTheRequest(t *testing.T) { - c := newSaveRequestClient(t, nil, finishedWithDeleteAfter("save-typo", "tomorrow")) - r := &SaveRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} +func TestCommitRequestTTL_AnUnreadableTimeKeepsTheRequest(t *testing.T) { + c := newCommitRequestClient(t, nil, finishedWithDeleteAfter("save-typo", "tomorrow")) + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: &fakeFinalizer{}} - res := reconcileSaveRequest(t, r, "save-typo") + res := reconcileCommitRequest(t, r, "save-typo") assert.Zero(t, res.RequeueAfter) - assert.True(t, saveRequestExists(t, c, "save-typo")) + assert.True(t, commitRequestExists(t, c, "save-typo")) } // A request still waiting for its outcome is neither stamped nor deleted; it is polled as before. -func TestSaveRequestTTL_LeavesARequestInProgressAlone(t *testing.T) { - cr := withInProgress(newSaveRequest("save-pending")) +func TestCommitRequestTTL_LeavesARequestInProgressAlone(t *testing.T) { + cr := withInProgress(newCommitRequest("save-pending")) cr.CreationTimestamp = metav1.Now() // inside the resolve window, so it polls rather than fails closed - c := newSaveRequestClient(t, nil, cr) + c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &SaveRequestReconciler{ + r := &CommitRequestReconciler{ Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice(), TTL: time.Nanosecond, } - res := reconcileSaveRequest(t, r, "save-pending") + res := reconcileCommitRequest(t, r, "save-pending") - assert.Equal(t, saveRequestPollInterval, res.RequeueAfter, "it keeps polling for the outcome") - assert.NotContains(t, fetchSaveRequest(t, c, "save-pending").Annotations, SaveRequestDeleteAfterAnnotation) + assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "it keeps polling for the outcome") + assert.NotContains(t, fetchCommitRequest(t, c, "save-pending").Annotations, CommitRequestDeleteAfterAnnotation) } // A person who extends delete-after in the moment between the terminal status write and the stamp // must keep the extension. The stamp is computed from the object the status write returned, so an // unconditional patch would overwrite the edit and delete the request early. -func TestSaveRequestTTL_StampDoesNotOverwriteAConcurrentExtension(t *testing.T) { +func TestCommitRequestTTL_StampDoesNotOverwriteAConcurrentExtension(t *testing.T) { const extended = "2099-01-01T00:00:00Z" - c := newSaveRequestClient(t, &interceptor.Funcs{ + c := newCommitRequestClient(t, &interceptor.Funcs{ SubResourceUpdate: func( ctx context.Context, cl client.Client, sub string, obj client.Object, opts ...client.SubResourceUpdateOption, ) error { if err := cl.SubResource(sub).Update(ctx, obj, opts...); err != nil { return err } - if !saveRequestIsTerminal(obj.(*configv1alpha3.SaveRequest)) { + if !commitRequestIsTerminal(obj.(*configv1alpha3.CommitRequest)) { return nil } // The edit lands right after the terminal status is published, before the stamp. - var live configv1alpha3.SaveRequest + var live configv1alpha3.CommitRequest require.NoError(t, cl.Get(ctx, client.ObjectKeyFromObject(obj), &live)) - live.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: extended} + live.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: extended} require.NoError(t, cl.Update(ctx, &live)) return nil }, - }, newSaveRequest("save-now")) - r := &SaveRequestReconciler{ + }, newCommitRequest("save-now")) + r := &CommitRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: time.Hour, } - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") assert.Equal(t, extended, - fetchSaveRequest(t, c, "save-now").Annotations[SaveRequestDeleteAfterAnnotation], + fetchCommitRequest(t, c, "save-now").Annotations[CommitRequestDeleteAfterAnnotation], "the extension made after the request finished survives the stamp") } // A submitter may choose its own deletion time up front; finishing does not replace it. -func TestSaveRequestTTL_AValueSetAtCreationIsKept(t *testing.T) { +func TestCommitRequestTTL_AValueSetAtCreationIsKept(t *testing.T) { const chosen = "2099-01-01T00:00:00Z" - cr := newSaveRequest("save-now") - cr.Annotations = map[string]string{SaveRequestDeleteAfterAnnotation: chosen} - c := newSaveRequestClient(t, nil, cr) - r := &SaveRequestReconciler{ + cr := newCommitRequest("save-now") + cr.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: chosen} + c := newCommitRequestClient(t, nil, cr) + r := &CommitRequestReconciler{ Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), TTL: time.Hour, } - reconcileSaveRequest(t, r, "save-now") + reconcileCommitRequest(t, r, "save-now") assert.Equal(t, chosen, - fetchSaveRequest(t, c, "save-now").Annotations[SaveRequestDeleteAfterAnnotation]) + fetchCommitRequest(t, c, "save-now").Annotations[CommitRequestDeleteAfterAnnotation]) } diff --git a/internal/controller/constants.go b/internal/controller/constants.go index 37cb2b7f..aaebcc87 100644 --- a/internal/controller/constants.go +++ b/internal/controller/constants.go @@ -86,19 +86,19 @@ const ( // still being established (the rule is blocked by validation), which must not be rendered as a // permanent failure. ConditionTypeSourceNamespaceAuthorized = "SourceNamespaceAuthorized" - // ConditionTypeAuthorAttributed indicates whether a SaveRequest's commit author + // ConditionTypeAuthorAttributed indicates whether a CommitRequest's commit author // was named from the submitter captured at admission. It is binary and immediately // settled (no Unknown, no timeout): True (AttributedFromAdmission) when the // validate-operator-types webhook recorded the submitter, False (CommitterFallback) when // capture ran but found no record, or False (AuthorCaptureDisabled) when capture is off. // False is not a failure and does not affect Ready: the // request claims no actor and can attach only to an unnamed watch window - // (docs/architecture.md#saverequest-finalize). + // (docs/architecture.md#commitrequest-finalize). ConditionTypeAuthorAttributed = "AuthorAttributed" - // ConditionTypePushed indicates whether a SaveRequest's commit reached the + // ConditionTypePushed indicates whether a CommitRequest's commit reached the // remote repository. ConditionTypePushed = "Pushed" - // ConditionTypeApplied reports, for a SaveRequest with catchUp: AfterPush, whether + // ConditionTypeApplied reports, for a CommitRequest with catchUp: AfterPush, whether // its GitTarget's applier applied a commit that holds the request's: Unknown while it applies or // retries, True once applied, False when there was nothing to apply. It never affects Ready. ConditionTypeApplied = "Applied" @@ -184,10 +184,10 @@ const ( // DefaultGitRefreshInterval is how often an IDLE branch's remote state is re-proved. A target // that is publishing renews its observation on every push and never waits for this. DefaultGitRefreshInterval = 10 * time.Minute - // DefaultSaveRequestTTL is how long a SaveRequest is kept once it finishes. Each save leaves + // DefaultCommitRequestTTL is how long a CommitRequest is kept once it finishes. Each save leaves // one object behind, so without an expiry they accumulate in etcd for as long as the namespace // lives. Two days leaves a weekend's worth of saves to inspect. - DefaultSaveRequestTTL = 48 * time.Hour + DefaultCommitRequestTTL = 48 * time.Hour // RequeueStreamSettleInterval is the requeue interval while a Ready GitTarget still // has streams pending replay completion. Stream status is computed during reconcile, so // this keeps status.streams fresh while watches converge. diff --git a/internal/controller/duration_fields_admission_test.go b/internal/controller/duration_fields_admission_test.go index d638e212..2a2b450a 100644 --- a/internal/controller/duration_fields_admission_test.go +++ b/internal/controller/duration_fields_admission_test.go @@ -186,18 +186,18 @@ var _ = Describe("Duration fields", func() { spec["gitTargetRef"] = map[string]any{"name": "any-target"} return &unstructured.Unstructured{Object: map[string]any{ "apiVersion": "configbutler.ai/v1alpha3", - "kind": "SaveRequest", + "kind": "CommitRequest", "metadata": map[string]any{"name": name, "namespace": "default"}, "spec": spec, }} } - It("defaults an omitted SaveRequest window and whenNothingToCommit, and keeps explicit zeros", func() { + It("defaults an omitted CommitRequest window and whenNothingToCommit, and keeps explicit zeros", func() { ctx := context.Background() - omitted := &configbutleraiv1alpha3.SaveRequest{ + omitted := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-omitted", Namespace: "default"}, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, }, } @@ -212,11 +212,11 @@ var _ = Describe("Duration fields", func() { // The pointers exist for exactly this: an explicit "0s" is a value, and defaulting must // not read it as an omission. - zero := &configbutleraiv1alpha3.SaveRequest{ + zero := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{Name: "window-zeros", Namespace: "default"}, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, - Window: &configbutleraiv1alpha3.SaveRequestWindow{ + Window: &configbutleraiv1alpha3.CommitRequestWindow{ AttachTimeout: &metav1.Duration{}, IdleTimeout: &metav1.Duration{}, MaxDuration: &metav1.Duration{}, @@ -230,7 +230,7 @@ var _ = Describe("Duration fields", func() { Expect(zero.Spec.Window.MaxDuration.Duration).To(BeZero()) }) - DescribeTable("refuse a SaveRequest window the rules forbid", + DescribeTable("refuse a CommitRequest window the rules forbid", func(name string, spec map[string]any, message string) { err := k8sClient.Create(context.Background(), newRequest(name, spec)) Expect(err).To(HaveOccurred()) @@ -265,12 +265,12 @@ var _ = Describe("Duration fields", func() { } }) - // SaveRequest carries the same hazard with immutability on top: its spec may not change + // CommitRequest carries the same hazard with immutability on top: its spec may not change // after creation, and a whole-object comparison compares a duration as a STRING. "1m" reads // back into Go and serializes as "1m0s", so before the rule compared durations instead of // spellings, every typed update — including one that only touches a label — was rejected as // an attempt to change an immutable spec. - It("lets a typed client update a SaveRequest created with a re-spelled duration", func() { + It("lets a typed client update a CommitRequest created with a re-spelled duration", func() { ctx := context.Background() // Created as UNSTRUCTURED, carrying the spelling a HUMAN writes. That is the whole @@ -290,7 +290,7 @@ var _ = Describe("Duration fields", func() { key := types.NamespacedName{Name: "window-respelled", Namespace: "default"} Eventually(func() error { - var typed configbutleraiv1alpha3.SaveRequest + var typed configbutleraiv1alpha3.CommitRequest if err := k8sClient.Get(ctx, key, &typed); err != nil { return err } @@ -303,15 +303,15 @@ var _ = Describe("Duration fields", func() { "the same duration spelled differently is not a spec change") }) - DescribeTable("still refuse a real change to an immutable SaveRequest spec", - func(name string, mutate func(*configbutleraiv1alpha3.SaveRequestSpec)) { + DescribeTable("still refuse a real change to an immutable CommitRequest spec", + func(name string, mutate func(*configbutleraiv1alpha3.CommitRequestSpec)) { ctx := context.Background() - request := &configbutleraiv1alpha3.SaveRequest{ + request := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, - Spec: configbutleraiv1alpha3.SaveRequestSpec{ + Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{Name: "any-target"}, Message: "save", - Window: &configbutleraiv1alpha3.SaveRequestWindow{ + Window: &configbutleraiv1alpha3.CommitRequestWindow{ MaxDuration: &metav1.Duration{Duration: time.Minute}, }, }, @@ -321,7 +321,7 @@ var _ = Describe("Duration fields", func() { key := types.NamespacedName{Name: request.Name, Namespace: request.Namespace} Eventually(func() string { - var current configbutleraiv1alpha3.SaveRequest + var current configbutleraiv1alpha3.CommitRequest if err := k8sClient.Get(ctx, key, ¤t); err != nil { return err.Error() } @@ -333,16 +333,16 @@ var _ = Describe("Duration fields", func() { }, "10s", "200ms").Should(ContainSubstring("immutable"), "comparing by value must not become comparing by nothing") }, - Entry("maxDuration", "immutable-max", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { + Entry("maxDuration", "immutable-max", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { spec.Window.MaxDuration = &metav1.Duration{Duration: 30 * time.Second} }), - Entry("attach", "immutable-attach", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { + Entry("attach", "immutable-attach", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { spec.Window.Attach = configbutleraiv1alpha3.AttachNext }), - Entry("an idleTimeout added later", "immutable-idle", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { + Entry("an idleTimeout added later", "immutable-idle", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { spec.Window.IdleTimeout = &metav1.Duration{Duration: time.Second} }), - Entry("whenNothingToCommit", "immutable-empty", func(spec *configbutleraiv1alpha3.SaveRequestSpec) { + Entry("whenNothingToCommit", "immutable-empty", func(spec *configbutleraiv1alpha3.CommitRequestSpec) { spec.WhenNothingToCommit = configbutleraiv1alpha3.NothingToCommitCommitEmpty }), ) diff --git a/internal/controller/gitbranch_controller.go b/internal/controller/gitbranch_controller.go index 9b56c0b5..fe86b80b 100644 --- a/internal/controller/gitbranch_controller.go +++ b/internal/controller/gitbranch_controller.go @@ -32,7 +32,7 @@ import ( ) // GitBranchFinalizer holds a GitBranch until its branch worker has given every pending write and -// SaveRequest an outcome. It never deletes the branch on the remote. +// CommitRequest an outcome. It never deletes the branch on the remote. const GitBranchFinalizer = "configbutler.ai/git-branch" const ( @@ -394,7 +394,7 @@ func (r *GitBranchReconciler) finalize(ctx context.Context, branch *configbutler // It is the one path for a GitBranch being deleted and for one that no grant admits any more, which // keeps its finalizer. The claim goes first, so no GitTarget reconcile can start this GitBranch's // worker again; then the worker is stopped, which closes its intake, pushes or refuses every pending -// write and settles every SaveRequest it holds (BranchWorker.Stop). Release, not a failed proof: +// write and settles every CommitRequest it holds (BranchWorker.Stop). Release, not a failed proof: // Observe keeps a holder's candidacy through a failed proof, and a revoked GitBranch must stop being // a candidate at all. Last, once the old worker has retired so pending work never moves on, the // GitBranches waiting for the branch are woken. Release handed it to none of them: each takes it only diff --git a/internal/controller/gitbranch_controller_test.go b/internal/controller/gitbranch_controller_test.go index 0125fffc..488a99e2 100644 --- a/internal/controller/gitbranch_controller_test.go +++ b/internal/controller/gitbranch_controller_test.go @@ -266,7 +266,7 @@ func TestGitBranch_AMissingOrUnreadyGitProvider(t *testing.T) { } // TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch: the finalizer stops the branch worker -// (which publishes or refuses its pending writes and settles its SaveRequests), releases the +// (which publishes or refuses its pending writes and settles its CommitRequests), releases the // claim, and goes. The next candidate then takes the branch. func TestGitBranch_DeletionStopsTheWorkerAndReleasesTheBranch(t *testing.T) { f := newGitBranchFixture(t, diff --git a/internal/controller/gittarget_publication_test.go b/internal/controller/gittarget_publication_test.go index cb26a1e0..2b7ab3fc 100644 --- a/internal/controller/gittarget_publication_test.go +++ b/internal/controller/gittarget_publication_test.go @@ -85,7 +85,7 @@ func TestPublicationReadiness_LeavesAMissingParentToTheParentGate(t *testing.T) // A save the worker holds says why it is held, with the same words the GitTarget uses, and the // status is written again when that reason changes, not only when the phase does. -func TestSaveRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { +func TestCommitRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { held := failingPublication(false).Message() reason, message := progressFor(git.PhaseWaitingForPush, held) assert.Equal(t, string(git.PhaseWaitingForPush), reason) @@ -95,11 +95,11 @@ func TestSaveRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { _, plain := progressFor(git.PhaseWaitingForPush, "") assert.Equal(t, waitingForPushMessage, plain, "a push that is merely pending says nothing more") - cr := &configbutleraiv1alpha3.SaveRequest{} - markSaveRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, "") + cr := &configbutleraiv1alpha3.CommitRequest{} + markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, "") assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, "")) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, held), "the outage is news to the save") - markSaveRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, held) + markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, held) assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, held)) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, ""), "and so is the recovery") } diff --git a/internal/controller/grant_defaults_apiserver_test.go b/internal/controller/grant_defaults_apiserver_test.go index 2c468543..ed521423 100644 --- a/internal/controller/grant_defaults_apiserver_test.go +++ b/internal/controller/grant_defaults_apiserver_test.go @@ -103,28 +103,28 @@ func TestGrantLists_APIServerDefaultsOmittedAndKeepsEmpty(t *testing.T) { assert.Empty(t, obj.Spec.AllowedApplierClaims) }) - target := func(name string, grants []configbutleraiv1alpha3.GitTargetSaveRequestGrant) *configbutleraiv1alpha3.GitTarget { + target := func(name string, grants []configbutleraiv1alpha3.GitTargetCommitRequestGrant) *configbutleraiv1alpha3.GitTarget { gt := &configbutleraiv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Namespace: "default", Name: name}, - Spec: configbutleraiv1alpha3.GitTargetSpec{Path: "apps", AllowedSaveRequests: grants}, + Spec: configbutleraiv1alpha3.GitTargetSpec{Path: "apps", AllowedCommitRequests: grants}, } gt.Spec.GitBranchRef.Name = "main" gt.Spec.KubeProviderRef.Name = "local" return gt } - t.Run("an omitted GitTarget allowedSaveRequests reads back as the default", func(t *testing.T) { + t.Run("an omitted GitTarget allowedCommitRequests reads back as the default", func(t *testing.T) { obj := target("omitted", nil) require.NoError(t, c.Create(ctx, obj)) require.NoError(t, c.Get(ctx, client.ObjectKeyFromObject(obj), obj)) - assert.Equal(t, configbutleraiv1alpha3.DefaultGitTargetSaveRequestGrants(), obj.Spec.AllowedSaveRequests) + assert.Equal(t, configbutleraiv1alpha3.DefaultGitTargetCommitRequestGrants(), obj.Spec.AllowedCommitRequests) }) - t.Run("an empty GitTarget allowedSaveRequests stays empty through an update", func(t *testing.T) { - obj := target("deny-all", []configbutleraiv1alpha3.GitTargetSaveRequestGrant{}) + t.Run("an empty GitTarget allowedCommitRequests stays empty through an update", func(t *testing.T) { + obj := target("deny-all", []configbutleraiv1alpha3.GitTargetCommitRequestGrant{}) require.NoError(t, c.Create(ctx, obj)) relabel(t, obj) - require.NotNil(t, obj.Spec.AllowedSaveRequests, "an update must not turn [] into the default") - assert.Empty(t, obj.Spec.AllowedSaveRequests) + require.NotNil(t, obj.Spec.AllowedCommitRequests, "an update must not turn [] into the default") + assert.Empty(t, obj.Spec.AllowedCommitRequests) }) } diff --git a/internal/controller/resource_condition_metrics.go b/internal/controller/resource_condition_metrics.go index 0c0dd7e9..4959cd39 100644 --- a/internal/controller/resource_condition_metrics.go +++ b/internal/controller/resource_condition_metrics.go @@ -19,10 +19,10 @@ import ( // bounds the series count and is the whole basis on which §7.1 of the metrics plan allows their // names as labels. // -// SaveRequest is deliberately absent. It is created once per save, so it is data-shaped rather +// CommitRequest is deliberately absent. It is created once per save, so it is data-shaped rather // than config-shaped: a gauge per object would churn series continuously — created, Ready, // finalized, deleted — and churn is the one cost Prometheus handles worst. Its state is answerable -// from git_commits_total{message_source="save_request"} and from the object's own conditions, and +// from git_commits_total{message_source="commit_request"} and from the object's own conditions, and // if a fleet view is ever wanted the right shape is a counter of terminal outcomes, because those // are events rather than state. Its reconciler does not use this status helper either, so the // exclusion holds at both ends. @@ -45,7 +45,7 @@ const ( // Which is why the kind does NOT come off statusObject, the accessor interface the status session // takes. That interface is about writing a status, and anything can be given the two methods it // asks for; hanging the kind on it would turn this list into an implication — implement the -// accessors over in api/v1alpha3 and you are on the metric — and SaveRequest would arrive here by +// accessors over in api/v1alpha3 and you are on the metric — and CommitRequest would arrive here by // a route nobody reviewing that file would see. func conditionMetricKind(obj client.Object) string { switch obj.(type) { diff --git a/internal/controller/resource_condition_metrics_test.go b/internal/controller/resource_condition_metrics_test.go index 97260270..c39e454d 100644 --- a/internal/controller/resource_condition_metrics_test.go +++ b/internal/controller/resource_condition_metrics_test.go @@ -116,7 +116,7 @@ func TestPublishConditionMetrics_ForgetsAnObjectThatIsGoing(t *testing.T) { assert.False(t, ok, "an object with no finalizer left is going now and must stop publishing") } -// The kind switch is the surface. SaveRequest is data-shaped — one per save — so it is excluded +// The kind switch is the surface. CommitRequest is data-shaped — one per save — so it is excluded // structurally rather than by a rule someone has to remember. func TestConditionMetricKind_CoversTheFiveConfigKindsAndNothingElse(t *testing.T) { assert.Equal(t, conditionKindGitTarget, conditionMetricKind(&configbutleraiv1alpha3.GitTarget{})) @@ -124,7 +124,7 @@ func TestConditionMetricKind_CoversTheFiveConfigKindsAndNothingElse(t *testing.T assert.Equal(t, conditionKindClusterWatchRule, conditionMetricKind(&configbutleraiv1alpha3.ClusterWatchRule{})) assert.Equal(t, conditionKindGitProvider, conditionMetricKind(&configbutleraiv1alpha3.GitProvider{})) assert.Equal(t, conditionKindKubeProvider, conditionMetricKind(&configbutleraiv1alpha3.KubeProvider{})) - assert.Empty(t, conditionMetricKind(&configbutleraiv1alpha3.SaveRequest{})) + assert.Empty(t, conditionMetricKind(&configbutleraiv1alpha3.CommitRequest{})) } // A condition written with an empty reason publishes a visibly wrong placeholder rather than an diff --git a/internal/controller/status.go b/internal/controller/status.go index fb4426f2..32f5f678 100644 --- a/internal/controller/status.go +++ b/internal/controller/status.go @@ -104,7 +104,7 @@ func (s *reconcileStatus) writeLost() bool { return s.writeLostToRace } // commit() drops the write, and the GenerationChangedPredicate on their For() filters the // status-only update that won, so nothing re-enqueues the object — it keeps the older answer for // its full steady interval. Fixing that at one call site would have left the same bug in the other -// four. SaveRequestReconciler carries neither half; see its Reconcile for why. +// four. CommitRequestReconciler carries neither half; see its Reconcile for why. func (s *reconcileStatus) requeueAfter(cadence time.Duration) time.Duration { if s.writeLostToRace { return RequeueWriteLostInterval @@ -123,7 +123,7 @@ func (s *reconcileStatus) requeueAfter(cadence time.Duration) time.Duration { // handing it one object's conditions and another object's metadata. // // It carries no kind name, on purpose. conditionMetricKind resolves that from its own list, -// because SaveRequest is excluded from the condition metric and an interface anything could +// because CommitRequest is excluded from the condition metric and an interface anything could // implement is not a place to keep an exclusion. See resource_condition_metrics.go. type statusObject interface { client.Object diff --git a/internal/controller/suite_test.go b/internal/controller/suite_test.go index ac0717f4..5e740d58 100644 --- a/internal/controller/suite_test.go +++ b/internal/controller/suite_test.go @@ -186,7 +186,7 @@ var _ = BeforeSuite(func() { // committer immediately, and a Finalizer that never resolves keeps requests in the // Progressing state. Neither ever completes, so these specs cover only the // initial in-progress stamp and the terminal short-circuit. - err = (&SaveRequestReconciler{ + err = (&CommitRequestReconciler{ Client: mgr.GetClient(), Scheme: mgr.GetScheme(), Finalizer: &fakeFinalizer{resolved: false}, diff --git a/internal/git/applier_claim.go b/internal/git/applier_claim.go index 80f97894..d956e233 100644 --- a/internal/git/applier_claim.go +++ b/internal/git/applier_claim.go @@ -251,9 +251,9 @@ type applierState struct { // lag is whether the converge pending is only scheduleConverges': nothing else asks for the tip, // so its attempt yields to an edit (convergeHolds). lag bool - // waitingSaves is the SaveRequests waiting for this applier's next apply that succeeds, each + // waitingSaves is the CommitRequests waiting for this applier's next apply that succeeds, each // with the commit it pushed (save_apply.go); the attempt sent next carries them. - waitingSaves map[saveRequestID]string + waitingSaves map[commitRequestID]string // appliedRevision is the revision the last operation that succeeded applied, as the applier // reported it, recovered on restart from the applier claim's status.operation. A converge of the commit // it names asks nothing. @@ -326,9 +326,9 @@ type inflightAttempt struct { refusals map[refusalKey]unsentRefusal // lag is whether it carries only scheduleConverges' converge. lag bool - // saves is the SaveRequests whose apply it carries, each with the commit it pushed + // saves is the CommitRequests whose apply it carries, each with the commit it pushed // (save_apply.go). - saves map[saveRequestID]string + saves map[commitRequestID]string // phase is the last report of its operation, empty before the first. phase ApplyOutcome // previous is the operation recorded before this attempt's, with the attempt it belonged to, @@ -357,7 +357,7 @@ func (s *applierState) hasReason(refusals map[refusalKey]unsentRefusal) bool { const barrierRecheckInterval = time.Second // writeBusy reports whether the loop is in a state in which no flush can be made, and so no request -// sent: a registered SaveRequest, whose save defines its own commit boundary (a save is announced +// sent: a registered CommitRequest, whose save defines its own commit boundary (a save is announced // before its writes arrive, and an apply landing between the two would revert its first edits before // they reach Git), a parent recovery, or a checkout a failed write left dirty. In the last two the // worker cannot say what is published. @@ -367,7 +367,7 @@ func (l *branchWorkerEventLoop) writeBusy() bool { // closeWindowForPushHint closes the open commit window early when a Git host has said the branch may // have moved, so its writes are published, and replayed onto the move, now rather than when the -// window's timers allow. A window a SaveRequest holds is left to its save. +// window's timers allow. A window a CommitRequest holds is left to its save. func (l *branchWorkerEventLoop) closeWindowForPushHint() { hinted := l.pushHinted l.pushHinted = false diff --git a/internal/git/applier_claim_test.go b/internal/git/applier_claim_test.go index 8d657f79..b4a986ee 100644 --- a/internal/git/applier_claim_test.go +++ b/internal/git/applier_claim_test.go @@ -150,7 +150,7 @@ func applierClaimLoop(t *testing.T, targets ...client.Object) (*branchWorkerEven // holdApplies puts the loop in a state in which no flush can be made, so no apply is sent: a save is // registered, and it defines its own commit boundary (writeBusy). releaseApplies ends it. func holdApplies(l *branchWorkerEventLoop) { - l.pendingCRs = map[saveRequestID]*pendingSaveRequest{{Namespace: "team-a", Name: "save"}: {}} + l.pendingCRs = map[commitRequestID]*pendingCommitRequest{{Namespace: "team-a", Name: "save"}: {}} } func releaseApplies(l *branchWorkerEventLoop) { l.pendingCRs = nil } diff --git a/internal/git/apply_attempt_test.go b/internal/git/apply_attempt_test.go index 7a6d4318..4a6059ac 100644 --- a/internal/git/apply_attempt_test.go +++ b/internal/git/apply_attempt_test.go @@ -349,12 +349,12 @@ func TestPushNotice_TheTipAlreadySeenDoesNotCutAWindow(t *testing.T) { } } -// TestWriteBusy_ASaveAnnouncedBeforeItsWrites: a registered SaveRequest makes the branch busy, so +// TestWriteBusy_ASaveAnnouncedBeforeItsWrites: a registered CommitRequest makes the branch busy, so // a catch-up cannot land between a save's announcement and its first writes. func TestWriteBusy_ASaveAnnouncedBeforeItsWrites(t *testing.T) { loop, _ := applierClaimLoop(t, applierTestTarget("alpha", true)) assert.False(t, loop.writeBusy()) - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{{}: {}} + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{{}: {}} assert.True(t, loop.writeBusy()) } diff --git a/internal/git/apply_converge_test.go b/internal/git/apply_converge_test.go index 07771d4e..edc3973c 100644 --- a/internal/git/apply_converge_test.go +++ b/internal/git/apply_converge_test.go @@ -198,20 +198,20 @@ func TestCatchUpAfter_AChangedIntervalAppliesFromTheNextWake(t *testing.T) { assert.Equal(t, "bbb", requests[0].Commit) } -// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a SaveRequest with catchUp: AfterPush +// TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce: a CommitRequest with catchUp: AfterPush // leaves a converge pending for its GitTarget's applier once its commit is pushed, without waiting // for catchUpAfter; one that pushed no commit asks nothing. It does not yield to an edit: it was // asked for. func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} save := func(name string, commit string) { - id := saveRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} - loop.pendingCRs[id] = &pendingSaveRequest{ + id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingCommitRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) } save("nothing-pushed", "") @@ -237,16 +237,16 @@ func TestCatchUpAfter_ASaveThatAsksCatchesUpAtOnce(t *testing.T) { func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true), applierTestTarget("loner", false)) convergedAt(t, loop, requester) - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} - save := func(name, target, commit string) saveRequestID { - id := saveRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} - loop.pendingCRs[id] = &pendingSaveRequest{ + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + save := func(name, target, commit string) commitRequestID { + id := commitRequestID{Namespace: "team-a", Name: name, UID: "uid-" + name} + loop.pendingCRs[id] = &pendingCommitRequest{ id: id, gitTargetName: target, gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: commit}) return id } - lookup := func(id saveRequestID) SaveApply { + lookup := func(id commitRequestID) SaveApply { apply, known := loop.w.LookupSaveApply(id.Namespace, id.Name, id.UID) require.True(t, known, "the worker knows %s", id.Name) return apply @@ -289,15 +289,15 @@ func TestSaveApply_FollowsTheAttemptThatCarriesIt(t *testing.T) { func TestSaveApply_AnApplierAlreadyAtTheTipSettlesIt(t *testing.T) { loop, requester := manualLoop(t, applierTestTarget("alpha", true)) convergedAt(t, loop, requester) - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} - id := saveRequestID{Namespace: "team-a", Name: "late", UID: "uid-late"} - loop.pendingCRs[id] = &pendingSaveRequest{ + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "late", UID: "uid-late"} + loop.pendingCRs[id] = &pendingCommitRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "aaa"}) + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "aaa"}) loop.serviceAppliers() assert.Empty(t, requester.taken()) @@ -332,12 +332,12 @@ func TestCatchUpAfter_TurningItOffCancelsAPendingConverge(t *testing.T) { assert.Empty(t, requester.taken(), "off: the timed converge is gone") assert.False(t, state.converge) - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} - id := saveRequestID{Namespace: "team-a", Name: "asks", UID: "uid-asks"} - loop.pendingCRs[id] = &pendingSaveRequest{ + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "asks", UID: "uid-asks"} + loop.pendingCRs[id] = &pendingCommitRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "bbb"}) + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: "bbb"}) loop.serviceAppliers() asked := requester.taken() require.Len(t, asked, 1, "a save that asks is no timed converge") @@ -403,12 +403,12 @@ func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { startup[0].Report(ApplyResult{ID: startup[0].ID, Outcome: ApplySucceeded, AppliedRevision: base}) loop.serviceAppliers() - loop.pendingCRs = map[saveRequestID]*pendingSaveRequest{} - id := saveRequestID{Namespace: "team-a", Name: "save", UID: "uid-save"} - loop.pendingCRs[id] = &pendingSaveRequest{ + loop.pendingCRs = map[commitRequestID]*pendingCommitRequest{} + id := commitRequestID{Namespace: "team-a", Name: "save", UID: "uid-save"} + loop.pendingCRs[id] = &pendingCommitRequest{ id: id, gitTargetName: "alpha", gitTargetNamespace: "team-a", catchUpNow: true, } - loop.resolveSaveRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: saved}) + loop.resolveCommitRequest(id, FinalizeResult{Outcome: FinalizeCommitted, Commit: saved}) tip, by := tc.tip(base, saved, later, rewritten) loop.w.inbox.observe(RemoteObservation{Commit: tip, By: by}) loop.serviceAppliers() @@ -435,6 +435,6 @@ func TestSaveApply_AppliedOnlyWhenTheRevisionHoldsItsCommit(t *testing.T) { func TestApplierState_ASaveIsNeverOnlyALag(t *testing.T) { state := &applierState{lag: true} assert.True(t, state.onlyLag(nil)) - state.waitSave(saveRequestID{Namespace: "team-a", Name: "save", UID: "uid"}, "ccc") + state.waitSave(commitRequestID{Namespace: "team-a", Name: "save", UID: "uid"}, "ccc") assert.False(t, state.onlyLag(nil)) } diff --git a/internal/git/attribution_outcome_test.go b/internal/git/attribution_outcome_test.go index d4390bf7..53117d3e 100644 --- a/internal/git/attribution_outcome_test.go +++ b/internal/git/attribution_outcome_test.go @@ -157,30 +157,30 @@ func TestOpenWindow_ResolvedWindowsStillSplitByAuthor(t *testing.T) { // THE REGRESSION THIS DESIGN EXISTS TO AVOID. // -// A SaveRequest that could not be attributed carries an empty author, and so does a window +// A CommitRequest that could not be attributed carries an empty author, and so does a window // whose attribution ran and named nobody — the sentinel is a git-signature rendering only and // never reaches the window's Author. The two therefore still match on the author, and the // outcomes must not be allowed to push them apart: the request's outcome comes from command // authorship and the window's from mirrored-resource attribution, which are configured // independently, so they routinely differ. See the full cross-subsystem matrix in // TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems. -func TestPendingSaveRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *testing.T) { +func TestPendingCommitRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *testing.T) { window := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) require.Empty(t, window.Author, "an unresolved window carries no author; the sentinel is applied at the write path") - request := &pendingSaveRequest{ - author: "", // an unnamed SaveRequest claims no actor + request := &pendingCommitRequest{ + author: "", // an unnamed CommitRequest claims no actor attribution: AttributionUnresolved, gitTargetName: "target", gitTargetNamespace: "default", } assert.True(t, request.matchesWindow(window), - "an unnamed SaveRequest must attach to the unresolved window it was meant for") + "an unnamed CommitRequest must attach to the unresolved window it was meant for") // The default deployment: command-author capture is off, so the request says "not // attempted" while the window says "unresolved". Neither names an actor, so they match. - notAttempted := &pendingSaveRequest{ + notAttempted := &pendingCommitRequest{ author: "", attribution: AttributionNotAttempted, gitTargetName: "target", @@ -191,10 +191,10 @@ func TestPendingSaveRequest_UnresolvedRequestAttachesToUnresolvedWindow(t *testi "requiring the two subsystems' outcomes to be equal drops the user's commit message") } -func TestPendingSaveRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { +func TestPendingCommitRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { resolvedWindow := newOpenWindow(attributedEvent("jane@acme.com", AttributionResolved), nil) - unresolvedRequest := &pendingSaveRequest{ + unresolvedRequest := &pendingCommitRequest{ author: "", attribution: AttributionUnresolved, gitTargetName: "target", @@ -203,7 +203,7 @@ func TestPendingSaveRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { assert.False(t, unresolvedRequest.matchesWindow(resolvedWindow), "an unattributed request must not claim a named actor's window") - resolvedRequest := &pendingSaveRequest{ + resolvedRequest := &pendingCommitRequest{ author: "jane@acme.com", attribution: AttributionResolved, gitTargetName: "target", @@ -211,7 +211,7 @@ func TestPendingSaveRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { } assert.True(t, resolvedRequest.matchesWindow(resolvedWindow)) - otherActor := &pendingSaveRequest{ + otherActor := &pendingCommitRequest{ author: "bob@acme.com", attribution: AttributionResolved, gitTargetName: "target", @@ -221,9 +221,9 @@ func TestPendingSaveRequest_DoesNotCrossAttributionOutcomes(t *testing.T) { "a resolved request still matches on the actor") } -func TestPendingSaveRequest_StillScopedToOneGitTarget(t *testing.T) { +func TestPendingCommitRequest_StillScopedToOneGitTarget(t *testing.T) { window := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) - request := &pendingSaveRequest{ + request := &pendingCommitRequest{ author: "", attribution: AttributionUnresolved, gitTargetName: "other-target", @@ -260,11 +260,11 @@ func TestRenderEventCommitMessage_UnresolvedUsernameStaysEmptyInTemplates(t *tes // A request that names its submitter claims a window whose writes name no author only when it was // allowed to, and never a window whose writes name an actor, nor one on another GitTarget. -func TestPendingSaveRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { +func TestPendingCommitRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { unattributed := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) named := newOpenWindow(attributedEvent("bob@acme.com", AttributionResolved), nil) - request := func(claim UnattributedClaim, target string) *pendingSaveRequest { - return &pendingSaveRequest{ + request := func(claim UnattributedClaim, target string) *pendingCommitRequest { + return &pendingCommitRequest{ author: "jane@acme.com", attribution: AttributionResolved, gitTargetName: target, @@ -291,12 +291,12 @@ func TestPendingSaveRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing } func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { - id := &saveRequestID{Namespace: "team-a", Name: "save-1"} + id := &commitRequestID{Namespace: "team-a", Name: "save-1"} assert.Equal(t, - "save: edit\n\nAuthor-Source: SaveRequest team-a/save-1; no audit record names who made these changes\n", + "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes\n", withTakeOverNote("save: edit\n", id), "the message's trailing newline is kept") assert.Equal(t, - "save: edit\n\nAuthor-Source: SaveRequest team-a/save-1; no audit record names who made these changes", + "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes", withTakeOverNote("save: edit", id)) } diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index 98e53268..f3aa556b 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -213,7 +213,7 @@ type BranchWorker struct { // to read it — and still answer "accepted". stoppingState bool // exitedState is set once the event loop has returned, for whatever reason, and the worker - // has settled every SaveRequest it knew. From then on nothing can act on a request, so a + // has settled every CommitRequest it knew. From then on nothing can act on a request, so a // withdraw is answered at once instead of being queued for a loop that is gone. Guarded by // pendingResyncsMu, like stoppingState. exitedState bool @@ -364,27 +364,27 @@ type BranchWorker struct { // Guarded by pendingResyncsMu. See queuedResyncsFor. queuedResyncs map[string]int - // crOutcomes holds resolved SaveRequest outcomes for the controller to poll - // via LookupSaveRequestOutcome. The event loop is the only writer (on its + // crOutcomes holds resolved CommitRequest outcomes for the controller to poll + // via LookupCommitRequestOutcome. The event loop is the only writer (on its // goroutine), the controller the only reader (on a reconcile goroutine), so the // mutex guards the cross-goroutine handoff. Entries are GC'd by age. The - // accessors live in save_request_attach_loop.go alongside the attach logic. + // accessors live in commit_request_attach_loop.go alongside the attach logic. crOutcomesMu sync.Mutex - crOutcomes map[saveRequestID]saveRequestOutcomeEntry - // crPhases holds where each unresolved SaveRequest stands, for the controller to report + crOutcomes map[commitRequestID]commitRequestOutcomeEntry + // crPhases holds where each unresolved CommitRequest stands, for the controller to report // while it waits. Guarded by crOutcomesMu; the event loop is the only writer. - crPhases map[saveRequestID]SaveRequestPhase - // crApplies holds where the apply each SaveRequest asked for stands (save_apply.go). Guarded by + crPhases map[commitRequestID]CommitRequestPhase + // crApplies holds where the apply each CommitRequest asked for stands (save_apply.go). Guarded by // crOutcomesMu; the event loop is the only writer. - crApplies map[saveRequestID]saveApplyEntry + crApplies map[commitRequestID]saveApplyEntry // crQueuedAttaches counts, per request, the attaches on the FIFO that the loop has not handled // yet. An outcome is not collected while any are queued: the outcome is what turns a late attach // into a no-op, so collecting it first would let that attach register a withdrawn request // again, however long the outcome had been kept. Guarded by crOutcomesMu. - crQueuedAttaches map[saveRequestID]int - // crOwners is the manager's table of which worker holds each request. See saveRequestOwners. + crQueuedAttaches map[commitRequestID]int + // crOwners is the manager's table of which worker holds each request. See commitRequestOwners. // Set by the WorkerManager before Start; nil for a worker built on its own. - crOwners *saveRequestOwners + crOwners *commitRequestOwners // clock is the time source of the retry schedule; nil is time.Now. See now(). clock func() time.Time @@ -561,7 +561,7 @@ func (w *BranchWorker) Start(parentCtx context.Context) error { go func() { defer w.wg.Done() w.processEvents() - w.settleSaveRequestsAfterExit() + w.settleCommitRequestsAfterExit() }() return nil @@ -596,10 +596,10 @@ func (w *BranchWorker) Enqueue(event Event) bool { return w.enqueueRequest(&WriteRequest{Events: []Event{event}}) } -// EnqueueWithdraw asks the loop to cancel a SaveRequest it has not acted on. It never blocks: a +// EnqueueWithdraw asks the loop to cancel a CommitRequest it has not acted on. It never blocks: a // full queue or a stopping worker drops it, and the controller sends it again on its next poll. A // worker whose loop has exited answers at once, because nothing is left that could act on it. -func (w *BranchWorker) EnqueueWithdraw(req *AttachSaveRequest) { +func (w *BranchWorker) EnqueueWithdraw(req *AttachCommitRequest) { if req == nil { return } @@ -620,17 +620,17 @@ func (w *BranchWorker) EnqueueWithdraw(req *AttachSaveRequest) { case w.eventQueue <- WorkItem{Withdraw: req}: default: w.releaseInflightItem() - w.Log.V(1).Info("Queue full, SaveRequest withdraw dropped (the controller re-sends)", + w.Log.V(1).Info("Queue full, CommitRequest withdraw dropped (the controller re-sends)", "request", req.Namespace+"/"+req.Name) } } -// EnqueueAttach adds a SaveRequest attach to this worker's queue. Riding the +// EnqueueAttach adds a CommitRequest attach to this worker's queue. Riding the // same queue as resource events is what makes it process in audit order, after // every earlier write. The attach is fire-and-forget: the controller polls the -// outcome via LookupSaveRequestOutcome and re-sends idempotently, so a queue- +// outcome via LookupCommitRequestOutcome and re-sends idempotently, so a queue- // full drop is recovered by the next poll rather than a synchronous reply. -func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { +func (w *BranchWorker) EnqueueAttach(req *AttachCommitRequest) { if req == nil { return } @@ -648,14 +648,14 @@ func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.V(1).Info("Worker is stopping, SaveRequest attach refused (the controller re-sends)") + w.Log.V(1).Info("Worker is stopping, CommitRequest attach refused (the controller re-sends)") return } if !w.admitPayload(charge) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.V(1).Info("SaveRequest attach refused while the branch has paused intake "+ + w.Log.V(1).Info("CommitRequest attach refused while the branch has paused intake "+ "(the controller re-sends)", "request", req.Namespace+"/"+req.Name) return } @@ -666,7 +666,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { w.intake.release(charge) w.pendingResyncsMu.Unlock() w.releaseInflightItem() - w.Log.Info("SaveRequest attach refused: another branch worker holds the request", + w.Log.Info("CommitRequest attach refused: another branch worker holds the request", "request", req.Namespace+"/"+req.Name) return } @@ -675,7 +675,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { select { case w.eventQueue <- WorkItem{Attach: req, charge: charge}: w.pendingResyncsMu.Unlock() - w.Log.Info("SaveRequest attach enqueued", + w.Log.Info("CommitRequest attach enqueued", "request", req.Namespace+"/"+req.Name, "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName, @@ -690,7 +690,7 @@ func (w *BranchWorker) EnqueueAttach(req *AttachSaveRequest) { w.pendingResyncsMu.Unlock() w.releaseInflightItem() w.recordQueueRefusal(queueRefusalAttach) - w.Log.Error(nil, "Event queue full, SaveRequest attach refused (controller will re-send)") + w.Log.Error(nil, "Event queue full, CommitRequest attach refused (controller will re-send)") } } @@ -1049,15 +1049,15 @@ type branchWorkerEventLoop struct { // See apply_ownership.go. ownershipLogged map[string]struct{} - // pendingCRs holds SaveRequests registered (via AttachSaveRequest) but not + // pendingCRs holds CommitRequests registered (via AttachCommitRequest) but not // yet resolved, keyed by identity. A request is parked here until a same-author // window opens (then it attaches to it) or its attach deadline passes. // Loop-goroutine only. - pendingCRs map[saveRequestID]*pendingSaveRequest - // savedCatchUps is the SaveRequests, by GitTarget, whose pushed commit asked their applier to + pendingCRs map[commitRequestID]*pendingCommitRequest + // savedCatchUps is the CommitRequests, by GitTarget, whose pushed commit asked their applier to // catch up at once (catchUp: AfterPush), with that commit, until the appliers are next listed // (takeReasons, save_apply.go). - savedCatchUps map[itypes.ResourceReference]map[saveRequestID]string + savedCatchUps map[itypes.ResourceReference]map[commitRequestID]string // crSeq numbers registrations, so competing requests attach first come, first served. crSeq uint64 // refusalTimer fires at the earliest deadline in refusalPending. See refusal_reapply.go. @@ -1097,7 +1097,7 @@ type branchWorkerEventLoop struct { flushMark uint64 flushEdits map[itypes.ResourceReference]map[string]time.Time - // attachTimer fires at the earliest attach deadline of a waiting SaveRequest, so a request + // attachTimer fires at the earliest attach deadline of a waiting CommitRequest, so a request // that no window reaches still resolves on time. An attached request needs no timer of its own: // its window's timers close it. attachTimer *time.Timer @@ -1165,7 +1165,7 @@ func (l *branchWorkerEventLoop) run() { case <-attachC: l.attachTimer = nil // The work (attach waiting requests, finalize due ones) is done by - // serviceSaveRequests below. + // serviceCommitRequests below. case <-refusalC: l.refusalTimer = nil l.flushPendingReapply() @@ -1195,7 +1195,7 @@ func (l *branchWorkerEventLoop) handleQueueItem(item WorkItem) { // // - Release the live events held for a GitTarget whose render fidelity is now established, or drop // them on a divergence or past the hold's bounds (serviceHeldWrites). -// - Bind any waiting SaveRequest to an open window, resolve any whose attach deadline has +// - Bind any waiting CommitRequest to an open window, resolve any whose attach deadline has // passed, and re-arm the deadline timer. // - Close the open window early if a Git host reported a push (closeWindowForPushHint). // - Drive the pending writes: commit what the wake decided, schedule its push (advance). @@ -1206,7 +1206,7 @@ func (l *branchWorkerEventLoop) handleQueueItem(item WorkItem) { // - Publish the result, moving the handled item's charge to what the loop holds. func (l *branchWorkerEventLoop) endWake(released int64) { l.serviceHeldWrites() - l.serviceSaveRequests() + l.serviceCommitRequests() l.closeWindowForPushHint() l.advance() l.serviceAppliers() @@ -1259,9 +1259,9 @@ func (l *branchWorkerEventLoop) totalRetainedBytes() int64 { func (l *branchWorkerEventLoop) dispatch(item WorkItem) int64 { switch { case item.Attach != nil: - l.handleAttachSaveRequest(item.Attach) + l.handleAttachCommitRequest(item.Attach) case item.Withdraw != nil: - l.handleWithdrawSaveRequest(item.Withdraw) + l.handleWithdrawCommitRequest(item.Withdraw) case item.ApplyBarrier != nil: l.handleApplyBarrier(item.ApplyBarrier) case item.Resync != nil: @@ -1309,7 +1309,7 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { // diagnosable: the common cause is an incoming event whose author is empty // (an unattributed update, e.g. a /status change that carries no audit fact) // arriving against a non-empty window author, which forces a finalize and - // can split a SaveRequest collect window. windowAuthor vs eventAuthor makes + // can split a CommitRequest collect window. windowAuthor vs eventAuthor makes // that visible at a glance. l.w.Log.Info("Window identity change forces finalize before appending event", "windowAuthor", l.openWindow.Author, @@ -1345,13 +1345,13 @@ func (l *branchWorkerEventLoop) appendLiveEvent(event Event) { l.windowBytes += eventRetainedCharge(&event) if opened { - // A waiting SaveRequest gets the new window BEFORE anything can close it: before the + // A waiting CommitRequest gets the new window BEFORE anything can close it: before the // target's timers are applied, and before a buffer-limit flush. With idleTimeout or // maxDuration at 0s, or a write that trips the cap, the window would otherwise close in // this very step, and a waiting request would miss the write it was waiting for — or, // waiting on another author, never see the window that refuses it. l.noteForeignWindow() - l.attachWaitingSaveRequests() + l.attachWaitingCommitRequests() if l.openWindow == nil { return // the attached request's maxDuration: 0s closed it already } @@ -1430,24 +1430,24 @@ func (l *branchWorkerEventLoop) handleShutdown() { // worker exits, even if a push was just sent. The push commits what was just decided. l.pushPending() } - l.failUnpushedSaveRequests() + l.failUnpushedCommitRequests() l.drainUnhandledQueueItems() } -// failUnpushedSaveRequests settles every SaveRequest still riding a write the exiting loop +// failUnpushedCommitRequests settles every CommitRequest still riding a write the exiting loop // could not push. // // A request that reached a local commit is resolved by the push and by nothing else, so a // shutdown whose final push failed would leave it in flight with no loop left to settle it. The // controller would poll until its own safety window expired and then fail closed on a timeout, // which says nothing about what happened. Say what happened instead. -func (l *branchWorkerEventLoop) failUnpushedSaveRequests() { +func (l *branchWorkerEventLoop) failUnpushedCommitRequests() { for i := range l.pendingWrites { - id := l.pendingWrites[i].SaveRequest + id := l.pendingWrites[i].CommitRequest if id == nil { continue } - l.resolveSaveRequest(*id, FinalizeResult{ + l.resolveCommitRequest(*id, FinalizeResult{ Err: errors.New("worker stopped before the commit reached the remote"), }) } @@ -1458,7 +1458,7 @@ func (l *branchWorkerEventLoop) drainUnhandledQueueItems() { l.w.drainQueue() } // drainQueue clears items no loop will handle. Each was counted into inflightItems at enqueue and // decremented only after handling, so without this the final depth would stay non-zero forever: -// the loop has stopped, so nothing republishes a corrected value. A buffered SaveRequest attach +// the loop has stopped, so nothing republishes a corrected value. A buffered CommitRequest attach // is dropped, and the request given back unless the worker knows it otherwise; the controller // re-sends on its next poll, to whichever worker holds it then. func (w *BranchWorker) drainQueue() { @@ -1495,10 +1495,10 @@ func (l *branchWorkerEventLoop) resetCommitTimer(delay time.Duration) { } // finalizeOpenWindowWithReason closes the live event window into one decided write. The attached -// SaveRequest message overrides the live template. +// CommitRequest message overrides the live template. // // The closed window is a decided write (decide): a remote that cannot be reached leaves it pending -// for the publication retry, and its SaveRequest held. Only a failure of the write itself +// for the publication retry, and its CommitRequest held. Only a failure of the write itself // drops it, and resolves that request Failed (settleFailed). func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinalizeReason) { if l.openWindow == nil { @@ -1573,11 +1573,11 @@ func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinali "pendingWrites", len(l.pendingWrites)) } -// carryRequestOnWrite carries the attached SaveRequest onto the write so its result follows the +// carryRequestOnWrite carries the attached CommitRequest onto the write so its result follows the // data: it is resolved Committed once this write is pushed (§6.5). The request's // whenNothingToCommit rides along, so a window that changed nothing can still record its message. -func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, pendingCR *saveRequestID) { - pendingWrite.SaveRequest = pendingCR +func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, pendingCR *commitRequestID) { + pendingWrite.CommitRequest = pendingCR if pendingCR == nil { return } @@ -1596,8 +1596,8 @@ func (l *branchWorkerEventLoop) carryRequestOnWrite(pendingWrite *PendingWrite, // target's scan stays fresh, but it commits nothing, and the push would report that as // AlreadyPresent, so only the request fails. See write_gate.go. func (l *branchWorkerEventLoop) gateBuiltWindow( - pendingCR *saveRequestID, target ResolvedTargetMetadata, reason windowFinalizeReason, -) (*saveRequestID, bool) { + pendingCR *commitRequestID, target ResolvedTargetMetadata, reason windowFinalizeReason, +) (*commitRequestID, bool) { if err := l.w.targetWriteRefusal(target.Name, target.Namespace, false); err != nil { l.w.Log.V(1).Info("Discarding built window: render fidelity closed while it was finalized", "reason", string(reason), "gitTarget", target.Namespace+"/"+target.Name) @@ -1605,19 +1605,19 @@ func (l *branchWorkerEventLoop) gateBuiltWindow( return nil, false } if pendingCR != nil && target.Suspend { - l.resolveSaveRequest(*pendingCR, FinalizeResult{Err: errTargetSuspended}) + l.resolveCommitRequest(*pendingCR, FinalizeResult{Err: errTargetSuspended}) return nil, true } return pendingCR, true } // dropOpenWindow discards a window whose finalize failed before it was decided, resolving any -// attached SaveRequest as Failed so the controller does not poll forever. -func (l *branchWorkerEventLoop) dropOpenWindow(pendingCR *saveRequestID, cause error) { +// attached CommitRequest as Failed so the controller does not poll forever. +func (l *branchWorkerEventLoop) dropOpenWindow(pendingCR *commitRequestID, cause error) { l.openWindow = nil l.windowBytes = 0 if pendingCR != nil { - l.resolveSaveRequest(*pendingCR, FinalizeResult{Branch: l.w.Branch, Err: cause}) + l.resolveCommitRequest(*pendingCR, FinalizeResult{Branch: l.w.Branch, Err: cause}) } } @@ -1684,15 +1684,15 @@ func (l *branchWorkerEventLoop) pushPending() { if err != nil { // Leave pendingWrites in place; do NOT advance lastPushAt — the // design specifies lastPushAt only advances on a successful push. A - // SaveRequest riding a retained write stays unresolved while we retry. + // CommitRequest riding a retained write stays unresolved while we retry. l.publicationFailed(err, "Push failed; pending writes retained for retry") l.settleReplayRefusals(refused) return } - // The writes are now on the remote: resolve every SaveRequest riding one with + // The writes are now on the remote: resolve every CommitRequest riding one with // the pushed commit's own SHA (§6.5) — "Committed" means "on the remote". - l.resolvePushedSaveRequests() + l.resolvePushedCommitRequests() l.pendingWrites = nil l.pendingWritesBytes = 0 @@ -1710,7 +1710,7 @@ func (l *branchWorkerEventLoop) publicationFailed(err error, message string) { l.w.Log.Error(err, message, "pendingWrites", len(l.pendingWrites), "retryAt", l.retry.due) } -// resolvePushedSaveRequests settles every SaveRequest carried by a just-pushed write, now +// resolvePushedCommitRequests settles every CommitRequest carried by a just-pushed write, now // that the remote has spoken. // // Two outcomes, decided by whether the write ended up with a commit of its own. A real SHA is @@ -1719,27 +1719,27 @@ func (l *branchWorkerEventLoop) publicationFailed(err error, message string) { // push confirmed the remote agreed, which is the difference between this and resolving at // finalize time. If the remote had moved, the replay would have turned this write into a real // commit before we got here, and it would resolve Committed instead. -func (l *branchWorkerEventLoop) resolvePushedSaveRequests() { +func (l *branchWorkerEventLoop) resolvePushedCommitRequests() { for i := range l.pendingWrites { pw := l.pendingWrites[i] - if pw.SaveRequest == nil { + if pw.CommitRequest == nil { continue } if pw.CommitSHA.IsZero() { - l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{Outcome: FinalizeAlreadyPresent}) + l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{Outcome: FinalizeAlreadyPresent}) continue } if pw.emptyCommitted { // Nothing changed, and the request asked to record its message anyway. The outcome // keeps the cause, and the commit proves the message is in Git. - l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{ + l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{ Outcome: pw.nothingToCommitOutcome(), Commit: pw.CommitSHA.String(), Branch: l.w.Branch, }) continue } - l.resolveSaveRequest(*pw.SaveRequest, FinalizeResult{ + l.resolveCommitRequest(*pw.CommitRequest, FinalizeResult{ Outcome: FinalizeCommitted, Commit: pw.CommitSHA.String(), Branch: l.w.Branch, @@ -2858,8 +2858,8 @@ func (w *BranchWorker) recordCommitFailure(reason string) { // Commit window timer sources, reported as the commit_windows_total `timer_source` label. const ( - windowTimerSourceTarget = "target" - windowTimerSourceSaveRequest = "save_request" + windowTimerSourceTarget = "target" + windowTimerSourceCommitRequest = "commit_request" ) // metricLabel is the commit_windows_total `close_reason` value for this reason. The log field keeps @@ -2906,7 +2906,7 @@ func (w *BranchWorker) recordWindowClosed(window *openWindow, reason windowFinal if telemetry.GitCommitWindowsTotal != nil { source := windowTimerSourceTarget if window.pendingCR != nil { - source = windowTimerSourceSaveRequest + source = windowTimerSourceCommitRequest } telemetry.GitCommitWindowsTotal.Add(ctx, 1, metric.WithAttributes(append(target, attribute.String("close_reason", reason.metricLabel()), diff --git a/internal/git/branch_worker_empty_repo_test.go b/internal/git/branch_worker_empty_repo_test.go index 53e5092e..3795235d 100644 --- a/internal/git/branch_worker_empty_repo_test.go +++ b/internal/git/branch_worker_empty_repo_test.go @@ -107,7 +107,7 @@ func branchOnDisk(t *testing.T, repoDir, branch, content string) plumbing.Hash { // standbyRequest waits for a window and closes it at once: the write it attaches to is committed // and, in a cooldown, retained. -func standbyRequest() *AttachSaveRequest { +func standbyRequest() *AttachCommitRequest { req := attachReq("alice", time.Hour) req.GitTargetName = newBranchTarget req.MaxDuration = 0 diff --git a/internal/git/branch_worker_metrics_test.go b/internal/git/branch_worker_metrics_test.go index 876f5491..02313b25 100644 --- a/internal/git/branch_worker_metrics_test.go +++ b/internal/git/branch_worker_metrics_test.go @@ -242,7 +242,7 @@ func TestHandleShutdown_DrainsBufferedItemsToZeroDepth(t *testing.T) { assert.Equal(t, int64(0), w.queueDepth(), "a drained, exiting worker must read depth 0") } -// A SaveRequest attach still buffered at shutdown is fire-and-forget: it is +// A CommitRequest attach still buffered at shutdown is fire-and-forget: it is // simply drained from the inflight count (the controller re-sends on its next // poll), so the exiting worker's depth gauge settles to 0. func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { @@ -250,7 +250,7 @@ func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { require.NoError(t, err) w := newMetricsTestWorker() - w.EnqueueAttach(&AttachSaveRequest{Namespace: "default", Name: "save", Author: "alice"}) + w.EnqueueAttach(&AttachCommitRequest{Namespace: "default", Name: "save", Author: "alice"}) require.Equal(t, int64(1), w.inflightItems.Load()) loop := newBranchWorkerEventLoop(w, time.Second) @@ -260,7 +260,7 @@ func TestHandleShutdown_DrainsBufferedAttach(t *testing.T) { "a buffered attach must be drained from the inflight count on shutdown") } -// The message_source label is what separates a commit a person named through a SaveRequest from +// The message_source label is what separates a commit a person named through a CommitRequest from // automatic mirroring, so it is asserted on the COLLECTED samples rather than only on // messageSource(): a label that never reaches the exporter is not observable. func TestRecordPendingWritesMetrics_LabelsCommitsByMessageSource(t *testing.T) { @@ -287,9 +287,9 @@ func TestRecordPendingWritesMetrics_LabelsCommitsByMessageSource(t *testing.T) { }, 3) for source, want := range map[string]int64{ - messageSourceLive: 1, - messageSourceSaveRequest: 1, - messageSourceReconcile: 1, + messageSourceLive: 1, + messageSourceCommitRequest: 1, + messageSourceReconcile: 1, } { labels := workerLabels() labels["message_source"] = source @@ -333,7 +333,7 @@ func TestRecordPendingWritesMetrics_MessageSourceSplitsOneAuthorKind(t *testing. require.True(t, ok) assert.Equal(t, int64(1), live) - labels["message_source"] = messageSourceSaveRequest + labels["message_source"] = messageSourceCommitRequest request, ok := telemetry.CollectInt64Sum(reader, commitsTotalMetric, labels) require.True(t, ok) assert.Equal(t, int64(1), request) diff --git a/internal/git/branch_worker_queue_depth_test.go b/internal/git/branch_worker_queue_depth_test.go index 20da5b06..f76f03aa 100644 --- a/internal/git/branch_worker_queue_depth_test.go +++ b/internal/git/branch_worker_queue_depth_test.go @@ -24,7 +24,7 @@ import ( // queueDepth() sums the in-flight count and the retained-work flag, so an item that opened a commit // window is covered by the first until the second is published. Releasing the count first leaves a // window where neither covers it, and the loop does real work inside that window -// (serviceSaveRequests, advance, serviceAppliers) before the flag is published. A scrape +// (serviceCommitRequests, advance, serviceAppliers) before the flag is published. A scrape // landing there reports a drained queue for a worker holding an uncommitted event, which can satisfy // a drain gate early. The loop releases the count only after the wake that handled the item. func TestHandledItem_PublishesRetainedWorkBeforeReleasingInflight(t *testing.T) { diff --git a/internal/git/commit.go b/internal/git/commit.go index 165fc47c..d2b82b7e 100644 --- a/internal/git/commit.go +++ b/internal/git/commit.go @@ -55,7 +55,7 @@ func renderLiveCommitMessage(pendingWrite PendingWrite, config CommitConfig) (st return renderCommitTemplate("live", config.Message.LiveTemplate, pendingWrite.liveMessageData()) } -// renderRequestCommitMessage frames an attached SaveRequest's message with the target's +// renderRequestCommitMessage frames an attached CommitRequest's message with the target's // requestTemplate. The literal rides in as .RequestMessage rather than being parsed, so nothing a // requester wrote is ever executed. func renderRequestCommitMessage(pendingWrite PendingWrite, config CommitConfig) (string, error) { @@ -265,7 +265,7 @@ func validateRequestTemplate(config CommitConfig, events []Event) error { // requester's reason reaches the commit whatever the window happened to contain. if !strings.Contains(rendered, probe) { return errors.New("requestTemplate must render {{.RequestMessage}}: as written it would " + - "drop the SaveRequest's message from the commit. Omit requestTemplate to commit that " + + "drop the CommitRequest's message from the commit. Omit requestTemplate to commit that " + "message verbatim") } return nil diff --git a/internal/git/commit_executor.go b/internal/git/commit_executor.go index b279fe1d..df76d030 100644 --- a/internal/git/commit_executor.go +++ b/internal/git/commit_executor.go @@ -81,7 +81,7 @@ func (w *BranchWorker) runPendingWrites( refusals := map[int]error{} // Index over the slice so the per-write commit hash is written back onto the - // caller's PendingWrite (§6.5): a SaveRequest riding a write resolves to this + // caller's PendingWrite (§6.5): a CommitRequest riding a write resolves to this // SHA on push, and a rebase-replay (which re-runs this loop on the retained // writes) refreshes it to the post-rebase hash. for i := range pendingWrites { @@ -164,11 +164,11 @@ func restoreWorktree(repo *gogit.Repository, start plumbing.Hash) error { // retained reports whether the loop keeps this write once it is committed: a write that changes // nothing in Git does not exist (docs/design/applier-echoes.md, F2), so a write whose plan left no // action, a window or a resync alike, is neither retained nor pushed. A save is the exception: it -// keeps its objects, and the push decides its outcome (resolvePushedSaveRequests). The checkout +// keeps its objects, and the push decides its outcome (resolvePushedCommitRequests). The checkout // counts the same writes (checkoutApplied), so the two can never disagree. func (p PendingWrite) retained() bool { switch { - case p.Kind == PendingWriteRequestRecord || p.SaveRequest != nil: + case p.Kind == PendingWriteRequestRecord || p.CommitRequest != nil: return true case p.Kind == PendingWriteResync: return p.resyncActions == nil || !p.resyncActions.empty() @@ -179,7 +179,7 @@ func (p PendingWrite) retained() bool { // keepsEveryObject reports whether a write keeps the objects whose plan changed nothing: only a // save does (docs/design/applier-echoes.md, decision 9). -func (p PendingWrite) keepsEveryObject() bool { return p.SaveRequest != nil || p.AllowEmpty } +func (p PendingWrite) keepsEveryObject() bool { return p.CommitRequest != nil || p.AllowEmpty } func (p PendingWrite) path() string { if targetPath := p.Target().Path; targetPath != "" { @@ -203,12 +203,12 @@ const ( messageResolutionUnsupported messageResolution = iota // messageResolutionPreRendered is a resync, which renders the target's reconcile template // itself and hands the result over as the write's message. That is generated text, not a - // request's override, so the SaveRequest literal contract does not bind it: an operator's + // request's override, so the CommitRequest literal contract does not bind it: an operator's // reconcile template may legitimately be long or contain a tab. messageResolutionPreRendered - // messageResolutionRequest is a message a SaveRequest supplied, used verbatim. + // messageResolutionRequest is a message a CommitRequest supplied, used verbatim. messageResolutionRequest - // messageResolutionRequestTemplate is a SaveRequest's message framed by the target's + // messageResolutionRequestTemplate is a CommitRequest's message framed by the target's // requestTemplate. The request's own bytes are never parsed — they ride in as .RequestMessage — // so the formatting policy is the operator's while the message stays the requester's. messageResolutionRequestTemplate @@ -226,7 +226,7 @@ func (p PendingWrite) resolveMessage() messageResolution { switch { case p.Kind == PendingWriteResync: return messageResolutionPreRendered - // Framing is bound to the GROUPED kind, not merely to "carries a message". A SaveRequest + // Framing is bound to the GROUPED kind, not merely to "carries a message". A CommitRequest // attaches its message to an open live window, which finalizes as PendingWriteCommit, so that // is the only write requestTemplate describes. case p.Kind == PendingWriteCommit && @@ -246,11 +246,11 @@ func (p PendingWrite) resolveMessage() messageResolution { func (r messageResolution) label() string { switch r { case messageResolutionRequest: - return messageSourceSaveRequest + return messageSourceCommitRequest case messageResolutionRequestTemplate: - return messageSourceSaveRequestFramed + return messageSourceCommitRequestFramed case messageResolutionRequestFallback: - return messageSourceSaveRequestFallback + return messageSourceCommitRequestFallback case messageResolutionPreRendered: return messageSourceReconcile case messageResolutionLiveTemplate: @@ -294,7 +294,7 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes // Validated on THIS arm too, not just the verbatim one. The literal reaches Git either // way — framed on success, and by itself on the fallback below — so framing must not // become a route that smuggles a message past the check the verbatim path enforces. - // The controller validates earlier (saverequest_controller.go), which makes this + // The controller validates earlier (commitrequest_controller.go), which makes this // belt-and-braces rather than reachable today; it is exactly the kind of invariant that // stops being true once a second producer of PendingWrite appears. if err = ValidateLiteralCommitMessage(p.CommitMessage); err == nil { @@ -315,7 +315,7 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes } switch { case p.TakenOver: - message = withTakeOverNote(message, p.SaveRequest) + message = withTakeOverNote(message, p.CommitRequest) case p.SavedBy.Username != "": message = withSavedByNote(message, p.SavedBy) } @@ -323,17 +323,17 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes return message, commitOptionsFor(p, p.CommitConfig, p.Signer, time.Now()), resolution, nil } -// withTakeOverNote ends the message of a commit whose SaveRequest took over a window with no author +// withTakeOverNote ends the message of a commit whose CommitRequest took over a window with no author // named (PendingWrite.TakenOver) with a trailer that says the author is the request's submitter. -func withTakeOverNote(message string, request *saveRequestID) string { - name := "a SaveRequest" +func withTakeOverNote(message string, request *commitRequestID) string { + name := "a CommitRequest" if request != nil { - name = "SaveRequest " + request.Namespace + "/" + request.Name + name = "CommitRequest " + request.Namespace + "/" + request.Name } return withTrailer(message, "Author-Source: "+name+"; no audit record names who made these changes") } -// withSavedByNote ends the message of a commit whose SaveRequest gave a window with no author named +// withSavedByNote ends the message of a commit whose CommitRequest gave a window with no author named // its message (PendingWrite.SavedBy) with a trailer naming the request's submitter, in the shape of a // Co-authored-by trailer. func withSavedByNote(message string, savedBy UserInfo) string { @@ -346,7 +346,7 @@ func withTrailer(message, trailer string) string { return body + "\n\n" + trailer + message[len(body):] } -// framedRequestMessage renders the target's requestTemplate around an attached SaveRequest's +// framedRequestMessage renders the target's requestTemplate around an attached CommitRequest's // message, falling back to that message verbatim if the render fails. // // The fallback deviates from liveTemplate, where a render failure fails the write, and the @@ -357,7 +357,7 @@ func withTrailer(message, trailer string) string { // harmless. // // It is never silent. The Error log names the target, and the returned resolution moves the commit -// to message_source="save_request_fallback" — because a fallback is a SUCCESSFUL commit, so no +// to message_source="commit_request_fallback" — because a fallback is a SUCCESSFUL commit, so no // condition moves, nothing is refused, and a rate is the only way an operator sees a template that // has quietly stopped applying. // @@ -365,7 +365,7 @@ func withTrailer(message, trailer string) string { func (p PendingWrite) framedRequestMessage() (string, messageResolution) { framed, err := renderRequestCommitMessage(p, p.CommitConfig) if err != nil { - log.Log.Error(err, "requestTemplate failed to render; committing the SaveRequest message verbatim", + log.Log.Error(err, "requestTemplate failed to render; committing the CommitRequest message verbatim", "gitTarget", p.Target().Namespace+"/"+p.Target().Name) return p.CommitMessage, messageResolutionRequestFallback } @@ -385,7 +385,7 @@ func (w *BranchWorker) executePendingWrite( case PendingWriteRequestRecord: return w.executeRequestRecord(ctx, worktree, *pendingWrite) case PendingWriteResync: - // Resync writes never carry a SaveRequest, so their commit hash is unused; + // Resync writes never carry a CommitRequest, so their commit hash is unused; // report ZeroHash to keep the per-write SHA bookkeeping uniform. created, err := w.executeResyncPendingWrite(ctx, repo, worktree, pendingWrite) return created, plumbing.ZeroHash, messageResolutionPreRendered, err @@ -442,7 +442,7 @@ func (w *BranchWorker) executeEventWrite( if err != nil { return 0, plumbing.ZeroHash, messageResolutionUnsupported, err } - // A window whose writes changed nothing still commits when its SaveRequest asked to record + // A window whose writes changed nothing still commits when its CommitRequest asked to record // its message: whenNothingToCommit: CommitEmpty. commitOptions.AllowEmptyCommits = recordEmpty @@ -546,7 +546,7 @@ func (w *BranchWorker) planPendingWriteEvents( return anyChanges, changedBy, nil } -// executeRequestRecord commits a SaveRequest's message with an untouched tree: the record a +// executeRequestRecord commits a CommitRequest's message with an untouched tree: the record a // request that ran out of time to attach leaves when it asked for whenNothingToCommit: CommitEmpty. func (w *BranchWorker) executeRequestRecord( ctx context.Context, @@ -563,7 +563,7 @@ func (w *BranchWorker) executeRequestRecord( return 0, plumbing.ZeroHash, messageResolutionUnsupported, fmt.Errorf("failed to create request record commit: %w", err) } - log.FromContext(ctx).Info("Empty commit created to record a SaveRequest that found nothing to commit", + log.FromContext(ctx).Info("Empty commit created to record a CommitRequest that found nothing to commit", "gitTarget", pendingWrite.GitTargetNamespace+"/"+pendingWrite.GitTargetName, "branch", w.Branch, "commit", hash.String()) return 1, hash, source, nil diff --git a/internal/git/commit_executor_test.go b/internal/git/commit_executor_test.go index 315a207d..927544de 100644 --- a/internal/git/commit_executor_test.go +++ b/internal/git/commit_executor_test.go @@ -205,7 +205,7 @@ func TestExecutor_AppliesEncryptionFromPendingWrite_NotFromWorker(t *testing.T) } // A resync renders the target's reconcile template and hands the result over as the pending -// write's message. That generated text is not a SaveRequest literal override, so the request +// write's message. That generated text is not a CommitRequest literal override, so the request // contract's length and control-character limits must not reject an operator's own template. func TestCommitMetadata_ResyncRenderedMessageIsNotHeldToTheLiteralRequestContract(t *testing.T) { longMessage := "chore: reconcile " + strings.Repeat("x", 1200) @@ -247,13 +247,13 @@ func TestMessageSource_MatchesTheMessageActuallyRendered(t *testing.T) { want: messageSourceLive, }, { - name: "an attached request message is save_request", + name: "an attached request message is commit_request", write: PendingWrite{ Kind: PendingWriteCommit, Events: []Event{makeEvent("api")}, CommitMessage: "fix(api): correct the port", }, - want: messageSourceSaveRequest, + want: messageSourceCommitRequest, message: "fix(api): correct the port", }, { diff --git a/internal/git/save_request_attach.go b/internal/git/commit_request_attach.go similarity index 86% rename from internal/git/save_request_attach.go rename to internal/git/commit_request_attach.go index c068e057..3f84231a 100644 --- a/internal/git/save_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -31,7 +31,7 @@ var ErrResyncDiscarded = errors.New("resync snapshot discarded: an apply made it // it stays pending in its place and is applied with the writes around it once the remote answers. var ErrResyncDeferred = errors.New("resync deferred until the remote can be reached") -// FinalizeOutcome is the terminal result of resolving a SaveRequest. +// FinalizeOutcome is the terminal result of resolving a CommitRequest. type FinalizeOutcome string const ( @@ -45,7 +45,7 @@ const ( // window this request could attach to. It is a refusal a human can see: the author's own edits // went into somebody else's commit, under a generated message rather than theirs. // - // Raised at expiry from pendingSaveRequest.sawForeignWindow, never at attach time: under + // Raised at expiry from pendingCommitRequest.sawForeignWindow, never at attach time: under // eager attach a request WAITS for a matching window rather than being refused on sight, so // "a foreign window is open right now" is not yet an outcome. FinalizeWindowMismatch FinalizeOutcome = "WindowMismatch" @@ -60,38 +60,38 @@ const ( FinalizeSaveInProgress FinalizeOutcome = "SaveInProgress" ) -// SaveRequestPhase is where an unresolved SaveRequest stands on the worker. It is what the +// CommitRequestPhase is where an unresolved CommitRequest stands on the worker. It is what the // controller reports while it waits, and it comes from the worker because sending an attach does // not prove the request was registered, attached or committed. -type SaveRequestPhase string +type CommitRequestPhase string const ( // PhaseWaitingForWorker is a request whose GitTarget has no branch worker yet, at startup or // before the target's first reconcile. It is reported by the router, since there is no worker to // report it, and the request is sent again on the controller's next poll. - PhaseWaitingForWorker SaveRequestPhase = "WaitingForWorker" + PhaseWaitingForWorker CommitRequestPhase = "WaitingForWorker" // PhaseWaitingForWindow is a registered request waiting for a window to attach to. - PhaseWaitingForWindow SaveRequestPhase = "WaitingForWindow" + PhaseWaitingForWindow CommitRequestPhase = "WaitingForWindow" // PhaseCollectingWindow is an attached request collecting writes until the window's timers close it. - PhaseCollectingWindow SaveRequestPhase = "CollectingWindow" + PhaseCollectingWindow CommitRequestPhase = "CollectingWindow" // PhaseWaitingForPush is a request whose write is decided and not yet confirmed by the remote. - PhaseWaitingForPush SaveRequestPhase = "WaitingForPush" + PhaseWaitingForPush CommitRequestPhase = "WaitingForPush" ) // Held reports whether the worker has acted on a request in this phase: it is attached to a window // or committed. From there only the worker can say how it ends, so the controller waits for it // however long that takes, instead of failing the request on a clock of its own. -func (p SaveRequestPhase) Held() bool { +func (p CommitRequestPhase) Held() bool { return p == PhaseCollectingWindow || p == PhaseWaitingForPush } -// ErrSaveRequestWithdrawn is the outcome of a request the controller withdrew before the worker +// ErrCommitRequestWithdrawn is the outcome of a request the controller withdrew before the worker // acted on it. The worker records it, so a late re-send of the attach can never register the // request again, and the controller's failure and the worker's answer are the same answer. -var ErrSaveRequestWithdrawn = errors.New("the SaveRequest was withdrawn before the worker acted on it") +var ErrCommitRequestWithdrawn = errors.New("the CommitRequest was withdrawn before the worker acted on it") -// FinalizeResult carries the resolved outcome of a SaveRequest back to the -// controller, polled via LookupSaveRequestOutcome. +// FinalizeResult carries the resolved outcome of a CommitRequest back to the +// controller, polled via LookupCommitRequestOutcome. type FinalizeResult struct { // Outcome is set when Err is nil. Outcome FinalizeOutcome @@ -103,19 +103,19 @@ type FinalizeResult struct { Err error // Phase is where an UNRESOLVED request stands on the worker; empty once resolved, and empty // before the worker has registered it. - Phase SaveRequestPhase + Phase CommitRequestPhase // Held is why a request the worker holds has not reached the remote yet: the worker's // publication report while it cannot publish, empty otherwise. See PublicationStatus. Held string } -// AttachSaveRequest is the "attach this SaveRequest to the author's window, with its +// AttachCommitRequest is the "attach this CommitRequest to the author's window, with its // message and its timers" work item. It rides the same per-worker FIFO // event queue as resource events, so by audit-stream ordering it is processed // after every earlier write for that worker. Re-sends are idempotent: the worker // keys pending requests by identity and keeps everything the first delivery set. -type AttachSaveRequest struct { - // Namespace, Name, UID identify the SaveRequest. UID may be empty (a +type AttachCommitRequest struct { + // Namespace, Name, UID identify the CommitRequest. UID may be empty (a // Metadata-level audit policy can omit it); identity then keys on // namespace/name only. Namespace string @@ -127,7 +127,7 @@ type AttachSaveRequest struct { // matches is attached; this binds "the open window" to "the requesting // author's open window". Author string - // Attribution is the outcome of attributing THIS SaveRequest, from the command-authorship + // Attribution is the outcome of attributing THIS CommitRequest, from the command-authorship // path. It is matched alongside Author rather than inferred from it, because an empty Author // alone cannot say whether an actor was sought: it is both "attribution is off" and // "attribution ran and named nobody". Only the NamesActor half is compared against the @@ -166,7 +166,7 @@ type AttachSaveRequest struct { Unattributed UnattributedClaim } -// UnattributedClaim is what a SaveRequest may claim of a commit window whose writes name no author. +// UnattributedClaim is what a CommitRequest may claim of a commit window whose writes name no author. type UnattributedClaim string const ( @@ -180,24 +180,24 @@ const ( UnattributedClaimTakeOver UnattributedClaim = "TakeOver" ) -// saveRequestID is the worker-local key for a SaveRequest: its namespaced -// name plus UID when available. Two SaveRequests with the same name but +// commitRequestID is the worker-local key for a CommitRequest: its namespaced +// name plus UID when available. Two CommitRequests with the same name but // different UIDs (a delete-and-recreate) are distinct. -type saveRequestID struct { +type commitRequestID struct { Namespace string Name string UID string } -func (a AttachSaveRequest) id() saveRequestID { - return saveRequestID{Namespace: a.Namespace, Name: a.Name, UID: a.UID} +func (a AttachCommitRequest) id() commitRequestID { + return commitRequestID{Namespace: a.Namespace, Name: a.Name, UID: a.UID} } -// pendingSaveRequest is a SaveRequest registered with the worker and not yet +// pendingCommitRequest is a CommitRequest registered with the worker and not yet // resolved: waiting for a same-author window to attach to, attached and collecting until the // window's timers close it, or committed locally and awaiting the push that settles it. -type pendingSaveRequest struct { - id saveRequestID +type pendingCommitRequest struct { + id commitRequestID author string attribution AttributionOutcome gitTargetName string @@ -219,7 +219,7 @@ type pendingSaveRequest struct { // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. catchUpNow bool // unattributed is what the request may claim of a window whose writes name no author - // (AttachSaveRequest.Unattributed). + // (AttachCommitRequest.Unattributed). unattributed UnattributedClaim // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays @@ -249,9 +249,9 @@ type pendingSaveRequest struct { // to save" and "someone else held the window the whole time" are different events with the same // shape, and only the second one silently substitutes a generated commit message for the sentence // the request's author typed. FinalizeWindowMismatch has always been declared, surfaced by the -// controller as the WindowMismatch reason, and counted by save_requests_total; from the eager +// controller as the WindowMismatch reason, and counted by commit_requests_total; from the eager // attach onwards nothing produced it, so every one of those refusals reported as NoOpenWindow. -func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { +func (p *pendingCommitRequest) expiryOutcome() FinalizeOutcome { if p.sawForeignWindow { return FinalizeWindowMismatch } @@ -279,7 +279,7 @@ func (p *pendingSaveRequest) expiryOutcome() FinalizeOutcome { // outcome fields being right, so any path that leaves an outcome unset while the author IS set // would let one author's request finalize another's window. Comparing both costs nothing and // keeps "bob never attaches to alice's window" true regardless of what the outcomes say. -func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { +func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { if p == nil || w == nil { return false } @@ -291,21 +291,21 @@ func (p *pendingSaveRequest) matchesWindow(w *openWindow) bool { // sameSaver reports whether two requests would save one window: the same GitTarget and the same // author, by the comparison matchesWindow makes. -func (p *pendingSaveRequest) sameSaver(other *pendingSaveRequest) bool { +func (p *pendingCommitRequest) sameSaver(other *pendingCommitRequest) bool { return p.gitTargetName == other.gitTargetName && p.gitTargetNamespace == other.gitTargetNamespace && p.author == other.author && p.attribution.NamesActor() == other.attribution.NamesActor() } // claims reports whether the request matches a window on its GitTarget whose writes name no author // only because it may claim such a window: it names its submitter, and asked for more than its own -// windows (AttachSaveRequest.Unattributed). -func (p *pendingSaveRequest) claims(w *openWindow) bool { +// windows (AttachCommitRequest.Unattributed). +func (p *pendingCommitRequest) claims(w *openWindow) bool { return p.unattributed != UnattributedClaimNone && p.attribution.NamesActor() && !w.Attribution.NamesActor() } -// saveRequestOutcomeEntry is a resolved outcome retained for the controller to +// commitRequestOutcomeEntry is a resolved outcome retained for the controller to // poll, GC'd by age. -type saveRequestOutcomeEntry struct { +type commitRequestOutcomeEntry struct { result FinalizeResult resolvedAt time.Time } diff --git a/internal/git/save_request_attach_loop.go b/internal/git/commit_request_attach_loop.go similarity index 74% rename from internal/git/save_request_attach_loop.go rename to internal/git/commit_request_attach_loop.go index 1f9997cf..fab96e00 100644 --- a/internal/git/save_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -11,14 +11,14 @@ import ( "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// This file holds the worker-loop side of SaveRequest attach. All methods run on the single +// This file holds the worker-loop side of CommitRequest attach. All methods run on the single // event loop goroutine, so pendingCRs needs no locking; only the resolved-outcome and phase tables // (BranchWorker.crOutcomes, crPhases) cross goroutines and are mutex-guarded. // -// The lifecycle of one SaveRequest, worker-side. A window is only ever opened by a write; a +// The lifecycle of one CommitRequest, worker-side. A window is only ever opened by a write; a // request attaches to one and never opens one. // -// register (handleAttachSaveRequest) → attach: Next closes the author's open window first +// register (handleAttachCommitRequest) → attach: Next closes the author's open window first // → [matching window open, CurrentOrNext] Attached // → otherwise WaitingForWindow // a same-author window opens before the attach deadline → Attached (first come, first served) @@ -27,35 +27,35 @@ import ( // attach deadline passes while WaitingForWindow → resolved NoOpenWindow, or WindowMismatch when // a window it could not attach to was open meanwhile; never a later window -// saveRequestOutcomeTTL bounds how long a resolved SaveRequest outcome is +// commitRequestOutcomeTTL bounds how long a resolved CommitRequest outcome is // retained for the controller to poll before it is GC'd. It comfortably exceeds // the controller's poll cadence. It is not what keeps a withdrawn request withdrawn: // an outcome is never collected while an attach for it is still queued (crQueuedAttaches), // and the controller sends no attach once it has started withdrawing. -const saveRequestOutcomeTTL = 15 * time.Minute +const commitRequestOutcomeTTL = 15 * time.Minute // ErrBranchWorkerStopped is the outcome of a request whose branch worker stopped before it could // finish it. Once the loop has exited nothing can publish the request, so this answer is final. -var ErrBranchWorkerStopped = errors.New("the GitTarget's branch worker stopped before it finished the SaveRequest") +var ErrBranchWorkerStopped = errors.New("the GitTarget's branch worker stopped before it finished the CommitRequest") -// recordSaveRequestOutcome stores a resolved outcome and GCs stale entries. The +// recordCommitRequestOutcome stores a resolved outcome and GCs stale entries. The // event loop is the only caller, but it takes the mutex because the controller -// reads concurrently via LookupSaveRequestOutcome. -func (w *BranchWorker) recordSaveRequestOutcome(id saveRequestID, result FinalizeResult) { +// reads concurrently via LookupCommitRequestOutcome. +func (w *BranchWorker) recordCommitRequestOutcome(id commitRequestID, result FinalizeResult) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - w.recordSaveRequestOutcomeLocked(id, result) + w.recordCommitRequestOutcomeLocked(id, result) } -// recordSaveRequestOutcomeLocked is recordSaveRequestOutcome for a caller holding crOutcomesMu. -func (w *BranchWorker) recordSaveRequestOutcomeLocked(id saveRequestID, result FinalizeResult) { +// recordCommitRequestOutcomeLocked is recordCommitRequestOutcome for a caller holding crOutcomesMu. +func (w *BranchWorker) recordCommitRequestOutcomeLocked(id commitRequestID, result FinalizeResult) { if w.crOutcomes == nil { - w.crOutcomes = map[saveRequestID]saveRequestOutcomeEntry{} + w.crOutcomes = map[commitRequestID]commitRequestOutcomeEntry{} } now := time.Now() - w.crOutcomes[id] = saveRequestOutcomeEntry{result: result, resolvedAt: now} + w.crOutcomes[id] = commitRequestOutcomeEntry{result: result, resolvedAt: now} for k, entry := range w.crOutcomes { - if now.Sub(entry.resolvedAt) > saveRequestOutcomeTTL && w.crQueuedAttaches[k] == 0 { + if now.Sub(entry.resolvedAt) > commitRequestOutcomeTTL && w.crQueuedAttaches[k] == 0 { delete(w.crOutcomes, k) w.crOwners.release(k, w) } @@ -65,11 +65,11 @@ func (w *BranchWorker) recordSaveRequestOutcomeLocked(id saveRequestID, result F // countQueuedAttach records an attach entering the FIFO. The caller holds pendingResyncsMu, so the // count is raised before the loop can receive the item. -func (w *BranchWorker) countQueuedAttach(id saveRequestID) { +func (w *BranchWorker) countQueuedAttach(id commitRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if w.crQueuedAttaches == nil { - w.crQueuedAttaches = map[saveRequestID]int{} + w.crQueuedAttaches = map[commitRequestID]int{} } w.crQueuedAttaches[id]++ } @@ -77,7 +77,7 @@ func (w *BranchWorker) countQueuedAttach(id saveRequestID) { // uncountQueuedAttach records an attach leaving the FIFO: handled by the loop, dropped on a full // queue, or drained by an exiting worker. A request the worker then knows nothing else about is // given back, so the controller's next poll finds the worker its GitTarget names. -func (w *BranchWorker) uncountQueuedAttach(id saveRequestID) { +func (w *BranchWorker) uncountQueuedAttach(id commitRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if n := w.crQueuedAttaches[id]; n > 1 { @@ -92,7 +92,7 @@ func (w *BranchWorker) uncountQueuedAttach(id saveRequestID) { } } -// settleSaveRequestsAfterExit runs once the event loop has returned: after a shutdown, after a +// settleCommitRequestsAfterExit runs once the event loop has returned: after a shutdown, after a // retirement, or when the loop never started because its GitProvider could not be read. Either way // nothing will act on a request again, and every request the worker knew must get an answer // rather than wait for one that cannot come. @@ -103,7 +103,7 @@ func (w *BranchWorker) uncountQueuedAttach(id saveRequestID) { // controller is free to send it to the worker the GitTarget names now, or to withdraw it. // // From then on a withdraw is answered at once (answerWithdrawAfterExit). -func (w *BranchWorker) settleSaveRequestsAfterExit() { +func (w *BranchWorker) settleCommitRequestsAfterExit() { w.pendingResyncsMu.Lock() w.stoppingState = true w.pendingResyncsMu.Unlock() @@ -114,7 +114,7 @@ func (w *BranchWorker) settleSaveRequestsAfterExit() { w.crOutcomesMu.Lock() for id, phase := range w.crPhases { if phase.Held() { - w.recordSaveRequestOutcomeLocked(id, FinalizeResult{Branch: w.Branch, Err: ErrBranchWorkerStopped}) + w.recordCommitRequestOutcomeLocked(id, FinalizeResult{Branch: w.Branch, Err: ErrBranchWorkerStopped}) } else { w.crOwners.release(id, w) } @@ -129,30 +129,30 @@ func (w *BranchWorker) settleSaveRequestsAfterExit() { // answerWithdrawAfterExit resolves a withdraw sent to a worker whose loop has exited. Nothing can // act on the request any more, so it is withdrawn for good, unless it already has an outcome. -func (w *BranchWorker) answerWithdrawAfterExit(id saveRequestID) { +func (w *BranchWorker) answerWithdrawAfterExit(id commitRequestID) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if _, resolved := w.crOutcomes[id]; resolved { return } - w.recordSaveRequestOutcomeLocked(id, FinalizeResult{ + w.recordCommitRequestOutcomeLocked(id, FinalizeResult{ Branch: w.Branch, - Err: errors.Join(ErrSaveRequestWithdrawn, ErrBranchWorkerStopped), + Err: errors.Join(ErrCommitRequestWithdrawn, ErrBranchWorkerStopped), }) } -// LookupSaveRequestOutcome returns a resolved SaveRequest outcome, or ok=false +// LookupCommitRequestOutcome returns a resolved CommitRequest outcome, or ok=false // when the request is still in flight (or already GC'd). The controller polls this -// after sending its AttachSaveRequest. -func (w *BranchWorker) LookupSaveRequestOutcome(namespace, name, uid string) (FinalizeResult, bool) { +// after sending its AttachCommitRequest. +func (w *BranchWorker) LookupCommitRequestOutcome(namespace, name, uid string) (FinalizeResult, bool) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - entry, ok := w.crOutcomes[saveRequestID{Namespace: namespace, Name: name, UID: uid}] + entry, ok := w.crOutcomes[commitRequestID{Namespace: namespace, Name: name, UID: uid}] return entry.result, ok } -// setSaveRequestPhase records where an unresolved request stands; an empty phase forgets it. -func (w *BranchWorker) setSaveRequestPhase(id saveRequestID, phase SaveRequestPhase) { +// setCommitRequestPhase records where an unresolved request stands; an empty phase forgets it. +func (w *BranchWorker) setCommitRequestPhase(id commitRequestID, phase CommitRequestPhase) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if phase == "" { @@ -160,40 +160,40 @@ func (w *BranchWorker) setSaveRequestPhase(id saveRequestID, phase SaveRequestPh return } if w.crPhases == nil { - w.crPhases = map[saveRequestID]SaveRequestPhase{} + w.crPhases = map[commitRequestID]CommitRequestPhase{} } w.crPhases[id] = phase } -// LookupSaveRequestPhase returns where an unresolved request stands on this worker, or "" when +// LookupCommitRequestPhase returns where an unresolved request stands on this worker, or "" when // the worker has not registered it (yet) or it is resolved. -func (w *BranchWorker) LookupSaveRequestPhase(namespace, name, uid string) SaveRequestPhase { +func (w *BranchWorker) LookupCommitRequestPhase(namespace, name, uid string) CommitRequestPhase { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - return w.crPhases[saveRequestID{Namespace: namespace, Name: name, UID: uid}] + return w.crPhases[commitRequestID{Namespace: namespace, Name: name, UID: uid}] } -// hasSaveRequestOutcome reports whether a request is already resolved, so a late +// hasCommitRequestOutcome reports whether a request is already resolved, so a late // idempotent re-send of its attach is a no-op. -func (w *BranchWorker) hasSaveRequestOutcome(id saveRequestID) bool { +func (w *BranchWorker) hasCommitRequestOutcome(id commitRequestID) bool { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() _, ok := w.crOutcomes[id] return ok } -// handleAttachSaveRequest registers a SaveRequest with the worker. Everything is stamped once: +// handleAttachCommitRequest registers a CommitRequest with the worker. Everything is stamped once: // an idempotent re-send restarts nothing and closes nothing. -func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) { +func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitRequest) { id := req.id() // Uncounted only once the request is registered or resolved, so the worker never looks as if it // had forgotten a request it is in the middle of taking. defer l.w.uncountQueuedAttach(id) - if l.w.hasSaveRequestOutcome(id) { + if l.w.hasCommitRequestOutcome(id) { return // already resolved: a late idempotent re-send. } if l.pendingCRs == nil { - l.pendingCRs = map[saveRequestID]*pendingSaveRequest{} + l.pendingCRs = map[commitRequestID]*pendingCommitRequest{} } if _, exists := l.pendingCRs[id]; exists { return // idempotent re-send: keep the first registration. @@ -202,7 +202,7 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) // delayed ingestion pipeline this lets attachTimeout cover only the inter-stream spread // instead of the absolute latency. l.crSeq++ - pcr := &pendingSaveRequest{ + pcr := &pendingCommitRequest{ id: id, author: req.Author, attribution: req.Attribution, @@ -219,15 +219,15 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) unattributed: req.Unattributed, } if earlier := l.saveInProgress(pcr, req.Attach); earlier != nil { - l.w.recordSaveRequestOutcome(id, FinalizeResult{Branch: l.w.Branch, Outcome: FinalizeSaveInProgress}) - l.w.Log.Info("SaveRequest refused: an earlier save of the same author on the GitTarget is in progress", + l.w.recordCommitRequestOutcome(id, FinalizeResult{Branch: l.w.Branch, Outcome: FinalizeSaveInProgress}) + l.w.Log.Info("CommitRequest refused: an earlier save of the same author on the GitTarget is in progress", "request", id.Namespace+"/"+id.Name, "earlier", earlier.id.Namespace+"/"+earlier.id.Name, "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName) return } l.pendingCRs[id] = pcr - l.w.setSaveRequestPhase(id, PhaseWaitingForWindow) - l.w.Log.Info("SaveRequest registered with worker", + l.w.setCommitRequestPhase(id, PhaseWaitingForWindow) + l.w.Log.Info("CommitRequest registered with worker", "request", id.Namespace+"/"+id.Name, "author", req.Author, "target", req.GitTargetNamespace+"/"+req.GitTargetName, @@ -265,8 +265,8 @@ func (l *branchWorkerEventLoop) handleAttachSaveRequest(req *AttachSaveRequest) // is collecting: it closes that window under the earlier request's message first. A request already // waiting for its push holds no window, so it races nothing. func (l *branchWorkerEventLoop) saveInProgress( - pcr *pendingSaveRequest, attach v1alpha3.AttachPolicy, -) *pendingSaveRequest { + pcr *pendingCommitRequest, attach v1alpha3.AttachPolicy, +) *pendingCommitRequest { now := time.Now() for _, earlier := range l.pendingCRs { if !earlier.sameSaver(pcr) { @@ -284,31 +284,31 @@ func (l *branchWorkerEventLoop) saveInProgress( return nil } -// serviceSaveRequests runs after every loop wake: note foreign windows, resolve waiting requests +// serviceCommitRequests runs after every loop wake: note foreign windows, resolve waiting requests // whose attach deadline passed, attach the next waiting request to an open window, and re-arm the // attach timer. // // Expiry runs BEFORE attach. When a request's deadline and a matching write are ready on the same // wake, the request has already run out: it resolves, and the write keeps its own window. -func (l *branchWorkerEventLoop) serviceSaveRequests() { +func (l *branchWorkerEventLoop) serviceCommitRequests() { if len(l.pendingCRs) == 0 { l.stopAttachTimer() return } l.noteForeignWindow() - l.expireWaitingSaveRequests() - l.attachWaitingSaveRequests() + l.expireWaitingCommitRequests() + l.attachWaitingCommitRequests() l.rearmAttachTimer() } // waiting reports whether a request is still looking for a window it may attach to at now. -func (p *pendingSaveRequest) waiting(now time.Time) bool { +func (p *pendingCommitRequest) waiting(now time.Time) bool { return !p.attached && p.attachDeadline.After(now) } // noteForeignWindow records, on every waiting request, that a window it cannot attach to is open. // -// It runs BEFORE attachWaitingSaveRequests and independently of it, because that function +// It runs BEFORE attachWaitingCommitRequests and independently of it, because that function // returns early when the window is already taken — so the scan inside it cannot be the place // this is observed. It also runs on every pass rather than at expiry: the foreign window is // finalized on its own timer, usually before this request's wait runs out. @@ -331,7 +331,7 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { continue } pcr.sawForeignWindow = true - l.w.Log.Info("SaveRequest is waiting on a window that belongs to someone else", + l.w.Log.Info("CommitRequest is waiting on a window that belongs to someone else", "request", pcr.id.Namespace+"/"+pcr.id.Name, "requestAuthor", pcr.author, "requestTarget", pcr.gitTargetNamespace+"/"+pcr.gitTargetName, @@ -340,15 +340,15 @@ func (l *branchWorkerEventLoop) noteForeignWindow() { } } -// attachWaitingSaveRequests attaches the first-registered waiting request that matches the open +// attachWaitingCommitRequests attaches the first-registered waiting request that matches the open // window, when the window carries none yet. A window carries at most one request; a second waits // for the next window, and a request whose deadline has passed takes none. -func (l *branchWorkerEventLoop) attachWaitingSaveRequests() { +func (l *branchWorkerEventLoop) attachWaitingCommitRequests() { if l.openWindow == nil || l.openWindow.pendingCR != nil { return } now := time.Now() - var first *pendingSaveRequest + var first *pendingCommitRequest for _, pcr := range l.pendingCRs { if !pcr.waiting(now) || !pcr.matchesWindow(l.openWindow) { continue @@ -366,7 +366,7 @@ func (l *branchWorkerEventLoop) attachWaitingSaveRequests() { // with the request's own: the request's maxDuration runs from this attach, and so does its first // idle interval. A zero maxDuration closes the window right here. The message is the window's to // keep from now on, so it moves into the window's charge. -func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { +func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { now := time.Now() l.openWindow.pendingMessage = pcr.message id := pcr.id @@ -381,12 +381,12 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { l.openWindow.lastWriteAt = now if pcr.claims(l.openWindow) { l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} - l.w.Log.Info("SaveRequest claimed a window whose writes name no author", + l.w.Log.Info("CommitRequest claimed a window whose writes name no author", "request", id.Namespace+"/"+id.Name, "author", pcr.author, "claim", string(pcr.unattributed), "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) } - l.w.setSaveRequestPhase(id, PhaseCollectingWindow) - l.w.Log.Info("SaveRequest attached to open window", + l.w.setCommitRequestPhase(id, PhaseCollectingWindow) + l.w.Log.Info("CommitRequest attached to open window", "request", id.Namespace+"/"+id.Name, "author", pcr.author, "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName, @@ -394,52 +394,52 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingSaveRequest) { l.closeOrArmWindow() } -// expireWaitingSaveRequests resolves every request whose attach deadline passed while it was +// expireWaitingCommitRequests resolves every request whose attach deadline passed while it was // still waiting. An attached request is not here: its window's timers close it, and whichever path // closes the window resolves it. -func (l *branchWorkerEventLoop) expireWaitingSaveRequests() { +func (l *branchWorkerEventLoop) expireWaitingCommitRequests() { now := time.Now() for id, pcr := range l.pendingCRs { if pcr.attached || pcr.attachDeadline.After(now) { continue } // Which of the two refusals this is depends on whether anything was open that this - // request could not have: see pendingSaveRequest.expiryOutcome. + // request could not have: see pendingCommitRequest.expiryOutcome. outcome := pcr.expiryOutcome() if outcome == FinalizeNoOpenWindow { // NoWindow says the save saw no writes. On a target that refuses them that is not the // cause, and with CommitEmpty the record would say so in Git: see write_gate.go. if err := l.w.requestWriteRefusal(l.w.ctx, pcr); err != nil { - l.resolveSaveRequest(id, FinalizeResult{Err: err}) + l.resolveCommitRequest(id, FinalizeResult{Err: err}) continue } } if outcome == FinalizeNoOpenWindow && pcr.commitEmpty { - recorded, err := l.recordSaveRequest(pcr) + recorded, err := l.recordCommitRequest(pcr) if err != nil { // The request asked for its message to be recorded and it was not: a failure, not // a benign NoWindow that would report Ready=True for a save that left no trace. - l.resolveSaveRequest(id, FinalizeResult{Err: err}) + l.resolveCommitRequest(id, FinalizeResult{Err: err}) continue } if recorded { continue // it now rides the record write, and the push resolves it } } - l.resolveSaveRequest(id, FinalizeResult{Outcome: outcome}) + l.resolveCommitRequest(id, FinalizeResult{Outcome: outcome}) } } -// recordSaveRequest decides an empty commit recording a request's message, for a request that ran +// recordCommitRequest decides an empty commit recording a request's message, for a request that ran // out of time to attach and asked for whenNothingToCommit: CommitEmpty. Every reason no record can // be made is returned, including a write gate (an empty commit is a write too), so the request fails // instead of reporting a save it did not make. Once decided, the request rides the record write // like an attached request rides its window: a remote that cannot be reached holds it, and only a // failure of the commit itself fails it. -func (l *branchWorkerEventLoop) recordSaveRequest(pcr *pendingSaveRequest) (bool, error) { +func (l *branchWorkerEventLoop) recordCommitRequest(pcr *pendingCommitRequest) (bool, error) { pendingWrite, err := l.w.buildRequestRecordWrite(l.w.ctx, pcr) if err != nil { - l.w.Log.Error(err, "Cannot build the empty commit recording a SaveRequest", + l.w.Log.Error(err, "Cannot build the empty commit recording a CommitRequest", "request", pcr.id.Namespace+"/"+pcr.id.Name) return false, fmt.Errorf("record the message in an empty commit: %w", err) } @@ -454,7 +454,7 @@ func (l *branchWorkerEventLoop) recordSaveRequest(pcr *pendingSaveRequest) (bool // gets none, and the refusal is returned (see write_gate.go). func (w *BranchWorker) buildRequestRecordWrite( ctx context.Context, - pcr *pendingSaveRequest, + pcr *pendingCommitRequest, ) (*PendingWrite, error) { metadata, err := w.resolveTargetMetadata(ctx, pcr.gitTargetName, pcr.gitTargetNamespace) if err != nil { @@ -485,41 +485,41 @@ func (w *BranchWorker) buildRequestRecordWrite( }, RequestAuthor: UserInfo{Username: pcr.author}, RequestAttribution: pcr.attribution, - SaveRequest: &id, + CommitRequest: &id, }, nil } -// handleWithdrawSaveRequest cancels a request the worker has not acted on: one that is only +// handleWithdrawCommitRequest cancels a request the worker has not acted on: one that is only // waiting for a window, or that was never registered at all. A request attached to a window or // already committed is held, and withdrawing it is a no-op; the controller reads the held phase and // keeps waiting. Either way the worker's answer and the controller's agree. -func (l *branchWorkerEventLoop) handleWithdrawSaveRequest(req *AttachSaveRequest) { +func (l *branchWorkerEventLoop) handleWithdrawCommitRequest(req *AttachCommitRequest) { id := req.id() - if l.w.hasSaveRequestOutcome(id) { + if l.w.hasCommitRequestOutcome(id) { return // already resolved, or already withdrawn. } if pcr, ok := l.pendingCRs[id]; ok && pcr.attached { - l.w.Log.Info("SaveRequest withdraw ignored: the worker already holds it", + l.w.Log.Info("CommitRequest withdraw ignored: the worker already holds it", "request", id.Namespace+"/"+id.Name) return } - l.resolveSaveRequest(id, FinalizeResult{Err: ErrSaveRequestWithdrawn}) + l.resolveCommitRequest(id, FinalizeResult{Err: ErrCommitRequestWithdrawn}) l.rearmAttachTimer() } -// resolveSaveRequest records a request's terminal outcome for the controller to +// resolveCommitRequest records a request's terminal outcome for the controller to // poll and forgets it from the pending set. -func (l *branchWorkerEventLoop) resolveSaveRequest(id saveRequestID, result FinalizeResult) { +func (l *branchWorkerEventLoop) resolveCommitRequest(id commitRequestID, result FinalizeResult) { if result.Branch == "" { result.Branch = l.w.Branch } - l.w.recordSaveRequestOutcome(id, result) + l.w.recordCommitRequestOutcome(id, result) if pcr := l.pendingCRs[id]; pcr != nil && pcr.catchUpNow { l.catchUpAfterSave(pcr, result.Commit) } delete(l.pendingCRs, id) - l.w.setSaveRequestPhase(id, "") - l.w.Log.Info("SaveRequest resolved", + l.w.setCommitRequestPhase(id, "") + l.w.Log.Info("CommitRequest resolved", "request", id.Namespace+"/"+id.Name, "outcome", string(result.Outcome), "commit", result.Commit, diff --git a/internal/git/save_request_attach_test.go b/internal/git/commit_request_attach_test.go similarity index 96% rename from internal/git/save_request_attach_test.go rename to internal/git/commit_request_attach_test.go index b7176e4c..ba2aff5c 100644 --- a/internal/git/save_request_attach_test.go +++ b/internal/git/commit_request_attach_test.go @@ -23,7 +23,7 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// crName is the single SaveRequest name these tests register; each test uses one. +// crName is the single CommitRequest name these tests register; each test uses one. // crTarget is the GitTarget the worker serves (see createPlainGitTarget). const ( crName = "save" @@ -32,15 +32,15 @@ const ( // serviceAttach drives one attach + servicing pass on the loop, the way run() // does after dequeuing an attach work item. -func serviceAttach(loop *branchWorkerEventLoop, req *AttachSaveRequest) { - loop.handleAttachSaveRequest(req) +func serviceAttach(loop *branchWorkerEventLoop, req *AttachCommitRequest) { + loop.handleAttachCommitRequest(req) loop.endWake(0) } // attachReq builds a CurrentOrNext request that waits up to d for a window and, once attached, // collects for d with no idle close: the defaults' shape, at any length. -func attachReq(author string, d time.Duration) *AttachSaveRequest { - return &AttachSaveRequest{ +func attachReq(author string, d time.Duration) *AttachCommitRequest { + return &AttachCommitRequest{ Namespace: "default", Name: crName, UID: "uid-" + crName, @@ -57,7 +57,7 @@ func attachReq(author string, d time.Duration) *AttachSaveRequest { // waiting request's attach deadline passes, and an attached request's window reaches its // maxDuration and closes. func forceDue(loop *branchWorkerEventLoop) { - id := saveRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} + id := commitRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} past := time.Now().Add(-time.Millisecond) if pcr := loop.pendingCRs[id]; pcr != nil && !pcr.attached { pcr.attachDeadline = past @@ -71,7 +71,7 @@ func forceDue(loop *branchWorkerEventLoop) { func outcome(t *testing.T, w *BranchWorker) (FinalizeResult, bool) { t.Helper() - return w.LookupSaveRequestOutcome("default", crName, "uid-"+crName) + return w.LookupCommitRequestOutcome("default", crName, "uid-"+crName) } // TestEnqueueAttach_Success verifies an attach lands on the worker's event queue. @@ -246,7 +246,7 @@ func TestAttach_TheAttachStartsTheRequestsMaxDuration(t *testing.T) { const maxDuration = 60 * time.Second serviceAttach(loop, attachReq("alice", maxDuration)) - id := saveRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} + id := commitRequestID{Namespace: "default", Name: crName, UID: "uid-" + crName} // Almost all of the wait spent: the writes arrive with a second of it left. loop.pendingCRs[id].attachDeadline = time.Now().Add(time.Second) @@ -374,7 +374,7 @@ func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) require.NoError(t, err) assert.Equal(t, "jane@acme.com", commit.Author.Name, "the submitter authors the commit") - assert.Equal(t, "save: from the editor\n\nAuthor-Source: SaveRequest default/save; "+ + assert.Equal(t, "save: from the editor\n\nAuthor-Source: CommitRequest default/save; "+ "no audit record names who made these changes", commit.Message) } @@ -508,14 +508,14 @@ func TestAttach_ForeignWindowClosingBeforeExpiryIsStillAMismatch(t *testing.T) { // once, SaveInProgress, instead of losing its message or its edits to the first. It is not a // mismatch, and it holds nothing. func TestAttach_ASecondSaveOfTheSameAuthorIsRefusedWhileTheFirstIsInProgress(t *testing.T) { - second := func() *AttachSaveRequest { + second := func() *AttachCommitRequest { req := attachReq("alice", 60*time.Second) req.Name, req.UID = "save-second", "uid-save-second" return req } secondOutcome := func(t *testing.T, w *BranchWorker) (FinalizeResult, bool) { t.Helper() - return w.LookupSaveRequestOutcome("default", "save-second", "uid-save-second") + return w.LookupCommitRequestOutcome("default", "save-second", "uid-save-second") } t.Run("while the first collects the window", func(t *testing.T) { @@ -634,7 +634,7 @@ func TestAttach_IdempotentReSendKeepsFirstDeadline(t *testing.T) { defer loop.stopTimers() serviceAttach(loop, attachReq("alice", 60*time.Second)) - id := saveRequestID{Namespace: "default", Name: "save", UID: "uid-save"} + id := commitRequestID{Namespace: "default", Name: "save", UID: "uid-save"} require.Contains(t, loop.pendingCRs, id) firstDeadline := loop.pendingCRs[id].attachDeadline @@ -686,7 +686,7 @@ func TestAttach_AnUnreachableRemoteHoldsTheRequest(t *testing.T) { _, resolved := outcome(t, worker) assert.False(t, resolved, "a request whose write can still land is not failed") - assert.Equal(t, PhaseWaitingForPush, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForPush, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) assert.Nil(t, loop.openWindow, "the window closed on its own timers") require.Len(t, loop.pendingWrites, 1, "the decided window stays pending") assert.False(t, loop.pendingWrites[0].committedOnce, "nothing could commit it yet") @@ -734,7 +734,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) loop := newBranchWorkerEventLoop(worker, time.Hour) defer loop.stopTimers() - // First, commit the ConfigMap (no SaveRequest) and push it, so it is already + // First, commit the ConfigMap (no CommitRequest) and push it, so it is already // present in Git. loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("present", "alice", "team-a")}, @@ -746,7 +746,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) // Hold the push back, so the next assertion is about the finalize and nothing else. loop.lastPushAt = time.Now() - // A second window re-asserts the SAME object: no diff. Attach a SaveRequest and + // A second window re-asserts the SAME object: no diff. Attach a CommitRequest and // finalize it (maxDuration 0s). loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("present", "alice", "team-a")}, @@ -759,7 +759,7 @@ func TestAttach_NoDiffResolvesAlreadyPresentOnceTheRemoteConfirms(t *testing.T) "a no-diff finalize must NOT resolve before the remote has been consulted") require.Len(t, loop.pendingWrites, 1, "the no-diff write is retained so it reaches the push") require.True(t, loop.pendingWrites[0].CommitSHA.IsZero(), "no diff means no commit of its own") - require.NotNil(t, loop.pendingWrites[0].SaveRequest, "the request rides the retained write") + require.NotNil(t, loop.pendingWrites[0].CommitRequest, "the request rides the retained write") // The push reads the advertisement, finds the remote already holds this state, and that is // what settles the request. @@ -910,14 +910,14 @@ func TestAttributionOutcome_ZeroValueIsNotAttempted(t *testing.T) { } // TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems is the permanent regression test for -// the merge blocker: in the DEFAULT deployment no SaveRequest could attach to any window, so +// the merge blocker: in the DEFAULT deployment no CommitRequest could attach to any window, so // the user's commit message was silently dropped and the change landed under the generated // message with no error anywhere. // // The window's outcome and the request's outcome come from two subsystems configured by two // different flags (--author-attribution vs --admission-webhook, cmd/main.go:311-316), so this // walks every pair either side can actually produce and names the deployment that produces it. -// TestSaveRequest_OutcomesAgree in internal/controller pins that these are the real +// TestCommitRequest_OutcomesAgree in internal/controller pins that these are the real // values the two producers emit; here we pin what the matching rule does with them. func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { const otherAuthor = "bob" @@ -987,7 +987,7 @@ func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - p := &pendingSaveRequest{ + p := &pendingCommitRequest{ author: tc.reqAuth, attribution: tc.request, gitTargetName: crTarget, @@ -1008,7 +1008,7 @@ func TestMatchesWindow_AcrossIndependentlyConfiguredSubsystems(t *testing.T) { // rework: no attribution pairing may let a request finalize another target's window. func TestMatchesWindow_GitTargetAlwaysScopes(t *testing.T) { for _, o := range []AttributionOutcome{AttributionNotAttempted, AttributionUnresolved, AttributionResolved} { - p := &pendingSaveRequest{attribution: o, gitTargetName: crTarget, gitTargetNamespace: "default"} + p := &pendingCommitRequest{attribution: o, gitTargetName: crTarget, gitTargetNamespace: "default"} assert.False(t, p.matchesWindow(&openWindow{ Attribution: o, GitTarget: "team-b", GitTargetNamespace: "default", }), "outcome %q must not match across GitTarget names", o) @@ -1018,7 +1018,7 @@ func TestMatchesWindow_GitTargetAlwaysScopes(t *testing.T) { } } -// The re-attach contract. A SaveRequest is settled by the push and by nothing else, so between +// The re-attach contract. A CommitRequest is settled by the push and by nothing else, so between // its window's finalize and that push it is neither pending nor resolved. The controller re-sends // its attach every couple of seconds throughout that gap, and these four tests pin what the worker // must do with those re-sends. @@ -1043,11 +1043,11 @@ func TestAttach_ReSentAttachDuringPushCooldownDoesNotResolve(t *testing.T) { serviceAttach(loop, attachReq("alice", 0)) require.Len(t, loop.pendingWrites, 1, "the window committed locally and is retained") - require.NotNil(t, loop.pendingWrites[0].SaveRequest, "the request rides that write") + require.NotNil(t, loop.pendingWrites[0].CommitRequest, "the request rides that write") _, resolved := outcome(t, worker) require.False(t, resolved, "nothing has reached the remote yet") - // The controller re-sends, twice, exactly as it does every saveRequestPollInterval while it + // The controller re-sends, twice, exactly as it does every commitRequestPollInterval while it // has no outcome to read. for range 2 { serviceAttach(loop, attachReq("alice", 0)) diff --git a/internal/git/save_request_empty_test.go b/internal/git/commit_request_empty_test.go similarity index 99% rename from internal/git/save_request_empty_test.go rename to internal/git/commit_request_empty_test.go index f3142f55..9e78f3e0 100644 --- a/internal/git/save_request_empty_test.go +++ b/internal/git/commit_request_empty_test.go @@ -21,7 +21,7 @@ import ( // request that ends with nothing to commit records its message in an empty commit, the outcome // keeps the cause, and a window that belonged to someone else never falls back to one. -func commitEmptyReq(author, message string) *AttachSaveRequest { +func commitEmptyReq(author, message string) *AttachCommitRequest { req := attachReq(author, time.Hour) req.Message = message req.CommitEmpty = true @@ -244,7 +244,7 @@ func TestCommitEmpty_AFailedPushTreatsEveryCommitShapeAlike(t *testing.T) { _, resolved := outcome(t, worker) require.False(t, resolved, "the remote has neither accepted nor refused it") assert.Equal(t, PhaseWaitingForPush, - worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) require.Len(t, loop.pendingWrites, 1, "and the re-send made no second write") } diff --git a/internal/git/save_request_owners.go b/internal/git/commit_request_owners.go similarity index 80% rename from internal/git/save_request_owners.go rename to internal/git/commit_request_owners.go index 25736d54..132b2131 100644 --- a/internal/git/save_request_owners.go +++ b/internal/git/commit_request_owners.go @@ -4,7 +4,7 @@ package git import "sync" -// saveRequestOwners records which branch worker accepted each SaveRequest's attach, so the +// commitRequestOwners records which branch worker accepted each CommitRequest's attach, so the // controller's questions about a request reach the worker that can still act on it. // // The GitTarget cannot answer that. A worker serves every GitTarget on its (provider, branch), so it @@ -20,14 +20,14 @@ import "sync" // // It is shared by the manager and every worker it creates. The lock is a leaf: callers may hold // a worker's pendingResyncsMu or crOutcomesMu when they take it, never the other way round. -type saveRequestOwners struct { +type commitRequestOwners struct { mu sync.Mutex - owners map[saveRequestID]*BranchWorker + owners map[commitRequestID]*BranchWorker } // owner returns the worker holding the request, if any. A nil table (a worker built outside a // manager) knows no owners. -func (o *saveRequestOwners) owner(id saveRequestID) (*BranchWorker, bool) { +func (o *commitRequestOwners) owner(id commitRequestID) (*BranchWorker, bool) { if o == nil { return nil, false } @@ -39,7 +39,7 @@ func (o *saveRequestOwners) owner(id saveRequestID) (*BranchWorker, bool) { // claim makes w the request's owner, unless another worker already is. It reports whether w owns // the request afterwards. A nil table lets every worker claim. -func (o *saveRequestOwners) claim(id saveRequestID, w *BranchWorker) bool { +func (o *commitRequestOwners) claim(id commitRequestID, w *BranchWorker) bool { if o == nil { return true } @@ -49,14 +49,14 @@ func (o *saveRequestOwners) claim(id saveRequestID, w *BranchWorker) bool { return current == w } if o.owners == nil { - o.owners = map[saveRequestID]*BranchWorker{} + o.owners = map[commitRequestID]*BranchWorker{} } o.owners[id] = w return true } // release forgets the request, if w is still its owner. -func (o *saveRequestOwners) release(id saveRequestID, w *BranchWorker) { +func (o *commitRequestOwners) release(id commitRequestID, w *BranchWorker) { if o == nil { return } diff --git a/internal/git/save_request_withdraw_test.go b/internal/git/commit_request_withdraw_test.go similarity index 83% rename from internal/git/save_request_withdraw_test.go rename to internal/git/commit_request_withdraw_test.go index 95f6c1da..f2ffa423 100644 --- a/internal/git/save_request_withdraw_test.go +++ b/internal/git/commit_request_withdraw_test.go @@ -2,7 +2,7 @@ package git -// The controller withdraws a SaveRequest before failing it closed. These pin the worker's half of +// The controller withdraws a CommitRequest before failing it closed. These pin the worker's half of // that agreement: a request the worker has not acted on is cancelled for good, and a request it // holds (attached, or committed) is not, so a request reported failed is never committed later. @@ -28,14 +28,14 @@ func TestWithdraw_AWaitingRequestIsCancelledForGood(t *testing.T) { req := attachReq("alice", time.Hour) serviceAttach(loop, req) - require.Equal(t, PhaseWaitingForWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + require.Equal(t, PhaseWaitingForWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) loop.handleQueueItem(WorkItem{Withdraw: req}) res, ok := outcome(t, worker) require.True(t, ok, "a withdrawn request resolves") - require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) - assert.Empty(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) + assert.Empty(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) // A late re-send of the attach, and a window it would have attached to, change nothing. serviceAttach(loop, req) @@ -46,7 +46,7 @@ func TestWithdraw_AWaitingRequestIsCancelledForGood(t *testing.T) { assert.Nil(t, loop.openWindow.pendingCR, "a withdrawn request never attaches") assert.Empty(t, loop.pendingCRs) res, _ = outcome(t, worker) - require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) } // A withdraw that arrives before the worker ever saw the attach leaves a tombstone, so the attach @@ -62,7 +62,7 @@ func TestWithdraw_AnUnregisteredRequestCannotRegisterLater(t *testing.T) { res, ok := outcome(t, worker) require.True(t, ok) - require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) assert.Empty(t, loop.pendingCRs) } @@ -79,13 +79,13 @@ func TestWithdraw_AnAttachedRequestIsHeldAndStillCommits(t *testing.T) { }}) req := attachReq("alice", time.Hour) serviceAttach(loop, req) - require.Equal(t, PhaseCollectingWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + require.Equal(t, PhaseCollectingWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) loop.handleQueueItem(WorkItem{Withdraw: req}) _, resolved := outcome(t, worker) require.False(t, resolved, "a held request is not withdrawn") - assert.True(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName).Held()) + assert.True(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName).Held()) forceDue(loop) loop.pushPending() @@ -136,8 +136,8 @@ func TestWithdraw_AnOutcomeIsKeptWhileAnAttachForItIsQueued(t *testing.T) { worker.EnqueueAttach(req) // a late attach, still on the FIFO // The outcome runs past its TTL while the attach waits, and a GC pass runs. - ageSaveRequestOutcome(worker, req.id(), 2*saveRequestOutcomeTTL) - worker.recordSaveRequestOutcome(saveRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) + ageCommitRequestOutcome(worker, req.id(), 2*commitRequestOutcomeTTL) + worker.recordCommitRequestOutcome(commitRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) _, kept := outcome(t, worker) require.True(t, kept, "an outcome with an attach still queued is not collected") @@ -145,11 +145,11 @@ func TestWithdraw_AnOutcomeIsKeptWhileAnAttachForItIsQueued(t *testing.T) { loop.endWake(0) assert.Empty(t, loop.pendingCRs, "the late attach does not register the withdrawn request") res, _ := outcome(t, worker) - require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) // With nothing queued for it any more, the outcome is collected as usual. - ageSaveRequestOutcome(worker, req.id(), 2*saveRequestOutcomeTTL) - worker.recordSaveRequestOutcome(saveRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) + ageCommitRequestOutcome(worker, req.id(), 2*commitRequestOutcomeTTL) + worker.recordCommitRequestOutcome(commitRequestID{Namespace: "default", Name: "other"}, FinalizeResult{}) _, kept = outcome(t, worker) assert.False(t, kept) } @@ -162,7 +162,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { require.NoError(t, configv1alpha3.AddToScheme(scheme)) worker := NewBranchWorker(fake.NewClientBuilder().WithScheme(scheme).Build(), logr.Discard(), "no-such-provider", "default", "main", RepoIdentity{URL: "file:///nowhere"}, nil, BranchWorkerLimits{}) - owners := &saveRequestOwners{} + owners := &commitRequestOwners{} worker.crOwners = owners req := attachReq("alice", time.Hour) @@ -181,7 +181,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { worker.EnqueueWithdraw(req) res, resolved := outcome(t, worker) require.True(t, resolved, "a withdraw is answered at once") - require.ErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + require.ErrorIs(t, res.Err, ErrCommitRequestWithdrawn) require.ErrorIs(t, res.Err, ErrBranchWorkerStopped) assert.Zero(t, worker.inflightItems.Load()) } @@ -192,7 +192,7 @@ func TestWithdraw_AWorkerThatExitedAtStartupAnswers(t *testing.T) { func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { worker, _, _ := setupCommitPushSplitWorker(t) createPlainGitTarget(t, worker, "team-a", "team-a") - owners := &saveRequestOwners{} + owners := &commitRequestOwners{} worker.crOwners = owners require.NoError(t, worker.Start(context.Background())) @@ -200,7 +200,7 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { req := attachReq("alice", time.Hour) worker.EnqueueAttach(req) require.Eventually(t, func() bool { - return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName).Held() + return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName).Held() }, 5*time.Second, 10*time.Millisecond) worker.Stop() @@ -210,7 +210,7 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { assert.Same(t, worker, owner) res, resolved := outcome(t, worker) require.True(t, resolved, "its shutdown settled the request, by its last push or by failing it") - require.NotErrorIs(t, res.Err, ErrSaveRequestWithdrawn) + require.NotErrorIs(t, res.Err, ErrCommitRequestWithdrawn) worker.EnqueueWithdraw(req) after, _ := outcome(t, worker) @@ -222,29 +222,29 @@ func TestWithdraw_ARetiredWorkerStillAnswersForWhatItHeld(t *testing.T) { func TestWithdraw_AStoppedWorkerGivesBackWhatItNeverActedOn(t *testing.T) { worker, _, _ := setupCommitPushSplitWorker(t) createPlainGitTarget(t, worker, "team-a", "team-a") - owners := &saveRequestOwners{} + owners := &commitRequestOwners{} worker.crOwners = owners require.NoError(t, worker.Start(context.Background())) req := attachReq("alice", time.Hour) worker.EnqueueAttach(req) require.Eventually(t, func() bool { - return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName) == PhaseWaitingForWindow + return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName) == PhaseWaitingForWindow }, 5*time.Second, 10*time.Millisecond) worker.Stop() _, owned := owners.owner(req.id()) assert.False(t, owned) - assert.Empty(t, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + assert.Empty(t, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) _, resolved := outcome(t, worker) assert.False(t, resolved, "giving it back is not an outcome") } -func TestSaveRequestOwners_OneOwnerAtATime(t *testing.T) { +func TestCommitRequestOwners_OneOwnerAtATime(t *testing.T) { a, b := &BranchWorker{}, &BranchWorker{} - id := saveRequestID{Namespace: "default", Name: crName} - owners := &saveRequestOwners{} + id := commitRequestID{Namespace: "default", Name: crName} + owners := &commitRequestOwners{} require.True(t, owners.claim(id, a)) assert.True(t, owners.claim(id, a), "claiming again is idempotent") @@ -256,7 +256,7 @@ func TestSaveRequestOwners_OneOwnerAtATime(t *testing.T) { owners.release(id, a) assert.True(t, owners.claim(id, b)) - var none *saveRequestOwners + var none *commitRequestOwners assert.True(t, none.claim(id, a), "a worker outside a manager owns what it is sent") _, ok = none.owner(id) assert.False(t, ok) @@ -264,7 +264,7 @@ func TestSaveRequestOwners_OneOwnerAtATime(t *testing.T) { } func TestEnqueueAttach_AnotherWorkersRequestIsRefused(t *testing.T) { - owners := &saveRequestOwners{} + owners := &commitRequestOwners{} holder := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} other := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} req := attachReq("alice", 0) @@ -279,7 +279,7 @@ func TestEnqueueAttach_AnotherWorkersRequestIsRefused(t *testing.T) { // A dropped attach gives the request back unless the worker knows it otherwise. func TestEnqueueAttach_AQueueFullDropGivesTheRequestBack(t *testing.T) { - owners := &saveRequestOwners{} + owners := &commitRequestOwners{} w := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 1), crOwners: owners} w.eventQueue <- WorkItem{} // saturate @@ -291,7 +291,7 @@ func TestEnqueueAttach_AQueueFullDropGivesTheRequestBack(t *testing.T) { assert.Empty(t, w.crQueuedAttaches) } -func ageSaveRequestOutcome(w *BranchWorker, id saveRequestID, by time.Duration) { +func ageCommitRequestOutcome(w *BranchWorker, id commitRequestID, by time.Duration) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() entry := w.crOutcomes[id] @@ -299,10 +299,10 @@ func ageSaveRequestOutcome(w *BranchWorker, id saveRequestID, by time.Duration) w.crOutcomes[id] = entry } -func TestSaveRequestPhase_Held(t *testing.T) { +func TestCommitRequestPhase_Held(t *testing.T) { assert.True(t, PhaseCollectingWindow.Held()) assert.True(t, PhaseWaitingForPush.Held()) assert.False(t, PhaseWaitingForWindow.Held()) assert.False(t, PhaseWaitingForWorker.Held()) - assert.False(t, SaveRequestPhase("").Held()) + assert.False(t, CommitRequestPhase("").Held()) } diff --git a/internal/git/commit_window_metrics_test.go b/internal/git/commit_window_metrics_test.go index b3be11e1..e58ce24c 100644 --- a/internal/git/commit_window_metrics_test.go +++ b/internal/git/commit_window_metrics_test.go @@ -112,7 +112,7 @@ func TestCommitWindowMetrics_AnAttachKeepsTheOpeningTimeAndCountsTheRequestsTime assert.Equal(t, opened, loop.openWindow.openedAt, "the attach restarts the timers, not the opening time") forceDue(loop) - assert.Equal(t, int64(1), windowsClosed(t, reader, "max_duration", windowTimerSourceSaveRequest)) + assert.Equal(t, int64(1), windowsClosed(t, reader, "max_duration", windowTimerSourceCommitRequest)) assert.Zero(t, windowsClosed(t, reader, "max_duration", windowTimerSourceTarget)) requireOneObservationPerWindow(t, reader, 1) sum, _ := telemetry.CollectHistogramSum(reader, commitWindowDurationMetric, diff --git a/internal/git/commit_window_timers_test.go b/internal/git/commit_window_timers_test.go index 95eba69e..38e27f5e 100644 --- a/internal/git/commit_window_timers_test.go +++ b/internal/git/commit_window_timers_test.go @@ -13,7 +13,7 @@ import ( ) // These tests pin the clock rules of docs/design/commit-timing-surface.md: a window opened by a -// write takes the GitTarget's timers, and a SaveRequest that attaches replaces them, shorter or +// write takes the GitTarget's timers, and a CommitRequest that attaches replaces them, shorter or // longer. Each one drives the loop directly, so no test waits on a real timer longer than a few // milliseconds. @@ -25,7 +25,7 @@ func writeTo(loop *branchWorkerEventLoop, name string) { }}) } -func namedAttachReq(name string, d time.Duration) *AttachSaveRequest { +func namedAttachReq(name string, d time.Duration) *AttachCommitRequest { req := attachReq("alice", d) req.Name = name req.UID = "uid-" + name @@ -229,7 +229,7 @@ func TestAttachSelection_TheWorkerReportsEachPhase(t *testing.T) { createPlainGitTarget(t, worker, "team-a", "team-a") loop := newBranchWorkerEventLoop(worker, time.Hour) defer loop.stopTimers() - phase := func() SaveRequestPhase { return worker.LookupSaveRequestPhase("default", crName, "uid-"+crName) } + phase := func() CommitRequestPhase { return worker.LookupCommitRequestPhase("default", crName, "uid-"+crName) } assert.Empty(t, phase(), "nothing is reported before the worker registers the request") loop.lastPushAt = time.Now() // keep the push behind its cooldown, so WaitingForPush is observable @@ -266,7 +266,7 @@ func TestWindowTimers_AWaitingRequestWithMaxDurationZeroCommitsExactlyTheNextWri req.Message = "save: exactly the next change" serviceAttach(loop, req) require.Nil(t, loop.openWindow, "the request waits; it opens nothing") - assert.Equal(t, PhaseWaitingForWindow, worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForWindow, worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) writeTo(loop, "the-change") loop.endWake(0) diff --git a/internal/git/decided_write_test.go b/internal/git/decided_write_test.go index 85281321..806c74ac 100644 --- a/internal/git/decided_write_test.go +++ b/internal/git/decided_write_test.go @@ -76,7 +76,7 @@ func TestDecidedWrite_SurvivesAFailedRebuildAndLandsThroughTheRetry(t *testing.T _, resolved := outcome(t, f.worker) require.False(t, resolved, "the save is held, not failed: its write can still land") - assert.Equal(t, PhaseWaitingForPush, f.worker.LookupSaveRequestPhase("default", crName, "uid-"+crName)) + assert.Equal(t, PhaseWaitingForPush, f.worker.LookupCommitRequestPhase("default", crName, "uid-"+crName)) require.Len(t, loop.pendingWrites, 2, "the window is decided and kept pending") assert.False(t, loop.pendingWrites[1].committedOnce) require.True(t, loop.retry.pending(), "the failure armed the publication retry") @@ -328,7 +328,7 @@ func TestDecidedWrite_SameObjectEditsReplayAsDecidedOntoARemoteHoldingTheLast(t assert.Equal(t, string(second), fileAt(t, replayed[1], gitPath), "the second applies it again") assert.Equal(t, "set value to 2", replayed[1].Message) - outcome, ok := f.worker.LookupSaveRequestOutcome(save.Namespace, save.Name, save.UID) + outcome, ok := f.worker.LookupCommitRequestOutcome(save.Namespace, save.Name, save.UID) require.True(t, ok) assert.Equal(t, FinalizeCommitted, outcome.Outcome, "the save resolves on its own write's commit") assert.Equal(t, replayed[1].Hash.String(), outcome.Commit) diff --git a/internal/git/dirty_worktree_recovery_test.go b/internal/git/dirty_worktree_recovery_test.go index 923ef984..4bd07f75 100644 --- a/internal/git/dirty_worktree_recovery_test.go +++ b/internal/git/dirty_worktree_recovery_test.go @@ -120,7 +120,7 @@ func TestEveryLoopCommitPathRecoversADirtyWorktree(t *testing.T) { }, }, { - name: "empty commit recording a SaveRequest", + name: "empty commit recording a CommitRequest", commit: func(t *testing.T, loop *branchWorkerEventLoop) { t.Helper() serviceAttach(loop, commitEmptyReq("alice", "save: nothing changed")) diff --git a/internal/git/intake_test.go b/internal/git/intake_test.go index b84ccd89..7a7854fa 100644 --- a/internal/git/intake_test.go +++ b/internal/git/intake_test.go @@ -78,7 +78,7 @@ func TestIntake_EveryKindOfHeldWorkCounts(t *testing.T) { req := attachReq("alice", time.Hour) req.GitTargetName = ledgerTargetName req.Message = "save once the window opens" - loop.handleAttachSaveRequest(req) + loop.handleAttachCommitRequest(req) require.Contains(t, loop.pendingCRs, req.id()) assert.GreaterOrEqual(t, loop.heldBytes()-held, int64(pendingWriteOverheadBytes), "a registered save is charged for what it keeps") @@ -155,7 +155,7 @@ func TestIntake_AWithdrawalProgressesWhilePaused(t *testing.T) { f, loop, _ := outageLoop(t, "intake-withdraw-while-paused") req := attachReq("alice", time.Hour) req.GitTargetName = ledgerTargetName - loop.handleAttachSaveRequest(req) + loop.handleAttachCommitRequest(req) f.worker.branchBufferMaxBytes = 1 loop.publishLoopState(0) require.NotNil(t, f.worker.IntakePaused()) @@ -163,9 +163,9 @@ func TestIntake_AWithdrawalProgressesWhilePaused(t *testing.T) { f.worker.EnqueueWithdraw(req) require.Len(t, f.worker.eventQueue, 1, "a withdrawal is not refused by the pause") loop.handleQueueItem(<-f.worker.eventQueue) - outcome, ok := f.worker.LookupSaveRequestOutcome(req.Namespace, req.Name, req.UID) + outcome, ok := f.worker.LookupCommitRequestOutcome(req.Namespace, req.Name, req.UID) require.True(t, ok) - assert.ErrorIs(t, outcome.Err, ErrSaveRequestWithdrawn) + assert.ErrorIs(t, outcome.Err, ErrCommitRequestWithdrawn) } // bigConfigMap is a ConfigMap whose data is about size bytes. diff --git a/internal/git/network_bound_test.go b/internal/git/network_bound_test.go index 1549edcc..797a711c 100644 --- a/internal/git/network_bound_test.go +++ b/internal/git/network_bound_test.go @@ -298,8 +298,8 @@ func TestPushCycle_ALostReplyThatLandedIsPublishedNotReplayed(t *testing.T) { f := newLedgerFixture(t, "lost-reply", true) f.createLedgerTarget("apps", nil) - record, err := f.worker.buildRequestRecordWrite(f.worker.ctx, &pendingSaveRequest{ - id: saveRequestID{Namespace: "default", Name: "save", UID: "save-uid"}, + record, err := f.worker.buildRequestRecordWrite(f.worker.ctx, &pendingCommitRequest{ + id: commitRequestID{Namespace: "default", Name: "save", UID: "save-uid"}, author: "alice", gitTargetName: ledgerTargetName, gitTargetNamespace: "default", diff --git a/internal/git/open_window.go b/internal/git/open_window.go index 087f3e42..a1fe5299 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -16,8 +16,8 @@ type openWindow struct { // author agree, so an unresolved event never coalesces with a resolved one that happens // to carry an empty username. Exact enum equality is right HERE — canAppend compares two // events from the same watch pipeline, so the outcomes are directly comparable. It is not - // right when comparing against a SaveRequest, which a different subsystem attributes; - // see pendingSaveRequest.matchesWindow. + // right when comparing against a CommitRequest, which a different subsystem attributes; + // see pendingCommitRequest.matchesWindow. Attribution AttributionOutcome // GitTarget is the target this window is bound to. Each finalized commit // covers exactly one target so per-target encryption and bootstrap @@ -39,28 +39,28 @@ type openWindow struct { // window collapse onto the same path key. writer eventContentWriter - // pendingMessage is the SaveRequest message attached to this window (§6.4). + // pendingMessage is the CommitRequest message attached to this window (§6.4). // Once set, whichever path finalizes the window uses it instead of the // generated grouped-commit message, so an early cut-off still carries intent. pendingMessage string - // pendingCR identifies the SaveRequest attached to this window; at most one. On + // pendingCR identifies the CommitRequest attached to this window; at most one. On // finalize its outcome is resolved (Committed once the carrying write pushes). - pendingCR *saveRequestID - // claim and claimedBy say what the attached SaveRequest claimed of the window, whose writes - // name no author, and who submitted it (pendingSaveRequest.claims): its message (SavedBy), or + pendingCR *commitRequestID + // claim and claimedBy say what the attached CommitRequest claimed of the window, whose writes + // name no author, and who submitted it (pendingCommitRequest.claims): its message (SavedBy), or // the window, which the submitter then authors (TakeOver). Empty when it claimed nothing. claim UnattributedClaim claimedBy UserInfo // timers decide when the window closes. They are the GitTarget's, snapshotted when the window - // opened, until a SaveRequest attaches and replaces them with its own. + // opened, until a CommitRequest attaches and replaces them with its own. timers windowTimers // openedAt is when the write that opened the window arrived. Unlike the timers and lastWriteAt it - // never moves, not even when a SaveRequest attaches, so it is what the window's collection time + // never moves, not even when a CommitRequest attaches, so it is what the window's collection time // is measured from. openedAt time.Time // lastWriteAt is when the idle interval last restarted: the latest collected write, or the - // attach of a SaveRequest. + // attach of a CommitRequest. lastWriteAt time.Time } @@ -68,7 +68,7 @@ type openWindow struct { type windowTimers struct { // idle is the silence that closes the window. Ignored when noIdle is set. idle time.Duration - // noIdle disables the idle close: a SaveRequest that set no idleTimeout collects until + // noIdle disables the idle close: a CommitRequest that set no idleTimeout collects until // maxAt alone. noIdle bool // maxAt closes the window whatever keeps arriving: the window's opening (or the attach) plus @@ -149,9 +149,9 @@ func windowPathKey(e Event, writer eventContentWriter) string { // window contributes the LAST routed event's resourceVersion, which is the state the commit // actually writes. // -// requestMessage is the attached SaveRequest's message, or empty when no request attached. It is +// requestMessage is the attached CommitRequest's message, or empty when no request attached. It is // carried through unaltered — never parsed, never trimmed — because the request's literal bytes -// reaching the commit is the audit property the whole SaveRequest contract rests on. +// reaching the commit is the audit property the whole CommitRequest contract rests on. func buildLiveCommitMessageData( author, gitTarget, requestMessage string, events []Event, diff --git a/internal/git/parent_recovery_test.go b/internal/git/parent_recovery_test.go index 100d0102..0abc2e83 100644 --- a/internal/git/parent_recovery_test.go +++ b/internal/git/parent_recovery_test.go @@ -325,12 +325,12 @@ func TestParentRecovery_AParentChangeMakesTheProbeDue(t *testing.T) { assert.False(t, f.worker.awaitingParentProbe(), "a new parent is looked for at once") } -// A SaveRequest that rides a write held back by a missing parent stays held through the whole +// A CommitRequest that rides a write held back by a missing parent stays held through the whole // recovery, however long it takes, and resolves Committed when the work lands. This runs the real // event loop on its own goroutine; the clock stands in for a recovery far longer than the bound the // controller used to fail requests on (attachTimeout + maxDuration + 120s). -func TestParentRecovery_AHeldSaveRequestIsCommittedAfterALongRecovery(t *testing.T) { - f := newRealServerNewBranch(t, "recovery-held-save-request", func(repoDir string) { +func TestParentRecovery_AHeldCommitRequestIsCommittedAfterALongRecovery(t *testing.T) { + f := newRealServerNewBranch(t, "recovery-held-commit-request", func(repoDir string) { simulateClientCommitOnDisk(t, repoDir, "main", "README.md", "main\n") gitIn(t, repoDir, "branch", "release", "main") }) @@ -350,8 +350,8 @@ func TestParentRecovery_AHeldSaveRequestIsCommittedAfterALongRecovery(t *testing require.True(t, f.worker.Enqueue(configMapTargetEvent("cm1", "alice", newBranchTarget))) req := standbyRequest() f.worker.EnqueueAttach(req) - phase := func() SaveRequestPhase { - return f.worker.LookupSaveRequestPhase(req.Namespace, req.Name, req.UID) + phase := func() CommitRequestPhase { + return f.worker.LookupCommitRequestPhase(req.Namespace, req.Name, req.UID) } require.Eventually(t, func() bool { open, _ := f.worker.ParentRecovery() diff --git a/internal/git/pending_writes.go b/internal/git/pending_writes.go index c404b42e..71cebe44 100644 --- a/internal/git/pending_writes.go +++ b/internal/git/pending_writes.go @@ -273,22 +273,22 @@ const ( // Commit message sources, reported as the commits_total `message_source` label. Each names // where the message came from, so they read as one set alongside each other. const ( - // messageSourceSaveRequest is a message a SaveRequest supplied and that was used - // verbatim. It counts commits that USED such a message, not SaveRequests: a request + // messageSourceCommitRequest is a message a CommitRequest supplied and that was used + // verbatim. It counts commits that USED such a message, not CommitRequests: a request // omitting spec.message takes the target's liveTemplate and counts as live. - messageSourceSaveRequest = "save_request" - // messageSourceSaveRequestFramed is a SaveRequest message rendered through the target's - // requestTemplate. Split from save_request rather than folded into it, even though the text + messageSourceCommitRequest = "commit_request" + // messageSourceCommitRequestFramed is a CommitRequest message rendered through the target's + // requestTemplate. Split from commit_request rather than folded into it, even though the text // originated with the request either way: the difference is not HOW the text was produced (the // distinction reconcile deliberately collapses, below) but WHETHER the operator's configured // framing applied at all, which is a question an operator actually asks. - messageSourceSaveRequestFramed = "save_request_framed" - // messageSourceSaveRequestFallback is a requestTemplate that failed to render, so the + messageSourceCommitRequestFramed = "commit_request_framed" + // messageSourceCommitRequestFallback is a requestTemplate that failed to render, so the // request's message was committed verbatim instead. It is the ONLY signal for that: the commit // succeeds, nothing is refused, and no condition moves, so without a counter a template that // has quietly stopped applying looks exactly like one that was never configured. Alert on its // rate; see docs/interpreting-metrics.md. - messageSourceSaveRequestFallback = "save_request_fallback" + messageSourceCommitRequestFallback = "commit_request_fallback" // messageSourceLive is a live window rendered through the target's liveTemplate. messageSourceLive = "live" // messageSourceReconcile is a resync rendered through reconcileTemplate. diff --git a/internal/git/pending_writes_loop.go b/internal/git/pending_writes_loop.go index f5a3ee39..68cdb6c5 100644 --- a/internal/git/pending_writes_loop.go +++ b/internal/git/pending_writes_loop.go @@ -88,9 +88,9 @@ func (l *branchWorkerEventLoop) decide(pendingWrite PendingWrite) { pendingWrite.decidedAt = l.w.now() l.pendingWrites = append(l.pendingWrites, pendingWrite) l.pendingWritesBytes += pendingWrite.charge - if id := pendingWrite.SaveRequest; id != nil { - l.w.setSaveRequestPhase(*id, PhaseWaitingForPush) - // Bound to its write from here on: see pendingSaveRequest.attached. + if id := pendingWrite.CommitRequest; id != nil { + l.w.setCommitRequestPhase(*id, PhaseWaitingForPush) + // Bound to its write from here on: see pendingCommitRequest.attached. if pcr := l.pendingCRs[*id]; pcr != nil { pcr.attached = true } @@ -430,7 +430,7 @@ func (l *branchWorkerEventLoop) settleDropped(pendingWrite PendingWrite, err err l.w.Log.Error(err, "Resync commit failed after its caller was answered; dropping it", "resources", len(req.Desired)) case pendingWrite.Kind == PendingWriteRequestRecord: - l.w.Log.Error(err, "The empty commit recording a SaveRequest failed", + l.w.Log.Error(err, "The empty commit recording a CommitRequest failed", "gitTarget", pendingWrite.GitTargetNamespace+"/"+pendingWrite.GitTargetName) l.failDecidedRequest(pendingWrite, fmt.Errorf("record the message in an empty commit: %w", err)) default: @@ -476,7 +476,7 @@ func (l *branchWorkerEventLoop) replyToDeferredResyncs(err error) { // failDecidedRequest fails the request a dropped write carried. func (l *branchWorkerEventLoop) failDecidedRequest(pendingWrite PendingWrite, cause error) { - if pendingWrite.SaveRequest != nil { - l.resolveSaveRequest(*pendingWrite.SaveRequest, FinalizeResult{Branch: l.w.Branch, Err: cause}) + if pendingWrite.CommitRequest != nil { + l.resolveCommitRequest(*pendingWrite.CommitRequest, FinalizeResult{Branch: l.w.Branch, Err: cause}) } } diff --git a/internal/git/render_fidelity_gate.go b/internal/git/render_fidelity_gate.go index cb46f2eb..fe1e56eb 100644 --- a/internal/git/render_fidelity_gate.go +++ b/internal/git/render_fidelity_gate.go @@ -146,7 +146,7 @@ func (g *RenderFidelityGate) Reconcile( // An EMPTY plan restarts every scope vacuously, and a vacuous full restart is the absence // of a measurement rather than a fresh one. Without the length guard a target whose rules // stopped selecting anything would clear a write divergence it never repaired, and the - // writes that do not come from a watch — a SaveRequest — would be + // writes that do not come from a watch — a CommitRequest — would be // admitted again on the strength of nothing. if len(scopes) > 0 && fresh == len(scopes) { state.writeDivergence = nil diff --git a/internal/git/render_fidelity_gate_test.go b/internal/git/render_fidelity_gate_test.go index f1517946..3c47ba6d 100644 --- a/internal/git/render_fidelity_gate_test.go +++ b/internal/git/render_fidelity_gate_test.go @@ -255,7 +255,7 @@ func TestRenderFidelityGate_DroppedScopeStopsCounting(t *testing.T) { // A plan that selects nothing restarts every scope only vacuously, which is the absence of a // measurement rather than a fresh one. Clearing on it would readmit the writes that do not come -// from a watch — a SaveRequest — on the strength of nothing. +// from a watch — a CommitRequest — on the strength of nothing. func TestRenderFidelityGate_AnEmptyPlanDoesNotClearAWriteDivergence(t *testing.T) { gate := NewRenderFidelityGate() target := types.NewResourceReference("apps", "default") @@ -405,7 +405,7 @@ func TestRenderFidelityGate_InvalidateAdvancesEveryScopeRevision(t *testing.T) { // TestRenderFidelityGate_InvalidateHoldsATargetWithNoScopes is the vacuous case, and it is the one // an unregistered target got wrong: with no scopes to report, an empty plan reduces to True, so a // target whose rules select nothing would have been writable against a repository nothing has -// looked at. A SaveRequest does not come from a watch. +// looked at. A CommitRequest does not come from a watch. func TestRenderFidelityGate_InvalidateHoldsATargetWithNoScopes(t *testing.T) { gate := NewRenderFidelityGate() target := types.NewResourceReference("apps", "default") diff --git a/internal/git/replay_failure_test.go b/internal/git/replay_failure_test.go index 2d833c3f..b11d2d26 100644 --- a/internal/git/replay_failure_test.go +++ b/internal/git/replay_failure_test.go @@ -12,7 +12,7 @@ package git // A push in that state is silently wrong rather than noisily wrong. The local branch is exactly // the remote tip, so validatePushState reports "already up to date" and PushAtomic returns nil // before it ever compares the cycle's root hash. The worker then counts the writes as published, -// clears them, and resolves any SaveRequest riding one as Committed, naming a SHA that is not on +// clears them, and resolves any CommitRequest riding one as Committed, naming a SHA that is not on // the remote and never will be. // // worktreeDirty is false throughout: the reset cleared it, and the worktree really is clean. It is @@ -114,9 +114,9 @@ func TestReplayFailure_DoesNotLetThePushSettleWorkThatWasReset(t *testing.T) { "the write survived the failed replay and reached the remote on the retry") } -// TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded is the same hazard seen from the +// TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded is the same hazard seen from the // caller's side, which is where it does real damage: a user is told their save landed, with a SHA. -func TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded(t *testing.T) { +func TestReplayFailure_DoesNotResolveACommitRequestThatNeverLanded(t *testing.T) { f := newLedgerFixture(t, "replay-failure-cr", true) f.createLedgerTarget("team-a", nil) f.publish("prime") @@ -134,7 +134,7 @@ func TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded(t *testing.T) { req.Message = "save: this must not be reported as committed" serviceAttach(loop, req) require.Len(t, loop.pendingWrites, 1) - require.NotNil(t, loop.pendingWrites[0].SaveRequest) + require.NotNil(t, loop.pendingWrites[0].CommitRequest) restoreAPI := failGitTargetReads(t, f.worker, errors.New("etcdserver: request timed out")) require.Error(t, f.worker.refreshRemoteAndRebuildPendingWrites( @@ -142,7 +142,7 @@ func TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded(t *testing.T) { loop.pushPending() - _, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) + _, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) require.False(t, resolved, "the request must stay in flight: nothing of its commit is on the remote") @@ -151,7 +151,7 @@ func TestReplayFailure_DoesNotResolveASaveRequestThatNeverLanded(t *testing.T) { restoreAPI() loop.pushPending() - res, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) + res, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) require.True(t, resolved) assert.Equal(t, FinalizeCommitted, res.Outcome) assert.Contains(t, remoteFileNames(t, f.repoDir), "saved-by-hand") diff --git a/internal/git/replay_refusal_test.go b/internal/git/replay_refusal_test.go index a917a0bf..7a175e80 100644 --- a/internal/git/replay_refusal_test.go +++ b/internal/git/replay_refusal_test.go @@ -59,7 +59,7 @@ func retainTwoWrites(t *testing.T, loop *branchWorkerEventLoop) { req.Message = "save: lands in a folder that is refused before it is pushed" serviceAttach(loop, req) require.Len(t, loop.pendingWrites, 1) - require.NotNil(t, loop.pendingWrites[0].SaveRequest) + require.NotNil(t, loop.pendingWrites[0].CommitRequest) loop.handleQueueItem(WorkItem{Request: &WriteRequest{ Events: []Event{configMapTargetEvent("kept-behind-it", "bob", replayRefusalTargetB)}, @@ -91,7 +91,7 @@ func assertOnlyTheRefusedWriteWasDropped( require.Len(t, *refusals, 1, "the refusal is reported on the target's status, once") assert.Equal(t, itypes.NewResourceReference(ledgerTargetName, "default"), (*refusals)[0].target) - res, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) + res, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) require.True(t, resolved, "the refused write's save is answered rather than held") require.Error(t, res.Err, "and it fails: its write will never land") assert.Contains(t, res.Err.Error(), "kustomization.yaml", "naming what refused it") @@ -140,7 +140,7 @@ func TestReplayRefusal_AnotherFailureStillHoldsEveryWrite(t *testing.T) { require.Len(t, loop.pendingWrites, 2, "an aborted replay settles nothing") assert.Empty(t, *refusals, "and reports no refusal it did not finish deciding") - _, resolved := f.worker.LookupSaveRequestOutcome("default", crName, "uid-"+crName) + _, resolved := f.worker.LookupCommitRequestOutcome("default", crName, "uid-"+crName) assert.False(t, resolved, "the save stays held") assert.True(t, loop.retry.pending(), "the retry is scheduled") diff --git a/internal/git/request_template_test.go b/internal/git/request_template_test.go index 28219d6d..41d4cfd6 100644 --- a/internal/git/request_template_test.go +++ b/internal/git/request_template_test.go @@ -57,7 +57,7 @@ func TestFramedRequestMessage_ComposesTheReasonWithWhatWasSaved(t *testing.T) { // The contract PR 1 of this series must not break: the request's bytes reach the commit unaltered. // A target with no requestTemplate commits exactly what was supplied — no trimming, no framing, no -// reformatting — which is what makes a SaveRequest auditable against the commit it produced. +// reformatting — which is what makes a CommitRequest auditable against the commit it produced. func TestCommitMetadata_WithoutARequestTemplateTheLiteralIsCommittedByteForByte(t *testing.T) { literal := " fix(api): correct the service port \n\n Route traffic to the API container. " write := requestWrite(literal, ResolveCommitConfig(nil)) @@ -67,7 +67,7 @@ func TestCommitMetadata_WithoutARequestTemplateTheLiteralIsCommittedByteForByte( assert.Equal(t, literal, message, "an unframed request message must not be altered at all") assert.Equal(t, messageResolutionRequest, resolution) - assert.Equal(t, messageSourceSaveRequest, resolution.label()) + assert.Equal(t, messageSourceCommitRequest, resolution.label()) } // A requestTemplate that fails at render time must not cost the window. The literal already passed @@ -88,7 +88,7 @@ func TestFramedRequestMessage_RenderFailureFallsBackToTheLiteralAndSaysSo(t *tes assert.Equal(t, "fix(api): correct the service port", message) assert.Equal(t, messageResolutionRequestFallback, resolution) - assert.Equal(t, messageSourceSaveRequestFallback, resolution.label()) + assert.Equal(t, messageSourceCommitRequestFallback, resolution.label()) } // The framed and fallback arms must stay distinguishable in commits_total. Folding them together @@ -96,15 +96,15 @@ func TestFramedRequestMessage_RenderFailureFallsBackToTheLiteralAndSaysSo(t *tes func TestMessageSource_SeparatesFramedFromFallbackAndPlainRequest(t *testing.T) { framedConfig := requestConfig(requestBodyTemplate) - assert.Equal(t, messageSourceSaveRequest, + assert.Equal(t, messageSourceCommitRequest, requestWrite("save", ResolveCommitConfig(nil)).messageSource()) - assert.Equal(t, messageSourceSaveRequestFramed, + assert.Equal(t, messageSourceCommitRequestFramed, requestWrite("save", framedConfig).messageSource()) // Stamped at commit time, which is the only place the fallback is known. stamped := requestWrite("save", framedConfig) stamped.committedMessageSource = messageResolutionRequestFallback - assert.Equal(t, messageSourceSaveRequestFallback, stamped.messageSource()) + assert.Equal(t, messageSourceCommitRequestFallback, stamped.messageSource()) } // Precedence: a configured requestTemplate outranks the verbatim arm, and a window with no request @@ -244,7 +244,7 @@ func TestNewRequestTemplateProbe_IsFreshEachTime(t *testing.T) { assert.NotEmpty(t, first) } -// requestTemplate describes the commit a SaveRequest's message produces, and a SaveRequest +// requestTemplate describes the commit a CommitRequest's message produces, and a CommitRequest // attaches to a live window, which finalizes as PendingWriteCommit. A resync arrives pre-rendered // and is never framed, whatever requestTemplate says. func TestResolveMessage_AResyncIsNeverFramed(t *testing.T) { diff --git a/internal/git/resync_push_test.go b/internal/git/resync_push_test.go index 9942c6d4..03e24b21 100644 --- a/internal/git/resync_push_test.go +++ b/internal/git/resync_push_test.go @@ -308,7 +308,7 @@ func TestEnqueueResync_ALabelExitFencesTheSelectedSnapshot(t *testing.T) { } // TestHandleResyncRequest_ClosedWindowIsPushedEvenWhenNoOpResync pins the -// stranded-write fix in the SaveRequest window contract: a +// stranded-write fix in the CommitRequest window contract: a // resync that closes a live commit window but commits nothing of its own must // still schedule the window's commit for push. Before the fix maybeSchedulePush // ran only when the resync itself committed, so a window the resync closed was @@ -438,7 +438,7 @@ func TestEnqueueResync_AReleasedMarkerRunsTheRequestCoalescedIntoIt(t *testing.T } // TestEnqueueResync_DoesNotCoalescePastQueuedAttach pins the same fence for a -// SaveRequest attach, which carries no resource identity of its own but decides +// CommitRequest attach, which carries no resource identity of its own but decides // which commit window later work joins. func TestEnqueueResync_DoesNotCoalescePastQueuedAttach(t *testing.T) { w := &BranchWorker{Log: logr.Discard(), Branch: "main", eventQueue: make(chan WorkItem, 4)} @@ -448,7 +448,7 @@ func TestEnqueueResync_DoesNotCoalescePastQueuedAttach(t *testing.T) { GitTargetNamespace: "ns", GitTargetName: "target", ResourceVersion: "100", Result: first, })) - w.EnqueueAttach(&AttachSaveRequest{ + w.EnqueueAttach(&AttachCommitRequest{ Namespace: "ns", Name: "cr", GitTargetNamespace: "ns", GitTargetName: "target", }) require.True(t, w.EnqueueResync(&ResyncRequest{ diff --git a/internal/git/retry.go b/internal/git/retry.go index c88269bc..13e1df5b 100644 --- a/internal/git/retry.go +++ b/internal/git/retry.go @@ -10,7 +10,7 @@ import "time" // The cooldown in maybeSchedulePush spaces SUCCESSFUL pushes; it says nothing about failed ones, // because lastPushAt advances only on success. Without a schedule of its own a failure had two // outcomes, both wrong: a branch that then went quiet kept its work indefinitely (and any -// SaveRequest riding it stayed WaitingForPush, since the controller never fails a request the +// CommitRequest riding it stayed WaitingForPush, since the controller never fails a request the // worker holds), and a branch that kept receiving commits spent one failed attempt on each of them. // // So a failed attempt schedules the next one: due one backoff later, 10s doubling to 5m, on a timer diff --git a/internal/git/retry_test.go b/internal/git/retry_test.go index 93cd2356..18ed1438 100644 --- a/internal/git/retry_test.go +++ b/internal/git/retry_test.go @@ -5,7 +5,7 @@ package git // A failed publication schedules its own retry: a branch that goes quiet after the failure still // publishes, a branch that keeps receiving commits does not spend a push on each of them, and a // failure caused by a missing parent stays with parent recovery's schedule. These pin all three, -// and the SaveRequest that rides the retained write. +// and the CommitRequest that rides the retained write. import ( "context" @@ -198,15 +198,15 @@ func TestPublicationRetry_AFailureAfterTheParentReturnsWaitsForRecovery(t *testi assert.Equal(t, int32(2), attempts.Load(), "recovery's next deadline does") } -// The user-facing half, on the real event loop: a SaveRequest riding a write whose push failed is +// The user-facing half, on the real event loop: a CommitRequest riding a write whose push failed is // held (the controller never fails a held request), so before this it stayed WaitingForPush until // some unrelated write came along. It now resolves Committed on its own. -func TestPublicationRetry_AHeldSaveRequestIsCommittedWithoutAnotherWrite(t *testing.T) { +func TestPublicationRetry_AHeldCommitRequestIsCommittedWithoutAnotherWrite(t *testing.T) { original := retryInitialBackoff retryInitialBackoff = 50 * time.Millisecond t.Cleanup(func() { retryInitialBackoff = original }) - f := newRealServerNewBranch(t, "retry-held-save-request", seedMain(t)) + f := newRealServerNewBranch(t, "retry-held-commit-request", seedMain(t)) attempts := failNextPushes(t, 1) require.NoError(t, f.worker.Start(context.Background())) t.Cleanup(f.worker.Stop) diff --git a/internal/git/save_apply.go b/internal/git/save_apply.go index d9a47a7b..e9149957 100644 --- a/internal/git/save_apply.go +++ b/internal/git/save_apply.go @@ -13,7 +13,7 @@ import ( itypes "github.com/ConfigButler/gitops-reverser/internal/types" ) -// A SaveRequest with catchUp: AfterPush asks its GitTarget's applier to apply the tip as soon as its +// A CommitRequest with catchUp: AfterPush asks its GitTarget's applier to apply the tip as soon as its // commit is pushed, and the controller reports how that went in its Applied condition. The save's own // outcome is decided by the push alone; this is reported beside it and never changes it. // @@ -35,7 +35,7 @@ const ( SaveApplyCommitDropped = "CommitDropped" ) -// SaveApply is how the apply a SaveRequest asked for stands. +// SaveApply is how the apply a CommitRequest asked for stands. type SaveApply struct { // Done is whether it is settled: it changes no more. Done bool @@ -55,56 +55,56 @@ type saveApplyEntry struct { } // recordSaveApply stores where a save's apply stands, for the controller to read. The event loop is -// the only writer; the entries are collected with the SaveRequest outcomes. -func (w *BranchWorker) recordSaveApply(id saveRequestID, apply SaveApply) { +// the only writer; the entries are collected with the CommitRequest outcomes. +func (w *BranchWorker) recordSaveApply(id commitRequestID, apply SaveApply) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() if w.crApplies == nil { - w.crApplies = map[saveRequestID]saveApplyEntry{} + w.crApplies = map[commitRequestID]saveApplyEntry{} } w.crApplies[id] = saveApplyEntry{apply: apply, updatedAt: time.Now()} } -// LookupSaveApply reports where the apply a SaveRequest asked for stands, and false while the +// LookupSaveApply reports where the apply a CommitRequest asked for stands, and false while the // worker knows nothing of it. func (w *BranchWorker) LookupSaveApply(namespace, name, uid string) (SaveApply, bool) { w.crOutcomesMu.Lock() defer w.crOutcomesMu.Unlock() - entry, ok := w.crApplies[saveRequestID{Namespace: namespace, Name: name, UID: uid}] + entry, ok := w.crApplies[commitRequestID{Namespace: namespace, Name: name, UID: uid}] return entry.apply, ok } // collectSaveApplies drops the entries older than the outcome TTL. The caller holds crOutcomesMu. func (w *BranchWorker) collectSaveApplies(now time.Time) { for id, entry := range w.crApplies { - if now.Sub(entry.updatedAt) > saveRequestOutcomeTTL { + if now.Sub(entry.updatedAt) > commitRequestOutcomeTTL { delete(w.crApplies, id) } } } -// catchUpAfterSave is called as a SaveRequest that asked to catch up resolves. With a pushed +// catchUpAfterSave is called as a CommitRequest that asked to catch up resolves. With a pushed // commit, its GitTarget's applier gets a converge when the appliers are next listed, at the end of // this wake (takeSavedCatchUps); with none, there is nothing to apply. -func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingSaveRequest, commit string) { +func (l *branchWorkerEventLoop) catchUpAfterSave(pcr *pendingCommitRequest, commit string) { if commit == "" { l.w.recordSaveApply(pcr.id, SaveApply{Done: true, Applied: true, Reason: SaveApplyNothingToApply, Message: "the request pushed no commit: what it saved was in Git already, and the cluster holds it"}) return } if l.savedCatchUps == nil { - l.savedCatchUps = map[itypes.ResourceReference]map[saveRequestID]string{} + l.savedCatchUps = map[itypes.ResourceReference]map[commitRequestID]string{} } ref := itypes.NewResourceReference(pcr.gitTargetName, pcr.gitTargetNamespace) if l.savedCatchUps[ref] == nil { - l.savedCatchUps[ref] = map[saveRequestID]string{} + l.savedCatchUps[ref] = map[commitRequestID]string{} } l.savedCatchUps[ref][pcr.id] = commit l.w.recordSaveApply(pcr.id, SaveApply{Reason: SaveApplyApplying, Message: "commit " + commit + " is pushed; its applier is asked to apply it"}) } -// takeSavedCatchUps leaves a converge pending for every applier a pushed SaveRequest asked to catch +// takeSavedCatchUps leaves a converge pending for every applier a pushed CommitRequest asked to catch // up, and records the request as waiting on it. It was asked for, so it is no lag converge: it does // not yield to an edit (convergeHolds). A request whose GitTarget names no applier is settled. func (l *branchWorkerEventLoop) takeSavedCatchUps() { @@ -122,7 +122,7 @@ func (l *branchWorkerEventLoop) takeSavedCatchUps() { for id, commit := range saves { state.waitSave(id, commit) } - l.w.Log.Info("A SaveRequest asked its applier to catch up with the pushed commit", + l.w.Log.Info("A CommitRequest asked its applier to catch up with the pushed commit", "branch", l.w.Branch, "applier", state.ref.String(), "gitTarget", ref.String(), "requests", len(saves)) } l.savedCatchUps = nil @@ -140,11 +140,11 @@ func (l *branchWorkerEventLoop) applierNaming(ref itypes.ResourceReference) *app return nil } -// waitSave records a SaveRequest, which pushed commit, as waiting for the applier's next apply that +// waitSave records a CommitRequest, which pushed commit, as waiting for the applier's next apply that // succeeds. -func (s *applierState) waitSave(id saveRequestID, commit string) { +func (s *applierState) waitSave(id commitRequestID, commit string) { if s.waitingSaves == nil { - s.waitingSaves = map[saveRequestID]string{} + s.waitingSaves = map[commitRequestID]string{} } s.waitingSaves[id] = commit } @@ -157,7 +157,7 @@ func (s *applierState) waitSave(id saveRequestID, commit string) { // - one whose commit the branch holds and revision does not waits for the converge left pending; // - one the checkout cannot tell about, because it lacks a commit, waits as well, and the remote is // looked at: revision is a commit pushed elsewhere, and its fetch brings the apply that settles it. -func (l *branchWorkerEventLoop) settleSaves(state *applierState, saves map[saveRequestID]string, revision string) { +func (l *branchWorkerEventLoop) settleSaves(state *applierState, saves map[commitRequestID]string, revision string) { if len(saves) == 0 { return } @@ -235,7 +235,7 @@ func commitHolds(repo *gogit.Repository, descendant, commit string) (bool, bool) } // retrySaves records the saves an attempt that did not succeed gave back as waiting again, with why. -func (l *branchWorkerEventLoop) retrySaves(saves map[saveRequestID]string, why string) { +func (l *branchWorkerEventLoop) retrySaves(saves map[commitRequestID]string, why string) { for id := range saves { l.w.recordSaveApply(id, SaveApply{Reason: SaveApplyRetrying, Message: "the apply did not complete, and is asked again: " + why}) @@ -248,7 +248,7 @@ func (l *branchWorkerEventLoop) dropSaves(state *applierState) { saves := maps.Clone(state.waitingSaves) if state.inflight != nil { if saves == nil { - saves = map[saveRequestID]string{} + saves = map[commitRequestID]string{} } maps.Copy(saves, state.inflight.saves) } diff --git a/internal/git/types.go b/internal/git/types.go index 9469e3ec..0c2edde0 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -37,7 +37,7 @@ const ( // // Deliberately the EMPTY string so it is also the ZERO VALUE: the many paths that never assign // Attribution (reconcile, resync, bootstrap) all mean exactly this. Any other value makes the - // zero value a silent fourth state, which is the bug that stopped every SaveRequest + // zero value a silent fourth state, which is the bug that stopped every CommitRequest // attaching. TestAttributionZeroValueIsNotAttempted pins it. AttributionNotAttempted AttributionOutcome = "" // AttributionResolved means an audit fact named the actor. @@ -272,7 +272,7 @@ const ( // content-derived mark-and-sweep against the worktree (upsert every desired // resource, drop every watched managed document the snapshot did not contain). PendingWriteResync PendingWriteKind = "resync" - // PendingWriteRequestRecord carries no content. It records a SaveRequest's message + // PendingWriteRequestRecord carries no content. It records a CommitRequest's message // in an empty commit when the request ran out of time to attach, and asked for that with // whenNothingToCommit: CommitEmpty. It is not a window: only writes open windows. PendingWriteRequestRecord PendingWriteKind = "request_record" @@ -376,7 +376,7 @@ type PendingWrite struct { Committed *bool // AllowEmpty makes a grouped window commit even when its events changed nothing, because the - // SaveRequest attached to it asked for whenNothingToCommit: CommitEmpty. A suspended target + // CommitRequest attached to it asked for whenNothingToCommit: CommitEmpty. A suspended target // still writes nothing. AllowEmpty bool // RequestAuthor and RequestAttribution author a PendingWriteRequestRecord, which has no @@ -384,11 +384,11 @@ type PendingWrite struct { // window's author is. RequestAuthor UserInfo RequestAttribution AttributionOutcome - // TakenOver says the SaveRequest attached to this grouped write took over a window whose + // TakenOver says the CommitRequest attached to this grouped write took over a window whose // writes named no author: RequestAuthor and RequestAttribution, the submitter's, author the // commit, and its message ends with an Author-Source trailer (withTakeOverNote). TakenOver bool - // SavedBy is the submitter of the SaveRequest that gave this grouped write its message without + // SavedBy is the submitter of the CommitRequest that gave this grouped write its message without // claiming its writes, which named no author: the commit stays unattributed, and its message ends // with a Saved-By trailer naming them (withSavedByNote). Zero when no request did. SavedBy UserInfo @@ -398,10 +398,10 @@ type PendingWrite struct { // message". Re-stamped on a rebase-replay, where the same write may now change files. emptyCommitted bool - // SaveRequest, when set, is the SaveRequest attached to this write: it is + // CommitRequest, when set, is the CommitRequest attached to this write: it is // resolved Committed (with CommitSHA) once this write is pushed. It rides the write through the // push cooldown and the conflict rebase-replay, so the result follows the data. - SaveRequest *saveRequestID + CommitRequest *commitRequestID // committedMessageSource is the message source this write actually committed under, stamped by // executePendingWrites the way CommitSHA is and for the same reason: publishCommitsForPush runs // after the push, and recomputing the source from the write cannot know that a requestTemplate @@ -411,7 +411,7 @@ type PendingWrite struct { // actionsDecided is set once the write was planned on the tree it was first committed on. A // write carries the actions that plan decided, never a picture of the cluster to plan again // (docs/design/applier-echoes.md, F2): from then on Events holds only the objects whose plan - // changed that tree, except in a save (SaveRequest), which keeps every object it captured, + // changed that tree, except in a save (CommitRequest), which keeps every object it captured, // and a resync carries resyncActions instead of its snapshot. A replay applies those actions to // the newer tip, and plans nothing else again. Loop-goroutine only. actionsDecided bool @@ -454,18 +454,18 @@ type PendingWrite struct { type WorkItem struct { // Request is a resource-write request. Request *WriteRequest - // Attach is a SaveRequest attach: bind a message to the author's window and + // Attach is a CommitRequest attach: bind a message to the author's window and // let its timers close that window. - Attach *AttachSaveRequest + Attach *AttachCommitRequest // Resync is the FIFO position of a streaming-snapshot resync request (M8): a synchronous // request/reply that materialises a GitTarget's complete desired set. Resync *resyncMarker // Refresh asks the worker to re-prove where its branch is on the remote. It is the only // work item that never writes anything: see RefreshRequest. Refresh *RefreshRequest - // Withdraw cancels a SaveRequest the worker has not acted on yet. It rides the same FIFO as + // Withdraw cancels a CommitRequest the worker has not acted on yet. It rides the same FIFO as // the attach, so it is always handled after every attach sent before it. - Withdraw *AttachSaveRequest + Withdraw *AttachCommitRequest // ApplyBarrier is one answer to the barrier an apply window took (apply_barrier.go). A stream's // answer rides the FIFO behind every event that stream routed before it. ApplyBarrier *ApplyBarrier @@ -753,7 +753,7 @@ type Event struct { UserInfo UserInfo // Attribution is the authority for author rendering, the author_kind metric, and - // SaveRequest window matching. None may infer the outcome from UserInfo: an empty username + // CommitRequest window matching. None may infer the outcome from UserInfo: an empty username // cannot distinguish "attribution is off" from "it ran and found nothing". attachAuthor is // the only assignment outside tests. Attribution AttributionOutcome @@ -832,7 +832,7 @@ type CommitterConfig struct { type CommitMessageConfig struct { LiveTemplate string ReconcileTemplate string - // RequestTemplate frames a SaveRequest's message. Empty — the default — commits that + // RequestTemplate frames a CommitRequest's message. Empty — the default — commits that // message verbatim, which is the behaviour every target had before the field existed. RequestTemplate string } @@ -992,7 +992,7 @@ type LiveCommitMessageData struct { // Resources is the per-resource list, deduplicated by file path so the // final state is what's being committed. Resources []ResourceRef - // RequestMessage is the message an attached SaveRequest supplied, unaltered. It is empty for + // RequestMessage is the message an attached CommitRequest supplied, unaltered. It is empty for // every window no request attached to, which is the ordinary case and why liveTemplate may // reference it freely. // diff --git a/internal/git/worker_manager.go b/internal/git/worker_manager.go index a0630cc8..16d49f00 100644 --- a/internal/git/worker_manager.go +++ b/internal/git/worker_manager.go @@ -104,11 +104,11 @@ type WorkerManager struct { // the scrape goroutine. Take it BEFORE mu, never the other way round. lifecycleMu sync.Mutex workers map[BranchKey]*BranchWorker - // saveRequests records which worker holds each SaveRequest. It outlives the map above on + // commitRequests records which worker holds each CommitRequest. It outlives the map above on // purpose: a retired worker leaves the map before it stops, and keeps answering for the - // requests it held. See saveRequestOwners. - saveRequests *saveRequestOwners - ctx context.Context + // requests it held. See commitRequestOwners. + commitRequests *commitRequestOwners + ctx context.Context // mapper is the GVK->GVR resolver injected into every worker so store scans build a // resource-identity inventory. It is set once at startup (SetMapper) before any // worker is created; a nil mapper keeps workers structure-only. It is the LOCAL cluster's @@ -223,7 +223,7 @@ func NewWorkerManager( limits: limits.withDefaults(), sensitiveResources: sensitiveResources, workers: make(map[BranchKey]*BranchWorker), - saveRequests: &saveRequestOwners{}, + commitRequests: &commitRequestOwners{}, remotes: make(map[BranchKey]RemoteObservation), replacements: make(map[BranchKey]struct{}), renderFidelityGate: NewRenderFidelityGate(), @@ -538,7 +538,7 @@ func (m *WorkerManager) startWorkerLocked(key BranchKey, req WorkerRequest, gitB worker.replacementPending = func() bool { return m.ReplacementPending(key) } worker.applierAdmitted = m.applierAdmitted worker.renderFidelityGate = m.renderFidelityGate - worker.crOwners = m.saveRequests + worker.crOwners = m.commitRequests if err := worker.Start(m.ctx); err != nil { return fmt.Errorf("failed to start worker for %s: %w", key.String(), err) @@ -685,7 +685,7 @@ func (m *WorkerManager) removeWorkersLocked(keys []BranchKey, reason string) { // RetireGitBranch stops the worker writing for gitBranch, if one is live, and returns once it has // stopped. Stopping closes the worker's intake, pushes or refuses every pending write and settles -// every SaveRequest it holds, so a GitBranch being deleted calls this before its finalizer goes. +// every CommitRequest it holds, so a GitBranch being deleted calls this before its finalizer goes. // What the manager kept about the branch goes with it: the next holder starts from the remote. func (m *WorkerManager) RetireGitBranch(gitBranch k8stypes.NamespacedName) { m.lifecycleMu.Lock() @@ -736,12 +736,12 @@ func (m *WorkerManager) WorkerForTarget(target *configv1alpha3.GitTarget) (*Bran return m.WorkerForGitBranch(target.GitBranchKey()) } -// SaveRequestOwner returns the worker that holds a SaveRequest: the one that accepted its +// CommitRequestOwner returns the worker that holds a CommitRequest: the one that accepted its // attach and has not given it back. It is found whatever has happened to the request's GitTarget // since, including a worker retired and still finishing its last push, so a caller that finds no // owner knows that no worker can still commit the request. -func (m *WorkerManager) SaveRequestOwner(namespace, name, uid string) (*BranchWorker, bool) { - return m.saveRequests.owner(saveRequestID{Namespace: namespace, Name: name, UID: uid}) +func (m *WorkerManager) CommitRequestOwner(namespace, name, uid string) (*BranchWorker, bool) { + return m.commitRequests.owner(commitRequestID{Namespace: namespace, Name: name, UID: uid}) } // afterOrphanSelection runs between choosing the workers to retire and retiring them. It is nil in diff --git a/internal/git/worker_repo_identity_test.go b/internal/git/worker_repo_identity_test.go index 29180177..11ab2fde 100644 --- a/internal/git/worker_repo_identity_test.go +++ b/internal/git/worker_repo_identity_test.go @@ -261,7 +261,7 @@ func TestEnqueue_RefusesOnceTheWorkerIsStopping(t *testing.T) { "and a resync must be answered rather than left waiting on a reply that never comes") // The two fire-and-forget paths report nothing, so they must at least leave no trace: an // increment here would be work the depth gauge counts and nothing will ever finish. - worker.EnqueueAttach(&AttachSaveRequest{Namespace: "ns", Name: "cr"}) + worker.EnqueueAttach(&AttachCommitRequest{Namespace: "ns", Name: "cr"}) worker.EnqueueRefresh(&RefreshRequest{}) assert.Equal(t, inflight, worker.inflightItems.Load(), diff --git a/internal/git/write_gate.go b/internal/git/write_gate.go index 5baea32f..f849c7f6 100644 --- a/internal/git/write_gate.go +++ b/internal/git/write_gate.go @@ -9,12 +9,12 @@ import ( // The write gates are the two reasons a GitTarget may not be written at all: spec.suspend, and a // render-fidelity gate that has not established render-vs-live for it. They are rules about the -// TARGET, and every path that decides a SaveRequest's outcome asks targetWriteRefusal before it +// TARGET, and every path that decides a CommitRequest's outcome asks targetWriteRefusal before it // reports one, so a save on a target that refuses writes fails with the reason instead of resolving // Ready=True with nothing written. That holds whichever path ends the request and whatever its // whenNothingToCommit says: // -// - a request no window reached (expireWaitingSaveRequests, before the record it may make); +// - a request no window reached (expireWaitingCommitRequests, before the record it may make); // - the record itself (buildRequestRecordWrite); // - a window a request is attached to (finalizeOpenWindowWithReason before it builds the write, // and gateBuiltWindow after, right before the commit). A render-fidelity refusal seen at @@ -59,7 +59,7 @@ func (w *BranchWorker) targetWriteRefusal(name, namespace string, suspended bool // spec.suspend from the live GitTarget. Only evidence refuses: a target that cannot be read is // left to the path that writes, which fails on it, and a request that writes nothing (Resolve) // keeps its answer. -func (w *BranchWorker) requestWriteRefusal(ctx context.Context, pcr *pendingSaveRequest) error { +func (w *BranchWorker) requestWriteRefusal(ctx context.Context, pcr *pendingCommitRequest) error { suspended := false if target, err := w.getGitTarget(ctx, pcr.gitTargetName, pcr.gitTargetNamespace); err == nil { suspended = target.Spec.Suspend diff --git a/internal/git/write_gate_test.go b/internal/git/write_gate_test.go index 3f33d4e8..49f01db2 100644 --- a/internal/git/write_gate_test.go +++ b/internal/git/write_gate_test.go @@ -21,7 +21,7 @@ import ( "github.com/ConfigButler/gitops-reverser/internal/types" ) -// These tests pin the write gates every SaveRequest outcome passes (write_gate.go): a request +// These tests pin the write gates every CommitRequest outcome passes (write_gate.go): a request // that ends on a target that refuses writes fails with the reason, whatever path it ends on and // whatever whenNothingToCommit says. They drive the transitions — a gate that closes after the // attach, after the local commit, on a sibling target, across a failed push — and not only the @@ -46,7 +46,7 @@ func closeRenderGate(worker *BranchWorker) { } // requestFor is a request named name, for target, by alice. -func requestFor(name, target string, commitEmpty bool) *AttachSaveRequest { +func requestFor(name, target string, commitEmpty bool) *AttachCommitRequest { req := attachReq("alice", time.Hour) req.Name, req.UID = name, "uid-"+name req.GitTargetName = target @@ -127,17 +127,17 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { endings := []struct { name string // run closes the gate at the point this ending puts it, and ends the request. - run func(t *testing.T, loop *branchWorkerEventLoop, closeGate func(), req *AttachSaveRequest) + run func(t *testing.T, loop *branchWorkerEventLoop, closeGate func(), req *AttachCommitRequest) }{ {"no window before the attach timeout", func(_ *testing.T, loop *branchWorkerEventLoop, closeGate func(), - req *AttachSaveRequest) { + req *AttachCommitRequest) { closeGate() serviceAttach(loop, req) expire(loop) loop.pushPending() }}, {"attached, the gate closing before the window closes", func(t *testing.T, loop *branchWorkerEventLoop, - closeGate func(), req *AttachSaveRequest) { + closeGate func(), req *AttachCommitRequest) { serviceAttach(loop, req) writeTo(loop, "new") require.NotNil(t, loop.openWindow, "the request is attached to an open window") @@ -146,7 +146,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { loop.pushPending() }}, {"attached to a window that changed nothing", func(t *testing.T, loop *branchWorkerEventLoop, - closeGate func(), req *AttachSaveRequest) { + closeGate func(), req *AttachCommitRequest) { writeTo(loop, "existing") serviceAttach(loop, req) require.NotNil(t, loop.openWindow) @@ -170,7 +170,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { ending.run(t, loop, func() { gate.close(t, worker) }, req) - res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.True(t, ok, "the request is resolved") require.ErrorIs(t, res.Err, gate.want, "a refused save is a failure, never Ready=True") assert.Empty(t, res.Commit) @@ -183,7 +183,7 @@ func TestWriteGate_ARequestOnARefusingTargetFails(t *testing.T) { } // forceDueNamed closes req's attached window at its maxDuration, as forceDue does for crName. -func forceDueNamed(loop *branchWorkerEventLoop, req *AttachSaveRequest) { +func forceDueNamed(loop *branchWorkerEventLoop, req *AttachCommitRequest) { id := req.id() if loop.openWindow != nil && loop.openWindow.pendingCR != nil && *loop.openWindow.pendingCR == id { loop.openWindow.timers.maxAt = time.Now().Add(-time.Millisecond) @@ -207,7 +207,7 @@ func TestWriteGate_ASaveWhoseWritesTheRenderGateDroppedSaysSo(t *testing.T) { expire(loop) loop.pushPending() - res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.ErrorIs(t, res.Err, errRenderFidelityClosed) assert.Equal(t, before, remoteHead(t, serverRepo), "no empty commit claims the save saw nothing") @@ -227,15 +227,15 @@ func TestWriteGate_AGateClosingAfterTheLocalCommitDoesNotRetractIt(t *testing.T) } commits := []struct { name string - commit func(loop *branchWorkerEventLoop, req *AttachSaveRequest) + commit func(loop *branchWorkerEventLoop, req *AttachCommitRequest) outcome FinalizeOutcome }{ - {"a window that changed files", func(loop *branchWorkerEventLoop, req *AttachSaveRequest) { + {"a window that changed files", func(loop *branchWorkerEventLoop, req *AttachCommitRequest) { serviceAttach(loop, req) writeTo(loop, "new") forceDueNamed(loop, req) }, FinalizeCommitted}, - {"the record of a request no window reached", func(loop *branchWorkerEventLoop, req *AttachSaveRequest) { + {"the record of a request no window reached", func(loop *branchWorkerEventLoop, req *AttachCommitRequest) { serviceAttach(loop, req) expire(loop) }, FinalizeNoOpenWindow}, @@ -255,12 +255,12 @@ func TestWriteGate_AGateClosingAfterTheLocalCommitDoesNotRetractIt(t *testing.T) gate.close(t, worker) loop.pushPending() - _, resolved := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + _, resolved := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.False(t, resolved, "the gate closing does not resolve a request still waiting for its push") restore() loop.pushPending() - res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.NoError(t, res.Err) assert.Equal(t, c.outcome, res.Outcome) @@ -299,11 +299,11 @@ func TestWriteGate_SiblingTargetsAreGatedApart(t *testing.T) { expire(loop) loop.pushPending() - res, ok := worker.LookupSaveRequestOutcome("default", refused.Name, refused.UID) + res, ok := worker.LookupCommitRequestOutcome("default", refused.Name, refused.UID) require.True(t, ok) require.ErrorIs(t, res.Err, gate.want) - res, ok = worker.LookupSaveRequestOutcome("default", served.Name, served.UID) + res, ok = worker.LookupCommitRequestOutcome("default", served.Name, served.UID) require.True(t, ok) require.NoError(t, res.Err, "a sibling's gate is not this target's") assert.Equal(t, FinalizeNoOpenWindow, res.Outcome) @@ -324,7 +324,7 @@ func TestWriteGate_AWindowMismatchIsStillAMismatch(t *testing.T) { writeTo(loop, "alices-edit") expire(loop) - res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.NoError(t, res.Err) assert.Equal(t, FinalizeWindowMismatch, res.Outcome) @@ -376,7 +376,7 @@ func TestWriteGate_ARenderGateClosingDuringFinalizeStopsTheWholeWindow(t *testin assert.Empty(t, loop.pendingWrites, "nothing is retained") assert.Equal(t, before, remoteHead(t, serverRepo), "nothing reached the remote") if withRequest { - res, ok := worker.LookupSaveRequestOutcome("default", req.Name, req.UID) + res, ok := worker.LookupCommitRequestOutcome("default", req.Name, req.UID) require.True(t, ok) require.ErrorIs(t, res.Err, errRenderFidelityClosed) } diff --git a/internal/telemetry/exporter.go b/internal/telemetry/exporter.go index 7a362919..7bde4072 100644 --- a/internal/telemetry/exporter.go +++ b/internal/telemetry/exporter.go @@ -44,7 +44,7 @@ var ( // GitTarget: a resync can apply every document and then abort on a precondition, writing // nothing at all. GitDocumentsTotal metric.Int64Counter - // SaveRequestsTotal counts SaveRequests reaching a terminal outcome, labelled by + // CommitRequestsTotal counts CommitRequests reaching a terminal outcome, labelled by // {outcome, gittarget_namespace, gittarget_name}. outcome is committed / no_window / // window_mismatch / already_present / failed. // @@ -58,13 +58,13 @@ var ( // // An UNRESOLVED target publishes the empty pair rather than the requested name. spec.gitTargetRef // is whatever the client wrote, so labelling it would let anyone with create rights on - // saverequests mint unbounded series by naming targets that do not exist; the requested + // commitrequests mint unbounded series by naming targets that do not exist; the requested // reference stays on the object's conditions and in the logs. // // It is the fleet view resource_condition deliberately refuses to give this kind. A - // SaveRequest is created once per save, so a gauge per object would churn a series + // CommitRequest is created once per save, so a gauge per object would churn a series // continuously; a counter over terminal outcomes is bounded by the enum instead. The pointer - // that used to stand in for it, git_commits_total{message_source="save_request"}, counts + // that used to stand in for it, git_commits_total{message_source="commit_request"}, counts // only the requests that succeeded AND were used, so the whole failure population had no // series at all — and this is the one object a human waits on synchronously. A request that // does not land still produces a green, pushed commit carrying a GENERATED message instead of @@ -75,19 +75,19 @@ var ( // // COUNTING SEMANTICS, because this counter cannot promise what it looks like it promises. It // counts one increment per TERMINAL DECISION ATTEMPT, recorded outside the status-write retry - // loop. That is not once per SaveRequest: a status write that fails permanently is requeued + // loop. That is not once per CommitRequest: a status write that fails permanently is requeued // and re-decides, and restart recovery re-reconciles any request whose terminal status never - // persisted (documented at SaveRequestReconciler.SetupWithManager as a knowingly-accepted + // persisted (documented at CommitRequestReconciler.SetupWithManager as a knowingly-accepted // gap). Deduplicating across invocations needs a durable marker on the object, which is an API // change rather than an instrument. Read rates and ratios from it, not exact request counts. - SaveRequestsTotal metric.Int64Counter + CommitRequestsTotal metric.Int64Counter // GitCommitsTotal counts commit batches that REACHED THE REMOTE, labelled by the recording // BranchWorker's identity {gitbranch_namespace, gitbranch_name, provider_namespace, // provider_name} (gitbranch_* and provider_* below) plus author_kind and // message_source. Live, snapshot and resync paths all feed this one counter; message_source - // (live / reconcile / save_request / save_request_framed / save_request_fallback) is what + // (live / reconcile / commit_request / commit_request_framed / commit_request_fallback) is what // tells them apart. It counts commits that USED a - // request-supplied message, not SaveRequests: a request omitting spec.message renders through + // request-supplied message, not CommitRequests: a request omitting spec.message renders through // liveTemplate and counts as live. // // It is recorded on a successful push, not on local commit creation. Recorded at creation it @@ -186,7 +186,7 @@ var ( // // This is the head-of-line signal, and it is a proven failure rather than a theoretical one: a // slow attribution resolution blocks the events queued BEHIND it on the same single-threaded - // stream, which broke a SaveRequest e2e spec and was only ever visible by correlating two + // stream, which broke a CommitRequest e2e spec and was only ever visible by correlating two // log lines by hand. The attribution wait histogram times each resolution in isolation and // cannot see the delay one imposes on its neighbours; occupancy can, because a stream that is // busy is a stream nothing else is being read from. @@ -257,7 +257,7 @@ var ( // (idle_timeout, max_duration, attach_next, identity_change, buffer_limit, resync_before_apply, // shutdown, foreign_commit for a Git host's push notice, apply_due for the // flush before an apply); timer_source says whose timers were in effect: the GitTarget's, - // or a SaveRequest's that attached and replaced them. Every window is counted exactly once, at + // or a CommitRequest's that attached and replaced them. Every window is counted exactly once, at // the start of its finalize, so a window that then produces no diff or fails to commit is counted // too. A push replay rebuilds commits and closes no window, and a CommitEmpty record has none. // @@ -580,7 +580,7 @@ func registerCounters() error { counters := []cSpec{ {"gitopsreverser_git_documents_total", &GitDocumentsTotal}, {"gitopsreverser_git_commits_total", &GitCommitsTotal}, - {"gitopsreverser_save_requests_total", &SaveRequestsTotal}, + {"gitopsreverser_commit_requests_total", &CommitRequestsTotal}, {"gitopsreverser_git_commit_failures_total", &GitCommitFailuresTotal}, {"gitopsreverser_git_writes_dropped_total", &GitWritesDroppedTotal}, {"gitopsreverser_git_commit_windows_total", &GitCommitWindowsTotal}, diff --git a/internal/watch/event_router.go b/internal/watch/event_router.go index 6744999e..2ac5f531 100644 --- a/internal/watch/event_router.go +++ b/internal/watch/event_router.go @@ -73,8 +73,8 @@ func NewEventRouter( } } -// ServiceSaveRequest is the controller's attach-then-poll seam (§6.4.3): it -// resolves the worker that holds the request, or else the GitTarget's branch worker, registers the SaveRequest attach +// ServiceCommitRequest is the controller's attach-then-poll seam (§6.4.3): it +// resolves the worker that holds the request, or else the GitTarget's branch worker, registers the CommitRequest attach // idempotently on that worker's FIFO event queue (attach it to the author's window // with its message and timers), and returns the request's current // outcome. resolved=false means the worker has not finished — the controller @@ -88,16 +88,16 @@ func NewEventRouter( // It is reported as pending and NOT as an error, because the GitTarget did resolve: the controller // reads a service error as "the target never resolved" and would count the request's eventual // failure without its GitTarget labels. A GitTarget that cannot be read is that error. -func (r *EventRouter) ServiceSaveRequest( +func (r *EventRouter) ServiceCommitRequest( ctx context.Context, - attach git.AttachSaveRequest, + attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { - worker, branch, err := r.saveRequestWorker(ctx, attach) + worker, branch, err := r.commitRequestWorker(ctx, attach) if err != nil { return git.FinalizeResult{}, false, err } if worker == nil { - r.Log.V(1).Info("ServiceSaveRequest: no branch worker for the GitTarget yet; will retry", + r.Log.V(1).Info("ServiceCommitRequest: no branch worker for the GitTarget yet; will retry", "gitTarget", attach.GitTargetNamespace+"/"+attach.GitTargetName) return git.FinalizeResult{Branch: branch, Phase: git.PhaseWaitingForWorker}, false, nil } @@ -105,17 +105,17 @@ func (r *EventRouter) ServiceSaveRequest( // Idempotent register (the worker keys by request identity and keeps the first // finalize deadline), then poll the outcome. worker.EnqueueAttach(&attach) - result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID) + result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID) if !resolved { // Report where the worker says the request stands, so the controller never has to infer // a phase from having sent the attach. - result.Phase = worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) + result.Phase = worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) result.Held = heldBecause(worker, result.Phase) } return result, resolved, nil } -// WithdrawSaveRequest is the controller's fail-closed seam: it cancels a request the worker has +// WithdrawCommitRequest is the controller's fail-closed seam: it cancels a request the worker has // not acted on, and reports the worker's answer, so the controller never fails a request the worker // could still commit. // @@ -126,14 +126,14 @@ func (r *EventRouter) ServiceSaveRequest( // - Nothing holds it, and the GitTarget is gone or names no worker: nothing can commit the // request, so it resolves at once as withdrawn. The controller sends no attach after it starts // withdrawing, so a worker that starts later never sees it. -func (r *EventRouter) WithdrawSaveRequest( +func (r *EventRouter) WithdrawCommitRequest( ctx context.Context, - attach git.AttachSaveRequest, + attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { - worker, branch, err := r.saveRequestWorker(ctx, attach) + worker, branch, err := r.commitRequestWorker(ctx, attach) if errors.Is(err, errGitTargetGone) { // No worker holds the request, and its GitTarget can never start one. - return git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, true, nil + return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil } if err != nil { return git.FinalizeResult{}, false, err @@ -142,32 +142,32 @@ func (r *EventRouter) WithdrawSaveRequest( return git.FinalizeResult{ Branch: branch, Phase: git.PhaseWaitingForWorker, - Err: git.ErrSaveRequestWithdrawn, + Err: git.ErrCommitRequestWithdrawn, }, true, nil } - if result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { + if result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { return result, true, nil } - phase := worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) + phase := worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) if phase.Held() { return git.FinalizeResult{Branch: branch, Phase: phase, Held: heldBecause(worker, phase)}, false, nil } worker.EnqueueWithdraw(&attach) // A worker whose loop has exited answers the withdraw at once. - if result, resolved := worker.LookupSaveRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { + if result, resolved := worker.LookupCommitRequestOutcome(attach.Namespace, attach.Name, attach.UID); resolved { return result, true, nil } return git.FinalizeResult{Branch: branch, Phase: phase}, false, nil } -// LookupSaveRequestApply reports where the apply a SaveRequest asked for stands, from the +// LookupCommitRequestApply reports where the apply a CommitRequest asked for stands, from the // branch worker of its GitTarget, and false while that worker knows nothing of it or none runs. -func (r *EventRouter) LookupSaveRequestApply( +func (r *EventRouter) LookupCommitRequestApply( ctx context.Context, - attach git.AttachSaveRequest, + attach git.AttachCommitRequest, ) (git.SaveApply, bool, error) { - worker, _, err := r.saveRequestWorker(ctx, attach) + worker, _, err := r.commitRequestWorker(ctx, attach) if errors.Is(err, errGitTargetGone) { return git.SaveApply{Done: true, Applied: true, Reason: git.SaveApplyApplierGone, Message: "the GitTarget is gone, so its applier is asked for nothing"}, true, nil @@ -181,14 +181,14 @@ func (r *EventRouter) LookupSaveRequestApply( // heldBecause is why a request waiting for its push has not reached the remote: the worker's // publication report while it cannot publish, empty otherwise. -func heldBecause(worker *git.BranchWorker, phase git.SaveRequestPhase) string { +func heldBecause(worker *git.BranchWorker, phase git.CommitRequestPhase) string { if phase != git.PhaseWaitingForPush { return "" } return worker.Publication().Message() } -// saveRequestWorker finds the worker to ask about a SaveRequest, and the branch it serves. +// commitRequestWorker finds the worker to ask about a CommitRequest, and the branch it serves. // // The worker that holds the request comes first, whatever its GitTarget says now: a shared worker // outlives a deleted target, and a retired one finishes its last push after the manager stopped @@ -196,11 +196,11 @@ func heldBecause(worker *git.BranchWorker, phase git.SaveRequestPhase) string { // nil while that worker has not started. A GitTarget that is gone is errGitTargetGone, and so is one // that replaced the GitTarget that admitted the request (attach.GitTargetUID): the replacement admitted // nothing. -func (r *EventRouter) saveRequestWorker( +func (r *EventRouter) commitRequestWorker( ctx context.Context, - attach git.AttachSaveRequest, + attach git.AttachCommitRequest, ) (*git.BranchWorker, string, error) { - if owner, ok := r.WorkerManager.SaveRequestOwner(attach.Namespace, attach.Name, attach.UID); ok { + if owner, ok := r.WorkerManager.CommitRequestOwner(attach.Namespace, attach.Name, attach.UID); ok { return owner, owner.Branch, nil } diff --git a/internal/watch/event_router_test.go b/internal/watch/event_router_test.go index 5977430c..3aafe2f6 100644 --- a/internal/watch/event_router_test.go +++ b/internal/watch/event_router_test.go @@ -32,8 +32,8 @@ func eventRouterScheme(t *testing.T) *runtime.Scheme { return scheme } -func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachSaveRequest { - return git.AttachSaveRequest{ +func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachCommitRequest { + return git.AttachCommitRequest{ Namespace: gitTargetNamespace, Name: "save", UID: "uid-save", @@ -43,7 +43,7 @@ func saveAttach(gitTargetName, gitTargetNamespace string) git.AttachSaveRequest } } -func TestServiceSaveRequest_GitTargetNotFound(t *testing.T) { +func TestServiceCommitRequest_GitTargetNotFound(t *testing.T) { scheme := eventRouterScheme(t) client := fake.NewClientBuilder().WithScheme(scheme).Build() workerManager := git.NewWorkerManager( @@ -55,7 +55,7 @@ func TestServiceSaveRequest_GitTargetNotFound(t *testing.T) { router := NewEventRouter(workerManager, nil, client, logr.Discard()) - _, resolved, err := router.ServiceSaveRequest(context.Background(), saveAttach("missing", "team-a")) + _, resolved, err := router.ServiceCommitRequest(context.Background(), saveAttach("missing", "team-a")) require.Error(t, err) assert.False(t, resolved) require.ErrorIs(t, err, errGitTargetGone) @@ -63,7 +63,7 @@ func TestServiceSaveRequest_GitTargetNotFound(t *testing.T) { // A save whose GitTarget is gone needs no apply: Applied=True with ApplierGone. Only a commit the // branch lost is ever Applied=False. -func TestLookupSaveRequestApply_GitTargetGoneIsApplied(t *testing.T) { +func TestLookupCommitRequestApply_GitTargetGoneIsApplied(t *testing.T) { scheme := eventRouterScheme(t) client := fake.NewClientBuilder().WithScheme(scheme).Build() workerManager := git.NewWorkerManager( @@ -74,7 +74,7 @@ func TestLookupSaveRequestApply_GitTargetGoneIsApplied(t *testing.T) { ) router := NewEventRouter(workerManager, nil, client, logr.Discard()) - apply, known, err := router.LookupSaveRequestApply(context.Background(), saveAttach("missing", "team-a")) + apply, known, err := router.LookupCommitRequestApply(context.Background(), saveAttach("missing", "team-a")) require.NoError(t, err) require.True(t, known) assert.True(t, apply.Done) @@ -316,7 +316,7 @@ func TestGitPathRefusalReason(t *testing.T) { // A GitTarget of the name an attach carries, but with another UID than the GitTarget that admitted // the request, replaced it and admitted nothing: no worker of it is asked to take the request. -func TestServiceSaveRequest_AReplacedGitTargetIsGone(t *testing.T) { +func TestServiceCommitRequest_AReplacedGitTargetIsGone(t *testing.T) { scheme := eventRouterScheme(t) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a", UID: "uid-second"}, @@ -335,25 +335,25 @@ func TestServiceSaveRequest_AReplacedGitTargetIsGone(t *testing.T) { attach := saveAttach("team-a-config", "team-a") attach.GitTargetUID = "uid-first" - _, resolved, err := router.ServiceSaveRequest(context.Background(), attach) + _, resolved, err := router.ServiceCommitRequest(context.Background(), attach) require.ErrorIs(t, err, errGitTargetGone) assert.False(t, resolved) - result, resolved, err := router.WithdrawSaveRequest(context.Background(), attach) + result, resolved, err := router.WithdrawCommitRequest(context.Background(), attach) require.NoError(t, err) assert.True(t, resolved, "nothing holds it, and the replacement can never take it") - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) attach.GitTargetUID = "uid-second" - _, _, err = router.ServiceSaveRequest(context.Background(), attach) + _, _, err = router.ServiceCommitRequest(context.Background(), attach) require.NoError(t, err, "the GitTarget that admitted it is asked") } -// TestServiceSaveRequest_NoWorkerKeepsTheRequestPending pins that a missing worker is not an -// outcome. At startup a SaveRequest can reach the controller before its GitTarget's worker exists; +// TestServiceCommitRequest_NoWorkerKeepsTheRequestPending pins that a missing worker is not an +// outcome. At startup a CommitRequest can reach the controller before its GitTarget's worker exists; // resolving it then would end the save before anything could collect its writes or record its // message, so it stays pending and the controller polls within its safety window. -func TestServiceSaveRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { +func TestServiceCommitRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { scheme := eventRouterScheme(t) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, @@ -371,7 +371,7 @@ func TestServiceSaveRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { router := NewEventRouter(workerManager, nil, client, logr.Discard()) - result, resolved, err := router.ServiceSaveRequest(context.Background(), saveAttach("team-a-config", "team-a")) + result, resolved, err := router.ServiceCommitRequest(context.Background(), saveAttach("team-a-config", "team-a")) require.NoError(t, err, "the GitTarget resolved; only its worker is missing") assert.False(t, resolved, "a worker that does not exist yet is a retry, not an answer") assert.Empty(t, result.Outcome) @@ -379,7 +379,7 @@ func TestServiceSaveRequest_NoWorkerKeepsTheRequestPending(t *testing.T) { assert.Equal(t, "main", result.Branch) } -func TestServiceSaveRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { +func TestServiceCommitRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -422,13 +422,13 @@ func TestServiceSaveRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { // No events routed and attachTimeout 0s, so the worker has no window: the attach // is enqueued, processed by the worker loop, and resolves NoOpenWindow. The - // controller polls via repeated (idempotent) ServiceSaveRequest calls. + // controller polls via repeated (idempotent) ServiceCommitRequest calls. attach := saveAttach("team-a-config", "team-a") var result git.FinalizeResult require.Eventually(t, func() bool { var resolved bool var err error - result, resolved, err = router.ServiceSaveRequest(context.Background(), attach) + result, resolved, err = router.ServiceCommitRequest(context.Background(), attach) require.NoError(t, err) return resolved }, 5*time.Second, 50*time.Millisecond, "the worker must resolve the attached request") @@ -436,10 +436,10 @@ func TestServiceSaveRequest_RegisteredWorkerResolvesNoOpenWindow(t *testing.T) { assert.Equal(t, "main", result.Branch) } -// TestWithdrawSaveRequest_NothingHoldsItWithoutAWorker: a deleted GitTarget, or one whose worker +// TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker: a deleted GitTarget, or one whose worker // never started, has nothing that could commit the request, so the withdraw resolves at once and // says why when it can. -func TestWithdrawSaveRequest_NothingHoldsItWithoutAWorker(t *testing.T) { +func TestWithdrawCommitRequest_NothingHoldsItWithoutAWorker(t *testing.T) { scheme := eventRouterScheme(t) gitTarget := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "team-a"}, @@ -452,24 +452,24 @@ func TestWithdrawSaveRequest_NothingHoldsItWithoutAWorker(t *testing.T) { client, logr.Discard(), git.BranchWorkerLimits{}, types.SensitiveResourcePolicy{}) router := NewEventRouter(workerManager, nil, client, logr.Discard()) - result, resolved, err := router.WithdrawSaveRequest(context.Background(), saveAttach("team-a-config", "team-a")) + result, resolved, err := router.WithdrawCommitRequest(context.Background(), saveAttach("team-a-config", "team-a")) require.NoError(t, err) require.True(t, resolved) - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) assert.Equal(t, git.PhaseWaitingForWorker, result.Phase) assert.Equal(t, "main", result.Branch) - result, resolved, err = router.WithdrawSaveRequest(context.Background(), saveAttach("deleted", "team-b")) + result, resolved, err = router.WithdrawCommitRequest(context.Background(), saveAttach("deleted", "team-b")) require.NoError(t, err) require.True(t, resolved) - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) assert.Empty(t, result.Branch, "a deleted GitTarget has no identity to report") } -// TestWithdrawSaveRequest_ALiveWorkerCancelsAWaitingRequest: the withdraw rides the FIFO behind the +// TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest: the withdraw rides the FIFO behind the // attach, cancels the request while it only waits for a window, and an attach sent afterwards // cannot bring it back. -func TestWithdrawSaveRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { +func TestWithdrawCommitRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -508,29 +508,29 @@ func TestWithdrawSaveRequest_ALiveWorkerCancelsAWaitingRequest(t *testing.T) { attach := saveAttach("team-a-config", "team-a") attach.AttachTimeout = time.Hour // it would wait an hour for a window - _, resolved, err := router.ServiceSaveRequest(context.Background(), attach) + _, resolved, err := router.ServiceCommitRequest(context.Background(), attach) require.NoError(t, err) require.False(t, resolved) var result git.FinalizeResult require.Eventually(t, func() bool { var err error - result, resolved, err = router.WithdrawSaveRequest(context.Background(), attach) + result, resolved, err = router.WithdrawCommitRequest(context.Background(), attach) require.NoError(t, err) return resolved }, 5*time.Second, 50*time.Millisecond, "the worker must answer the withdraw") - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) - result, resolved, err = router.ServiceSaveRequest(context.Background(), attach) + result, resolved, err = router.ServiceCommitRequest(context.Background(), attach) require.NoError(t, err) require.True(t, resolved, "a re-sent attach finds the withdrawn outcome") - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) } -// TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers: a worker is shared by every +// TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers: a worker is shared by every // GitTarget on its branch, so it outlives a deleted one. The withdraw must reach that worker, and // must never report success while the request is still registered there. -func TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) { +func TestWithdrawCommitRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) { scheme := eventRouterScheme(t) provider := &configv1alpha3.GitProvider{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-provider", Namespace: "team-a"}, @@ -570,10 +570,10 @@ func TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) router := NewEventRouter(workerManager, nil, client, logr.Discard()) attach := saveAttach("team-a-config", "team-a") attach.AttachTimeout = time.Hour - _, _, err := router.ServiceSaveRequest(context.Background(), attach) + _, _, err := router.ServiceCommitRequest(context.Background(), attach) require.NoError(t, err) require.Eventually(t, func() bool { - return worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID) == git.PhaseWaitingForWindow + return worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID) == git.PhaseWaitingForWindow }, 5*time.Second, 10*time.Millisecond) require.NoError(t, client.Delete(context.Background(), gitTarget)) @@ -581,15 +581,15 @@ func TestWithdrawSaveRequest_ADeletedGitTargetsWorkerStillAnswers(t *testing.T) var result git.FinalizeResult require.Eventually(t, func() bool { var resolved bool - result, resolved, err = router.WithdrawSaveRequest(context.Background(), attach) + result, resolved, err = router.WithdrawCommitRequest(context.Background(), attach) require.NoError(t, err) if resolved { - require.Empty(t, worker.LookupSaveRequestPhase(attach.Namespace, attach.Name, attach.UID), + require.Empty(t, worker.LookupCommitRequestPhase(attach.Namespace, attach.Name, attach.UID), "the withdraw reported success while the worker still held the registration") } return resolved }, 5*time.Second, 50*time.Millisecond) - require.ErrorIs(t, result.Err, git.ErrSaveRequestWithdrawn) + require.ErrorIs(t, result.Err, git.ErrCommitRequestWithdrawn) assert.Equal(t, "main", result.Branch, "the worker answered, not the missing GitTarget") } diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go index 5f1f75c2..46f7e36a 100644 --- a/internal/watch/remote_save_requests.go +++ b/internal/watch/remote_save_requests.go @@ -30,7 +30,7 @@ import ( // A RemoteSaveRequest is a save made on a source cluster, next to the edits it saves // (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose // spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way -// the SaveRequest controller serves a SaveRequest: it attaches the request to its creator's commit +// the CommitRequest controller serves a CommitRequest: it attaches the request to its creator's commit // window on the GitTarget its spec.gitTargetRef names, polls the branch worker for the outcome, and // writes it back to the request's status on the source cluster. // @@ -94,12 +94,12 @@ const ( saveReasonLost = "Lost" ) -// SaveFinalizer is what serves a save once it is attached: the SaveRequest service of the event +// SaveFinalizer is what serves a save once it is attached: the CommitRequest service of the event // router, which owns the branch workers. type SaveFinalizer interface { - ServiceSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) - WithdrawSaveRequest(ctx context.Context, attach git.AttachSaveRequest) (git.FinalizeResult, bool, error) - LookupSaveRequestApply(ctx context.Context, attach git.AttachSaveRequest) (git.SaveApply, bool, error) + ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) + LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) } // RemoteSaveRequestServer serves the RemoteSaveRequests of every KubeProvider that enables them. It runs on the @@ -495,8 +495,8 @@ func (s *RemoteSaveRequestServer) admit( // remoteSaveRequestAttach is the attach of a request to its GitTarget's window. func remoteSaveRequestAttach( request *sourcev1alpha1.RemoteSaveRequest, author string, attribution git.AttributionOutcome, -) git.AttachSaveRequest { - attach := git.AttachSaveRequest{ +) git.AttachCommitRequest { + attach := git.AttachCommitRequest{ Namespace: request.Namespace, Name: request.Name, UID: string(request.UID), @@ -533,13 +533,13 @@ func remoteSaveRequestAttach( // replacement admitted nothing, and its grants may not admit the request. func (s *RemoteSaveRequestServer) finalize( ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attach git.AttachSaveRequest, readmit func() (string, string, string, error), + attach git.AttachCommitRequest, readmit func() (string, string, string, error), ) bool { bound := time.Now().Add(attach.AttachTimeout + attach.MaxDuration + remoteSaveRequestPushAllowance) for { - ask := s.Finalizer.ServiceSaveRequest + ask := s.Finalizer.ServiceCommitRequest if time.Now().After(bound) { - ask = s.Finalizer.WithdrawSaveRequest + ask = s.Finalizer.WithdrawCommitRequest } result, done, err := ask(ctx, attach) if err == nil && done { @@ -572,7 +572,7 @@ func recordSaveOutcome( status.Branch = result.Branch if result.Err != nil { message := result.Err.Error() - if errors.Is(result.Err, git.ErrSaveRequestWithdrawn) { + if errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { message = "the save was not served in time and was withdrawn" } failSave(request, status, saveReasonFailed, message) @@ -612,7 +612,7 @@ func failSave( // Applied condition; a save its worker knows nothing of for remoteSaveRequestApplyLost is Lost. func (s *RemoteSaveRequestServer) followApply( ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attach git.AttachSaveRequest, write func(), + attach git.AttachCommitRequest, write func(), ) { if apimeta.FindStatusCondition(status.Conditions, "Applied") == nil { setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, @@ -621,7 +621,7 @@ func (s *RemoteSaveRequestServer) followApply( } started := time.Now() for { - apply, known, err := s.Finalizer.LookupSaveRequestApply(ctx, attach) + apply, known, err := s.Finalizer.LookupCommitRequestApply(ctx, attach) switch { case err == nil && known && (apply.Done || apply.Applied): status.AppliedRevision = apply.AppliedRevision diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go index b58075b3..b2b48fd6 100644 --- a/internal/watch/remote_save_requests_test.go +++ b/internal/watch/remote_save_requests_test.go @@ -35,11 +35,11 @@ type saveFinalizer struct { mu sync.Mutex result git.FinalizeResult apply git.SaveApply - attaches []git.AttachSaveRequest + attaches []git.AttachCommitRequest } -func (f *saveFinalizer) ServiceSaveRequest( - _ context.Context, attach git.AttachSaveRequest, +func (f *saveFinalizer) ServiceCommitRequest( + _ context.Context, attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { f.mu.Lock() defer f.mu.Unlock() @@ -47,14 +47,14 @@ func (f *saveFinalizer) ServiceSaveRequest( return f.result, true, nil } -func (f *saveFinalizer) WithdrawSaveRequest( - _ context.Context, _ git.AttachSaveRequest, +func (f *saveFinalizer) WithdrawCommitRequest( + _ context.Context, _ git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { - return git.FinalizeResult{Err: git.ErrSaveRequestWithdrawn}, true, nil + return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil } -func (f *saveFinalizer) LookupSaveRequestApply( - _ context.Context, _ git.AttachSaveRequest, +func (f *saveFinalizer) LookupCommitRequestApply( + _ context.Context, _ git.AttachCommitRequest, ) (git.SaveApply, bool, error) { return f.apply, true, nil } @@ -333,8 +333,8 @@ type replacingFinalizer struct { replacement *configv1alpha3.GitTarget } -func (f *replacingFinalizer) ServiceSaveRequest( - ctx context.Context, attach git.AttachSaveRequest, +func (f *replacingFinalizer) ServiceCommitRequest( + ctx context.Context, attach git.AttachCommitRequest, ) (git.FinalizeResult, bool, error) { var current configv1alpha3.GitTarget if err := f.client.Get(ctx, client.ObjectKey{Namespace: "team-a", Name: "app"}, ¤t); err != nil { @@ -353,7 +353,7 @@ func (f *replacingFinalizer) ServiceSaveRequest( f.replacement = nil return git.FinalizeResult{Phase: git.PhaseWaitingForWorker}, false, nil } - return f.saveFinalizer.ServiceSaveRequest(ctx, attach) + return f.saveFinalizer.ServiceCommitRequest(ctx, attach) } // A GitTarget recreated while a save waits for its worker admitted nothing: the save is attached to it @@ -636,7 +636,7 @@ func TestRemoteSaveRequest_RecordsEachOutcome(t *testing.T) { {Outcome: git.FinalizeSaveInProgress}: {saveReasonSaveInProgress, metav1.ConditionFalse}, {Outcome: "Unknown"}: {saveReasonFailed, metav1.ConditionFalse}, {Err: errors.New("boom")}: {saveReasonFailed, metav1.ConditionFalse}, - {Err: git.ErrSaveRequestWithdrawn}: {saveReasonFailed, metav1.ConditionFalse}, + {Err: git.ErrCommitRequestWithdrawn}: {saveReasonFailed, metav1.ConditionFalse}, } { request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) status := &sourcev1alpha1.RemoteSaveRequestStatus{} diff --git a/internal/watch/resource_policy.go b/internal/watch/resource_policy.go index e7854998..06cc1619 100644 --- a/internal/watch/resource_policy.go +++ b/internal/watch/resource_policy.go @@ -24,7 +24,7 @@ func isDefaultResourceExcluded(group, resource string) bool { groupResourceKey("flowcontrol.apiserver.k8s.io", "prioritylevelconfigurations"), groupResourceKey("batch", "jobs"), groupResourceKey("batch", "cronjobs"), - // A save is a request to GitOps Reverser, never state to mirror (save_requests.go). + // A save is a request to GitOps Reverser, never state to mirror (commit_requests.go). groupResourceKey("source.configbutler.ai", "remotesaverequests"): return true default: diff --git a/internal/watch/target_watch.go b/internal/watch/target_watch.go index e6ff0d9f..f64a74c1 100644 --- a/internal/watch/target_watch.go +++ b/internal/watch/target_watch.go @@ -1320,7 +1320,7 @@ func (m *Manager) attachAuthor( Terminating: u.GetDeletionTimestamp() != nil, }) // Stamp the outcome even when no actor was named: an unresolved attribution is a fact the - // writer, the author_kind metric, and SaveRequest matching all need. Leaving it at the + // writer, the author_kind metric, and CommitRequest matching all need. Leaving it at the // zero value would say "attribution was never attempted", which is exactly the conflation // that made this loss invisible. event.Attribution = outcome diff --git a/internal/webhook/validate_operator_types_handler.go b/internal/webhook/validate_operator_types_handler.go index 3ad1b138..ef09336b 100644 --- a/internal/webhook/validate_operator_types_handler.go +++ b/internal/webhook/validate_operator_types_handler.go @@ -19,7 +19,7 @@ import ( // ValidateOperatorTypesPath is the validating admission endpoint scoped to our own // operator CRDs. Today its one job is command authorship — capturing the submitter of a -// command kind (a SaveRequest) into Redis and always allowing — but it is the intended +// command kind (a CommitRequest) into Redis and always allowing — but it is the intended // home for per-our-type admission generally (e.g. config validation of WatchRule / // GitProvider / GitTarget later), which would be added as additional webhook-config // entries with their own rules and failurePolicy. Distinct from the broad observe-all @@ -41,7 +41,7 @@ const ( // case here plus one rule in the webhook config — the handler body does not change. func isCommandKind(gr metav1.GroupResource) bool { switch gr { - case metav1.GroupResource{Group: "configbutler.ai", Resource: "saverequests"}: + case metav1.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}: return true // future: case metav1.GroupResource{Group: "configbutler.ai", Resource: ""}: default: @@ -59,11 +59,11 @@ type CommandAuthorRecorder interface { // ValidateOperatorTypesHandler is the admission handler for our operator CRDs. It dispatches on the // resource under review and does two unrelated things: // -// - For a COMMAND kind (a SaveRequest) it captures the authenticated submitter into the +// - For a COMMAND kind (a CommitRequest) it captures the authenticated submitter into the // CommandAuthorStore and always allows — pure observation with a single side effect (a Redis // upsert), never a rejection, so a user's command never depends on it succeeding (a missed // capture leaves the request without a claimed actor; see -// docs/spec/saverequest-admission-authorship.md). +// docs/spec/commitrequest-admission-authorship.md). // - For a WatchRule it validates the one cross-object rule admission can usefully give feedback // on: a second source namespace against a GitTarget that declared its folder namespace-free. // That one CAN reject, and it is the only thing here that does. @@ -137,7 +137,7 @@ func (h *ValidateOperatorTypesHandler) Handle(ctx context.Context, req admission } // Info (not V(1)) on purpose: command CREATEs are rare, and this line proves the // webhook was actually invoked and the author captured — the first thing to check - // when a SaveRequest unexpectedly reports AuthorAttributed=False. + // when a CommitRequest unexpectedly reports AuthorAttributed=False. log.Info("recorded command author at admission", "resource", gr.String(), "namespace", req.Namespace, "name", req.Name, "uid", uid, "author", author.Author) diff --git a/internal/webhook/validate_operator_types_handler_test.go b/internal/webhook/validate_operator_types_handler_test.go index bb871f9e..7d184517 100644 --- a/internal/webhook/validate_operator_types_handler_test.go +++ b/internal/webhook/validate_operator_types_handler_test.go @@ -37,8 +37,8 @@ func (f *fakeCommandAuthorRecorder) RecordCommandAuthor( return f.err } -func saveRequestResource() metav1.GroupVersionResource { - return metav1.GroupVersionResource{Group: "configbutler.ai", Version: "v1alpha3", Resource: "saverequests"} +func commitRequestResource() metav1.GroupVersionResource { + return metav1.GroupVersionResource{Group: "configbutler.ai", Version: "v1alpha3", Resource: "commitrequests"} } // commandReview builds an admission.Request for a command CREATE with the given raw @@ -72,7 +72,7 @@ func TestValidateOperatorTypesHandler_RecordsCommandAuthor(t *testing.T) { }, } resp := h.Handle(context.Background(), - commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, user, nil)) + commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, user, nil)) assert.True(t, resp.Allowed) require.Len(t, rec.calls, 1) @@ -90,7 +90,7 @@ func TestValidateOperatorTypesHandler_NilStoreNoOp(t *testing.T) { h := &ValidateOperatorTypesHandler{} resp := h.Handle(context.Background(), - commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, + commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, nil)) assert.True(t, resp.Allowed, "a missing author store never blocks a command") @@ -102,7 +102,7 @@ func TestValidateOperatorTypesHandler_ServiceAccountSubmitter(t *testing.T) { user := authnv1.UserInfo{Username: "system:serviceaccount:team-a:deployer"} resp := h.Handle(context.Background(), - commandReview(saveRequestResource(), `{"metadata":{"uid":"sa-uid"}}`, user, nil)) + commandReview(commitRequestResource(), `{"metadata":{"uid":"sa-uid"}}`, user, nil)) assert.True(t, resp.Allowed) require.Len(t, rec.calls, 1) @@ -129,7 +129,7 @@ func TestValidateOperatorTypesHandler_DryRunIsNotRecorded(t *testing.T) { resp := h.Handle( context.Background(), commandReview( - saveRequestResource(), + commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, &dryRun, @@ -145,7 +145,7 @@ func TestValidateOperatorTypesHandler_MissingUIDIsNotRecorded(t *testing.T) { h := &ValidateOperatorTypesHandler{Store: rec} resp := h.Handle(context.Background(), - commandReview(saveRequestResource(), `{"metadata":{}}`, authnv1.UserInfo{Username: "alice"}, nil)) + commandReview(commitRequestResource(), `{"metadata":{}}`, authnv1.UserInfo{Username: "alice"}, nil)) assert.True(t, resp.Allowed) assert.Empty(t, rec.calls, "without a uid the record cannot be keyed") @@ -156,7 +156,7 @@ func TestValidateOperatorTypesHandler_NoUserIsNotRecorded(t *testing.T) { h := &ValidateOperatorTypesHandler{Store: rec} resp := h.Handle(context.Background(), - commandReview(saveRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{}, nil)) + commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{}, nil)) assert.True(t, resp.Allowed) assert.Empty(t, rec.calls, "an unauthenticated request names no author") @@ -171,7 +171,7 @@ func TestValidateOperatorTypesHandler_RecordErrorStillAllows(t *testing.T) { resp := h.Handle( context.Background(), commandReview( - saveRequestResource(), + commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{Username: "alice"}, nil, diff --git a/test/e2e/Taskfile-e2e.yml b/test/e2e/Taskfile-e2e.yml index 42c6d658..f30e5a50 100644 --- a/test/e2e/Taskfile-e2e.yml +++ b/test/e2e/Taskfile-e2e.yml @@ -1328,7 +1328,7 @@ tasks: - | CTX="{{.CTX}}" KUBECTL="{{.KUBECTL}}" bash hack/e2e/wait-crds-established.sh 60 \ crd/kubeproviders.configbutler.ai crd/clusterwatchrules.configbutler.ai \ - crd/saverequests.configbutler.ai crd/gitproviders.configbutler.ai \ + crd/commitrequests.configbutler.ai crd/gitproviders.configbutler.ai \ crd/gitbranches.configbutler.ai crd/gitreceivers.configbutler.ai \ crd/gittargets.configbutler.ai crd/watchrules.configbutler.ai \ crd/fluxclaims.configbutler.ai crd/argocdclaims.configbutler.ai diff --git a/test/e2e/bi_directional_echoes_e2e_test.go b/test/e2e/bi_directional_echoes_e2e_test.go index de4b5259..3e1b6e4a 100644 --- a/test/e2e/bi_directional_echoes_e2e_test.go +++ b/test/e2e/bi_directional_echoes_e2e_test.go @@ -395,15 +395,15 @@ func echoScenarios(run func() *echoRun) { r.pushFromOutside("saved", "s0") r.eventuallyEverywhere("saved", "s0", echoApplyBudget) - By("saving an API edit with a SaveRequest while a Git-side push changes the same object") - name := applySaveRequestWithGenerateName(r.testNs, "echo-save-", r.targetName, "e2e: save the edit") + By("saving an API edit with a CommitRequest while a Git-side push changes the same object") + name := applyCommitRequestWithGenerateName(r.testNs, "echo-save-", r.targetName, "e2e: save the edit") r.editThroughAPI("saved", "s-save") r.pushFromOutside("saved", "s-git") Eventually(func(g Gomega) { - g.Expect(saveRequestCondition(g, r.testNs, name, "Ready")).To(Equal("True")) - g.Expect(saveRequestField(g, r.testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(BeElementOf(saveRequestReasonCommitted, "AlreadyPresent")) + g.Expect(commitRequestCondition(g, r.testNs, name, "Ready")).To(Equal("True")) + g.Expect(commitRequestField(g, r.testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(BeElementOf(commitRequestReasonCommitted, "AlreadyPresent")) }, echoApplyBudget, 2*time.Second).Should(Succeed()) r.eventuallyEverywhere("saved", "s-save", echoApplyBudget) r.consistentlyEverywhere("saved", "s-save") diff --git a/test/e2e/cluster/audit/policy.yaml b/test/e2e/cluster/audit/policy.yaml index 8ac915be..bfc033c8 100644 --- a/test/e2e/cluster/audit/policy.yaml +++ b/test/e2e/cluster/audit/policy.yaml @@ -52,8 +52,8 @@ rules: - delete - deletecollection # No resources listed = Matches everything else (including CRDs). - # This also captures `saverequests.configbutler.ai` create events, which + # This also captures `commitrequests.configbutler.ai` create events, which # drive the "finalize the open commit window now" path. If this catch-all # is ever narrowed, keep at least Metadata-level capture for - # `saverequests` create events — the consumer reads the object body from + # `commitrequests` create events — the consumer reads the object body from # the apiserver, so Metadata is sufficient. diff --git a/test/e2e/save_request_e2e_test.go b/test/e2e/commit_request_e2e_test.go similarity index 73% rename from test/e2e/save_request_e2e_test.go rename to test/e2e/commit_request_e2e_test.go index 2800bb31..595dd969 100644 --- a/test/e2e/save_request_e2e_test.go +++ b/test/e2e/commit_request_e2e_test.go @@ -13,12 +13,12 @@ import ( . "github.com/onsi/gomega" ) -// The SaveRequest suite exercises the "save" signal: a SaveRequest +// The CommitRequest suite exercises the "save" signal: a CommitRequest // object finalizes the open commit window for a GitTarget immediately, instead // of waiting for the rolling silence timer. The GitProvider is configured with -// a deliberately long commitWindow so that, without the SaveRequest, the +// a deliberately long commitWindow so that, without the CommitRequest, the // edit would not be committed for minutes — observing the commit promptly is -// what proves the save-request path works. +// what proves the commit-request path works. // // Not Serial: this spec owns a dedicated Gitea repo (its own GitProvider → // GitTarget → namespace-scoped WatchRule), so the only writer to its main @@ -26,7 +26,7 @@ import ( // namespace. The HEAD/SHA assertions below therefore read back only this spec's // own commit; concurrent audit traffic for other GitTargets lands in other // repos and cannot move this repo's HEAD. See docs/spec/e2e-serial-registry.md. -var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordered, func() { +var _ = Describe("Save Request", Label("commit-request", "audit-consumer"), Ordered, func() { var ( testNs string repo *RepoArtifacts @@ -40,7 +40,7 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere const commitWindow = "300s" BeforeAll(func() { - p := setupWindowedDeploymentPipeline("save-request", "e2e/save-request-test", commitWindow) + p := setupWindowedDeploymentPipeline("commit-request", "e2e/commit-request-test", commitWindow) testNs, repo, gitProvName, gitTargetName, watchRuleName = p.ns, p.repo, p.gitProvider, p.gitTarget, p.watchRule }) @@ -50,10 +50,10 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere }) It("finalizes the open commit window on demand and reports the resulting SHA", func() { - basePath := "e2e/save-request-test" + basePath := "e2e/commit-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("save-request-deploy-%d", seed) - saveRequestName := fmt.Sprintf("save-request-save-%d", seed) + deployName := fmt.Sprintf("commit-request-deploy-%d", seed) + commitRequestName := fmt.Sprintf("commit-request-save-%d", seed) message := fmt.Sprintf("save: save request from e2e seed %d", seed) By("creating a Deployment to open a commit window") @@ -69,16 +69,16 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere "the open commit window must hold the edit; main must not exist until the window is finalized") }, 10*time.Second, 2*time.Second).Should(Succeed()) - By("creating a SaveRequest to finalize the open window now") - applySaveRequest(testNs, saveRequestName, gitTargetName, message) + By("creating a CommitRequest to finalize the open window now") + applyCommitRequest(testNs, commitRequestName, gitTargetName, message) - By("waiting for the SaveRequest to commit and push") + By("waiting for the CommitRequest to commit and push") var reportedSHA string Eventually(func(g Gomega) { - reportedSHA = expectSaveRequestCommitted(g, testNs, saveRequestName, + reportedSHA = expectCommitRequestCommitted(g, testNs, commitRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) - branch := saveRequestField(g, testNs, saveRequestName, "{.status.branch}") + branch := commitRequestField(g, testNs, commitRequestName, "{.status.branch}") g.Expect(branch).To(Equal("main")) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -116,11 +116,11 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere // is: the commit above is verified in Git, so a missing series is a broken exporter rather // than a spec that did not run. Both queries are scoped to this suite's own GitTarget, so // they need no run bookkeeping to isolate from the other parallel processes. - By("verifying this save reached the SaveRequest outcome counter") + By("verifying this save reached the CommitRequest outcome counter") ensurePrometheusClient() waitForMetric( fmt.Sprintf( - `sum(gitopsreverser_save_requests_total{outcome="committed",`+ + `sum(gitopsreverser_commit_requests_total{outcome="committed",`+ `gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, testNs, gitTargetName), func(v float64) bool { return v > 0 }, @@ -133,7 +133,7 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere waitForMetric( fmt.Sprintf( `sum(gitopsreverser_git_commit_windows_total{close_reason="max_duration",`+ - `timer_source="save_request",gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, + `timer_source="commit_request",gittarget_namespace=%q,gittarget_name=%q}) or vector(0)`, testNs, gitTargetName), func(v float64) bool { return v > 0 }, "the save's window is counted with the timer that closed it", @@ -159,20 +159,20 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere "the join series maps this GitTarget to the GitBranch its commits are labelled by", ) - By("cleaning up the test Deployment and SaveRequest") + By("cleaning up the test Deployment and CommitRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") }) // The companion path: once a branch HAS been established (the previous spec's finalize created // main), a fresh edit must STILL be held in the open window — an existing branch must not - // advance until a SaveRequest finalizes it. Together with the spec above this covers both + // advance until a CommitRequest finalizes it. Together with the spec above this covers both // cases of "defer committing as long as possible": an absent branch and an established one. It("holds a new edit in the open window without advancing an already-established branch", func() { - basePath := "e2e/save-request-test" + basePath := "e2e/commit-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("save-request-hold-%d", seed) - saveRequestName := fmt.Sprintf("save-request-hold-save-%d", seed) + deployName := fmt.Sprintf("commit-request-hold-%d", seed) + commitRequestName := fmt.Sprintf("commit-request-hold-save-%d", seed) message := fmt.Sprintf("save: held edit from e2e seed %d", seed) By("capturing the branch HEAD the previous spec established") @@ -192,10 +192,10 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere "the open commit window must hold the new edit; main must not advance until finalized") }, 10*time.Second, 2*time.Second).Should(Succeed()) - By("creating a SaveRequest and confirming the branch then advances with the held edit") - applySaveRequest(testNs, saveRequestName, gitTargetName, message) + By("creating a CommitRequest and confirming the branch then advances with the held edit") + applyCommitRequest(testNs, commitRequestName, gitTargetName, message) Eventually(func(g Gomega) { - expectSaveRequestCommitted(g, testNs, saveRequestName, + expectCommitRequestCommitted(g, testNs, commitRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) g.Expect(remoteBranchHead(g, repo.CheckoutDir)).NotTo(Equal(baseSHA), "finalizing must advance main past the previously-established HEAD") @@ -211,9 +211,9 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere fmt.Sprintf("the held Deployment should exist after finalize at %s", expected)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the held Deployment and SaveRequest") + By("cleaning up the held Deployment and CommitRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") }) // generateName authorship is now captured at admission, not from the audit @@ -221,26 +221,26 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere // validating admission runs, so the validate-operator-types webhook records the // submitter keyed by uid with no response-body name recovery (the old audit // generateName headache is gone, see - // docs/spec/saverequest-admission-authorship.md). This spec proves a - // generateName SaveRequest still finalizes and becomes Ready. It is skipped in + // docs/spec/commitrequest-admission-authorship.md). This spec proves a + // generateName CommitRequest still finalizes and becomes Ready. It is skipped in // configured-author mode, where the edit's window is committer-authored and the // named admission author would not match it end to end. - It("finalizes a SaveRequest created with metadata.generateName", func() { - basePath := "e2e/save-request-test" + It("finalizes a CommitRequest created with metadata.generateName", func() { + basePath := "e2e/commit-request-test" seed := GinkgoRandomSeed() - deployName := fmt.Sprintf("save-request-gen-deploy-%d", seed) - saveRequestPrefix := fmt.Sprintf("save-request-gen-%d-", seed) + deployName := fmt.Sprintf("commit-request-gen-deploy-%d", seed) + commitRequestPrefix := fmt.Sprintf("commit-request-gen-%d-", seed) message := fmt.Sprintf("save: generateName save request from e2e seed %d", seed) By("creating a Deployment to open a commit window") applyScaleTestDeployment(testNs, deployName, 0) - By("creating a SaveRequest with metadata.generateName") - generatedName := applySaveRequestWithGenerateName(testNs, saveRequestPrefix, gitTargetName, message) + By("creating a CommitRequest with metadata.generateName") + generatedName := applyCommitRequestWithGenerateName(testNs, commitRequestPrefix, gitTargetName, message) - By("waiting for the generated-name SaveRequest to commit") + By("waiting for the generated-name CommitRequest to commit") Eventually(func(g Gomega) { - expectSaveRequestCommitted(g, testNs, generatedName, + expectCommitRequestCommitted(g, testNs, generatedName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -261,24 +261,24 @@ var _ = Describe("Save Request", Label("save-request", "audit-consumer"), Ordere recentCommitDiagnostics(repo.CheckoutDir, basePath)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the generateName Deployment and SaveRequest") + By("cleaning up the generateName Deployment and CommitRequest") _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", deployName, "--ignore-not-found=true") - _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", generatedName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", generatedName, "--ignore-not-found=true") }) }) -// The UC2 suite exercises a `kubectl apply` bundle that includes a SaveRequest +// The UC2 suite exercises a `kubectl apply` bundle that includes a CommitRequest // as its FIRST document — the deliberately-hard ordering where the save intent -// arrives before the work it is meant to save (docs/spec/saverequest-design.md). The request +// arrives before the work it is meant to save (docs/spec/commitrequest-design.md). The request // waits (spec.window.attachTimeout) for the first Deployment to open a window, attaches, and // collects (spec.window.maxDuration from the attach) while the rest of the bundle arrives, so the -// whole bundle lands in ONE commit carrying the SaveRequest's message. +// whole bundle lands in ONE commit carrying the CommitRequest's message. // // Its own dedicated Gitea repo (own GitProvider → GitTarget → namespace-scoped // Deployment WatchRule) makes the one-commit assertion unambiguous: main does not // exist until the bundle is finalized, so the bundle commit is the only commit on // main. -var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consumer"), Ordered, func() { +var _ = Describe("Save Request Bundle (UC2)", Label("commit-request", "audit-consumer"), Ordered, func() { var ( testNs string repo *RepoArtifacts @@ -288,11 +288,11 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu ) // A long spec.commit.window so the silence timer can never be what produces the - // commit — only the SaveRequest finalize may (A1). + // commit — only the CommitRequest finalize may (A1). const commitWindow = "300s" BeforeAll(func() { - p := setupWindowedDeploymentPipeline("save-request-bundle", "e2e/save-request-bundle", commitWindow) + p := setupWindowedDeploymentPipeline("commit-request-bundle", "e2e/commit-request-bundle", commitWindow) testNs, repo, gitProvName, gitTargetName, watchRuleName = p.ns, p.repo, p.gitProvider, p.gitTarget, p.watchRule }) @@ -301,11 +301,11 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu cleanupNamespace(testNs) }) - It("lands a kubectl-apply bundle (SaveRequest first) in one commit with the request's message", func() { - basePath := "e2e/save-request-bundle" + It("lands a kubectl-apply bundle (CommitRequest first) in one commit with the request's message", func() { + basePath := "e2e/commit-request-bundle" seed := GinkgoRandomSeed() message := fmt.Sprintf("bundle save: apply from e2e seed %d", seed) - saveRequestName := fmt.Sprintf("save-request-bundle-save-%d", seed) + commitRequestName := fmt.Sprintf("commit-request-bundle-save-%d", seed) deployNames := []string{ fmt.Sprintf("bundle-deploy-a-%d", seed), fmt.Sprintf("bundle-deploy-b-%d", seed), @@ -318,25 +318,25 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu "main must not exist before the bundle is finalized") }, 5*time.Second, 1*time.Second).Should(Succeed()) - By("applying a bundle whose FIRST document is a SaveRequest, then three Deployments") + By("applying a bundle whose FIRST document is a CommitRequest, then three Deployments") // The window is sized to comfortably exceed the bundle's per-type ingestion spread, so the // collection after the attach is deterministic. var bundle strings.Builder - bundle.WriteString(saveRequestManifest(testNs, saveRequestName, gitTargetName, message, "8s")) + bundle.WriteString(commitRequestManifest(testNs, commitRequestName, gitTargetName, message, "8s")) for _, name := range deployNames { bundle.WriteString("---\n") bundle.WriteString(deploymentManifest(testNs, name)) } _, err := kubectlRunWithStdin(testNs, bundle.String(), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to apply the SaveRequest+Deployments bundle") + Expect(err).NotTo(HaveOccurred(), "failed to apply the CommitRequest+Deployments bundle") - By("waiting for the bundle's SaveRequest to commit the collected window") + By("waiting for the bundle's CommitRequest to commit the collected window") var reportedSHA string Eventually(func(g Gomega) { - reportedSHA = expectSaveRequestCommitted(g, testNs, saveRequestName, + reportedSHA = expectCommitRequestCommitted(g, testNs, commitRequestName, recentCommitDiagnostics(repo.CheckoutDir, basePath)) - branch := saveRequestField(g, testNs, saveRequestName, "{.status.branch}") + branch := commitRequestField(g, testNs, commitRequestName, "{.status.branch}") g.Expect(branch).To(Equal("main")) }, 2*time.Minute, 3*time.Second).Should(Succeed()) @@ -345,7 +345,7 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu pullLatestRepoState(g, repo.CheckoutDir) // Exactly one commit on the fresh repo's main: the entire bundle — applied - // across the SaveRequest's attribution and the per-type Deployment stream — + // across the CommitRequest's attribution and the per-type Deployment stream — // collapsed into a single commit. g.Expect(mustCommitCount(repo.CheckoutDir)).To(Equal(1), "the whole bundle must land in exactly one commit\n%s", @@ -354,7 +354,7 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu subject, logErr := gitRun(repo.CheckoutDir, "log", "-1", "--pretty=%B") g.Expect(logErr).NotTo(HaveOccurred()) g.Expect(strings.TrimSpace(subject)).To(Equal(message), - "the single commit must carry the SaveRequest's message verbatim\n%s", + "the single commit must carry the CommitRequest's message verbatim\n%s", recentCommitDiagnostics(repo.CheckoutDir, basePath)) headSHA, shaErr := gitRun(repo.CheckoutDir, "rev-parse", "HEAD") @@ -372,20 +372,20 @@ var _ = Describe("Save Request Bundle (UC2)", Label("save-request", "audit-consu } }, 2*time.Minute, 3*time.Second).Should(Succeed()) - By("cleaning up the bundle Deployments and SaveRequest") + By("cleaning up the bundle Deployments and CommitRequest") for _, name := range deployNames { _, _ = kubectlRunInNamespace(testNs, "delete", "deployment", name, "--ignore-not-found=true") } - _, _ = kubectlRunInNamespace(testNs, "delete", "saverequest", saveRequestName, "--ignore-not-found=true") + _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") }) }) -// saveRequestManifest renders a single SaveRequest document with an explicit message and a +// commitRequestManifest renders a single CommitRequest document with an explicit message and a // window that waits and then collects for the given Go duration each. It is used to build -// multi-document `kubectl apply` bundles where the SaveRequest is the first document. -func saveRequestManifest(namespace, name, gitTargetName, message, window string) string { +// multi-document `kubectl apply` bundles where the CommitRequest is the first document. +func commitRequestManifest(namespace, name, gitTargetName, message, window string) string { return fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: %s namespace: %s @@ -424,11 +424,11 @@ spec: `, name, namespace, name, name) } -// applySaveRequestWithGenerateName creates a SaveRequest using +// applyCommitRequestWithGenerateName creates a CommitRequest using // metadata.generateName and returns the server-allocated name. The spec creates its work and this -// SaveRequest back-to-back: authorship is settled synchronously at admission (no controller-side +// CommitRequest back-to-back: authorship is settled synchronously at admission (no controller-side // wait), so a zero wait would race the work's watch event and could resolve NoWindow before the -// window opens. Waiting to attach is the documented mechanism for a SaveRequest issued +// window opens. Waiting to attach is the documented mechanism for a CommitRequest issued // concurrently with its work (UC2). // // The wait is deliberately tight: it covers audit delivery (--audit-webhook-batch-max-wait=1s) and @@ -437,10 +437,10 @@ spec: // attributes its events in order, so a work event held behind another event that waits out the whole // grace fails the spec with NoWindow instead of passing slowly. Do not widen it to make such a // failure go away: that wait is the bug. -func applySaveRequestWithGenerateName(namespace, prefix, gitTargetName, message string) string { +func applyCommitRequestWithGenerateName(namespace, prefix, gitTargetName, message string) string { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: generateName: %s namespace: %s @@ -455,19 +455,19 @@ spec: out, err := kubectlRunWithStdin(namespace, manifest, "create", "-f", "-", "-o", "jsonpath={.metadata.name}") Expect(err).NotTo(HaveOccurred(), - fmt.Sprintf("failed to create SaveRequest with generateName=%s", prefix)) + fmt.Sprintf("failed to create CommitRequest with generateName=%s", prefix)) name := strings.TrimSpace(out) Expect(name).NotTo(BeEmpty(), "kubectl create must return the server-allocated name") Expect(name).To(HavePrefix(prefix), "the allocated name must start with the requested prefix") return name } -// applySaveRequest creates a SaveRequest object that targets the given +// applyCommitRequest creates a CommitRequest object that targets the given // GitTarget with an optional commit message. -func applySaveRequest(namespace, name, gitTargetName, message string) { +func applyCommitRequest(namespace, name, gitTargetName, message string) { GinkgoHelper() manifest := fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: %s namespace: %s @@ -477,37 +477,37 @@ spec: message: %q `, name, namespace, gitTargetName, message) _, err := kubectlRunWithStdin(namespace, manifest, "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to apply SaveRequest %s/%s", namespace, name)) + Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to apply CommitRequest %s/%s", namespace, name)) } -// saveRequestField reads a jsonpath field off a SaveRequest object. -func saveRequestField(g Gomega, namespace, name, jsonPath string) string { - out, err := kubectlRunInNamespace(namespace, "get", "saverequest", name, +// commitRequestField reads a jsonpath field off a CommitRequest object. +func commitRequestField(g Gomega, namespace, name, jsonPath string) string { + out, err := kubectlRunInNamespace(namespace, "get", "commitrequest", name, "-o", "jsonpath="+jsonPath) - g.Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to read %s of SaveRequest %s", jsonPath, name)) + g.Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to read %s of CommitRequest %s", jsonPath, name)) return strings.TrimSpace(out) } -// saveRequestCondition reads the status of one condition off a SaveRequest. -func saveRequestCondition(g Gomega, namespace, name, conditionType string) string { - return saveRequestField(g, namespace, name, +// commitRequestCondition reads the status of one condition off a CommitRequest. +func commitRequestCondition(g Gomega, namespace, name, conditionType string) string { + return commitRequestField(g, namespace, name, fmt.Sprintf(`{.status.conditions[?(@.type=="%s")].status}`, conditionType)) } -// saveRequestReasonCommitted is the one Ready reason that means a commit actually happened. -// It mirrors crReasonCommitted in internal/controller/saverequest_finalize.go. -const saveRequestReasonCommitted = "Committed" +// commitRequestReasonCommitted is the one Ready reason that means a commit actually happened. +// It mirrors crReasonCommitted in internal/controller/commitrequest_finalize.go. +const commitRequestReasonCommitted = "Committed" -// saveRequestOutcome is the terminal shape of a SaveRequest: not just whether it finished, +// commitRequestOutcome is the terminal shape of a CommitRequest: not just whether it finished, // but WHICH ending it reached. // // Ready=True alone cannot answer that, and reading it as "it committed" is how this suite went // blind. A benign rejection — no window collected in the grace, a foreign author's window, a // change already present on the remote — is deliberately Ready=True with Pushed=False and an -// empty status.commit, so that kstatus reads Current rather than Failed (see rejectSaveRequest). +// empty status.commit, so that kstatus reads Current rather than Failed (see rejectCommitRequest). // A spec asserting Ready=True therefore PASSES on a request that committed nothing, then fails // seconds later on the empty sha with ": not to be empty" and no reason attached. -type saveRequestOutcome struct { +type commitRequestOutcome struct { Ready string Reason string Message string @@ -518,59 +518,59 @@ type saveRequestOutcome struct { } // String renders the outcome as the diagnostic the bare-sha assertion never produced. -func (o saveRequestOutcome) String() string { +func (o commitRequestOutcome) String() string { return fmt.Sprintf( - "SaveRequest status: Ready=%s (reason=%q) Stalled=%s Pushed=%s commit=%q branch=%q\n message: %s", + "CommitRequest status: Ready=%s (reason=%q) Stalled=%s Pushed=%s commit=%q branch=%q\n message: %s", o.Ready, o.Reason, o.Stalled, o.Pushed, o.Commit, o.Branch, o.Message) } // isTerminal reports whether the controller has stopped working on this request: Ready=True -// (committed or benignly rejected) or Stalled=True (failed). It mirrors saveRequestIsTerminal +// (committed or benignly rejected) or Stalled=True (failed). It mirrors commitRequestIsTerminal // in the controller, and it is what makes an early give-up safe. -func (o saveRequestOutcome) isTerminal() bool { +func (o commitRequestOutcome) isTerminal() bool { return o.Ready == "True" || o.Stalled == "True" } -// readSaveRequestOutcome reads the whole terminal picture in one API call, so every field in a +// readCommitRequestOutcome reads the whole terminal picture in one API call, so every field in a // failure report is from the same observation rather than five reads a poll apart. -func readSaveRequestOutcome(g Gomega, namespace, name string) saveRequestOutcome { +func readCommitRequestOutcome(g Gomega, namespace, name string) commitRequestOutcome { const readyPath = `{.status.conditions[?(@.type=="Ready")]` - return saveRequestOutcome{ - Ready: saveRequestField(g, namespace, name, readyPath+`.status}`), - Reason: saveRequestField(g, namespace, name, readyPath+`.reason}`), - Message: saveRequestField(g, namespace, name, readyPath+`.message}`), - Stalled: saveRequestCondition(g, namespace, name, "Stalled"), - Pushed: saveRequestCondition(g, namespace, name, "Pushed"), - Commit: saveRequestField(g, namespace, name, "{.status.commit}"), - Branch: saveRequestField(g, namespace, name, "{.status.branch}"), + return commitRequestOutcome{ + Ready: commitRequestField(g, namespace, name, readyPath+`.status}`), + Reason: commitRequestField(g, namespace, name, readyPath+`.reason}`), + Message: commitRequestField(g, namespace, name, readyPath+`.message}`), + Stalled: commitRequestCondition(g, namespace, name, "Stalled"), + Pushed: commitRequestCondition(g, namespace, name, "Pushed"), + Commit: commitRequestField(g, namespace, name, "{.status.commit}"), + Branch: commitRequestField(g, namespace, name, "{.status.branch}"), } } -// expectSaveRequestCommitted asserts the request reached the COMMITTED ending, and returns the +// expectCommitRequestCommitted asserts the request reached the COMMITTED ending, and returns the // commit it reported. // // It gives up early on any other terminal ending rather than polling to the timeout. A terminal // outcome is final — the controller will not revisit it — so continuing to re-read it cannot // change the result; it only delays the report and buries the reason. Failing at the moment the // ending is known means the message names it: "NoWindow" instead of an empty string. -func expectSaveRequestCommitted(g Gomega, namespace, name, diagnostics string) string { - outcome := readSaveRequestOutcome(g, namespace, name) - if outcome.isTerminal() && outcome.Reason != saveRequestReasonCommitted { +func expectCommitRequestCommitted(g Gomega, namespace, name, diagnostics string) string { + outcome := readCommitRequestOutcome(g, namespace, name) + if outcome.isTerminal() && outcome.Reason != commitRequestReasonCommitted { StopTrying(fmt.Sprintf( - "the SaveRequest reached a terminal outcome that is not a commit, so waiting longer "+ + "the CommitRequest reached a terminal outcome that is not a commit, so waiting longer "+ "cannot help.\n%s\n%s", outcome, diagnostics)).Now() } - g.Expect(outcome.Ready).To(Equal("True"), "SaveRequest is not Ready yet\n%s\n%s", outcome, diagnostics) - g.Expect(outcome.Reason).To(Equal(saveRequestReasonCommitted), + g.Expect(outcome.Ready).To(Equal("True"), "CommitRequest is not Ready yet\n%s\n%s", outcome, diagnostics) + g.Expect(outcome.Reason).To(Equal(commitRequestReasonCommitted), "Ready=True must mean a commit, not a benign rejection\n%s\n%s", outcome, diagnostics) g.Expect(outcome.Pushed).To(Equal("True"), - "a committed SaveRequest must report Pushed=True\n%s", outcome) - g.Expect(outcome.Commit).NotTo(BeEmpty(), "a committed SaveRequest must report status.commit\n%s", outcome) + "a committed CommitRequest must report Pushed=True\n%s", outcome) + g.Expect(outcome.Commit).NotTo(BeEmpty(), "a committed CommitRequest must report status.commit\n%s", outcome) return outcome.Commit } -// windowedDeploymentPipeline is the pipeline a SaveRequest suite saves through: a namespace and +// windowedDeploymentPipeline is the pipeline a CommitRequest suite saves through: a namespace and // Gitea repo of its own, a GitProvider, a GitTarget with a long spec.commit.window, and a WatchRule // on Deployments. type windowedDeploymentPipeline struct { diff --git a/test/e2e/e2e_suite_test.go b/test/e2e/e2e_suite_test.go index c69cd47e..21a81de4 100644 --- a/test/e2e/e2e_suite_test.go +++ b/test/e2e/e2e_suite_test.go @@ -128,17 +128,17 @@ var _ = SynchronizedAfterSuite(func() {}, func() { // Diagnostic first, so its numbers reach the artifacts even when the gating // audit invariant below fails the suite. reportAttributionStats() - reportSaveRequestOutcomes() + reportCommitRequestOutcomes() assertNoAnomalousAuditOutcomes() }) -// saveRequestOutcomes is the bounded outcome set of gitopsreverser_save_requests_total, in the +// commitRequestOutcomes is the bounded outcome set of gitopsreverser_commit_requests_total, in the // order a reader wants them: the success, the two refusals, the no-op, then the fault. -var saveRequestOutcomes = []string{ +var commitRequestOutcomes = []string{ "committed", "window_mismatch", "no_window", "already_present", "failed", } -// reportSaveRequestOutcomes prints the run's SaveRequest outcome distribution. +// reportCommitRequestOutcomes prints the run's CommitRequest outcome distribution. // // DIAGNOSTIC, not a gate. A suite-wide "none failed" assertion would first have to establish that // the last save had been scraped, and until that is handled a green result would mean less than it @@ -148,14 +148,14 @@ var saveRequestOutcomes = []string{ // The breakdown is the point. `committed` is what the happy path looks like and the only value // these specs produce: `window_mismatch` needs two authors contending for one branch's window, // which nothing here arranges. The per-run reset means these numbers are this run's. -func reportSaveRequestOutcomes() { - By("reporting the run's SaveRequest terminal outcomes") +func reportCommitRequestOutcomes() { + By("reporting the run's CommitRequest terminal outcomes") ensurePrometheusClient() verifyPrometheusAvailable() - for _, oc := range saveRequestOutcomes { + for _, oc := range commitRequestOutcomes { n, err := queryPrometheus(fmt.Sprintf( - `sum(max_over_time(gitopsreverser_save_requests_total{outcome=%q}[2h])) or vector(0)`, oc)) + `sum(max_over_time(gitopsreverser_commit_requests_total{outcome=%q}[2h])) or vector(0)`, oc)) if err == nil { _, _ = fmt.Fprintf(GinkgoWriter, " save request outcome %-16s = %.0f\n", oc, n) } diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index f0d97649..8a85d720 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -359,14 +359,14 @@ data: }) // A save that asks to catch up is applied as soon as it is pushed, with catchUpAfter off, and its - // SaveRequest reports the apply: Applied=True with the revision Flux applied. - It("applies a save that asks to catch up, and reports it on the SaveRequest", func() { + // CommitRequest reports the apply: Applied=True with the revision Flux applied. + It("applies a save that asks to catch up, and reports it on the CommitRequest", func() { const configMap, request = "applier-converge-edit", "applier-save-catch-up" setFluxClaimCatchUpAfter(testNs, fluxName, "0s") By("saving an API edit with catchUp: AfterPush") _, err := kubectlRunWithStdin(testNs, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: %s namespace: %s @@ -387,17 +387,17 @@ spec: By("the save is pushed, and Ready says only that") var commit string Eventually(func(g Gomega) { - g.Expect(saveRequestCondition(g, testNs, request, "Ready")).To(Equal("True")) - g.Expect(saveRequestField(g, testNs, request, "{.status.conditions[?(@.type==\"Ready\")].reason}")). - To(Equal(saveRequestReasonCommitted)) - commit = saveRequestField(g, testNs, request, "{.status.commit}") + g.Expect(commitRequestCondition(g, testNs, request, "Ready")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, request, "{.status.conditions[?(@.type==\"Ready\")].reason}")). + To(Equal(commitRequestReasonCommitted)) + commit = commitRequestField(g, testNs, request, "{.status.commit}") g.Expect(commit).NotTo(BeEmpty()) }, 2*time.Minute, 2*time.Second).Should(Succeed()) By("the applier applies it at once, and Applied reports the revision") Eventually(func(g Gomega) { - g.Expect(saveRequestCondition(g, testNs, request, "Applied")).To(Equal("True")) - g.Expect(saveRequestField(g, testNs, request, "{.status.appliedRevision}")).To(ContainSubstring(commit)) + g.Expect(commitRequestCondition(g, testNs, request, "Applied")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, request, "{.status.appliedRevision}")).To(ContainSubstring(commit)) g.Expect(fluxLastAppliedRevision(g, fluxName)).To(ContainSubstring(commit)) g.Expect(readClaimOperation(g, testNs, fluxClaimResource, fluxName).Reason).To(Equal("CatchUp")) }, 2*time.Minute, 2*time.Second).Should(Succeed()) diff --git a/test/e2e/helpers.go b/test/e2e/helpers.go index 5606b495..064617f1 100644 --- a/test/e2e/helpers.go +++ b/test/e2e/helpers.go @@ -543,18 +543,18 @@ func dumpFailureDiagnostics() { _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get Kubernetes events: %s\n", err) } - By("Fetching SaveRequest diagnostics") - saveRequestsOutput, err := kubectlRun( + By("Fetching CommitRequest diagnostics") + commitRequestsOutput, err := kubectlRun( "get", - "saverequests.configbutler.ai", + "commitrequests.configbutler.ai", "--all-namespaces", "-o", "yaml", ) if err == nil { - _, _ = fmt.Fprintf(GinkgoWriter, "SaveRequests:\n%s", saveRequestsOutput) + _, _ = fmt.Fprintf(GinkgoWriter, "CommitRequests:\n%s", commitRequestsOutput) } else { - _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get SaveRequests: %s\n", err) + _, _ = fmt.Fprintf(GinkgoWriter, "Failed to get CommitRequests: %s\n", err) } podNames, err := controllerPodNames() diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 930481f1..f124e212 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -36,7 +36,7 @@ import ( // the GitTarget mirrors that workspace, and its FluxClaim holds the Kustomization through the // install's own provider. It shows the startup converge landing in the workspace, an edit made // there withheld from the apply's revert by the REMOTE object's managedFields, two -// SaveRequests that take over their edits' windows (the workspace sends no audit facts) and +// CommitRequests that take over their edits' windows (the workspace sends no audit facts) and // report the apply, and a RemoteSaveRequest created in the workspace itself. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), @@ -822,7 +822,7 @@ spec: p.targetName, "--type=merge", "-p", - `{"spec":{"allowedSaveRequests":[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"TakeOver"}]}}`, + `{"spec":{"allowedCommitRequests":[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"TakeOver"}]}}`, ) Expect(err).NotTo(HaveOccurred()) names := []string{"sc-flux-save-1", "sc-flux-save-2"} @@ -832,10 +832,10 @@ spec: applyCatchUpSave(testNs, name, p.targetName) r.editThroughAPI(timingConfigMap, value) Eventually(func(g Gomega) { - g.Expect(saveRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) - g.Expect(saveRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(Equal(saveRequestReasonCommitted)) - commits[name] = saveRequestField(g, testNs, name, "{.status.commit}") + g.Expect(commitRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) + g.Expect(commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal(commitRequestReasonCommitted)) + commits[name] = commitRequestField(g, testNs, name, "{.status.commit}") g.Expect(commits[name]).NotTo(BeEmpty()) }, 2*time.Minute, 2*time.Second).Should(Succeed()) committed, inGit := r.committedValue(Default, timingConfigMap) @@ -843,26 +843,26 @@ spec: Expect(committed).To(Equal(value), "the save's commit holds its edit") message, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commits[name]) Expect(err).NotTo(HaveOccurred()) - Expect(message).To(ContainSubstring("Author-Source: SaveRequest "+testNs+"/"+name), + Expect(message).To(ContainSubstring("Author-Source: CommitRequest "+testNs+"/"+name), "the commit says its author is the request's submitter") } last := commits[names[len(names)-1]] Eventually(func(g Gomega) { for _, name := range names { - applied := saveRequestCondition(g, testNs, name, "Applied") - reason := saveRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) + applied := commitRequestCondition(g, testNs, name, "Applied") + reason := commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) if applied == "False" { - StopTrying(fmt.Sprintf("SaveRequest %s is Applied=False (%s): only a commit the branch "+ + StopTrying(fmt.Sprintf("CommitRequest %s is Applied=False (%s): only a commit the branch "+ "lost may be", name, reason)).Now() } - g.Expect(applied).To(Equal("True"), "SaveRequest %s Applied (%s)", name, reason) + g.Expect(applied).To(Equal("True"), "CommitRequest %s Applied (%s)", name, reason) g.Expect(reason).To(Equal("Applied")) // A later save's commit holds the earlier one's, so one apply may settle both. - g.Expect(saveRequestField(g, testNs, name, "{.status.appliedRevision}")). + g.Expect(commitRequestField(g, testNs, name, "{.status.appliedRevision}")). To(Or(ContainSubstring(commits[name]), ContainSubstring(last))) } - g.Expect(saveRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). + g.Expect(commitRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). To(ContainSubstring(last)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) @@ -1032,13 +1032,13 @@ spec: }, 3*time.Minute, 2*time.Second).Should(Succeed(), "the Kustomization must apply its folder into the workspace") } -// applyCatchUpSave creates the SaveRequest name on gitTarget, asking its applier to catch up once +// applyCatchUpSave creates the CommitRequest name on gitTarget, asking its applier to catch up once // it is pushed. It waits 45s for its window: a write from a cluster with no audit stream reaches the // window only after the attribution grace. func applyCatchUpSave(namespace, name, gitTarget string) { GinkgoHelper() _, err := kubectlRunWithStdin(namespace, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: SaveRequest +kind: CommitRequest metadata: name: %s namespace: %s @@ -1052,5 +1052,5 @@ spec: attachTimeout: "45s" maxDuration: "4s" `, name, namespace, gitTarget, name), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to create SaveRequest %s", name) + Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s", name) } From 448a10892f6361526464764bc75be5af5bb61b37 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 09:36:54 +0000 Subject: [PATCH 24/33] feat!: remove the admission webhook and window takeover A CommitRequest's author came from the validate-operator-types webhook, captured into a Redis command-author store. Both are gone, with the always-allow /validate-all endpoint the e2e system under test installed, the --admission-webhook* flags, the chart's servers.admission values, its Certificate and ValidatingWebhookConfiguration, and the WatchRule feedback the endpoint gave (the GitTarget still reports it as MultipleSourceNamespaces). The AuthorAttributed condition goes with them. Takeover existed only because admission could name a taker: a request's spec.window.author (SavedBy, TakeOver), a grant's unattributedWindows, the TakeOverNotAllowed reason and the Saved-By and Author-Source commit trailers are removed. Until the shared core takes a request's author from audit, a CommitRequest names no author. BREAKING CHANGE: the admission webhook is removed. Drop the --admission-webhook, --admission-webhook-bind-address and --admission-webhook-cert-* flags and the chart's servers.admission values; CommitRequest loses spec.window.author and the AuthorAttributed condition, and allowedCommitRequests grants lose unattributedWindows. Co-Authored-By: Claude Opus 5.5 --- api/v1alpha3/access_grants.go | 64 +---- api/v1alpha3/commitrequest_types.go | 49 +--- api/v1alpha3/gittarget_types.go | 31 +- charts/gitops-reverser/templates/_helpers.tpl | 14 - .../admission-server-certificate.yaml | 28 -- .../gitops-reverser/templates/deployment.yaml | 23 -- .../gitops-reverser/templates/services.yaml | 6 - .../validate-operator-types-webhook.yaml | 64 ----- .../templates/validate-redis.yaml | 3 - charts/gitops-reverser/values.schema.json | 26 -- charts/gitops-reverser/values.yaml | 30 -- cmd/main.go | 158 ++--------- cmd/main_audit_server_test.go | 36 +-- cmd/main_redis_flags_test.go | 14 - config/certs/certificates.yaml | 25 -- .../bases/configbutler.ai_commitrequests.yaml | 39 +-- .../crd/bases/configbutler.ai_gittargets.yaml | 34 +-- config/deployment.yaml | 11 - config/kustomization.yaml | 23 -- config/service.yaml | 4 - config/webhook/kustomization.yaml | 4 - config/webhook/validating-webhook.yaml | 120 -------- docs/INDEX.md | 3 +- docs/architecture.md | 8 +- docs/components.md | 1 - docs/configuration.md | 3 +- docs/design/branch-worker-event-model.md | 2 +- .../design/commitrequest-save-wait-options.md | 4 +- .../support-boundary/admission-consent.md | 3 +- docs/finished/redis-key-schema-v3.md | 3 +- docs/spec/README.md | 1 - docs/spec/attribution.md | 5 +- .../commitrequest-admission-authorship.md | 68 ----- docs/spec/commitrequest-design.md | 3 +- docs/spec/where-validation-lives.md | 27 +- hack/crdfields/fields.yaml | 7 +- hack/e2e/deploy-controller.sh | 4 +- internal/authz/commitrequest_admission.go | 15 +- .../authz/commitrequest_admission_test.go | 37 +-- .../controller/commitrequest_apply_test.go | 8 +- ...ommitrequest_attribution_agreement_test.go | 152 ---------- .../controller/commitrequest_controller.go | 227 ++++----------- .../commitrequest_controller_test.go | 8 +- .../commitrequest_controller_unit_test.go | 267 ++---------------- internal/controller/commitrequest_finalize.go | 110 +------- .../controller/commitrequest_kstatus_test.go | 20 +- .../commitrequest_outcome_metric_test.go | 18 +- internal/controller/commitrequest_ttl_test.go | 10 +- internal/controller/constants.go | 9 - .../controller/gittarget_publication_test.go | 4 +- internal/controller/suite_test.go | 13 +- internal/git/attribution_outcome_test.go | 49 ---- internal/git/branch_worker.go | 8 - internal/git/commit_executor.go | 30 -- internal/git/commit_request_attach.go | 32 +-- internal/git/commit_request_attach_loop.go | 8 - internal/git/commit_request_attach_test.go | 75 ----- internal/git/open_window.go | 5 - internal/git/pending_writes.go | 4 +- internal/git/types.go | 8 - internal/queue/command_author_store.go | 84 ------ internal/queue/command_author_store_test.go | 83 ------ internal/queue/key_prefix_test.go | 4 - internal/queue/redis_store.go | 8 - internal/webhook/admission_allow_handler.go | 26 -- .../webhook/admission_allow_handler_test.go | 18 -- internal/webhook/fuzz_test.go | 14 +- .../validate_operator_types_handler.go | 175 ------------ .../validate_operator_types_handler_test.go | 190 ------------- .../watchrule_source_namespace_admission.go | 138 --------- ...tchrule_source_namespace_admission_test.go | 182 ------------ test/e2e/author_mode_test.go | 2 +- test/e2e/commit_request_e2e_test.go | 12 +- test/e2e/gittarget_overlap_e2e_test.go | 3 +- test/e2e/quickstart_framework_e2e_test.go | 10 +- test/e2e/source_cluster_e2e_test.go | 29 +- 76 files changed, 240 insertions(+), 2803 deletions(-) delete mode 100644 charts/gitops-reverser/templates/admission-server-certificate.yaml delete mode 100644 charts/gitops-reverser/templates/validate-operator-types-webhook.yaml delete mode 100644 config/webhook/kustomization.yaml delete mode 100644 config/webhook/validating-webhook.yaml delete mode 100644 docs/spec/commitrequest-admission-authorship.md delete mode 100644 internal/controller/commitrequest_attribution_agreement_test.go delete mode 100644 internal/queue/command_author_store.go delete mode 100644 internal/queue/command_author_store_test.go delete mode 100644 internal/webhook/admission_allow_handler.go delete mode 100644 internal/webhook/admission_allow_handler_test.go delete mode 100644 internal/webhook/validate_operator_types_handler.go delete mode 100644 internal/webhook/validate_operator_types_handler_test.go delete mode 100644 internal/webhook/watchrule_source_namespace_admission.go delete mode 100644 internal/webhook/watchrule_source_namespace_admission_test.go diff --git a/api/v1alpha3/access_grants.go b/api/v1alpha3/access_grants.go index 6cf7234c..ab7d749b 100644 --- a/api/v1alpha3/access_grants.go +++ b/api/v1alpha3/access_grants.go @@ -220,64 +220,19 @@ type KubeProviderApplierClaimGrant struct { } // GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to -// save a GitTarget's commit windows, and bounds what they may claim of a -// window whose writes name no author. +// save a GitTarget's commit windows. type GitTargetCommitRequestGrant struct { // ConsumerNamespaces are the namespaces whose CommitRequests this grant // admits. from: "Same" is this GitTarget's namespace. // +required ConsumerNamespaces ConsumerNamespaces `json:"consumerNamespaces"` - - // UnattributedWindows says how much an admitted CommitRequest may claim of a - // commit window whose writes name no author, when it asks in - // spec.window.author. It acts only while the GitTarget's KubeProvider has - // never received an audit fact; on a cluster that has, a write naming no - // author is never claimed. Defaults to "Keep". - // - // - "Keep": a request attaches only to its creator's own window. - // - "SavedBy": a request may give such a window its message. The commit - // names no author, and its message ends with a Saved-By line naming the - // request's creator. - // - "TakeOver": a request may also take such a window over. Its creator - // authors the commit, and the message ends with an Author-Source line. - // - // Every write that names no author shares one window, whoever made it, so - // a takeover names the creator as the author of all of them. Allow it only - // where one person edits the mirrored namespaces at a time. A request - // admitted by several grants may claim the most any of them allows. - // +optional - // +kubebuilder:validation:Enum=Keep;SavedBy;TakeOver - // +kubebuilder:default=Keep - UnattributedWindows UnattributedWindowPolicy `json:"unattributedWindows,omitempty"` } -// CommitRequestAdmission is what a GitTarget's allowedCommitRequests give a -// CommitRequest. -// +kubebuilder:object:generate=false -type CommitRequestAdmission struct { - // Admitted reports whether some grant admits the request's namespace. - Admitted bool - // UnattributedWindows is the most the admitting grants allow the request to - // claim of a window whose writes name no author. - UnattributedWindows UnattributedWindowPolicy -} - -// AdmitsCommitRequest reports what spec.allowedCommitRequests gives a CommitRequest in namespace, by the -// labels that namespace carries in the control cluster. -func (t *GitTarget) AdmitsCommitRequest(namespace string, nsLabels map[string]string) (CommitRequestAdmission, error) { - grants := t.commitRequestGrants() - admitting, err := admitAll(consumersOf(grants), t.Namespace, namespace, nsLabels) - if err != nil || len(admitting) == 0 { - return CommitRequestAdmission{}, err - } - out := CommitRequestAdmission{Admitted: true, UnattributedWindows: UnattributedWindowsKeep} - for _, i := range admitting { - policy := grants[i].UnattributedWindows - if policy != "" && !out.UnattributedWindows.Allows(WindowAuthor(policy)) { - out.UnattributedWindows = policy - } - } - return out, nil +// AdmitsCommitRequest reports whether spec.allowedCommitRequests admits a CommitRequest in namespace, +// by the labels that namespace carries in the control cluster. +func (t *GitTarget) AdmitsCommitRequest(namespace string, nsLabels map[string]string) (bool, error) { + admitting, err := admitAll(consumersOf(t.commitRequestGrants()), t.Namespace, namespace, nsLabels) + return len(admitting) > 0, err } // DescribeAllowedCommitRequests names the namespaces spec.allowedCommitRequests admits. @@ -370,12 +325,9 @@ func DefaultKubeProviderApplierClaimGrants() []KubeProviderApplierClaimGrant { } // DefaultGitTargetCommitRequestGrants is GitTarget.spec.allowedCommitRequests when it is omitted: -// CommitRequests in the GitTarget's own namespace, claiming nothing of an unattributed window. +// CommitRequests in the GitTarget's own namespace. func DefaultGitTargetCommitRequestGrants() []GitTargetCommitRequestGrant { - return []GitTargetCommitRequestGrant{{ - ConsumerNamespaces: ConsumerNamespaces{From: GrantFromSame}, - UnattributedWindows: UnattributedWindowsKeep, - }} + return []GitTargetCommitRequestGrant{{ConsumerNamespaces: ConsumerNamespaces{From: GrantFromSame}}} } // orDefault is grants, or def when grants is nil (omitted rather than empty). diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 4f7ae964..1630eb48 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -60,7 +60,7 @@ type CommitRequestSpec struct { // write does. Omitted, the request attaches to the author's current or next // window within 2s and closes it 2s later. // +optional - // +kubebuilder:default={attach: "CurrentOrNext", author: "Own", attachTimeout: "2s", maxDuration: "2s"} + // +kubebuilder:default={attach: "CurrentOrNext", attachTimeout: "2s", maxDuration: "2s"} Window *CommitRequestWindow `json:"window,omitempty"` // WhenNothingToCommit decides what happens when the request ends with nothing @@ -124,20 +124,6 @@ const ( AttachNext AttachPolicy = "Next" ) -// WindowAuthor is whose commit window a CommitRequest may attach to. -type WindowAuthor string - -const ( - // WindowAuthorOwn attaches only to the creator's own window. - WindowAuthorOwn WindowAuthor = "Own" - // WindowAuthorSavedBy may also give the request's message to a window whose writes name no - // author, naming no author. - WindowAuthorSavedBy WindowAuthor = "SavedBy" - // WindowAuthorTakeOver may also take a window whose writes name no author over, and the - // creator authors its commit. - WindowAuthorTakeOver WindowAuthor = "TakeOver" -) - // CommitRequestWindow is how a CommitRequest attaches to a commit window and // when that window closes. A window is only ever opened by a write; a request // waits for one and never opens one. @@ -145,7 +131,7 @@ const ( // Every duration shares the shape GitTargetCommitSpec documents: a Go duration // string behind a pattern and a CEL bound. The immutability rule compares them // as durations, for the reason CommitRequestSpec gives. -// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && self.author == oldSelf.author && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" +// +kubebuilder:validation:XValidation:rule="self.attach == oldSelf.attach && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="CommitRequest spec is immutable after creation" // +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" // +kubebuilder:validation:XValidation:rule="self.attach != 'Next' || duration(self.attachTimeout) > duration('0s')",message="attach: Next needs a positive attachTimeout; with 0s it would close the author's window and give up in the same instant" type CommitRequestWindow struct { @@ -159,18 +145,6 @@ type CommitRequestWindow struct { // +kubebuilder:default=CurrentOrNext Attach AttachPolicy `json:"attach,omitempty"` - // Author says whose window the request may attach to. "Own" attaches only - // to the creator's own window. "SavedBy" may also give its message to a - // window whose writes name no author, naming no author. "TakeOver" may - // also take such a window over, and the creator authors the commit. The - // unattributedWindows of a grant in the GitTarget's spec.allowedCommitRequests - // that admits this namespace must allow what is asked, or the request fails - // with reason TakeOverNotAllowed. Defaults to "Own". - // +optional - // +kubebuilder:validation:Enum=Own;SavedBy;TakeOver - // +kubebuilder:default=Own - Author WindowAuthor `json:"author,omitempty"` - // AttachTimeout is how long the request waits for a window, as a Go duration, // from when GitOps Reverser picks the request up. "0s" attaches only to a // window already open. A request whose wait ran out never takes a later @@ -214,14 +188,6 @@ type CommitRequestWindow struct { // - Reconciling / Stalled: the kstatus progress / blocked pair. // Reconciling=True while finalizing; Stalled=True when the finalize failed // and needs attention. -// - AuthorAttributed (domain): binary and settled immediately. True -// (AttributedFromAdmission) when the submitter captured at admission named -// the commit author; False (CommitterFallback) when capture ran but no -// admission record exists, or False (AuthorCaptureDisabled) when capture -// is disabled. In either False case the request claims no actor. False is -// not a failure and does not affect Ready; the final Git author remains -// the matching watch window's author, unless the request took over a -// window whose writes name no author (spec.window.author "TakeOver"). // - Pushed (domain): True once the commit is in the remote repository. // - Applied (domain): with spec.catchUp "AfterPush", whether the applier // caught up with the commit. It never affects Ready. @@ -231,9 +197,7 @@ type CommitRequestStatus struct { ObservedGeneration int64 `json:"observedGeneration,omitempty"` // Conditions are the latest observations of this CommitRequest's state. - // Reconciling and Stalled follow kstatus. AuthorAttributed says whether the - // request's creator, recorded at admission, names the commit author; False - // is not a failure and does not affect Ready. + // Reconciling and Stalled follow kstatus. // // Three conditions say how far the save got: // @@ -290,7 +254,6 @@ type CommitRequestStatus struct { // +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` // +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` // +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.commit` -// +kubebuilder:printcolumn:name="AuthorAttributed",type=string,JSONPath=`.status.conditions[?(@.type=="AuthorAttributed")].status`,priority=1 // +kubebuilder:printcolumn:name="Pushed",type=string,JSONPath=`.status.conditions[?(@.type=="Pushed")].status`,priority=1 // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 // +kubebuilder:printcolumn:name="Branch",type=string,JSONPath=`.status.branch`,priority=1 @@ -299,10 +262,8 @@ type CommitRequestStatus struct { // CommitRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and -// gives the commit its message. With spec.window.author, and where the -// GitTarget allows it, it may also claim a window whose writes name no -// author. Its spec is immutable, and status.commit reports the commit once -// it is on the remote. +// gives the commit its message. Its spec is immutable, and status.commit +// reports the commit once it is on the remote. type CommitRequest struct { metav1.TypeMeta `json:",inline"` diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 435e46ba..f8d0b0c9 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -3,8 +3,6 @@ package v1alpha3 import ( - "slices" - meta "github.com/fluxcd/pkg/apis/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" @@ -187,11 +185,10 @@ type GitTargetSpec struct { Commit *GitTargetCommitSpec `json:"commit,omitempty"` // AllowedCommitRequests says which CommitRequests may save this GitTarget's - // commit windows, by the namespace they are in, and how much each may - // claim of a window whose writes name no author: + // commit windows, by the namespace they are in: // // - Omitted: the CommitRequests in this GitTarget's namespace, as one grant - // with from: "Same" that claims nothing. + // with from: "Same". // - Empty list ([]): no CommitRequest. // - A list of grants: the CommitRequests that at least one grant allows. // @@ -201,7 +198,7 @@ type GitTargetSpec struct { // +optional // +listType=atomic // +kubebuilder:validation:MaxItems=64 - // +kubebuilder:default={{consumerNamespaces: {from: Same}, unattributedWindows: Keep}} + // +kubebuilder:default={{consumerNamespaces: {from: Same}}} AllowedCommitRequests []GitTargetCommitRequestGrant `json:"allowedCommitRequests"` // Consent lives on the GitTarget, not on the KubeProvider that serves the requests: the provider's @@ -277,28 +274,6 @@ type GitTargetCommitSpec struct { Message *CommitMessageSpec `json:"message,omitempty"` } -// UnattributedWindowPolicy is how much a CommitRequest may claim of a commit -// window whose writes name no author. -type UnattributedWindowPolicy string - -const ( - // UnattributedWindowsKeep lets a request attach only to its creator's own window. - UnattributedWindowsKeep UnattributedWindowPolicy = "Keep" - // UnattributedWindowsSavedBy lets a request give such a window its message, naming no author. - UnattributedWindowsSavedBy UnattributedWindowPolicy = "SavedBy" - // UnattributedWindowsTakeOver also lets a request take such a window over and author its commit. - UnattributedWindowsTakeOver UnattributedWindowPolicy = "TakeOver" -) - -// Allows reports whether a CommitRequest asking for author may attach as it asks under this policy: -// "TakeOver" allows every ask, "SavedBy" allows "SavedBy" and "Own", and "Keep" (or empty) allows only -// "Own". An unknown value allows only "Own". -func (p UnattributedWindowPolicy) Allows(author WindowAuthor) bool { - // Weakest first; an unknown value ranks with the weakest (-1 + 1 = 0). - order := []string{string(UnattributedWindowsSavedBy), string(UnattributedWindowsTakeOver)} - return slices.Index(order, string(author)) <= slices.Index(order, string(p)) -} - // CommitWindow holds the two timers that close a GitTarget's commit windows. A // change to them applies to windows that open afterward; a window already open // keeps the timers it opened with. diff --git a/charts/gitops-reverser/templates/_helpers.tpl b/charts/gitops-reverser/templates/_helpers.tpl index 103ad0fb..741a3fe2 100644 --- a/charts/gitops-reverser/templates/_helpers.tpl +++ b/charts/gitops-reverser/templates/_helpers.tpl @@ -93,20 +93,6 @@ Default Secret name for the kube-apiserver audit client certificate. {{- .Values.servers.audit.tls.client.secretNameOverride | default (printf "%s-audit-client-cert" (include "gitops-reverser.fullname" .)) -}} {{- end }} -{{/* -Default Secret name for the validating admission (validate-operator-types) serving certificate. -*/}} -{{- define "gitops-reverser.admissionServerSecretName" -}} -{{- .Values.servers.admission.tls.secretNameOverride | default (printf "%s-admission-server-cert" (include "gitops-reverser.fullname" .)) -}} -{{- end }} - -{{/* -Name of the cert-manager Certificate that mints the admission serving cert. Used both -as the Certificate name and in the webhook's cert-manager.io/inject-ca-from annotation. -*/}} -{{- define "gitops-reverser.admissionServerCertName" -}} -{{- printf "%s-admission-server-cert" (include "gitops-reverser.fullname" .) -}} -{{- end }} {{/* Name of the push receiver Service. diff --git a/charts/gitops-reverser/templates/admission-server-certificate.yaml b/charts/gitops-reverser/templates/admission-server-certificate.yaml deleted file mode 100644 index e3dcfc4f..00000000 --- a/charts/gitops-reverser/templates/admission-server-certificate.yaml +++ /dev/null @@ -1,28 +0,0 @@ -{{- if and .Values.servers.admission.enabled .Values.servers.admission.tls.certManager .Values.certManager.enabled }} ---- -# Serving certificate for the validating admission (validate-operator-types) webhook. It is -# fronted by the controller Service, so its dnsNames are that Service's in-cluster names -# (the same the apiserver dials). The webhook config's cert-manager.io/inject-ca-from -# annotation injects this issuer's CA bundle into the webhook clientConfig automatically. -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: {{ include "gitops-reverser.admissionServerCertName" . }} - namespace: {{ .Release.Namespace }} - labels: - {{- include "gitops-reverser.labels" . | nindent 4 }} -spec: - dnsNames: - - {{ include "gitops-reverser.fullname" . }}.{{ .Release.Namespace }}.svc - - {{ include "gitops-reverser.fullname" . }}.{{ .Release.Namespace }}.svc.cluster.local - issuerRef: - kind: {{ .Values.certManager.issuer.kind }} - name: {{ .Values.certManager.issuer.name }} - secretName: {{ include "gitops-reverser.admissionServerSecretName" . }} - usages: - - digital signature - - key encipherment - - server auth - privateKey: - rotationPolicy: Always -{{- end }} diff --git a/charts/gitops-reverser/templates/deployment.yaml b/charts/gitops-reverser/templates/deployment.yaml index 4f74ba4f..f54832cb 100644 --- a/charts/gitops-reverser/templates/deployment.yaml +++ b/charts/gitops-reverser/templates/deployment.yaml @@ -92,13 +92,6 @@ spec: {{- if .Values.attribution.auditRouteAnnotationKey }} - --author-attribution-audit-route-annotation-key={{ .Values.attribution.auditRouteAnnotationKey }} {{- end }} - {{- if .Values.servers.admission.enabled }} - - --admission-webhook - - --admission-webhook-bind-address={{ .Values.servers.admission.bindAddress }} - - --admission-webhook-cert-path={{ .Values.servers.admission.tls.certPath }} - - --admission-webhook-cert-name={{ .Values.servers.admission.tls.certName }} - - --admission-webhook-cert-key={{ .Values.servers.admission.tls.certKey }} - {{- end }} {{- with .Values.controllerManager.additionalSensitiveResources }} - {{ printf "--additional-sensitive-resources=%s" (join "," .) | quote }} {{- end }} @@ -136,11 +129,6 @@ spec: - name: metrics containerPort: {{ .Values.servers.metrics.port }} protocol: TCP - {{- if .Values.servers.admission.enabled }} - - name: webhook - containerPort: {{ .Values.servers.admission.port }} - protocol: TCP - {{- end }} {{- if .Values.servers.receiver.enabled }} - name: receiver containerPort: {{ .Values.servers.receiver.port }} @@ -195,11 +183,6 @@ spec: mountPath: {{ .Values.servers.audit.tls.clientCAPath }} readOnly: true {{- end }} - {{- if .Values.servers.admission.enabled }} - - name: admission-cert - mountPath: {{ .Values.servers.admission.tls.certPath }} - readOnly: true - {{- end }} {{- with .Values.volumeMounts }} {{- toYaml . | nindent 12 }} {{- end }} @@ -222,12 +205,6 @@ spec: secretName: {{ include "gitops-reverser.auditRootCASecretName" . }} defaultMode: 420 {{- end }} - {{- if .Values.servers.admission.enabled }} - - name: admission-cert - secret: - secretName: {{ include "gitops-reverser.admissionServerSecretName" . }} - defaultMode: 420 - {{- end }} {{- with .Values.volumes }} {{- toYaml . | nindent 8 }} {{- end }} diff --git a/charts/gitops-reverser/templates/services.yaml b/charts/gitops-reverser/templates/services.yaml index e6c20408..3139e690 100644 --- a/charts/gitops-reverser/templates/services.yaml +++ b/charts/gitops-reverser/templates/services.yaml @@ -19,12 +19,6 @@ spec: port: {{ .Values.service.ports.metrics }} targetPort: {{ .Values.servers.metrics.port }} protocol: TCP - {{- if .Values.servers.admission.enabled }} - - name: webhook - port: {{ .Values.servers.admission.port }} - targetPort: {{ .Values.servers.admission.port }} - protocol: TCP - {{- end }} selector: {{- include "gitops-reverser.selectorLabels" . | nindent 4 }} control-plane: gitops-reverser diff --git a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml b/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml deleted file mode 100644 index 3d382ebf..00000000 --- a/charts/gitops-reverser/templates/validate-operator-types-webhook.yaml +++ /dev/null @@ -1,64 +0,0 @@ -{{- if .Values.servers.admission.enabled }} ---- -# Two unrelated jobs on one endpoint, both narrow by construction. -# -# It captures the authenticated submitter of our own command kinds (a CommitRequest today) -# into the command-author Redis corner the controller reads back with no wait; a miss -# leaves the request without a claimed actor (AuthorAttributed=False). See -# docs/spec/commitrequest-admission-authorship.md. -# -# It also rejects a WatchRule that would bring a second source namespace to a GitTarget -# declaring spec.serializeNamespace: false. That rejection is FEEDBACK, not enforcement: -# the writer refuses the same configuration at the write whether or not this webhook ran, -# which is why failurePolicy stays Ignore. See docs/layout/model.md. -apiVersion: admissionregistration.k8s.io/v1 -kind: ValidatingWebhookConfiguration -metadata: - name: {{ include "gitops-reverser.fullname" . }}-validate-operator-types - labels: - {{- include "gitops-reverser.labels" . | nindent 4 }} - {{- if and .Values.servers.admission.tls.certManager .Values.certManager.enabled }} - annotations: - cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "gitops-reverser.admissionServerCertName" . }} - {{- end }} -webhooks: - - name: validate-operator-types.configbutler.ai - admissionReviewVersions: - - v1 - clientConfig: - service: - name: {{ include "gitops-reverser.fullname" . }} - namespace: {{ .Release.Namespace }} - path: /validate-operator-types - port: {{ .Values.servers.admission.port }} - # Ignore, not Fail: a missed author capture must leave the request unnamed, never - # reject a user's CommitRequest, and the WatchRule check is feedback the write path - # already enforces on its own. - failurePolicy: Ignore - matchPolicy: Equivalent - rules: - - apiGroups: - - configbutler.ai - apiVersions: - - v1alpha3 - operations: - - CREATE - resources: - - commitrequests - scope: Namespaced - # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it - # was created, and that edit is the same mistake. - - apiGroups: - - configbutler.ai - apiVersions: - - v1alpha3 - operations: - - CREATE - - UPDATE - resources: - - watchrules - scope: Namespaced - # Redis write on real requests; nothing on dry-run (the handler honors this). - sideEffects: NoneOnDryRun - timeoutSeconds: {{ .Values.servers.admission.timeoutSeconds }} -{{- end }} diff --git a/charts/gitops-reverser/templates/validate-redis.yaml b/charts/gitops-reverser/templates/validate-redis.yaml index eb24062d..c6ccab7f 100644 --- a/charts/gitops-reverser/templates/validate-redis.yaml +++ b/charts/gitops-reverser/templates/validate-redis.yaml @@ -6,9 +6,6 @@ endpoint at all, and refusing it here would make a value this chart itself docum The memory transport's other requirement, a single replica, needs no check here: validate-replica-count.yaml already refuses replicaCount > 1 for the whole chart, so there is no configuration this could catch. - -The admission webhook is deliberately NOT gated here: it stays enabled without Redis and simply -no-ops command-author capture. */ -}} {{- if and .Values.attribution.enabled (eq .Values.attribution.transport "redis") (eq (trim .Values.queue.redis.addr) "") -}} {{- fail "attribution.enabled=true with attribution.transport=redis requires queue.redis.addr: the redis transport carries audit facts on Redis streams. Set queue.redis.addr, select attribution.transport=memory to run attribution in-process on a single replica, or leave attribution.enabled=false." -}} diff --git a/charts/gitops-reverser/values.schema.json b/charts/gitops-reverser/values.schema.json index 4eb69f9d..2f8bf2fe 100644 --- a/charts/gitops-reverser/values.schema.json +++ b/charts/gitops-reverser/values.schema.json @@ -221,32 +221,6 @@ } } }, - "admission": { - "type": "object", - "additionalProperties": false, - "properties": { - "enabled": { "type": "boolean" }, - "bindAddress": { "type": "string" }, - "port": { "$ref": "#/$defs/port" }, - "timeoutSeconds": { - "type": "integer", - "minimum": 1, - "maximum": 30, - "description": "The apiserver rejects a webhook timeout outside 1-30s." - }, - "tls": { - "type": "object", - "additionalProperties": false, - "properties": { - "certManager": { "type": "boolean" }, - "certPath": { "type": "string" }, - "certName": { "type": "string" }, - "certKey": { "type": "string" }, - "secretNameOverride": { "$ref": "#/$defs/secretNameOverride" } - } - } - } - }, "audit": { "type": "object", "additionalProperties": false, diff --git a/charts/gitops-reverser/values.yaml b/charts/gitops-reverser/values.yaml index e255e890..3924e12d 100644 --- a/charts/gitops-reverser/values.yaml +++ b/charts/gitops-reverser/values.yaml @@ -174,36 +174,6 @@ servers: # Leave empty to use the chart-generated Secret name. secretNameOverride: "" - # Validating admission server hosting the validate-operator-types webhook, which captures - # the submitter of each CommitRequest at admission and records it as the request's named actor - # (read back by the controller with no wait). Independent of `attribution` below: - # attribution governs *mirrored-resource* authorship (from audit), this governs - # *command* authorship (from admission). Off → CommitRequests claim no actor and report - # AuthorAttributed=False. See - # docs/spec/commitrequest-admission-authorship.md. - # - # Command-author capture is Redis-backed: the captured author is written to Redis at admission and - # read back at reconcile. Enabled by default so the webhook is always installed, but it is a no-op - # without Redis: the pod runs healthy and CommitRequests claim no actor - # (AuthorAttributed=False) until queue.redis.addr is set. - admission: - enabled: true - bindAddress: ":9443" - port: 9443 - # Bounded per-request wait before the apiserver gives up on the webhook. - # failurePolicy is Ignore, so a slow/unreachable backend admits the command anyway. - timeoutSeconds: 2 - tls: - # The admission server always serves TLS (the apiserver requires HTTPS); the only - # choice is who mints the cert. certManager=false means BYO via secretNameOverride - # (and patch the webhook caBundle out of band). - certManager: true - certPath: "/tmp/k8s-webhook-server/serving-certs" - certName: "tls.crt" - certKey: "tls.key" - # Leave empty to use the chart-generated Secret name. - secretNameOverride: "" - audit: # host:port the audit ingress server binds to (passed as --audit-bind-address). bindAddress: "0.0.0.0:9444" diff --git a/cmd/main.go b/cmd/main.go index 89fae16e..c78af423 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -34,8 +34,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/log/zap" "sigs.k8s.io/controller-runtime/pkg/metrics/filters" metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" - ctrlwebhook "sigs.k8s.io/controller-runtime/pkg/webhook" - ctrladmission "sigs.k8s.io/controller-runtime/pkg/webhook/admission" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/applier" @@ -58,15 +56,14 @@ var ( ) const ( - flagParseFailureExitCode = 2 - defaultAdmissionWebhookBindAddr = ":9443" - defaultAuditBindAddr = "0.0.0.0:9444" - defaultAuditMaxBodyBytes = int64(10 * 1024 * 1024) - defaultAuditReadTimeout = 15 * time.Second - defaultAuditWriteTimeout = 30 * time.Second - defaultAuditIdleTimeout = 60 * time.Second - defaultAuditShutdownTimeout = 10 * time.Second - defaultBranchBufferMaxSizeStr = "8Mi" + flagParseFailureExitCode = 2 + defaultAuditBindAddr = "0.0.0.0:9444" + defaultAuditMaxBodyBytes = int64(10 * 1024 * 1024) + defaultAuditReadTimeout = 15 * time.Second + defaultAuditWriteTimeout = 30 * time.Second + defaultAuditIdleTimeout = 60 * time.Second + defaultAuditShutdownTimeout = 10 * time.Second + defaultBranchBufferMaxSizeStr = "8Mi" ) func init() { @@ -92,9 +89,7 @@ func main() { "metricsAddr", cfg.metricsAddr, "metricsInsecure", cfg.metricsInsecure, "auditBindAddress", cfg.auditBindAddress, - "auditInsecure", cfg.auditInsecure, - "admissionWebhookEnabled", cfg.admissionWebhookEnabled, - "admissionWebhookBindAddress", cfg.admissionWebhookBindAddress) + "auditInsecure", cfg.auditInsecure) setupLog.Info("Sensitive resource policy", "resources", cfg.sensitiveResources.Entries()) // Initialize metrics @@ -113,7 +108,7 @@ func main() { ) // Manager - mgr := newManager(metricsServerOptions, cfg.probeAddr, cfg, tlsOpts) + mgr := newManager(metricsServerOptions, cfg.probeAddr) // Expose build metadata on the metrics server so an operator can confirm a // running pod is the build they expect (also logged at startup above). @@ -222,8 +217,8 @@ func main() { // Valkey/Redis is optional. When configured it holds each GitTarget's watch resume cursor so work // is re-picked up exactly where it left off after a restart or reconnect. When not configured the // WatchCursorStore stays nil and watches cold-replay from scratch on restart instead of resuming. - // Author attribution and the admission webhook both require Redis; validation has already rejected - // those combinations when redis-addr is empty. + // Author attribution over the Redis transport requires it; validation has already rejected that + // combination when redis-addr is empty. var redisStore *queue.RedisStore var redisGate *redisReadinessGate if cfg.redisAddr != "" { @@ -413,44 +408,16 @@ func main() { Follower: applyDispatcher, }).SetupWithManager(mgr), "unable to create controller", "controller", kind) } - // Command authorship is captured at admission by the validate-operator-types webhook and - // lives in its own Redis corner (author:v1:command), independent of - // --author-attribution (which governs mirrored-resource attribution). It is wired - // whenever the admission server is on; the controller reads the captured submitter - // back with no wait (docs/spec/commitrequest-admission-authorship.md). - // - // AuthorLookup must be a nil interface — not a non-nil interface wrapping a nil - // *CommandAuthorStore — when the webhook is off, so the controller's nil check - // selects the no-actor path immediately (AuthorAttributed=False) instead of - // dereferencing a nil store. - var commandAuthorStore *queue.CommandAuthorStore - var commandAuthorLookup controller.CommandAuthorLookup - if cfg.admissionWebhookEnabled { - if redisStore != nil { - commandAuthorStore = redisStore.CommandAuthorStore() - commandAuthorLookup = commandAuthorStore - setupLog.Info("validate-operator-types webhook enabled: command submitters are captured at admission " + - "and recorded as the request's named actor") - } else { - setupLog.Info("validate-operator-types webhook enabled without Redis: command author capture is a " + - "no-op; CommitRequests claim no actor (AuthorAttributed=False). " + - "Set --redis-addr to capture command authors.") - } - } if err := (&controller.CommitRequestReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - APIReader: mgr.GetAPIReader(), - Finalizer: eventRouter, - AuthorLookup: commandAuthorLookup, - TTL: cfg.commitRequestTTL, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + APIReader: mgr.GetAPIReader(), + Finalizer: eventRouter, + TTL: cfg.commitRequestTTL, }).SetupWithManager(mgr); err != nil { setupLog.Error(err, "unable to create controller", "controller", "CommitRequest") os.Exit(1) } - if cfg.admissionWebhookEnabled { - setupAdmissionWebhooks(mgr, commandAuthorStore) - } // +kubebuilder:scaffold:builder // Cert watchers (auditCertWatcher is nil in configured-author mode / --audit-insecure). @@ -477,11 +444,6 @@ type appConfig struct { metricsCertKey string probeAddr string metricsInsecure bool - admissionWebhookEnabled bool - admissionWebhookBindAddress string - admissionWebhookCertPath string - admissionWebhookCertName string - admissionWebhookCertKey string enableHTTP2 bool auditBindAddress string auditCertPath string @@ -548,20 +510,6 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { fs.StringVar(&cfg.probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.") fs.BoolVar(&cfg.metricsInsecure, "metrics-insecure", false, "Serve the metrics endpoint over plain HTTP instead of HTTPS (default false; HTTPS).") - fs.BoolVar(&cfg.admissionWebhookEnabled, "admission-webhook", false, - "Serve the validating admission webhook endpoint (default false; off). When true, the webhook "+ - "server binds --admission-webhook-bind-address and requires --admission-webhook-cert-path.") - fs.StringVar(&cfg.admissionWebhookBindAddress, "admission-webhook-bind-address", defaultAdmissionWebhookBindAddr, - "Address (host:port) the validating admission webhook HTTPS server binds to; "+ - "an empty host (\":9443\") binds all interfaces.") - bindServerCertFlags( - fs, - "admission-webhook", - "validating admission webhook TLS", - &cfg.admissionWebhookCertPath, - &cfg.admissionWebhookCertName, - &cfg.admissionWebhookCertKey, - ) bindServerCertFlags( fs, "metrics", @@ -595,8 +543,7 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { "and, with --author-attribution-transport=redis, the attribution fact streams. Leave empty to "+ "run without Redis: watches cold-replay on restart instead of resuming, and attribution needs "+ "--author-attribution-transport=memory. Required by --author-attribution-transport=redis when "+ - "attribution is on. --admission-webhook still runs without it, but command-author capture is a "+ - "no-op: CommitRequests claim no actor.") + "attribution is on.") fs.BoolVar(&cfg.authorAttribution, "author-attribution", true, "Name the real actor (human or service account) who caused each change as the Git commit author, "+ "resolved from matching audit facts; this runs the audit webhook ingress (default true). When "+ @@ -761,9 +708,6 @@ func parseFlagsWithArgs(fs *flag.FlagSet, args []string) (appConfig, error) { if err := validateAuditConfig(cfg); err != nil { return appConfig{}, err } - if err := validateAdmissionWebhookConfig(cfg); err != nil { - return appConfig{}, err - } if err := validateReceiverConfig(cfg); err != nil { return appConfig{}, err } @@ -852,8 +796,7 @@ func validateAuditConfig(cfg appConfig) error { if strings.TrimSpace(cfg.redisAddr) == "" { // Attribution no longer implies Redis. It needs a fact transport, and the in-memory one is a // supported configuration rather than a rejected one — which is the whole reason the transport - // is selectable. Redis stays required for the transport that uses it, and for the admission - // webhook's command-author capture, which has no in-memory counterpart. + // is selectable. Redis stays required for the transport that uses it. if cfg.authorAttribution && cfg.attributionTransport == transportRedis { return errors.New( "redis-addr is required when author-attribution is enabled with " + @@ -934,22 +877,6 @@ func validateAttributionTransport(cfg appConfig) error { return nil } -func validateAdmissionWebhookConfig(cfg appConfig) error { - if !cfg.admissionWebhookEnabled { - return nil - } - // No redis-addr requirement here: the admission webhook stays enabled without Redis and - // simply no-ops command-author capture (CommitRequests claim no actor). Redis is - // wired only when configured; see the commandAuthorStore setup. - if _, _, err := splitBindAddress(cfg.admissionWebhookBindAddress); err != nil { - return fmt.Errorf("invalid admission-webhook-bind-address %q: %w", cfg.admissionWebhookBindAddress, err) - } - if strings.TrimSpace(cfg.admissionWebhookCertPath) == "" { - return errors.New("admission-webhook-cert-path is required when admission webhook is enabled") - } - return nil -} - // fatalIfErr logs and exits the process if err is not nil. func fatalIfErr(err error, msg string, keysAndValues ...any) { if err != nil { @@ -1244,35 +1171,11 @@ func loadCertPoolFromPEMFile(path string) (*x509.CertPool, error) { } // newManager creates a new controller-runtime Manager with common options. -func newManager( - metricsOptions metricsserver.Options, - probeAddr string, - cfg appConfig, - baseTLS []func(*tls.Config), -) ctrl.Manager { - var webhookServer ctrlwebhook.Server - if cfg.admissionWebhookEnabled { - // The bind address was validated in validateAdmissionWebhookConfig; controller-runtime's - // webhook server takes Host and Port separately, so split it back here. - host, port, err := splitBindAddress(cfg.admissionWebhookBindAddress) - if err != nil { - setupLog.Error(err, "invalid --admission-webhook-bind-address") - os.Exit(1) - } - webhookServer = ctrlwebhook.NewServer(ctrlwebhook.Options{ - Host: host, - Port: port, - CertDir: cfg.admissionWebhookCertPath, - CertName: cfg.admissionWebhookCertName, - KeyName: cfg.admissionWebhookCertKey, - TLSOpts: baseTLS, - }) - } +func newManager(metricsOptions metricsserver.Options, probeAddr string) ctrl.Manager { mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ Scheme: scheme, Metrics: metricsOptions, HealthProbeBindAddress: probeAddr, - WebhookServer: webhookServer, // Never cache Secret values. The control plane reads a small set of named // Secrets (Git credentials, signing keys, age keys) directly by name; caching // them would start a cluster-wide Secret informer that retains every Secret @@ -1293,27 +1196,6 @@ func newManager( return mgr } -// setupAdmissionWebhooks registers both handlers on the one admission server: the -// always-allow observer (a future-policy extension point) and the validate-operator-types -// handler, which captures the submitter of our own command kinds into commandAuthorStore and -// validates a WatchRule's source namespaces against the GitTarget it names. -func setupAdmissionWebhooks(mgr ctrl.Manager, commandAuthorStore *queue.CommandAuthorStore) { - mgr.GetWebhookServer().Register( - webhookhandler.ValidateAllPath, - &ctrladmission.Webhook{Handler: webhookhandler.AdmissionAllowHandler{}}, - ) - // Leave Store as a nil interface when there is no Redis-backed store, so the handler - // no-ops rather than dereferencing a typed-nil *CommandAuthorStore. - operatorTypesHandler := &webhookhandler.ValidateOperatorTypesHandler{Client: mgr.GetClient()} - if commandAuthorStore != nil { - operatorTypesHandler.Store = commandAuthorStore - } - mgr.GetWebhookServer().Register( - webhookhandler.ValidateOperatorTypesPath, - &ctrladmission.Webhook{Handler: operatorTypesHandler}, - ) -} - // addCertWatchersToManager attaches optional certificate watchers to the manager. func addCertWatchersToManager( mgr ctrl.Manager, diff --git a/cmd/main_audit_server_test.go b/cmd/main_audit_server_test.go index 25f68931..9e5e7636 100644 --- a/cmd/main_audit_server_test.go +++ b/cmd/main_audit_server_test.go @@ -33,8 +33,6 @@ func TestParseFlagsWithArgs_Defaults(t *testing.T) { require.NoError(t, err) assert.False(t, cfg.metricsInsecure) - assert.False(t, cfg.admissionWebhookEnabled) - assert.Equal(t, ":9443", cfg.admissionWebhookBindAddress) assert.False(t, cfg.auditInsecure) assert.Equal(t, "0.0.0.0:9444", cfg.auditBindAddress) assert.Equal(t, "/tmp/k8s-audit-server/audit-client-ca", cfg.auditClientCAPath) @@ -56,26 +54,6 @@ func TestParseFlagsWithArgs_Defaults(t *testing.T) { assert.False(t, cfg.credentialPolicy.AllowInsecureGitHTTP) } -func TestParseFlagsWithArgs_AdmissionWebhookValues(t *testing.T) { - fs := flag.NewFlagSet("test-admission-webhook", flag.ContinueOnError) - args := []string{ - "--admission-webhook", - "--admission-webhook-bind-address=:9445", - "--admission-webhook-cert-path=/tmp/admission-certs", - "--admission-webhook-cert-name=cert.pem", - "--admission-webhook-cert-key=key.pem", - } - - cfg, err := parseFlagsWithArgs(fs, args) - require.NoError(t, err) - - assert.True(t, cfg.admissionWebhookEnabled) - assert.Equal(t, ":9445", cfg.admissionWebhookBindAddress) - assert.Equal(t, "/tmp/admission-certs", cfg.admissionWebhookCertPath) - assert.Equal(t, "cert.pem", cfg.admissionWebhookCertName) - assert.Equal(t, "key.pem", cfg.admissionWebhookCertKey) -} - func TestParseFlagsWithArgs_AuditUnsecure(t *testing.T) { fs := flag.NewFlagSet("test-audit-insecure", flag.ContinueOnError) args := []string{ @@ -282,16 +260,10 @@ func TestParseFlagsWithArgs_InvalidAuditSettings(t *testing.T) { args: []string{"--additional-sensitive-resources=example.io/v1/credentials"}, }, { - name: "invalid admission webhook bind address port", - args: []string{"--admission-webhook", "--admission-webhook-bind-address=:0"}, - }, - { - name: "missing admission webhook cert path", - args: []string{"--admission-webhook", "--admission-webhook-cert-path="}, - }, - { - name: "admission webhook without redis", - args: []string{"--admission-webhook", "--admission-webhook-cert-path=/tmp/certs", "--redis-addr="}, + // The admission webhook is gone, and so are its flags: an install that still passes one + // fails at startup rather than running without what it asked for. + name: "the removed admission webhook flag", + args: []string{"--admission-webhook"}, }, } diff --git a/cmd/main_redis_flags_test.go b/cmd/main_redis_flags_test.go index e04b8c76..0d65ac2b 100644 --- a/cmd/main_redis_flags_test.go +++ b/cmd/main_redis_flags_test.go @@ -34,20 +34,6 @@ func TestParseFlags_RedisAddrIsOnlyRequiredByAttribution(t *testing.T) { "--audit-insecure", }, }, - // The webhook is failurePolicy: Ignore by design and the controller is the real gate, - // so running it without Redis is a supported, degraded mode — not a usage error. - // (--admission-webhook-cert-path has no default and is required whenever the webhook - // is on; that requirement is about TLS, not about Redis.) - "admission webhook runs without redis": { - args: []string{ - "--redis-addr=", "--author-attribution=false", - "--admission-webhook", "--admission-webhook-cert-path=/tmp/certs", - }, - }, - "admission webhook still requires its cert path": { - args: []string{"--redis-addr=", "--author-attribution=false", "--admission-webhook"}, - wantErr: "admission-webhook-cert-path is required", - }, "configured-author runs without redis": { args: []string{"--redis-addr=", "--author-attribution=false"}, }, diff --git a/config/certs/certificates.yaml b/config/certs/certificates.yaml index c9e4d2fd..1f5c428c 100644 --- a/config/certs/certificates.yaml +++ b/config/certs/certificates.yaml @@ -77,28 +77,3 @@ spec: usages: - digital signature - client auth ---- -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - labels: - app.kubernetes.io/managed-by: kustomize - app.kubernetes.io/name: gitops-reverser - name: gitops-reverser-admission-server-cert - namespace: sut -spec: - dnsNames: - - gitops-reverser-service.sut.svc - - gitops-reverser-service.sut.svc.cluster.local - duration: 2160h - issuerRef: - kind: Issuer - name: gitops-reverser-audit-ca-issuer - privateKey: - rotationPolicy: Always - renewBefore: 360h - secretName: admission-server-cert - usages: - - digital signature - - key encipherment - - server auth diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index e33fa796..31f10e00 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -27,10 +27,6 @@ spec: - jsonPath: .status.commit name: Commit type: string - - jsonPath: .status.conditions[?(@.type=="AuthorAttributed")].status - name: AuthorAttributed - priority: 1 - type: string - jsonPath: .status.conditions[?(@.type=="Pushed")].status name: Pushed priority: 1 @@ -56,10 +52,8 @@ spec: description: |- CommitRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and - gives the commit its message. With spec.window.author, and where the - GitTarget allows it, it may also claim a window whose writes name no - author. Its spec is immutable, and status.commit reports the commit once - it is on the remote. + gives the commit its message. Its spec is immutable, and status.commit + reports the commit once it is on the remote. properties: apiVersion: description: |- @@ -155,7 +149,6 @@ spec: default: attach: CurrentOrNext attachTimeout: 2s - author: Own maxDuration: 2s description: |- Window says which commit window this request attaches to, how long it waits @@ -189,21 +182,6 @@ spec: x-kubernetes-validations: - message: spec.window.attachTimeout must not exceed 5m rule: duration(self) <= duration('5m') - author: - default: Own - description: |- - Author says whose window the request may attach to. "Own" attaches only - to the creator's own window. "SavedBy" may also give its message to a - window whose writes name no author, naming no author. "TakeOver" may - also take such a window over, and the creator authors the commit. The - unattributedWindows of a grant in the GitTarget's spec.allowedCommitRequests - that admits this namespace must allow what is asked, or the request fails - with reason TakeOverNotAllowed. Defaults to "Own". - enum: - - Own - - SavedBy - - TakeOver - type: string idleTimeout: description: |- IdleTimeout closes the attached window this long after its last write, as a @@ -228,11 +206,10 @@ spec: type: object x-kubernetes-validations: - message: CommitRequest spec is immutable after creation - rule: self.attach == oldSelf.attach && self.author == oldSelf.author - && duration(self.attachTimeout) == duration(oldSelf.attachTimeout) - && duration(self.maxDuration) == duration(oldSelf.maxDuration) - && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) - || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) + rule: self.attach == oldSelf.attach && duration(self.attachTimeout) + == duration(oldSelf.attachTimeout) && duration(self.maxDuration) + == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) + && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) - message: spec.window.idleTimeout must not exceed maxDuration rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' - message: 'attach: Next needs a positive attachTimeout; with 0s it @@ -275,9 +252,7 @@ spec: conditions: description: |- Conditions are the latest observations of this CommitRequest's state. - Reconciling and Stalled follow kstatus. AuthorAttributed says whether the - request's creator, recorded at admission, names the commit author; False - is not a failure and does not affect Ready. + Reconciling and Stalled follow kstatus. Three conditions say how far the save got: diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index 454a072d..f5a0521e 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -111,14 +111,12 @@ spec: default: - consumerNamespaces: from: Same - unattributedWindows: Keep description: |- AllowedCommitRequests says which CommitRequests may save this GitTarget's - commit windows, by the namespace they are in, and how much each may - claim of a window whose writes name no author: + commit windows, by the namespace they are in: - Omitted: the CommitRequests in this GitTarget's namespace, as one grant - with from: "Same" that claims nothing. + with from: "Same". - Empty list ([]): no CommitRequest. - A list of grants: the CommitRequests that at least one grant allows. @@ -128,8 +126,7 @@ spec: items: description: |- GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to - save a GitTarget's commit windows, and bounds what they may claim of a - window whose writes name no author. + save a GitTarget's commit windows. properties: consumerNamespaces: description: |- @@ -243,31 +240,6 @@ spec: rule: '!has(self.selector) || (has(self.selector.matchLabels) && size(self.selector.matchLabels) > 0) || (has(self.selector.matchExpressions) && size(self.selector.matchExpressions) > 0)' - unattributedWindows: - default: Keep - description: |- - UnattributedWindows says how much an admitted CommitRequest may claim of a - commit window whose writes name no author, when it asks in - spec.window.author. It acts only while the GitTarget's KubeProvider has - never received an audit fact; on a cluster that has, a write naming no - author is never claimed. Defaults to "Keep". - - - "Keep": a request attaches only to its creator's own window. - - "SavedBy": a request may give such a window its message. The commit - names no author, and its message ends with a Saved-By line naming the - request's creator. - - "TakeOver": a request may also take such a window over. Its creator - authors the commit, and the message ends with an Author-Source line. - - Every write that names no author shares one window, whoever made it, so - a takeover names the creator as the author of all of them. Allow it only - where one person edits the mirrored namespaces at a time. A request - admitted by several grants may claim the most any of them allows. - enum: - - Keep - - SavedBy - - TakeOver - type: string required: - consumerNamespaces type: object diff --git a/config/deployment.yaml b/config/deployment.yaml index 4cb63548..77d1516d 100644 --- a/config/deployment.yaml +++ b/config/deployment.yaml @@ -27,8 +27,6 @@ spec: - --metrics-bind-address=:8443 - --metrics-insecure - --health-probe-bind-address=:8081 - - --admission-webhook - - --admission-webhook-cert-path=/tmp/k8s-webhook-server/serving-certs - --audit-cert-path=/tmp/k8s-audit-server/audit-server-certs - --audit-client-ca-path=/tmp/k8s-audit-server/audit-client-ca - --redis-addr=valkey.valkey-e2e.svc.cluster.local:6379 @@ -95,9 +93,6 @@ spec: - containerPort: 9444 name: audit protocol: TCP - - containerPort: 9443 - name: webhook - protocol: TCP - containerPort: 8443 name: metrics protocol: TCP @@ -132,9 +127,6 @@ spec: - mountPath: /tmp/k8s-audit-server/audit-client-ca name: audit-client-ca readOnly: true - - mountPath: /tmp/k8s-webhook-server/serving-certs - name: admission-webhook-certs - readOnly: true - mountPath: /var/gocoverage name: gocoverage securityContext: @@ -152,9 +144,6 @@ spec: - name: audit-client-ca secret: secretName: gitops-reverser-audit-root-ca - - name: admission-webhook-certs - secret: - secretName: admission-server-cert # Coverage output for a -cover image (GOCOVERDIR). emptyDir keeps the # pod compliant with the restricted PodSecurity profile; it survives a # container restart, so `task e2e-coverage-collect` flushes by signalling diff --git a/config/kustomization.yaml b/config/kustomization.yaml index 0eeba6ae..98ab3d53 100644 --- a/config/kustomization.yaml +++ b/config/kustomization.yaml @@ -9,7 +9,6 @@ resources: - receiver-service.yaml - deployment.yaml - certs -- webhook - kubeprovider-local.yaml images: - name: gitops-reverser @@ -40,26 +39,4 @@ replacements: select: kind: Certificate name: gitops-reverser-audit-server-cert - - fieldPaths: - - spec.dnsNames.0 - - spec.dnsNames.1 - options: - delimiter: . - index: 1 - select: - kind: Certificate - name: gitops-reverser-admission-server-cert - - fieldPaths: - - metadata.annotations.[cert-manager.io/inject-ca-from] - options: - delimiter: / - index: 0 - select: - kind: ValidatingWebhookConfiguration - name: gitops-reverser-validating-webhook - - fieldPaths: - - webhooks.0.clientConfig.service.namespace - select: - kind: ValidatingWebhookConfiguration - name: gitops-reverser-validating-webhook namespace: gitops-reverser diff --git a/config/service.yaml b/config/service.yaml index 9c5d0a6c..bb7fa95c 100644 --- a/config/service.yaml +++ b/config/service.yaml @@ -13,10 +13,6 @@ spec: port: 8443 protocol: TCP targetPort: 8443 - - name: webhook - port: 9443 - protocol: TCP - targetPort: 9443 selector: app.kubernetes.io/name: gitops-reverser control-plane: gitops-reverser diff --git a/config/webhook/kustomization.yaml b/config/webhook/kustomization.yaml deleted file mode 100644 index 19451dfe..00000000 --- a/config/webhook/kustomization.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: -- validating-webhook.yaml diff --git a/config/webhook/validating-webhook.yaml b/config/webhook/validating-webhook.yaml deleted file mode 100644 index 37830cbc..00000000 --- a/config/webhook/validating-webhook.yaml +++ /dev/null @@ -1,120 +0,0 @@ -apiVersion: admissionregistration.k8s.io/v1 -kind: ValidatingWebhookConfiguration -metadata: - annotations: - cert-manager.io/inject-ca-from: sut/gitops-reverser-admission-server-cert - labels: - app.kubernetes.io/managed-by: kustomize - app.kubernetes.io/name: gitops-reverser - name: gitops-reverser-validating-webhook -# This ValidatingWebhookConfiguration is part of the kustomize SUT overlay (config/). -# It carries two webhooks served by the one admission server (port 9443, gated behind -# --admission-webhook): -# -# 1. validate-all.configbutler.ai — the broad '*' observer below. E2E-ONLY: -# it is intentionally NOT packaged in the Helm chart. The handler always allows -# (pure observation, see internal/webhook/admission_allow_handler.go); when a real, -# rejecting policy becomes product behavior, the chart must gain matching packaging -# and this note must change. -# 2. validate-operator-types.configbutler.ai — captures the submitter of our own command -# kinds (a CommitRequest today) at admission. This one IS product behavior and is -# packaged in the chart (charts/.../templates/validate-operator-types-webhook.yaml); the -# entry here is the e2e SUT's copy. See -# docs/spec/commitrequest-admission-authorship.md. -# -# The cert-manager.io/inject-ca-from annotation injects the admission server CA bundle -# for every webhook in this configuration. -webhooks: -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: gitops-reverser-service - namespace: sut - path: /validate-all - port: 9443 - # Ignore (not Fail) is mandatory while this matches '*': the webhook backend is - # the gitops-reverser pod itself, so a Fail policy on core resources would - # deadlock the cluster at bootstrap (the apiserver could not admit the pod that - # serves the webhook). When a real, rejecting policy lands, switch to Fail only - # together with a namespaceSelector that excludes kube-system and this pod's own - # namespace. - failurePolicy: Ignore - matchPolicy: Equivalent - name: validate-all.configbutler.ai - # Broad match for TESTING and as the future-policy extension point: observe - # writes to all real (top-level) resources, mirroring the intent of the audit - # capture policy (test/e2e/cluster/audit/policy.yaml). Two honest gaps vs that - # policy: - # * A ValidatingWebhookConfiguration matches by inclusion only — it cannot - # express "everything EXCEPT events/pods/leases/...", so this set is a - # superset of the audit set. The handler always allows, so the extra - # matches are harmless (just more requests observed). - # * resources: ['*'] is top-level only and deliberately excludes subresources, - # mirroring the audit policy dropping */status and apps/*/scale noise. - rules: - - apiGroups: - - '*' - apiVersions: - - '*' - operations: - - CREATE - - UPDATE - - DELETE - resources: - - '*' - scope: '*' - sideEffects: None - # 1s, not the apiserver max of 30s. failurePolicy is Ignore, so on a slow or - # unreachable backend the request is admitted anyway after the timeout — a - # large value only adds latency to every cluster write, it buys no safety. - timeoutSeconds: 1 -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: gitops-reverser-service - namespace: sut - path: /validate-operator-types - port: 9443 - # Ignore, not Fail: a missed author capture must degrade to the committer, never - # reject a user's CommitRequest. The handler also returns Allowed even when the Redis - # write errors, so the command's success never depends on this webhook at all — - # Ignore only covers the webhook being entirely unreachable. - # - # It stays Ignore now that the endpoint can also REJECT a WatchRule, because that - # rejection is feedback rather than enforcement: the writer's own precondition refuses - # the same configuration whether or not this webhook ran. Fail here would make every - # WatchRule write depend on the operator being up, to gate something already gated. - failurePolicy: Ignore - matchPolicy: Equivalent - name: validate-operator-types.configbutler.ai - # Narrow: our own command kinds (one rule per command kind), plus the one config kind - # this endpoint validates. The submitter of a command is captured into the command-author - # Redis corner and read back by the CommitRequest controller with no wait; a WatchRule is - # checked against the GitTarget it names and may be rejected. - rules: - - apiGroups: - - configbutler.ai - apiVersions: - - v1alpha3 - operations: - - CREATE - resources: - - commitrequests - scope: Namespaced - # UPDATE as well as CREATE: a rule can grow a second sourceNamespace long after it was - # created, and that edit is the same mistake. - - apiGroups: - - configbutler.ai - apiVersions: - - v1alpha3 - operations: - - CREATE - - UPDATE - resources: - - watchrules - scope: Namespaced - # Redis write on real requests; nothing on dry-run (the handler honors NoneOnDryRun). - sideEffects: NoneOnDryRun - timeoutSeconds: 2 diff --git a/docs/INDEX.md b/docs/INDEX.md index e0453767..f6287a1a 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -103,8 +103,7 @@ misled. Full list in [`spec/README.md`](spec/README.md); the ones that carry a | [`gitpath-foreign-content-stringency.md`](spec/gitpath-foreign-content-stringency.md) | refusing a path that shadows foreign content | | [`unsupported-folder-refusal-plan.md`](spec/unsupported-folder-refusal-plan.md) | `GitPathAccepted`, and refusing what we cannot own | | [`commitrequest-design.md`](spec/commitrequest-design.md) | the CommitRequest window and its conditions | -| [`commitrequest-admission-authorship.md`](spec/commitrequest-admission-authorship.md) | how command submitters are captured and matched to commit windows | -| [`where-validation-lives.md`](spec/where-validation-lives.md) | schema → CEL → **the reconciler**; a webhook only for what exists solely at admission | +| [`where-validation-lives.md`](spec/where-validation-lives.md) | schema → CEL → **the reconciler**; no admission webhook ships | | [`e2e-serial-registry.md`](spec/e2e-serial-registry.md) | which e2e specs must run Serial, and why | ## What is being decided now — [`design/`](design/) diff --git a/docs/architecture.md b/docs/architecture.md index c6789ac7..8609dc80 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1189,12 +1189,10 @@ CommitDate: Mon Jun 30 12:00:05 2026 +0000 ### Wiring OIDC author claims - **Audit-path extractor**: [internal/queue/audit_event_parsing.go](../internal/queue/audit_event_parsing.go) -- **Admission-path extractor**: - [internal/webhook/validate_operator_types_handler.go](../internal/webhook/validate_operator_types_handler.go) - **Signature construction**: [internal/git/commit.go](../internal/git/commit.go) (`authorName` / `authorEmail`) The operator never talks to the identity provider. It reads two fixed keys from the request's `user.extra` -map: the audit attribution path and the CommitRequest admission path resolve the **same** two keys: +map, from the audit event that names the actor: | `user.extra` key | OIDC claim it carries | Used for | |---|---|---| @@ -1763,10 +1761,6 @@ unsafely: logged per-resource, counted in the resync summary as `placementSkippe ## CommitRequest finalize - **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) -- **Admission handler**: - [internal/webhook/validate_operator_types_handler.go](../internal/webhook/validate_operator_types_handler.go) -- **Admission author cache**: - [internal/queue/command_author_store.go](../internal/queue/command_author_store.go) A `CommitRequest` finalizes the open commit window for its GitTarget. The request author is resolved from the `/validate-operator-types` validating admission webhook, which captures the authenticated submitter at diff --git a/docs/components.md b/docs/components.md index 3174d653..5d3f20e6 100644 --- a/docs/components.md +++ b/docs/components.md @@ -137,7 +137,6 @@ Optional, and deliberately unable to affect *what* is written. It only names the | Fact transport | [`internal/queue/fact_stream.go`](../internal/queue/fact_stream.go) | Redis Streams by default, an in-process ring with `--author-attribution-transport=memory` (which frees attribution from Redis, but not the watch resume cursors) | | Fact index and follower | [`internal/queue/fact_index.go`](../internal/queue/fact_index.go) | a bounded, TTL'd in-process index, followed per type while at least one watch needs it | | Author resolver | [`internal/watch/author_resolver.go`](../internal/watch/author_resolver.go) | joins a watch event to a fact within a bounded grace window | -| Command author store | [`internal/queue/command_author_store.go`](../internal/queue/command_author_store.go) | the `CommitRequest` submitter captured at admission | ### Support and tooling diff --git a/docs/configuration.md b/docs/configuration.md index 8dd671b3..eab697ec 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -119,7 +119,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | -| `allowedCommitRequests` | `[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"Keep"}]` | The namespaces whose `CommitRequest`s may save this target's windows, and how much each may claim of a window whose writes name no author. See [who may save](#who-may-save-specallowedcommitrequests) | +| `allowedCommitRequests` | `[{"consumerNamespaces":{"from":"Same"}}]` | The namespaces whose `CommitRequest`s may save this target's windows. See [who may save](#who-may-save-specallowedcommitrequests) | | `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and save its windows. See [`RemoteSaveRequest`](#remotesaverequest) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | @@ -181,7 +181,6 @@ The references are immutable: to hold another `Application`, delete the applier | `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedCommitRequests` must admit this one | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | -| `window.author` | `Own` | Whose window to attach to: `Own`, or also claim a window whose writes name no author with `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) | | `window.attachTimeout` | `2s` | How long to wait for a window to attach to | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | diff --git a/docs/design/branch-worker-event-model.md b/docs/design/branch-worker-event-model.md index 244a2af7..9fdd013f 100644 --- a/docs/design/branch-worker-event-model.md +++ b/docs/design/branch-worker-event-model.md @@ -260,7 +260,7 @@ collect entries older than 15 minutes when no attach is queued. Collection is no durable receipt proving that status was written. Authorship has a separate lifetime. The -[admission author store](../../internal/queue/command_author_store.go) retains records for an hour; +admission author store (since removed) retained records for an hour; the `CommitRequest` object cannot reconstruct the authenticated submitter by itself. Recovering a previously accepted save must preserve its resolved attribution, including an unresolved result. Re-running the lookup later can produce a different decision. diff --git a/docs/design/commitrequest-save-wait-options.md b/docs/design/commitrequest-save-wait-options.md index 73851628..bf33a53b 100644 --- a/docs/design/commitrequest-save-wait-options.md +++ b/docs/design/commitrequest-save-wait-options.md @@ -505,8 +505,8 @@ the participant's message on the commit. - [`../spec/commitrequest-design.md`](../spec/commitrequest-design.md), the current contract. The code depends on it, so each phase changes that page in the same commit. -- [`../spec/commitrequest-admission-authorship.md`](../spec/commitrequest-admission-authorship.md), - where the submitter comes from and why the lookup is present-or-never. +- `commitrequest-admission-authorship.md`, since removed with the admission webhook: where the + submitter came from and why the lookup was present-or-never. - [`../spec/watch-event-ordering-and-attribution-grace.md`](../spec/watch-event-ordering-and-attribution-grace.md), the head-of-line wait this page proposes to wait behind rather than shorten. - [`attribution-removal-wait-options.md`](attribution-removal-wait-options.md), the same shape of diff --git a/docs/design/support-boundary/admission-consent.md b/docs/design/support-boundary/admission-consent.md index 3416d1c2..b6395875 100644 --- a/docs/design/support-boundary/admission-consent.md +++ b/docs/design/support-boundary/admission-consent.md @@ -165,8 +165,7 @@ RBAC). This boundary is the same class of "no" as correctness — not lift­able Consent needs a synchronous surface, and admission is the only one: it is the sole point where the user's intent can be accepted or rejected **whole, before persistence** — the property the write-gating doc names as the real argument for the gate. The infrastructure exists (the operator -already runs admission webhooks, -[`validate_operator_types_handler.go`](../../../internal/webhook/validate_operator_types_handler.go)), +ran an admission webhook for `CommitRequest` authorship, since removed), and the preflight design specifies the baseline. Consent would add one branch: ```mermaid diff --git a/docs/finished/redis-key-schema-v3.md b/docs/finished/redis-key-schema-v3.md index ae2ad576..ee04bed7 100644 --- a/docs/finished/redis-key-schema-v3.md +++ b/docs/finished/redis-key-schema-v3.md @@ -5,7 +5,7 @@ > Status: **IMPLEMENTED** — 2026-06-29. The `v2 → v3` key-format bump landed for the live Redis families: > the audit author index, the watch-resume cursor, and the new command-author **store** (storage > mechanics only — the CommitRequest admission/controller rewrite in -> [commitrequest-admission-authorship.md](../spec/commitrequest-admission-authorship.md) is a separate round and +> `commitrequest-admission-authorship.md`, since removed, is a separate round and > is **not** wired up yet). Scope: (1) replace the opaque `attr` namespace with a top-level `author` > domain; (2) re-lay audit-sourced resource author facts as a readable, **object-grouped** hierarchy > (`:object::` + `:last`, plus a type-scoped `:rv:` escape hatch), carrying @@ -24,7 +24,6 @@ > `internal/queue/attribution_index.go`, > [`internal/queue/redis_store.go`](../../internal/queue/redis_store.go), > [`internal/watch/author_resolver.go`](../../internal/watch/author_resolver.go), -> [CommitRequest authorship from admission](../spec/commitrequest-admission-authorship.md), > [attribution](../spec/attribution.md), > [watch event ordering & attribution grace](../spec/watch-event-ordering-and-attribution-grace.md), > [watch-first ingestion architecture](watch-first-ingestion-architecture.md). diff --git a/docs/spec/README.md b/docs/spec/README.md index b1fc6b72..7ca32626 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -47,7 +47,6 @@ three lifecycle classes and was unreadable spread across them. The rows below st | [`scale-subresource-audit-rehydration.md`](scale-subresource-audit-rehydration.md) | history: the `/scale` field-patch design; a scale is now mirrored from the watch, and the field-patch writer is what remains | | [`commit-window-refactor.md`](commit-window-refactor.md) | one grouped commit = one (author, GitTarget) | | [`commitrequest-design.md`](commitrequest-design.md) | how a request binds to a same-actor commit window and reports its outcome | -| [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md) | a request's submitter comes from admission, present-or-never, apart from audit facts | | [`status-conditions-guide.md`](status-conditions-guide.md) | the conditions and reasons every kind reports | | [`gitpath-foreign-content-stringency.md`](gitpath-foreign-content-stringency.md) | a `GitTarget` owns its subtree and refuses foreign content; `.gittargetignore` | | [`unsupported-folder-refusal-plan.md`](unsupported-folder-refusal-plan.md) | history of the `GitPathAccepted` refusal of unsupported folder content | diff --git a/docs/spec/attribution.md b/docs/spec/attribution.md index 9b2b27a4..65985b1d 100644 --- a/docs/spec/attribution.md +++ b/docs/spec/attribution.md @@ -18,7 +18,6 @@ > [what attribution gives up, on purpose](#what-attribution-gives-up-on-purpose). > > Related: [`watch-event-ordering-and-attribution-grace.md`](watch-event-ordering-and-attribution-grace.md), -> [`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md), > [`../attribution-setup-guide.md`](../attribution-setup-guide.md), > [`../interpreting-metrics.md`](../interpreting-metrics.md). @@ -63,8 +62,8 @@ That is why `CommitRequest` has no `spec.author`, although one was asked for (co To commit under your own name, authenticate as yourself. Use your identity provider (OIDC, for example) so that the API server knows who you are. A request that cannot be attributed claims no -actor (`AuthorAttributed=False`); it never falls back to a name it was given. See -[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). +actor; it never falls back to a name it was given. A `CommitRequest`'s own author comes from its +audit event the same way, in [`commitrequest-design.md`](commitrequest-design.md). ## 1. Deletion is attributed at intent time diff --git a/docs/spec/commitrequest-admission-authorship.md b/docs/spec/commitrequest-admission-authorship.md deleted file mode 100644 index 9818f09f..00000000 --- a/docs/spec/commitrequest-admission-authorship.md +++ /dev/null @@ -1,68 +0,0 @@ -# CommitRequest authorship from admission - -> **spec** — current behaviour. The code depends on this document; change one, change the other. -> Index: [`../INDEX.md`](../INDEX.md) - -A `CommitRequest` is a command to close an existing GitTarget commit window. Its submitter is captured -by the `/validate-operator-types` validating admission webhook, while mirrored-resource attribution is -separately derived from kube-apiserver audit facts. The two paths answer different questions and must not -be conflated. - -## Contract - -| Concern | CommitRequest submitter | Mirrored-resource author | -|---|---|---| -| Source | validating admission request `userInfo` | post-persist audit fact joined to watch event | -| Key | CommitRequest UID | source provider, GVR, UID, and resourceVersion | -| Timing | written before the object persists | waits up to `--author-attribution-grace` after the watch event | -| Miss | request claims no actor | watch commit has the explicit unresolved author | - -The admission handler records the authenticated submitter in the Redis command-author store and always -allows the request. It is deliberately best-effort: a Redis failure or an unavailable webhook must not -reject a user's command. The controller reads the record when the persisted `CommitRequest` is first -reconciled. The result is **present-or-never**; waiting cannot create a record that admission did not write. - -## Attach and final Git author - -`AuthorAttributed=True` means admission captured the command submitter. The request can attach only to an -open window with that same named actor and GitTarget. - -`AuthorAttributed=False` (`CommitterFallback`) means capture ran but no admission record was available; -`AuthorCaptureDisabled` means capture was not configured. Neither is a failure and neither means the -eventual Git author is necessarily the configured committer. The request claims no actor and can attach only -to an unnamed window. That window determines the actual Git author: - -- configured-author mode or a replay/resync write: the configured committer; -- live attribution that ran but found no usable audit fact: - `unknown (attribution unresolved) `. - -The worker never closes another actor's window. If no matching window appears before `attachTimeout` -runs out, the request ends successfully with `Ready=True` and `NoWindow` or `WindowMismatch`. That is -`Pushed=False`, except for a `NoWindow` request with `whenNothingToCommit: CommitEmpty`, which records -its message in a pushed empty commit (`Pushed=True`). A `WindowMismatch` never falls back to one. - -## Conditions - -`AuthorAttributed` is binary and settled on the first reconcile; it has no audit-wait or `Unknown` state. - -| Condition | True | False | -|---|---|---| -| `AuthorAttributed` | `AttributedFromAdmission`: the command submitter was captured | `CommitterFallback`: capture ran but no command-author record; `AuthorCaptureDisabled`: capture is off; request claims no actor | -| `Pushed` | the attached window was committed and pushed | a benign no-commit or finalize failure | -| `Ready` | a pushed commit or benign no-commit | progress or a finalize failure | - -`Reconciling=True` is the normal in-progress state, with the phase the branch worker reports as its -reason: `Progressing` until the worker registers the request (`WaitingForWorker` while the GitTarget -has no branch worker yet), then `WaitingForWindow`, `CollectingWindow` and `WaitingForPush`. - -## Wiring - -The command-author store is wired when the admission webhook is enabled and Redis is configured. It is -independent of `--author-attribution`, which controls audit-backed authorship for live watched resources. -An installation may therefore have either, both, or neither source of author information. - -Related live contracts: - -- [architecture: CommitRequest Finalize](../architecture.md#commitrequest-finalize) -- [configuration: CommitRequest conditions](../configuration.md#when-there-is-nothing-to-commit) -- [status conditions guide](status-conditions-guide.md) diff --git a/docs/spec/commitrequest-design.md b/docs/spec/commitrequest-design.md index 46896136..7407e5cb 100644 --- a/docs/spec/commitrequest-design.md +++ b/docs/spec/commitrequest-design.md @@ -91,8 +91,7 @@ that: - a resolved live attribution fact uses the authenticated actor; and - a live attribution miss uses `unknown (attribution unresolved) `. -See [CommitRequest admission authorship](commitrequest-admission-authorship.md) for capture provenance and -[architecture: author identity](../architecture.md#author-and-committer-identity-in-git) for the three Git +See [architecture: author identity](../architecture.md#author-and-committer-identity-in-git) for the three Git author states. ## Lifecycle and outcomes diff --git a/docs/spec/where-validation-lives.md b/docs/spec/where-validation-lives.md index c003c44a..b121a68d 100644 --- a/docs/spec/where-validation-lives.md +++ b/docs/spec/where-validation-lives.md @@ -18,7 +18,7 @@ Validate at the **first** rung that can express the rule: **A validating admission webhook is not a rung on this ladder.** It is only correct when the information being validated *exists only at admission time* and is nowhere on the persisted -object. That is a narrow case, and we ship exactly one instance of it (below). +object. That is a narrow case, and GitOps Reverser ships no webhook at all (below). ## Why cross-object checks go to the reconciler, not a webhook @@ -48,23 +48,16 @@ the same rule: which namespace holds a route depends on every other provider car provider that loses it reports `Stalled=True` (`AuditRouteConflict`) on reconcile rather than being refused at admission. -## The one webhook we ship, and why it is not an exception to the rule +## No webhook ships -[`/validate-operator-types`](../../internal/webhook/validate_operator_types_handler.go#L27) -is **not a validation gate**. It captures the *submitter's identity* on a command kind -(`CommitRequest`) so a commit can be authored by a real Kubernetes user, and for a command it -**always allows**. Identity is the textbook case for admission: it exists in the `AdmissionRequest` -and on no persisted field, so no reconciler could recover it later. See -[`commitrequest-admission-authorship.md`](commitrequest-admission-authorship.md). - -The same endpoint carries one rejection, and it is feedback, not a gate: a `WatchRule` that would -bring a second source namespace to a `GitTarget` declaring `spec.serializeNamespace: false` is -rejected at the moment the mistake is made. The writer refuses the same configuration at the write -whether or not the webhook ran, which is why its `failurePolicy` is `Ignore`. This is the feedback -latency trade above, taken where a webhook is installed anyway, and never as the only gate. - -(`/validate-all`, in [`admission_allow_handler.go`](../../internal/webhook/admission_allow_handler.go#L17), -is an always-allow observation surface wired only by the e2e SUT. The chart does not ship it.) +GitOps Reverser installs no admission webhook. The one it had captured the creator of a +`CommitRequest` at admission, the textbook case for one: identity exists in the `AdmissionRequest` +and on no persisted field. It went once the audit pipeline could give a request's author the same +way it gives an edit's: from the request's own audit event, awaited from the same index with the +same grace ([`attribution.md`](attribution.md)). The same endpoint also rejected a `WatchRule` that +would bring a second source namespace to a `GitTarget` declaring `spec.serializeNamespace: false`. +That was feedback, never the gate: the writer refuses the configuration whether or not the webhook +ran, and the `GitTarget` reports it as `MultipleSourceNamespaces`. ## Applying this diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index fdc970ed..35ad4031 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -160,8 +160,7 @@ kinds: doc: How commits are phrased. See [commit messages](commit-messages.md) - path: allowedCommitRequests doc: >- - The namespaces whose `CommitRequest`s may save this target's windows, and how much each may - claim of a window whose writes name no author. See + The namespaces whose `CommitRequest`s may save this target's windows. See [who may save](#who-may-save-specallowedcommitrequests) - path: allowedRemoteSaveRequests default: none admitted @@ -315,10 +314,6 @@ kinds: doc: >- Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) - - path: window.author - doc: >- - Whose window to attach to: `Own`, or also claim a window whose writes name no author with - `SavedBy` or `TakeOver`. See [saving where writes name no author](#saving-where-writes-name-no-author) - path: window.attachTimeout doc: How long to wait for a window to attach to - path: window.idleTimeout diff --git a/hack/e2e/deploy-controller.sh b/hack/e2e/deploy-controller.sh index 500b8130..f26b36e6 100755 --- a/hack/e2e/deploy-controller.sh +++ b/hack/e2e/deploy-controller.sh @@ -75,8 +75,8 @@ else kubectl --context "${CTX}" -n "${NAMESPACE}" \ rollout restart "deployment/${deploy}" fi -# 300s: a fresh-namespace install waits on cert-manager issuing the admission -# cert before the pod can mount it, and CI runners near their disk-eviction +# 300s: a fresh-namespace install waits on cert-manager issuing the audit +# certs before the pod can mount them, and CI runners near their disk-eviction # threshold start pods slowly; 180s was too tight in the tail (CI run # 28623232380). On timeout, dump the state that explains *why* the pod never # became ready — node conditions expose DiskPressure, describe/events expose diff --git a/internal/authz/commitrequest_admission.go b/internal/authz/commitrequest_admission.go index 4d9ba182..ed14d7f2 100644 --- a/internal/authz/commitrequest_admission.go +++ b/internal/authz/commitrequest_admission.go @@ -12,23 +12,22 @@ import ( // CommitRequestAdmitted reports whether request may save the commit windows of target, the GitTarget // its spec.gitTargetRef names: a grant in target's spec.allowedCommitRequests admits the request's -// namespace, by the labels that namespace carries. The admission also says how much the request may -// claim of a window whose writes name no author. A non-NotFound read of the Namespace is returned as +// namespace, by the labels that namespace carries. A non-NotFound read of the Namespace is returned as // err, so the caller retries instead of refusing a request on a transient failure. func CommitRequestAdmitted( ctx context.Context, reader client.Reader, request *configv1alpha3.CommitRequest, target *configv1alpha3.GitTarget, -) (Decision, configv1alpha3.CommitRequestAdmission, error) { +) (Decision, error) { nsLabels, err := namespaceLabels(ctx, reader, request.Namespace) if err != nil { - return Decision{}, configv1alpha3.CommitRequestAdmission{}, err + return Decision{}, err } - admission, selErr := target.AdmitsCommitRequest(request.Namespace, nsLabels) - if !admission.Admitted { + admitted, selErr := target.AdmitsCommitRequest(request.Namespace, nsLabels) + if !admitted { return notAdmitted("GitTarget", client.ObjectKeyFromObject(target), target.DescribeAllowedCommitRequests(), - "spec.allowedCommitRequests", request.Namespace, selErr), admission, nil + "spec.allowedCommitRequests", request.Namespace, selErr), nil } - return Decision{Allowed: true}, admission, nil + return Decision{Allowed: true}, nil } diff --git a/internal/authz/commitrequest_admission_test.go b/internal/authz/commitrequest_admission_test.go index 193cf4c6..2bd4a1e1 100644 --- a/internal/authz/commitrequest_admission_test.go +++ b/internal/authz/commitrequest_admission_test.go @@ -18,8 +18,7 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" ) -// A GitTarget admits a CommitRequest by the namespace it is in and that namespace's labels, and gives it -// the most any admitting grant allows of a window whose writes name no author. +// A GitTarget admits a CommitRequest by the namespace it is in and that namespace's labels. func TestCommitRequestAdmitted(t *testing.T) { tenant := &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{ Name: testNS, Labels: map[string]string{"platform.example.com/tenant": "true"}, @@ -31,46 +30,32 @@ func TestCommitRequestAdmitted(t *testing.T) { Spec: configv1alpha3.GitTargetSpec{AllowedCommitRequests: grants}, } } - bySelector := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetCommitRequestGrant { - return configv1alpha3.GitTargetCommitRequestGrant{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{ - From: configv1alpha3.GrantFromSelector, - Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"platform.example.com/tenant": "true"}}, - }, - UnattributedWindows: policy, - } + bySelector := configv1alpha3.GitTargetCommitRequestGrant{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{ + From: configv1alpha3.GrantFromSelector, + Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"platform.example.com/tenant": "true"}}, + }, } - all := func(policy configv1alpha3.UnattributedWindowPolicy) configv1alpha3.GitTargetCommitRequestGrant { - return configv1alpha3.GitTargetCommitRequestGrant{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, - UnattributedWindows: policy, - } + all := configv1alpha3.GitTargetCommitRequestGrant{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, } reader := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithObjects(tenant).Build() - decision, _, err := CommitRequestAdmitted(context.Background(), reader, request, target()) + decision, err := CommitRequestAdmitted(context.Background(), reader, request, target()) require.NoError(t, err) assert.False(t, decision.Allowed, "the default admits only the GitTarget's own namespace") assert.Equal(t, ReasonNamespaceNotAuthorized, decision.Reason) assert.Contains(t, decision.Message, "spec.allowedCommitRequests") - decision, admission, err := CommitRequestAdmitted(context.Background(), reader, request, - target(bySelector(configv1alpha3.UnattributedWindowsSavedBy))) + decision, err = CommitRequestAdmitted(context.Background(), reader, request, target(bySelector)) require.NoError(t, err) assert.True(t, decision.Allowed, "the namespace's labels match") - assert.Equal(t, configv1alpha3.UnattributedWindowsSavedBy, admission.UnattributedWindows) - - _, admission, err = CommitRequestAdmitted(context.Background(), reader, request, - target(all(configv1alpha3.UnattributedWindowsKeep), bySelector(configv1alpha3.UnattributedWindowsTakeOver))) - require.NoError(t, err) - assert.Equal(t, configv1alpha3.UnattributedWindowsTakeOver, admission.UnattributedWindows, - "the most any admitting grant allows") failing := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithInterceptorFuncs(interceptor.Funcs{ Get: func(context.Context, client.WithWatch, client.ObjectKey, client.Object, ...client.GetOption) error { return errors.New("apiserver unavailable") }, }).Build() - _, _, err = CommitRequestAdmitted(context.Background(), failing, request, target(all(""))) + _, err = CommitRequestAdmitted(context.Background(), failing, request, target(all)) require.Error(t, err, "a namespace that cannot be read is retried, never refused") } diff --git a/internal/controller/commitrequest_apply_test.go b/internal/controller/commitrequest_apply_test.go index b8adbd8c..22496d66 100644 --- a/internal/controller/commitrequest_apply_test.go +++ b/internal/controller/commitrequest_apply_test.go @@ -27,7 +27,7 @@ func catchingUp(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest func TestCommitRequestReconcile_ReportsTheApplyItAskedFor(t *testing.T) { c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-1") assert.Zero(t, f.applyLookups, "the save resolves first") @@ -71,7 +71,7 @@ func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() f.apply, f.applyKnown = tc.apply, true - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-1") reconcileCommitRequest(t, r, "save-1") @@ -88,7 +88,7 @@ func TestCommitRequestReconcile_ASettledApplyWithoutAnApply(t *testing.T) { func TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { c := newCommitRequestClient(t, nil, catchingUp(newCommitRequest("save-1"))) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-1") reconcileCommitRequest(t, r, "save-1") @@ -108,7 +108,7 @@ func TestCommitRequestReconcile_AnApplyNobodyKnowsIsLost(t *testing.T) { func TestCommitRequestReconcile_NoCatchUpReportsNoApply(t *testing.T) { c := newCommitRequestClient(t, nil, newCommitRequest("save-1")) f := committedFinalizer() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-1") reconcileCommitRequest(t, r, "save-1") diff --git a/internal/controller/commitrequest_attribution_agreement_test.go b/internal/controller/commitrequest_attribution_agreement_test.go deleted file mode 100644 index 80b44bf5..00000000 --- a/internal/controller/commitrequest_attribution_agreement_test.go +++ /dev/null @@ -1,152 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package controller - -import ( - "context" - "testing" - "time" - - "github.com/go-logr/logr" - "github.com/stretchr/testify/assert" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime/schema" - k8stypes "k8s.io/apimachinery/pkg/types" - - configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/queue" - "github.com/ConfigButler/gitops-reverser/internal/watch" -) - -// absentLookup is an attribution index that never has a fact — the production shape of -// "attribution is enabled but nothing matched within the grace". -type absentLookup struct{} - -func (absentLookup) Await( - _ context.Context, - _ queue.FactQuery, - _ time.Duration, -) queue.AuthorResolution { - return queue.AuthorResolution{Result: queue.AttributionAbsent} -} - -// windowOutcomeAttributionOff is the outcome a commit window carries in configured-author mode. -// It is read off a zero git.Event on purpose rather than written as a constant: that IS the -// production value, because watch.Manager.attachAuthor returns early without assigning -// Attribution when the Manager's AuthorResolver is nil. -func windowOutcomeAttributionOff() git.AttributionOutcome { - return git.Event{}.Attribution -} - -// windowOutcomeAttributionMissed drives the real resolver over a lookup that has no fact, which -// is what a live watch event gets when attribution is on and nothing matched in the grace. -func windowOutcomeAttributionMissed(t *testing.T) git.AttributionOutcome { - t.Helper() - _, outcome := watch.NewAuthorResolver(absentLookup{}, 0, logr.Discard(), nil).ResolveAuthor( - context.Background(), - watch.AuthorQuery{ - AuditRoute: "default", - GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, - UID: k8stypes.UID("uid-1"), - ResourceVersion: "101", - ExactCapable: true, - }, - ) - return outcome -} - -// TestCommitRequest_OutcomesAgree is the cross-subsystem half of the P0 regression test. -// -// The gap that let the blocker through was that every existing test set both sides of the -// comparison to the same literal, so nothing ever checked that the two INDEPENDENT producers -// emit compatible values. This drives each side from its real source — the controller's -// gitOutcome projection, and the watch package's actual resolver / actual zero event — and -// asserts they agree on the only thing matching may depend on across that boundary. -// -// The default deployment is the first row: --admission-webhook defaults to false, so the -// controller emits attributionNotAttempted, while --author-attribution=false leaves the window's -// outcome at the zero value. When those two were different strings, no CommitRequest could -// attach to any window and the user's commit message was silently dropped. -func TestCommitRequest_OutcomesAgree(t *testing.T) { - tests := []struct { - name string - requestSide commitRequestAttribution - windowSide git.AttributionOutcome - wantNamesActor bool - }{ - { - name: "default deployment: webhook off, attribution off", - requestSide: attributionNotAttempted, - windowSide: windowOutcomeAttributionOff(), - }, - { - name: "webhook off, attribution on but missed", - requestSide: attributionNotAttempted, - windowSide: windowOutcomeAttributionMissed(t), - }, - { - name: "webhook on but no record, attribution off", - requestSide: attributionCommitter, - windowSide: windowOutcomeAttributionOff(), - }, - { - name: "webhook on but no record, attribution on but missed", - requestSide: attributionCommitter, - windowSide: windowOutcomeAttributionMissed(t), - }, - } - - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - requestOutcome := tc.requestSide.gitOutcome() - assert.Equal(t, tc.wantNamesActor, requestOutcome.NamesActor()) - assert.Equal(t, requestOutcome.NamesActor(), tc.windowSide.NamesActor(), - "the CommitRequest side produced %q and the window side produced %q; they must "+ - "agree on whether an actor was named, or the request cannot attach and the "+ - "user's commit message is silently dropped", - requestOutcome, tc.windowSide) - }) - } -} - -// TestAttributionOutcome_OffLeavesZeroOutcome states the invariant the default deployment rests on, -// separately from the agreement table so a regression names itself precisely. -func TestAttributionOutcome_OffLeavesZeroOutcome(t *testing.T) { - assert.Equal(t, git.AttributionNotAttempted, windowOutcomeAttributionOff(), - "configured-author mode must leave the event's outcome at AttributionNotAttempted") - assert.Equal(t, git.AttributionNotAttempted, attributionNotAttempted.gitOutcome(), - "a CommitRequest with command-author capture off must carry AttributionNotAttempted") - assert.Equal(t, git.AttributionUnresolved, windowOutcomeAttributionMissed(t), - "attribution that ran and found nothing must be AttributionUnresolved, not the zero value") -} - -// TestCommitRequestAttribution_GitOutcome keeps every controller outcome's projection explicit, -// including the defensive zero value. -func TestCommitRequestAttribution_GitOutcome(t *testing.T) { - tests := []struct { - name string - input commitRequestAttribution - want git.AttributionOutcome - }{ - {name: "unset is not attempted", input: attributionUnset, want: git.AttributionNotAttempted}, - {name: "admission capture is resolved", input: attributionFromAdmission, want: git.AttributionResolved}, - {name: "admission miss is unresolved", input: attributionCommitter, want: git.AttributionUnresolved}, - {name: "capture disabled is not attempted", input: attributionNotAttempted, want: git.AttributionNotAttempted}, - } - - for _, tc := range tests { - t.Run(tc.name, func(t *testing.T) { - assert.Equal(t, tc.want, tc.input.gitOutcome()) - }) - } -} - -// TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled keeps an unset decision visible and safe. -func TestCommitRequestAttribution_UnsetStatusIsCaptureDisabled(t *testing.T) { - var request configv1alpha3.CommitRequest - - setCommitRequestAttributed(&request, attributionUnset) - - requireCondition(t, request, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) -} diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index cf622506..e7b766b5 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -10,9 +10,7 @@ import ( "github.com/go-logr/logr" apierrors "k8s.io/apimachinery/pkg/api/errors" - apimeta "k8s.io/apimachinery/pkg/api/meta" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -21,7 +19,6 @@ import ( configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/authz" "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/queue" ) // CommitRequestFinalizer is the EventRouter seam the reconciler drives, using the @@ -45,16 +42,6 @@ type CommitRequestFinalizer interface { LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) } -// CommandAuthorLookup resolves the author of a CommitRequest from the submitter -// captured at admission by the validate-operator-types webhook, keyed by the persisted -// object's UID. *queue.CommandAuthorStore satisfies it without adaptation. The lookup -// is present-or-never (docs/spec/commitrequest-admission-authorship.md): a miss is -// immediate and final — the webhook is not configured (or a best-effort write missed) — -// and the request claims no actor with no wait. -type CommandAuthorLookup interface { - LookupCommandAuthor(ctx context.Context, uid types.UID) (queue.CommandAuthor, bool) -} - // windowMismatchMessage explains the author-bound refusal: an open window // existed but was not this requester's, so it was deliberately left alone. const saveInProgressMessage = "an earlier save of the same author on this GitTarget was still waiting for " + @@ -151,21 +138,12 @@ func (w requestWindow) resolveTimeout() time.Duration { // NoOpenWindow that never happened.) // // CommitRequestReconciler drives a CommitRequest through its state machine -// (docs/spec/commitrequest-design.md and -// docs/spec/commitrequest-admission-authorship.md): -// -// 1. ATTRIBUTE — a single synchronous read of the submitter captured at admission -// (present-or-never). A hit names that submitter as the author -// (AuthorAttributed=True); a miss claims no actor immediately -// (AuthorAttributed=False). There is no wait and no requeue for the -// author: the record is written before the object is visible, so waiting cannot -// help. -// 2. ATTACH + POLL — the instant the author is settled, send the attach to the -// GitTarget's worker, which attaches the request to the author's window with its -// message and timers, and poll the outcome. The attach wait is anchored at the worker's -// registration, so there is no controller-side delay. While it polls, the controller -// reports the phase the worker reports. No window before the attach deadline resolves -// NoWindow; a window belonging to someone else stays open for its own author. +// (docs/spec/commitrequest-design.md): it sends the attach to the GitTarget's worker, which +// attaches the request to the author's window with its message and timers, and polls the +// outcome. The attach wait is anchored at the worker's registration, so there is no +// controller-side delay. While it polls, the controller reports the phase the worker reports. No +// window before the attach deadline resolves NoWindow; a window belonging to someone else stays +// open for its own author. type CommitRequestReconciler struct { client.Client @@ -177,12 +155,8 @@ type CommitRequestReconciler struct { APIReader client.Reader // Finalizer attaches the request to the author-bound open window and reports - // its outcome; AuthorLookup resolves the submitter captured at admission. When - // AuthorLookup is nil (the validate-operator-types webhook is disabled), requests - // claim no actor immediately, with AuthorAttributed=False. The attached window - // determines the eventual Git author. - Finalizer CommitRequestFinalizer - AuthorLookup CommandAuthorLookup + // its outcome. The attached window determines the eventual Git author. + Finalizer CommitRequestFinalizer // TTL is how long a request is kept once it finishes. It is written onto the request as // CommitRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. @@ -229,17 +203,12 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques return ctrl.Result{}, err } - // 1. ATTRIBUTE: settle the request's actor synchronously (present-or-never). - // A hit names the admission submitter; a miss claims no actor. Either way the - // decision is final — there is no wait and no requeue for the author. - author, attribution := r.attributeAuthor(ctx, commitRequest) - - claim, admitted, refusal, err := r.admitAtFirstSight(ctx, log, commitRequest, attribution) + admitted, refusal, err := r.admitAtFirstSight(ctx, log, commitRequest) if err != nil || admitted == commitRequestRefused { return ctrl.Result{}, err } - if err := r.stampFirstSightConditions(ctx, commitRequest, attribution); err != nil { + if err := r.stampFirstSightConditions(ctx, commitRequest); err != nil { return ctrl.Result{}, err } @@ -249,7 +218,7 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques if err := git.ValidateLiteralCommitMessage(commitRequest.Spec.Message); err != nil { // The fallback identity: this fails before ServiceCommitRequest, so nothing has // confirmed the named GitTarget exists. - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, attribution, + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, commitRequestTarget{}) return ctrl.Result{}, nil } @@ -262,8 +231,7 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques Namespace: commitRequest.Namespace, Name: commitRequest.Name, UID: string(commitRequest.UID), - Author: author.Author, - Attribution: attribution.gitOutcome(), + Attribution: git.AttributionNotAttempted, GitTargetName: commitRequest.GitTargetKey().Name, GitTargetNamespace: commitRequest.GitTargetKey().Namespace, GitTargetUID: commitRequest.Status.GitTargetUID, @@ -274,12 +242,11 @@ func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Reques MaxDuration: window.maxDuration, CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, - Unattributed: claim, } if admitted == commitRequestReplaced { - return r.refuseOnceUnheld(ctx, log, req, commitRequest, attribution, attach, refusal) + return r.refuseOnceUnheld(ctx, log, req, commitRequest, attach, refusal) } - return r.sendCommitRequest(ctx, log, req, commitRequest, attribution, attach, + return r.sendCommitRequest(ctx, log, req, commitRequest, attach, time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout(), admitted == commitRequestTargetMissing) } @@ -292,7 +259,6 @@ func (r *CommitRequestReconciler) sendCommitRequest( log logr.Logger, req ctrl.Request, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, attach git.AttachCommitRequest, pastTheBound, targetMissing bool, ) (ctrl.Result, error) { @@ -300,11 +266,11 @@ func (r *CommitRequestReconciler) sendCommitRequest( case pastTheBound: // Past the bound the controller only withdraws: an attach sent now would sit behind the // withdraw on the worker's FIFO and could register the request after it was cancelled. - return r.withdrawCommitRequest(ctx, log, req, commitRequest, attribution, attach) + return r.withdrawCommitRequest(ctx, log, req, commitRequest, attach) case targetMissing: - return r.pollCommitRequest(ctx, commitRequest, attribution, git.PhaseWaitingForWorker, "") + return r.pollCommitRequest(ctx, commitRequest, git.PhaseWaitingForWorker, "") } - return r.attachCommitRequest(ctx, log, req, commitRequest, attribution, attach) + return r.attachCommitRequest(ctx, log, req, commitRequest, attach) } // commitRequestAdmission is how admitAtFirstSight left a request. @@ -333,38 +299,37 @@ func (r *CommitRequestReconciler) admitAtFirstSight( ctx context.Context, log logr.Logger, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, -) (git.UnattributedClaim, commitRequestAdmission, error, error) { - claim, targetUID, err := r.admitCommitRequest(ctx, commitRequest) +) (commitRequestAdmission, error, error) { + targetUID, err := r.admitCommitRequest(ctx, commitRequest) recorded := commitRequest.Status.GitTargetUID switch { case err == nil && targetUID == "": if recorded != "" { // Its GitTarget is gone since it admitted the request: the attach path reports that. - return claim, commitRequestAdmitted, nil, nil + return commitRequestAdmitted, nil, nil } - return claim, commitRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, commitRequest, attribution) + return commitRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, commitRequest) case err == nil: if recorded == targetUID { - return claim, commitRequestAdmitted, nil, nil + return commitRequestAdmitted, nil, nil } // Recorded before any attach is sent for this GitTarget, so a later refusal can tell the two // apart. At first sight the in-progress stamp that follows writes it. commitRequest.Status.GitTargetUID = targetUID if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) == nil { - return claim, commitRequestAdmitted, nil, nil + return commitRequestAdmitted, nil, nil } - return claim, commitRequestAdmitted, nil, r.Status().Update(ctx, commitRequest) - case !errors.Is(err, errCommitRequestNotAllowed) && !errors.Is(err, errTakeOverNotAllowed): - return claim, commitRequestAdmitted, nil, err + return commitRequestAdmitted, nil, r.Status().Update(ctx, commitRequest) + case !errors.Is(err, errCommitRequestNotAllowed): + return commitRequestAdmitted, nil, err case recorded == targetUID: - return git.UnattributedClaimNone, commitRequestAdmitted, nil, nil + return commitRequestAdmitted, nil, nil case recorded != "": - return git.UnattributedClaimNone, commitRequestReplaced, err, nil + return commitRequestReplaced, err, nil } - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, attribution, + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, resolvedCommitRequestTarget(commitRequest)) - return claim, commitRequestRefused, nil, nil + return commitRequestRefused, nil, nil } // refuseOnceUnheld fails a request that a replacement of its GitTarget refuses. The GitTarget it @@ -376,7 +341,6 @@ func (r *CommitRequestReconciler) refuseOnceUnheld( log logr.Logger, req ctrl.Request, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, attach git.AttachCommitRequest, refusal error, ) (ctrl.Result, error) { @@ -386,93 +350,46 @@ func (r *CommitRequestReconciler) refuseOnceUnheld( log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", "name", req.NamespacedName, "err", withdrawErr.Error()) } - return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) } finalizeErr := result.Err if errors.Is(finalizeErr, git.ErrCommitRequestWithdrawn) { result, finalizeErr = git.FinalizeResult{}, refusal } - r.writeTerminalStatus(ctx, log, commitRequest, result, finalizeErr, attribution, + r.writeTerminalStatus(ctx, log, commitRequest, result, finalizeErr, resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } -// errTakeOverNotAllowed is a request that asks to claim more of a window whose writes name no author -// than the grants admitting it allow. It is terminal: the spec is immutable. -var errTakeOverNotAllowed = errors.New("spec.window.author asks for more than the GitTarget allows") - // errCommitRequestNotAllowed is a request whose GitTarget's spec.allowedCommitRequests admits no grant for // its namespace. It is terminal: the spec is immutable, so a later grant reaches only a new request. var errCommitRequestNotAllowed = errors.New("the GitTarget does not admit CommitRequests from this namespace") -// admitCommitRequest decides whether the request may save its GitTarget's windows, and what it may claim -// of a window whose writes name no author (git.AttachCommitRequest.Unattributed). -// -// A grant in the GitTarget's spec.allowedCommitRequests must admit the request's namespace, or the -// request is errCommitRequestNotAllowed. It then claims what spec.window.author asks for, and only while -// the GitTarget's KubeProvider has never received an audit fact: its AuditFactsReceived condition is -// not True, so no write watched on that cluster can name its author. On a cluster that has sent one, a -// write naming no author means its fact went missing, and nothing is claimed. A request that asks for -// more than the admitting grants' unattributedWindows allow is errTakeOverNotAllowed. +// admitCommitRequest decides whether the request may save its GitTarget's windows: a grant in the +// GitTarget's spec.allowedCommitRequests must admit the request's namespace, or the request is +// errCommitRequestNotAllowed. It returns the UID of the GitTarget that decided. // // A GitTarget that does not exist admits nothing yet, and is reported with an empty UID. Any other // read error is returned as is, so the request is retried rather than refused. func (r *CommitRequestReconciler) admitCommitRequest( ctx context.Context, commitRequest *configbutleraiv1alpha3.CommitRequest, -) (git.UnattributedClaim, string, error) { +) (string, error) { var target configbutleraiv1alpha3.GitTarget if err := r.Get(ctx, commitRequest.GitTargetKey(), &target); err != nil { if apierrors.IsNotFound(err) { - return git.UnattributedClaimNone, "", nil + return "", nil } - return git.UnattributedClaimNone, "", fmt.Errorf("read GitTarget %s: %w", commitRequest.GitTargetKey(), err) + return "", fmt.Errorf("read GitTarget %s: %w", commitRequest.GitTargetKey(), err) } - uid := string(target.UID) - claim, err := r.claimOn(ctx, commitRequest, &target) - return claim, uid, err -} - -// claimOn is admitCommitRequest for a GitTarget that exists. -func (r *CommitRequestReconciler) claimOn( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - target *configbutleraiv1alpha3.GitTarget, -) (git.UnattributedClaim, error) { - decision, admission, err := authz.CommitRequestAdmitted(ctx, r.Client, commitRequest, target) + decision, err := authz.CommitRequestAdmitted(ctx, r.Client, commitRequest, &target) if err != nil { - return git.UnattributedClaimNone, err + return string(target.UID), err } if !decision.Allowed { - return git.UnattributedClaimNone, fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message) - } - ask := configbutleraiv1alpha3.WindowAuthorOwn - if window := commitRequest.Spec.Window; window != nil && window.Author != "" { - ask = window.Author - } - if ask == configbutleraiv1alpha3.WindowAuthorOwn { - return git.UnattributedClaimNone, nil + return string(target.UID), fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message) } - if !admission.UnattributedWindows.Allows(ask) { - return git.UnattributedClaimNone, fmt.Errorf("%w: spec.window.author is %q, and the grants of GitTarget %s "+ - "that admit namespace %q allow unattributedWindows %q", errTakeOverNotAllowed, ask, - commitRequest.GitTargetKey(), commitRequest.Namespace, admission.UnattributedWindows) - } - key := target.KubeProviderKey() - if key.Name == "" { - return git.UnattributedClaimNone, nil - } - var provider configbutleraiv1alpha3.KubeProvider - if err := r.Get(ctx, key, &provider); err != nil { - return git.UnattributedClaimNone, nil //nolint:nilerr // an unread provider claims nothing - } - if apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) { - return git.UnattributedClaimNone, nil - } - if ask == configbutleraiv1alpha3.WindowAuthorTakeOver { - return git.UnattributedClaimTakeOver, nil - } - return git.UnattributedClaimSavedBy, nil + return string(target.UID), nil } // attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then @@ -483,7 +400,6 @@ func (r *CommitRequestReconciler) attachCommitRequest( log logr.Logger, req ctrl.Request, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, attach git.AttachCommitRequest, ) (ctrl.Result, error) { result, resolved, serviceErr := r.Finalizer.ServiceCommitRequest(ctx, attach) @@ -492,14 +408,14 @@ func (r *CommitRequestReconciler) attachCommitRequest( log.V(1).Info("CommitRequest attach not yet serviceable; will retry", "name", req.NamespacedName, "err", serviceErr.Error()) } - return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) } if result.Err != nil { log.Error(result.Err, "CommitRequest finalize failed", "gitTarget", commitRequest.GitTargetKey().String(), "name", req.NamespacedName) } - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, + r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } @@ -513,7 +429,6 @@ func (r *CommitRequestReconciler) withdrawCommitRequest( log logr.Logger, req ctrl.Request, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, attach git.AttachCommitRequest, ) (ctrl.Result, error) { result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) @@ -523,13 +438,13 @@ func (r *CommitRequestReconciler) withdrawCommitRequest( "name", req.NamespacedName, "err", withdrawErr.Error()) } // Not answered yet, or the worker holds the request: keep polling. - return r.pollCommitRequest(ctx, commitRequest, attribution, result.Phase, result.Held) + return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) } if !errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { // The worker resolved it on its own before the withdraw: report that outcome, as the // attach path would have. - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, attribution, + r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, resolvedCommitRequestTarget(commitRequest)) return ctrl.Result{}, nil } @@ -550,7 +465,7 @@ func (r *CommitRequestReconciler) withdrawCommitRequest( } log.Info("CommitRequest was not acted on within the safety window; withdrawn and failed closed", "name", req.NamespacedName, "reason", message) - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, errors.New(message), attribution, target) + r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, errors.New(message), target) return ctrl.Result{}, nil } @@ -558,28 +473,26 @@ func (r *CommitRequestReconciler) withdrawCommitRequest( func (r *CommitRequestReconciler) pollCommitRequest( ctx context.Context, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, phase git.CommitRequestPhase, held string, ) (ctrl.Result, error) { - if err := r.recordWorkerPhase(ctx, commitRequest, attribution, phase, held); err != nil { + if err := r.recordWorkerPhase(ctx, commitRequest, phase, held); err != nil { return ctrl.Result{}, err } return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil } // stampFirstSightConditions stamps the still-running conditions the first time a request is seen, -// so the object reports its progress (kstatus InProgress) and AuthorAttributed is settled -// immediately. A disabled controller returns before this, so it never stamps. +// so the object reports its progress (kstatus InProgress). A disabled controller returns before +// this, so it never stamps. func (r *CommitRequestReconciler) stampFirstSightConditions( ctx context.Context, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, ) error { if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) != nil { return nil } - markCommitRequestProgressing(commitRequest, attribution, "", "") + markCommitRequestProgressing(commitRequest, "", "") if err := r.Status().Update(ctx, commitRequest); err != nil { return err } @@ -625,37 +538,6 @@ func (r *CommitRequestReconciler) refusePrunedGitTargetRef( return true, nil } -// attributeAuthor settles the commit author with a single synchronous lookup of the -// submitter captured at admission (present-or-never). It never waits: a nil -// AuthorLookup (the validate-operator-types webhook is disabled) or a miss both claim no -// actor immediately. The miss case is final — the record is written before the object is -// visible, so there is no asynchronous arrival to wait for. -// -// The lookup result is logged at Info: it is the counterpart to the admission handler's -// "recorded command author" line, so a hit/miss pair makes the whole capture→read path -// legible (the first thing to check when a request unexpectedly claims no actor). -func (r *CommitRequestReconciler) attributeAuthor( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (queue.CommandAuthor, commitRequestAttribution) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - if r.AuthorLookup == nil { - log.Info("command-author lookup disabled (validate-operator-types webhook off); request claims no actor", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) - // Capture is off, so nothing was attempted — distinct from a capture that ran and - // found no record, which is what attributionCommitter now means. - return queue.CommandAuthor{}, attributionNotAttempted - } - if author, ok := r.AuthorLookup.LookupCommandAuthor(ctx, commitRequest.UID); ok { - log.Info("command author resolved from admission record", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID, "author", author.Author) - return author, attributionFromAdmission - } - log.Info("no admission command-author record found; request claims no actor", - "name", client.ObjectKeyFromObject(commitRequest), "uid", commitRequest.UID) - return queue.CommandAuthor{}, attributionCommitter -} - // recordWorkerPhase reports the phase the worker says the request is in. It writes only on a // change, so polling does not re-write status every interval. An empty phase — the worker has not // registered the request yet — keeps whatever is shown, so the controller never reports a phase @@ -663,14 +545,13 @@ func (r *CommitRequestReconciler) attributeAuthor( func (r *CommitRequestReconciler) recordWorkerPhase( ctx context.Context, commitRequest *configbutleraiv1alpha3.CommitRequest, - attribution commitRequestAttribution, phase git.CommitRequestPhase, held string, ) error { if progressRecorded(commitRequest, phase, held) { return nil } - markCommitRequestProgressing(commitRequest, attribution, phase, held) + markCommitRequestProgressing(commitRequest, phase, held) return r.Status().Update(ctx, commitRequest) } @@ -840,7 +721,6 @@ func (r *CommitRequestReconciler) writeTerminalStatus( commitRequest *configbutleraiv1alpha3.CommitRequest, result git.FinalizeResult, finalizeErr error, - attribution commitRequestAttribution, target commitRequestTarget, ) { expectedUID := commitRequest.UID @@ -856,7 +736,7 @@ func (r *CommitRequestReconciler) writeTerminalStatus( current := commitRequest for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { - applyFinalizeResultToStatus(current, result, finalizeErr, attribution) + applyFinalizeResultToStatus(current, result, finalizeErr) err := r.Status().Update(ctx, current) if err == nil { @@ -871,7 +751,6 @@ func (r *CommitRequestReconciler) writeTerminalStatus( "ready", commitRequestConditionStatus(current, ConditionTypeReady), "reason", readyReason, "message", readyMessage, - "authorAttributed", commitRequestConditionStatus(current, ConditionTypeAuthorAttributed), "pushed", commitRequestConditionStatus(current, ConditionTypePushed), "branch", current.Status.Branch, "commit", current.Status.Commit, diff --git a/internal/controller/commitrequest_controller_test.go b/internal/controller/commitrequest_controller_test.go index 041a4624..6ee4e7ce 100644 --- a/internal/controller/commitrequest_controller_test.go +++ b/internal/controller/commitrequest_controller_test.go @@ -18,11 +18,9 @@ import ( var _ = Describe("CommitRequest controller", func() { const namespace = "default" - // The suite registers the reconciler in attributed-author mode (a non-nil AuthorLookup - // that never resolves) with a Finalizer that never resolves, so these specs cover - // the in-progress stamp and the terminal short-circuit only; the full - // attribute → attach → terminal flow and the configured-author path are covered by - // the unit tests in commitrequest_controller_unit_test.go. + // The suite registers the reconciler with a Finalizer that never resolves, so these specs + // cover the in-progress stamp and the terminal short-circuit only; the full attach → + // terminal flow is covered by the unit tests in commitrequest_controller_unit_test.go. It("stamps a freshly created CommitRequest with in-progress conditions", func() { commitRequest := &configbutleraiv1alpha3.CommitRequest{ ObjectMeta: metav1.ObjectMeta{ diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index ccdd2a07..9fbf94a4 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -26,7 +26,6 @@ import ( configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/queue" ) // fakeFinalizer records the reconciler's attach calls and replies with a canned @@ -78,26 +77,6 @@ func (f *fakeFinalizer) ServiceCommitRequest( return f.result, f.resolved, f.err } -// fakeAuthorLookup is the command-author source stub: found=false models a miss -// (the validate-operator-types webhook is not configured, or a best-effort write missed), -// which the controller resolves to the committer immediately — present-or-never. -type fakeAuthorLookup struct { - author queue.CommandAuthor - found bool - calls int -} - -func (f *fakeAuthorLookup) LookupCommandAuthor( - _ context.Context, _ types.UID, -) (queue.CommandAuthor, bool) { - f.calls++ - return f.author, f.found -} - -func attributedAlice() *fakeAuthorLookup { - return &fakeAuthorLookup{author: queue.CommandAuthor{Author: "alice"}, found: true} -} - // newCommitRequest builds a request created just now, inside its safety window, as the API server // would stamp it. pastTheBound ages one beyond it. func newCommitRequest(name string) *configv1alpha3.CommitRequest { @@ -235,14 +214,13 @@ func TestCommitRequestReconcile_Committed(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-1") got := fetchCommitRequest(t, c, "save-1") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) - requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionFalse, "") requireCondition(t, got, ConditionTypeStalled, metav1.ConditionFalse, "") assert.Equal(t, "abc123", got.Status.Commit) @@ -250,8 +228,7 @@ func TestCommitRequestReconcile_Committed(t *testing.T) { require.Len(t, f.calls, 1) call := f.calls[0] - assert.Equal(t, "alice", call.Author, - "the attach must carry the author captured at admission") + assert.Empty(t, call.Author, "with attribution off, the request names no author") assert.Equal(t, "team-a-config", call.GitTargetName) assert.Equal(t, "default", call.GitTargetNamespace) assert.Equal(t, "save: save-1", call.Message) @@ -266,7 +243,7 @@ func TestCommitRequestReconcile_NoOpenWindow(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-now") @@ -296,7 +273,7 @@ func TestCommitRequestReconcile_CommitEmptyRecordsTheMessageAndKeepsTheCause(t * result: git.FinalizeResult{Outcome: tc.outcome, Commit: "abc123", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-empty") @@ -322,7 +299,7 @@ func TestCommitRequestReconcile_WindowMismatchIsExplained(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-foreign") @@ -342,7 +319,7 @@ func TestCommitRequestReconcile_AlreadyPresentRejected(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-noop") @@ -360,7 +337,7 @@ func TestCommitRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { result: git.FinalizeResult{Err: errors.New("commit failed: unreachable remote")}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-fail") @@ -371,34 +348,6 @@ func TestCommitRequestReconcile_FinalizeErrorBecomesFailed(t *testing.T) { requireCondition(t, got, ConditionTypePushed, metav1.ConditionFalse, "") } -// TestCommitRequestReconcile_LookupMissClaimsNoActor verifies that a lookup miss claims no actor -// immediately — present-or-never, no wait. A freshly created CommitRequest attaches at once with a -// blank author and records AuthorAttributed=False (CommitterFallback). The matching window determines -// the Git author. -func TestCommitRequestReconcile_LookupMissClaimsNoActor(t *testing.T) { - cr := newCommitRequest("save-fresh") - cr.CreationTimestamp = metav1.Now() // fresh: the old path would have requeued here - c := newCommitRequestClient(t, nil, cr) - f := &fakeFinalizer{ - result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Branch: "main"}, - resolved: true, - } - lookup := &fakeAuthorLookup{found: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} - - res := reconcileCommitRequest(t, r, "save-fresh") - - assert.Zero(t, res.RequeueAfter, "a miss is final; the request must not poll for an author") - assert.Equal(t, 1, lookup.calls, "the author is looked up exactly once, synchronously") - require.Len(t, f.calls, 1, "the attach is sent immediately on a miss") - assert.Empty(t, f.calls[0].Author, "a miss attaches with a blank, unnamed actor") - - got := fetchCommitRequest(t, c, "save-fresh") - requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) - requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) - requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback) -} - // A request whose window was never defaulted must reach the worker carrying the schema defaults, // not zeros. The API server normally fills the block in, so nil arrives here only from a client // that bypassed defaulting — and a zero there would be the old defect: the request expires before @@ -409,7 +358,7 @@ func TestCommitRequestReconcile_AnOmittedWindowUsesTheDefaults(t *testing.T) { require.Nil(t, cr.Spec.Window, "the fixture must leave the block unset") c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-default") @@ -432,7 +381,7 @@ func TestCommitRequestReconcile_ExplicitZerosArePreservedFieldByField(t *testing } c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-now") @@ -452,7 +401,7 @@ func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) cr.Spec.Window = &configv1alpha3.CommitRequestWindow{MaxDuration: &metav1.Duration{Duration: 30 * time.Second}} c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} res := reconcileCommitRequest(t, r, "save-linger") @@ -463,7 +412,6 @@ func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) "the window is passed to the worker, not consumed here") got := fetchCommitRequest(t, c, "save-linger") - requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) assert.False(t, commitRequestIsTerminal(&got), "an unresolved request is not terminal") @@ -491,7 +439,7 @@ func TestCommitRequestReconcile_ServiceErrorPolls(t *testing.T) { cr.CreationTimestamp = metav1.Now() c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{err: errors.New("get GitTarget: not found")} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} res := reconcileCommitRequest(t, r, "save-transient") @@ -511,7 +459,7 @@ func TestCommitRequestReconcile_ResolveTimeoutFailsClosed(t *testing.T) { withdrawResult: git.FinalizeResult{Branch: "main", Err: git.ErrCommitRequestWithdrawn}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-stuck") @@ -531,7 +479,7 @@ func TestCommitRequestReconcile_AHeldRequestOutlivesTheBound(t *testing.T) { cr := pastTheBound(newCommitRequest("save-held")) c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{result: git.FinalizeResult{Phase: phase}} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} res := reconcileCommitRequest(t, r, "save-held") @@ -566,7 +514,7 @@ func TestCommitRequestReconcile_AnUnansweredWithdrawKeepsPolling(t *testing.T) { withdrawResult: reply, withdrawPending: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} res := reconcileCommitRequest(t, r, "save-race") @@ -586,7 +534,7 @@ func TestCommitRequestReconcile_AWithdrawThatLostTheRaceReportsTheRealOutcome(t withdrawResult: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-late") @@ -606,7 +554,7 @@ func TestCommitRequestReconcile_NoWorkerIsNamedWhenItFailsClosed(t *testing.T) { }, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-no-worker") @@ -619,12 +567,10 @@ func TestCommitRequestReconcile_TerminalShortCircuits(t *testing.T) { cr := withReadyCommitted(newCommitRequest("save-done")) c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{} - lookup := attributedAlice() - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: lookup} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-done") - assert.Zero(t, lookup.calls) assert.Empty(t, f.calls, "a terminal CommitRequest must never re-attach") } @@ -638,16 +584,15 @@ func TestCommitRequestReconcile_StaleCacheEchoDoesNotRefinalize(t *testing.T) { live := newCommitRequestClient(t, nil, terminal) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: cached, APIReader: live, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: cached, APIReader: live, Finalizer: f} reconcileCommitRequest(t, r, "save-echo") assert.Empty(t, f.calls) } -// Webhook-disabled mode (no AuthorLookup) never waits: a freshly created CommitRequest -// attaches immediately with a blank author and records AuthorAttributed=False -// (AuthorCaptureDisabled). The matching window determines the Git author. +// A freshly created CommitRequest attaches immediately with a blank author. The matching window +// determines the Git author. func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing.T) { cr := newCommitRequest("save-configured-author") cr.CreationTimestamp = metav1.Now() // fresh: a waiting path would requeue instead of commit @@ -656,7 +601,7 @@ func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} res := reconcileCommitRequest(t, r, "save-configured-author") @@ -664,41 +609,18 @@ func TestCommitRequestReconcile_ConfiguredAuthorCommitsWithoutWaiting(t *testing got := fetchCommitRequest(t, c, "save-configured-author") requireCondition(t, got, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, got, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) - requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) assert.Equal(t, "c0ffee", got.Status.Commit) require.Len(t, f.calls, 1, "the attach is sent immediately, with no attribution wait") assert.Empty(t, f.calls[0].Author, "webhook-disabled mode attaches with a blank author") } -// Webhook-disabled mode settles AuthorAttributed=False (AuthorCaptureDisabled) immediately -// and never parks the request in a "waiting for the author" state: even while the -// attach is still being polled, the request reports progress. -func TestCommitRequestReconcile_ConfiguredAuthorAttributedImmediately(t *testing.T) { - cr := newCommitRequest("save-committer-poll") - cr.CreationTimestamp = metav1.Now() - c := newCommitRequestClient(t, nil, cr) - f := &fakeFinalizer{resolved: false} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} // AuthorLookup nil - - res := reconcileCommitRequest(t, r, "save-committer-poll") - - assert.Equal(t, commitRequestPollInterval, res.RequeueAfter, "an unresolved attach is polled") - require.Len(t, f.calls, 1) - assert.Empty(t, f.calls[0].Author) - - got := fetchCommitRequest(t, c, "save-committer-poll") - requireCondition(t, got, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonAuthorCaptureDisabled) - requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) - assert.False(t, commitRequestIsTerminal(&got), "in progress, not terminal") -} - // With no Finalizer wired the controller is inert: it neither attaches, stamps any // condition, nor drives the request to a terminal outcome. (Production always wires // the Finalizer; this is the disabled guard at the top of Reconcile.) func TestCommitRequestReconcile_NilFinalizerIsInert(t *testing.T) { cr := newCommitRequest("save-no-finalizer") c := newCommitRequestClient(t, nil, cr) - r := &CommitRequestReconciler{Client: c, APIReader: c} // Finalizer nil, AuthorLookup nil + r := &CommitRequestReconciler{Client: c, APIReader: c} // Finalizer nil res := reconcileCommitRequest(t, r, "save-no-finalizer") @@ -711,7 +633,7 @@ func TestCommitRequestReconcile_NilFinalizerIsInert(t *testing.T) { func TestCommitRequestReconcile_ObjectDeletedIsBenign(t *testing.T) { c := newCommitRequestClient(t, nil) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "gone") @@ -746,7 +668,7 @@ func TestCommitRequestReconcile_TerminalWriteRetriesOnConflict(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "ddd111", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-retry") @@ -769,31 +691,28 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { Branch: "main", }, nil, - attributionFromAdmission, ) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonCommitted) requireCondition(t, cr, ConditionTypePushed, metav1.ConditionTrue, ReasonSucceeded) requireCondition(t, cr, ConditionTypeReconciling, metav1.ConditionFalse, "") requireCondition(t, cr, ConditionTypeStalled, metav1.ConditionFalse, "") - requireCondition(t, cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, crReasonAttributedFromAdmission) assert.Equal(t, "abc", cr.Status.Commit) assert.Equal(t, "main", cr.Status.Branch) }) - t.Run("no window in grace is a benign ready (no admission author)", func(t *testing.T) { + t.Run("no window in grace is a benign ready", func(t *testing.T) { var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, - git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, nil, attributionCommitter) + git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}, nil) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonNoWindow) requireCondition(t, cr, ConditionTypePushed, metav1.ConditionFalse, crReasonNoWindow) - requireCondition(t, cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, crReasonCommitterFallback) assert.Empty(t, cr.Status.Commit) }) t.Run("window mismatch surfaces the reason", func(t *testing.T) { var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, - git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, nil, attributionFromAdmission) + git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, nil) ready := requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonWindowMismatch) assert.Equal(t, windowMismatchMessage, ready.Message) }) @@ -801,7 +720,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { t.Run("already present is a benign ready", func(t *testing.T) { var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, - git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, nil, attributionFromAdmission) + git.FinalizeResult{Outcome: git.FinalizeAlreadyPresent, Branch: "main"}, nil) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionTrue, crReasonAlreadyPresent) assert.Empty(t, cr.Status.Commit) }) @@ -810,7 +729,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { var cr configv1alpha3.CommitRequest applyFinalizeResultToStatus(&cr, git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc"}, - errors.New("boom"), attributionFromAdmission) + errors.New("boom")) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionFalse, crReasonFinalizeFailed) stalled := requireCondition(t, cr, ConditionTypeStalled, metav1.ConditionTrue, crReasonFinalizeFailed) assert.Equal(t, "boom", stalled.Message) @@ -818,7 +737,7 @@ func TestApplyFinalizeResultToStatus(t *testing.T) { t.Run("unknown outcome stalls", func(t *testing.T) { var cr configv1alpha3.CommitRequest - applyFinalizeResultToStatus(&cr, git.FinalizeResult{}, nil, attributionFromAdmission) + applyFinalizeResultToStatus(&cr, git.FinalizeResult{}, nil) requireCondition(t, cr, ConditionTypeReady, metav1.ConditionFalse, crReasonUnexpectedOutcome) stalled := requireCondition(t, cr, ConditionTypeStalled, metav1.ConditionTrue, crReasonUnexpectedOutcome) assert.Contains(t, stalled.Message, "unexpected finalize outcome") @@ -915,132 +834,6 @@ func TestCommitRequestReconciler_LiteralValidation(t *testing.T) { } } -// A request claims a window whose writes name no author only when it asked, its GitTarget allows -// what it asked, and the GitTarget's KubeProvider has never received an audit fact. A request that -// asks for more than its GitTarget allows fails at once, before anything is attached. -func TestCommitRequestAttach_ClaimsUnattributedWindowsOnlyWhenAskedAllowedAndUnaudited(t *testing.T) { - target := func(policy configv1alpha3.UnattributedWindowPolicy) *configv1alpha3.GitTarget { - gt := &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, - Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{Name: "workspace", Namespace: "providers"}, - }, - } - if policy != "" { - gt.Spec.AllowedCommitRequests = []configv1alpha3.GitTargetCommitRequestGrant{{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromSame}, - UnattributedWindows: policy, - }} - } - return gt - } - provider := func(facts metav1.ConditionStatus) *configv1alpha3.KubeProvider { - p := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} - if facts != "" { - p.Status.Conditions = []metav1.Condition{{ - Type: KubeProviderConditionAuditFactsReceived, Status: facts, Reason: "Test", - }} - } - return p - } - unaudited := provider(metav1.ConditionUnknown) - cases := []struct { - name string - ask configv1alpha3.WindowAuthor - objects []client.Object - want git.UnattributedClaim - refused bool - }{ - { - "own asks nothing", - configv1alpha3.WindowAuthorOwn, - []client.Object{ - target(configv1alpha3.UnattributedWindowsTakeOver), - unaudited, - }, - git.UnattributedClaimNone, - false, - }, - { - "takeover allowed", - configv1alpha3.WindowAuthorTakeOver, - []client.Object{ - target(configv1alpha3.UnattributedWindowsTakeOver), - unaudited, - }, - git.UnattributedClaimTakeOver, - false, - }, - { - "saved-by under a takeover policy", - configv1alpha3.WindowAuthorSavedBy, - []client.Object{ - target(configv1alpha3.UnattributedWindowsTakeOver), - unaudited, - }, - git.UnattributedClaimSavedBy, - false, - }, - { - "saved-by allowed", - configv1alpha3.WindowAuthorSavedBy, - []client.Object{ - target(configv1alpha3.UnattributedWindowsSavedBy), - unaudited, - }, - git.UnattributedClaimSavedBy, - false, - }, - { - "author attribution off", - configv1alpha3.WindowAuthorTakeOver, - []client.Object{ - target(configv1alpha3.UnattributedWindowsTakeOver), - provider(""), - }, - git.UnattributedClaimTakeOver, - false, - }, - {"audit facts received", configv1alpha3.WindowAuthorTakeOver, - []client.Object{target(configv1alpha3.UnattributedWindowsTakeOver), provider(metav1.ConditionTrue)}, - git.UnattributedClaimNone, false}, - {"KubeProvider missing", configv1alpha3.WindowAuthorTakeOver, - []client.Object{target(configv1alpha3.UnattributedWindowsTakeOver)}, git.UnattributedClaimNone, false}, - {"takeover under a saved-by policy", configv1alpha3.WindowAuthorTakeOver, - []client.Object{target(configv1alpha3.UnattributedWindowsSavedBy), unaudited}, "", true}, - {"saved-by under the default policy", configv1alpha3.WindowAuthorSavedBy, - []client.Object{target(""), unaudited}, "", true}, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - cr := newCommitRequest("save") - cr.Spec.Window = &configv1alpha3.CommitRequestWindow{Author: tc.ask} - c := newCommitRequestClient(t, nil, append([]client.Object{cr}, tc.objects...)...) - f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - - reconcileCommitRequest(t, r, "save") - - if tc.refused { - assert.Empty(t, f.calls, "a refused request is never attached") - var got configv1alpha3.CommitRequest - require.NoError( - t, - c.Get(context.Background(), client.ObjectKey{Namespace: "default", Name: "save"}, &got), - ) - ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) - require.NotNil(t, ready) - assert.Equal(t, metav1.ConditionFalse, ready.Status) - assert.Equal(t, crReasonTakeOverNotAllowed, ready.Reason) - assert.True(t, apimeta.IsStatusConditionTrue(got.Status.Conditions, ConditionTypeStalled)) - return - } - require.NotEmpty(t, f.calls, "the request must be attached") - assert.Equal(t, tc.want, f.calls[0].Unattributed) - }) - } -} - // A CommitRequest may name a GitTarget in another namespace, which saves for it only when a grant in its // spec.allowedCommitRequests admits the request's namespace. Omitted, the list admits the GitTarget's own // namespace alone; a refused request fails at once, before anything is attached. diff --git a/internal/controller/commitrequest_finalize.go b/internal/controller/commitrequest_finalize.go index 6c03f362..fbbf6e67 100644 --- a/internal/controller/commitrequest_finalize.go +++ b/internal/controller/commitrequest_finalize.go @@ -18,25 +18,21 @@ import ( // CommitRequest condition reasons (CamelCase tokens surfaced on status.conditions). const ( - crReasonCommitted = "Committed" - crReasonNoWindow = "NoWindow" - crReasonWindowMismatch = "WindowMismatch" - crReasonAlreadyPresent = "AlreadyPresent" - crReasonFinalizeFailed = "FinalizeFailed" - crReasonUnexpectedOutcome = "UnexpectedOutcome" - crReasonAttributedFromAdmission = "AttributedFromAdmission" - crReasonCommitterFallback = "CommitterFallback" - crReasonAuthorCaptureDisabled = "AuthorCaptureDisabled" - crReasonGitTargetRefPruned = "GitTargetRefPruned" - crReasonTakeOverNotAllowed = "TakeOverNotAllowed" - crReasonNotAllowed = "NotAllowed" - crReasonSaveInProgress = "SaveInProgress" + crReasonCommitted = "Committed" + crReasonNoWindow = "NoWindow" + crReasonWindowMismatch = "WindowMismatch" + crReasonAlreadyPresent = "AlreadyPresent" + crReasonFinalizeFailed = "FinalizeFailed" + crReasonUnexpectedOutcome = "UnexpectedOutcome" + crReasonGitTargetRefPruned = "GitTargetRefPruned" + crReasonNotAllowed = "NotAllowed" + crReasonSaveInProgress = "SaveInProgress" ) // The bounded `outcome` values on gitopsreverser_commit_requests_total. // // They are derived from git.FinalizeOutcome rather than from the condition reasons above: the -// FinalizeResult is what DECIDES, and the reasons include non-terminal and attribution-only values +// FinalizeResult is what DECIDES, and the reasons include non-terminal values // that have no business on a terminal counter. The set is closed, and outcomeFailed absorbs every // hard failure so an unmapped result can never go uncounted. const ( @@ -134,46 +130,6 @@ const alreadyPresentMessage = "the change already matches the remote, so no comm // notStalledMessage is the boilerplate message for a False Stalled condition. const notStalledMessage = "the CommitRequest is not stalled" -// commitRequestAttribution is the settled author decision threaded from the -// attribute step into the terminal status write so the AuthorAttributed condition can -// record whether the commit author was named. It is binary and always settled at -// first sight (present-or-never, §2) — there is no pending or timed-out state. -type commitRequestAttribution int - -const ( - // attributionUnset is the defensive zero value. It must never report a resolved - // actor when a caller neglected to initialize the attribution decision. - attributionUnset commitRequestAttribution = iota - // attributionFromAdmission means the submitter was captured at admission by the - // validate-operator-types webhook and named as the commit author. - attributionFromAdmission - // attributionCommitter means command-author capture ran but no admission author record - // exists, so the request does not claim an actor. - attributionCommitter - // attributionNotAttempted means command-author capture is switched off entirely (the - // validate-operator-types webhook is not running), so no submitter was ever sought. It is - // distinct from attributionCommitter, where capture DID run and found nothing. - attributionNotAttempted -) - -// gitOutcome projects the controller's attribution state onto the git layer's outcome, which -// is what the worker matches windows on. Keeping the projection here — rather than passing the -// controller's own enum down — means the git package never learns about admission records. -func (a commitRequestAttribution) gitOutcome() git.AttributionOutcome { - switch a { - case attributionUnset: - return git.AttributionNotAttempted - case attributionFromAdmission: - return git.AttributionResolved - case attributionCommitter: - return git.AttributionUnresolved - case attributionNotAttempted: - return git.AttributionNotAttempted - default: - return git.AttributionNotAttempted - } -} - // setCommitRequestCondition upserts a condition keyed by type, stamping it with // the request's generation so kstatus can tell a current status from a stale one. func setCommitRequestCondition( @@ -232,66 +188,28 @@ func setCommitRequestProgress(cr *configv1alpha3.CommitRequest, reason, message // (Reconciling=True). An empty phase is the generic Progressing: sent, not yet confirmed. func markCommitRequestProgressing( cr *configv1alpha3.CommitRequest, - attribution commitRequestAttribution, phase git.CommitRequestPhase, held string, ) { cr.Status.ObservedGeneration = cr.Generation - setCommitRequestAttributed(cr, attribution) reason, message := progressFor(phase, held) setCommitRequestProgress(cr, reason, message) } -// setCommitRequestAttributed records the settled, binary author decision on the -// AuthorAttributed condition. A False reason distinguishes an absent record -// (CommitterFallback) from capture disabled (AuthorCaptureDisabled). Neither is a failure or -// affects Ready; the request claims no actor. -func setCommitRequestAttributed(cr *configv1alpha3.CommitRequest, attribution commitRequestAttribution) { - switch attribution { - case attributionUnset: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, - crReasonAuthorCaptureDisabled, - "command-author capture state was unset; the request can attach only to a window with no named actor") - case attributionFromAdmission: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionTrue, - crReasonAttributedFromAdmission, - "the submitter was captured at admission and named as the commit author") - case attributionCommitter: - // Capture RAN and found nothing. Distinct from attributionNotAttempted below, whose - // message this used to carry — reporting "the webhook is not configured" for a - // webhook that is configured and simply missed sends operators to the wrong place. - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, - crReasonCommitterFallback, - "no admission author record was found for this request; it can attach only to a "+ - "window with no named actor") - case attributionNotAttempted: - setCommitRequestCondition(cr, ConditionTypeAuthorAttributed, metav1.ConditionFalse, - crReasonAuthorCaptureDisabled, - "command-author capture is disabled (the validate-operator-types webhook is not "+ - "configured); the request can attach only to a window with no named actor") - } -} - -// applyFinalizeResultToStatus maps a FinalizeResult (or a finalize error) and the -// settled attribution onto a CommitRequest's terminal conditions. Ready is the -// summary; Reconciling/Stalled carry kstatus; Attributed records the author -// decision; Pushed records whether a commit reached the remote. +// applyFinalizeResultToStatus maps a FinalizeResult (or a finalize error) onto a +// CommitRequest's terminal conditions. Ready is the summary; Reconciling/Stalled +// carry kstatus; Pushed records whether a commit reached the remote. func applyFinalizeResultToStatus( cr *configv1alpha3.CommitRequest, result git.FinalizeResult, finalizeErr error, - attribution commitRequestAttribution, ) { cr.Status.ObservedGeneration = cr.Generation cr.Status.Branch = result.Branch - setCommitRequestAttributed(cr, attribution) if finalizeErr != nil { reason := crReasonFinalizeFailed - switch { - case errors.Is(finalizeErr, errTakeOverNotAllowed): - reason = crReasonTakeOverNotAllowed - case errors.Is(finalizeErr, errCommitRequestNotAllowed): + if errors.Is(finalizeErr, errCommitRequestNotAllowed) { reason = crReasonNotAllowed } failCommitRequest(cr, reason, finalizeErr.Error()) diff --git a/internal/controller/commitrequest_kstatus_test.go b/internal/controller/commitrequest_kstatus_test.go index c84221dc..1602c796 100644 --- a/internal/controller/commitrequest_kstatus_test.go +++ b/internal/controller/commitrequest_kstatus_test.go @@ -24,7 +24,7 @@ func TestCommitRequestKstatusContract(t *testing.T) { wantMsg string }{ { - name: "missing admission author in the close-delay wait (AuthorAttributed=False is not a failure)", + name: "in the close-delay wait", conds: []map[string]interface{}{ conditionMap(ConditionTypeReady, "False", string(git.PhaseWaitingForWindow), waitingForWindowMessage), conditionMap( @@ -34,23 +34,6 @@ func TestCommitRequestKstatusContract(t *testing.T) { waitingForWindowMessage, ), conditionMap(ConditionTypeStalled, "False", string(git.PhaseWaitingForWindow), notStalledMessage), - conditionMap(ConditionTypeAuthorAttributed, "False", crReasonCommitterFallback, "no admission record"), - conditionMap(ConditionTypePushed, "Unknown", string(git.PhaseWaitingForWindow), pushPendingMessage), - }, - wantStatus: kstatus.InProgressStatus, - }, - { - name: "admission-attributed in the close-delay wait", - conds: []map[string]interface{}{ - conditionMap(ConditionTypeReady, "False", string(git.PhaseWaitingForWindow), waitingForWindowMessage), - conditionMap( - ConditionTypeReconciling, - "True", - string(git.PhaseWaitingForWindow), - waitingForWindowMessage, - ), - conditionMap(ConditionTypeStalled, "False", string(git.PhaseWaitingForWindow), notStalledMessage), - conditionMap(ConditionTypeAuthorAttributed, "True", crReasonAttributedFromAdmission, "from admission"), conditionMap(ConditionTypePushed, "Unknown", string(git.PhaseWaitingForWindow), pushPendingMessage), }, wantStatus: kstatus.InProgressStatus, @@ -61,7 +44,6 @@ func TestCommitRequestKstatusContract(t *testing.T) { conditionMap(ConditionTypeReady, "True", crReasonCommitted, "closed, committed, and pushed"), conditionMap(ConditionTypeReconciling, "False", crReasonCommitted, "closed, committed, and pushed"), conditionMap(ConditionTypeStalled, "False", crReasonCommitted, notStalledMessage), - conditionMap(ConditionTypeAuthorAttributed, "True", crReasonAttributedFromAdmission, "from admission"), conditionMap(ConditionTypePushed, "True", ReasonSucceeded, "pushed"), }, wantStatus: kstatus.CurrentStatus, diff --git a/internal/controller/commitrequest_outcome_metric_test.go b/internal/controller/commitrequest_outcome_metric_test.go index d03dc32f..d25df234 100644 --- a/internal/controller/commitrequest_outcome_metric_test.go +++ b/internal/controller/commitrequest_outcome_metric_test.go @@ -66,7 +66,7 @@ func TestCommitRequestMetric_TwoTargetsAreDistinguishable(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-"+target) } @@ -90,7 +90,7 @@ func TestCommitRequestMetric_RepeatedSavesReuseOneSeries(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, name) } @@ -119,7 +119,7 @@ func TestCommitRequestMetric_UnresolvableTargetTakesTheFallback(t *testing.T) { withdrawResult: git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, withdrawResolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-bogus") @@ -201,7 +201,7 @@ func TestCommitRequestMetric_EveryOutcomeIsReachableThroughReconcile(t *testing. c := newCommitRequestClient(t, nil, newCommitRequest("save-outcome")) f := &fakeFinalizer{result: tc.result, err: tc.err, resolved: true} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-outcome") @@ -242,7 +242,7 @@ func TestCommitRequestMetric_ConflictRetriesIncrementOnce(t *testing.T) { result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-conflict") @@ -275,7 +275,7 @@ func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testi result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "abc", Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-lost") reconcileCommitRequest(t, r, "save-lost") @@ -302,7 +302,7 @@ func TestCommitRequestMetric_PrunedGitTargetRefCountsFailed(t *testing.T) { } c := newCommitRequestClient(t, nil, pruned) f := &fakeFinalizer{} - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-pruned") @@ -326,7 +326,7 @@ func TestCommitRequestMetric_RecreatedRequestCountsPerIncarnation(t *testing.T) result: git.FinalizeResult{Outcome: git.FinalizeWindowMismatch, Branch: "main"}, resolved: true, } - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} reconcileCommitRequest(t, r, "save-again") @@ -370,7 +370,7 @@ func TestCommitRequestMetric_AMissingWorkerKeepsTheTargetLabels(t *testing.T) { c := newCommitRequestClient(t, nil, fresh, stale, target, gitBranchOn("team-a-provider", "default", "main")) workers := git.NewWorkerManager(c, logr.Discard(), git.BranchWorkerLimits{}, itypes.SensitiveResourcePolicy{}) router := watch.NewEventRouter(workers, nil, c, logr.Discard()) - r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: router, AuthorLookup: attributedAlice()} + r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: router} // Inside the safety window: pending, and the status says why. res := reconcileCommitRequest(t, r, fresh.Name) diff --git a/internal/controller/commitrequest_ttl_test.go b/internal/controller/commitrequest_ttl_test.go index 5f84ba02..c35a1bd9 100644 --- a/internal/controller/commitrequest_ttl_test.go +++ b/internal/controller/commitrequest_ttl_test.go @@ -48,7 +48,7 @@ func committedFinalizer() *fakeFinalizer { func TestCommitRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) r := &CommitRequestReconciler{ - Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), + Client: c, APIReader: c, Finalizer: committedFinalizer(), TTL: 48 * time.Hour, } @@ -65,7 +65,7 @@ func TestCommitRequestTTL_FinishingStampsTheDeletionTime(t *testing.T) { func TestCommitRequestTTL_ZeroStampsNothing(t *testing.T) { c := newCommitRequestClient(t, nil, newCommitRequest("save-now")) r := &CommitRequestReconciler{ - Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), + Client: c, APIReader: c, Finalizer: committedFinalizer(), } reconcileCommitRequest(t, r, "save-now") @@ -128,7 +128,7 @@ func TestCommitRequestTTL_LeavesARequestInProgressAlone(t *testing.T) { c := newCommitRequestClient(t, nil, cr) f := &fakeFinalizer{resolved: false} r := &CommitRequestReconciler{ - Client: c, APIReader: c, Finalizer: f, AuthorLookup: attributedAlice(), TTL: time.Nanosecond, + Client: c, APIReader: c, Finalizer: f, TTL: time.Nanosecond, } res := reconcileCommitRequest(t, r, "save-pending") @@ -161,7 +161,7 @@ func TestCommitRequestTTL_StampDoesNotOverwriteAConcurrentExtension(t *testing.T }, }, newCommitRequest("save-now")) r := &CommitRequestReconciler{ - Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), + Client: c, APIReader: c, Finalizer: committedFinalizer(), TTL: time.Hour, } @@ -179,7 +179,7 @@ func TestCommitRequestTTL_AValueSetAtCreationIsKept(t *testing.T) { cr.Annotations = map[string]string{CommitRequestDeleteAfterAnnotation: chosen} c := newCommitRequestClient(t, nil, cr) r := &CommitRequestReconciler{ - Client: c, APIReader: c, Finalizer: committedFinalizer(), AuthorLookup: attributedAlice(), + Client: c, APIReader: c, Finalizer: committedFinalizer(), TTL: time.Hour, } diff --git a/internal/controller/constants.go b/internal/controller/constants.go index aaebcc87..600b3c26 100644 --- a/internal/controller/constants.go +++ b/internal/controller/constants.go @@ -86,15 +86,6 @@ const ( // still being established (the rule is blocked by validation), which must not be rendered as a // permanent failure. ConditionTypeSourceNamespaceAuthorized = "SourceNamespaceAuthorized" - // ConditionTypeAuthorAttributed indicates whether a CommitRequest's commit author - // was named from the submitter captured at admission. It is binary and immediately - // settled (no Unknown, no timeout): True (AttributedFromAdmission) when the - // validate-operator-types webhook recorded the submitter, False (CommitterFallback) when - // capture ran but found no record, or False (AuthorCaptureDisabled) when capture is off. - // False is not a failure and does not affect Ready: the - // request claims no actor and can attach only to an unnamed watch window - // (docs/architecture.md#commitrequest-finalize). - ConditionTypeAuthorAttributed = "AuthorAttributed" // ConditionTypePushed indicates whether a CommitRequest's commit reached the // remote repository. ConditionTypePushed = "Pushed" diff --git a/internal/controller/gittarget_publication_test.go b/internal/controller/gittarget_publication_test.go index 2b7ab3fc..5f2616d2 100644 --- a/internal/controller/gittarget_publication_test.go +++ b/internal/controller/gittarget_publication_test.go @@ -96,10 +96,10 @@ func TestCommitRequestProgress_AHeldSaveSaysWhyItIsHeld(t *testing.T) { assert.Equal(t, waitingForPushMessage, plain, "a push that is merely pending says nothing more") cr := &configbutleraiv1alpha3.CommitRequest{} - markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, "") + markCommitRequestProgressing(cr, git.PhaseWaitingForPush, "") assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, "")) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, held), "the outage is news to the save") - markCommitRequestProgressing(cr, attributionFromAdmission, git.PhaseWaitingForPush, held) + markCommitRequestProgressing(cr, git.PhaseWaitingForPush, held) assert.True(t, progressRecorded(cr, git.PhaseWaitingForPush, held)) assert.False(t, progressRecorded(cr, git.PhaseWaitingForPush, ""), "and so is the recovery") } diff --git a/internal/controller/suite_test.go b/internal/controller/suite_test.go index 5e740d58..25d5d0b0 100644 --- a/internal/controller/suite_test.go +++ b/internal/controller/suite_test.go @@ -182,15 +182,12 @@ var _ = BeforeSuite(func() { }).SetupWithManager(mgr) Expect(err).NotTo(HaveOccurred()) - // A non-nil AuthorLookup that misses (no admission record) resolves to the - // committer immediately, and a Finalizer that never resolves keeps requests in the - // Progressing state. Neither ever completes, so these specs cover only the - // initial in-progress stamp and the terminal short-circuit. + // A Finalizer that never resolves keeps requests in the Progressing state, so these specs + // cover only the initial in-progress stamp and the terminal short-circuit. err = (&CommitRequestReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - Finalizer: &fakeFinalizer{resolved: false}, - AuthorLookup: &fakeAuthorLookup{}, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + Finalizer: &fakeFinalizer{resolved: false}, }).SetupWithManager(mgr) Expect(err).NotTo(HaveOccurred()) diff --git a/internal/git/attribution_outcome_test.go b/internal/git/attribution_outcome_test.go index 53117d3e..a7a581df 100644 --- a/internal/git/attribution_outcome_test.go +++ b/internal/git/attribution_outcome_test.go @@ -257,52 +257,3 @@ func TestRenderEventCommitMessage_UnresolvedUsernameStaysEmptyInTemplates(t *tes require.NotNil(t, options.Author) assert.Equal(t, UnresolvedAuthorDisplayName, options.Author.Name) } - -// A request that names its submitter claims a window whose writes name no author only when it was -// allowed to, and never a window whose writes name an actor, nor one on another GitTarget. -func TestPendingCommitRequest_ClaimsOnlyAnUnattributedWindowWhenAllowed(t *testing.T) { - unattributed := newOpenWindow(attributedEvent("", AttributionUnresolved), nil) - named := newOpenWindow(attributedEvent("bob@acme.com", AttributionResolved), nil) - request := func(claim UnattributedClaim, target string) *pendingCommitRequest { - return &pendingCommitRequest{ - author: "jane@acme.com", - attribution: AttributionResolved, - gitTargetName: target, - gitTargetNamespace: "default", - unattributed: claim, - } - } - - assert.False(t, request(UnattributedClaimNone, "target").matchesWindow(unattributed), - "without a claim a named request keeps the author rule") - for _, claim := range []UnattributedClaim{UnattributedClaimSavedBy, UnattributedClaimTakeOver} { - assert.True(t, request(claim, "target").matchesWindow(unattributed), string(claim)) - assert.True(t, request(claim, "target").claims(unattributed), string(claim)) - assert.False(t, request(claim, "target").matchesWindow(named), - "a window whose writes name an actor is never claimed: %s", claim) - assert.False(t, request(claim, "other-target").matchesWindow(unattributed), - "a claim does not widen attachment across GitTargets: %s", claim) - } - - own := newOpenWindow(attributedEvent("jane@acme.com", AttributionResolved), nil) - assert.True(t, request(UnattributedClaimTakeOver, "target").matchesWindow(own)) - assert.False(t, request(UnattributedClaimTakeOver, "target").claims(own), - "the submitter's own window is not a claim") -} - -func TestWithTakeOverNote_EndsTheMessageWithATrailer(t *testing.T) { - id := &commitRequestID{Namespace: "team-a", Name: "save-1"} - assert.Equal(t, - "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes\n", - withTakeOverNote("save: edit\n", id), "the message's trailing newline is kept") - assert.Equal(t, - "save: edit\n\nAuthor-Source: CommitRequest team-a/save-1; no audit record names who made these changes", - withTakeOverNote("save: edit", id)) -} - -func TestWithSavedByNote_NamesTheSubmitterLikeACoAuthor(t *testing.T) { - assert.Equal(t, "save: edit\n\nSaved-By: jane@acme.com ", - withSavedByNote("save: edit", UserInfo{Username: "jane@acme.com"})) - assert.Equal(t, "save: edit\n\nSaved-By: Jane Doe \n", - withSavedByNote("save: edit\n", UserInfo{Username: "jane", DisplayName: "Jane Doe", Email: "jane@acme.com"})) -} diff --git a/internal/git/branch_worker.go b/internal/git/branch_worker.go index f3aa556b..ba477525 100644 --- a/internal/git/branch_worker.go +++ b/internal/git/branch_worker.go @@ -1547,14 +1547,6 @@ func (l *branchWorkerEventLoop) finalizeOpenWindowWithReason(reason windowFinali return } pendingWrite.CommitMessage = effectiveMessage - switch l.openWindow.claim { - case UnattributedClaimTakeOver: - pendingWrite.RequestAuthor, pendingWrite.RequestAttribution = l.openWindow.claimedBy, AttributionResolved - pendingWrite.TakenOver = true - case UnattributedClaimSavedBy: - pendingWrite.SavedBy = l.openWindow.claimedBy - case UnattributedClaimNone: - } pendingCR, proceed := l.gateBuiltWindow(pendingCR, pendingWrite.Target(), reason) if !proceed { return diff --git a/internal/git/commit_executor.go b/internal/git/commit_executor.go index df76d030..f892c207 100644 --- a/internal/git/commit_executor.go +++ b/internal/git/commit_executor.go @@ -8,7 +8,6 @@ import ( "fmt" "path/filepath" "slices" - "strings" "time" gogit "github.com/go-git/go-git/v6" @@ -313,39 +312,10 @@ func (p PendingWrite) commitMetadata() (string, *gogit.CommitOptions, messageRes if err != nil { return "", nil, messageResolutionUnsupported, err } - switch { - case p.TakenOver: - message = withTakeOverNote(message, p.CommitRequest) - case p.SavedBy.Username != "": - message = withSavedByNote(message, p.SavedBy) - } log.Log.V(1).Info("Selected commit message", "source", resolution.label()) return message, commitOptionsFor(p, p.CommitConfig, p.Signer, time.Now()), resolution, nil } -// withTakeOverNote ends the message of a commit whose CommitRequest took over a window with no author -// named (PendingWrite.TakenOver) with a trailer that says the author is the request's submitter. -func withTakeOverNote(message string, request *commitRequestID) string { - name := "a CommitRequest" - if request != nil { - name = "CommitRequest " + request.Namespace + "/" + request.Name - } - return withTrailer(message, "Author-Source: "+name+"; no audit record names who made these changes") -} - -// withSavedByNote ends the message of a commit whose CommitRequest gave a window with no author named -// its message (PendingWrite.SavedBy) with a trailer naming the request's submitter, in the shape of a -// Co-authored-by trailer. -func withSavedByNote(message string, savedBy UserInfo) string { - return withTrailer(message, "Saved-By: "+authorName(savedBy)+" <"+authorEmail(savedBy)+">") -} - -// withTrailer adds trailer as the message's last paragraph, keeping the message's trailing newline. -func withTrailer(message, trailer string) string { - body := strings.TrimRight(message, "\n") - return body + "\n\n" + trailer + message[len(body):] -} - // framedRequestMessage renders the target's requestTemplate around an attached CommitRequest's // message, falling back to that message verbatim if the render fails. // diff --git a/internal/git/commit_request_attach.go b/internal/git/commit_request_attach.go index 3f84231a..4b05f606 100644 --- a/internal/git/commit_request_attach.go +++ b/internal/git/commit_request_attach.go @@ -158,28 +158,8 @@ type AttachCommitRequest struct { // CatchUpNow asks the GitTarget's applier to apply the tip as soon as the request's commit is // pushed: catchUp: AfterPush. CatchUpNow bool - // Unattributed is what a request that names its submitter may claim of a window on its GitTarget - // whose writes name no author: nothing, its message (SavedBy), or the window (TakeOver). The - // controller sets more than nothing only when the request asked, its GitTarget allows it, and - // the GitTarget's KubeProvider has never received an audit fact. Once it has, a write naming no - // author means its fact went missing, and that window stays another's. - Unattributed UnattributedClaim } -// UnattributedClaim is what a CommitRequest may claim of a commit window whose writes name no author. -type UnattributedClaim string - -const ( - // UnattributedClaimNone attaches only to the submitter's own window. - UnattributedClaimNone UnattributedClaim = "" - // UnattributedClaimSavedBy gives the window the request's message and names no author: the - // commit stays unattributed, and its message ends with a Saved-By trailer. - UnattributedClaimSavedBy UnattributedClaim = "SavedBy" - // UnattributedClaimTakeOver takes the window over: the submitter authors the commit, and its - // message ends with an Author-Source trailer. - UnattributedClaimTakeOver UnattributedClaim = "TakeOver" -) - // commitRequestID is the worker-local key for a CommitRequest: its namespaced // name plus UID when available. Two CommitRequests with the same name but // different UIDs (a delete-and-recreate) are distinct. @@ -218,9 +198,6 @@ type pendingCommitRequest struct { commitEmpty bool // catchUpNow asks the GitTarget's applier for a converge once the request's commit is pushed. catchUpNow bool - // unattributed is what the request may claim of a window whose writes name no author - // (AttachCommitRequest.Unattributed). - unattributed UnattributedClaim // attached is true once this request is bound to a write: its message to the open window, or // its own empty-commit record. From then on only that write's outcome settles it, and it stays // set while the write is pending, waiting for the push. @@ -286,7 +263,7 @@ func (p *pendingCommitRequest) matchesWindow(w *openWindow) bool { if p.gitTargetName != w.GitTarget || p.gitTargetNamespace != w.GitTargetNamespace { return false } - return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() || p.claims(w) + return p.author == w.Author && p.attribution.NamesActor() == w.Attribution.NamesActor() } // sameSaver reports whether two requests would save one window: the same GitTarget and the same @@ -296,13 +273,6 @@ func (p *pendingCommitRequest) sameSaver(other *pendingCommitRequest) bool { p.author == other.author && p.attribution.NamesActor() == other.attribution.NamesActor() } -// claims reports whether the request matches a window on its GitTarget whose writes name no author -// only because it may claim such a window: it names its submitter, and asked for more than its own -// windows (AttachCommitRequest.Unattributed). -func (p *pendingCommitRequest) claims(w *openWindow) bool { - return p.unattributed != UnattributedClaimNone && p.attribution.NamesActor() && !w.Attribution.NamesActor() -} - // commitRequestOutcomeEntry is a resolved outcome retained for the controller to // poll, GC'd by age. type commitRequestOutcomeEntry struct { diff --git a/internal/git/commit_request_attach_loop.go b/internal/git/commit_request_attach_loop.go index fab96e00..9800534b 100644 --- a/internal/git/commit_request_attach_loop.go +++ b/internal/git/commit_request_attach_loop.go @@ -215,8 +215,6 @@ func (l *branchWorkerEventLoop) handleAttachCommitRequest(req *AttachCommitReque maxDuration: req.MaxDuration, commitEmpty: req.CommitEmpty, catchUpNow: req.CatchUpNow, - - unattributed: req.Unattributed, } if earlier := l.saveInProgress(pcr, req.Attach); earlier != nil { l.w.recordCommitRequestOutcome(id, FinalizeResult{Branch: l.w.Branch, Outcome: FinalizeSaveInProgress}) @@ -379,12 +377,6 @@ func (l *branchWorkerEventLoop) attachToOpenWindow(pcr *pendingCommitRequest) { } l.openWindow.timers = timers l.openWindow.lastWriteAt = now - if pcr.claims(l.openWindow) { - l.openWindow.claim, l.openWindow.claimedBy = pcr.unattributed, UserInfo{Username: pcr.author} - l.w.Log.Info("CommitRequest claimed a window whose writes name no author", - "request", id.Namespace+"/"+id.Name, "author", pcr.author, "claim", string(pcr.unattributed), - "target", pcr.gitTargetNamespace+"/"+pcr.gitTargetName) - } l.w.setCommitRequestPhase(id, PhaseCollectingWindow) l.w.Log.Info("CommitRequest attached to open window", "request", id.Namespace+"/"+id.Name, diff --git a/internal/git/commit_request_attach_test.go b/internal/git/commit_request_attach_test.go index ba2aff5c..b68e7473 100644 --- a/internal/git/commit_request_attach_test.go +++ b/internal/git/commit_request_attach_test.go @@ -340,81 +340,6 @@ func TestAttach_WritesAfterTheClaimJoinOneCommit(t *testing.T) { }, "both writes must land in the one commit carrying the message") } -// TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit pins the takeover from a source that -// sends no audit facts: the request attaches to a window whose writes name no author, its submitter -// authors the commit, and the message ends with a note that says so. -func TestAttach_TakesOverAnUnattributedWindowAndAuthorsItsCommit(t *testing.T) { - worker, _, _ := setupCommitPushSplitWorker(t) - createPlainGitTarget(t, worker, "team-a", "team-a") - - loop := newBranchWorkerEventLoop(worker, time.Hour) - defer loop.stopTimers() - - event := configMapTargetEvent("cm", "", "team-a") - event.Attribution = AttributionUnresolved - loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{event}}}) - require.NotNil(t, loop.openWindow) - - req := attachReq("jane@acme.com", 60*time.Second) - req.Attribution = AttributionResolved - req.Message = "save: from the editor" - req.Unattributed = UnattributedClaimTakeOver - serviceAttach(loop, req) - require.NotNil(t, loop.openWindow.pendingCR, "the request takes the window over") - - forceDue(loop) - loop.endWake(0) - res, ok := outcome(t, worker) - require.True(t, ok) - require.NoError(t, res.Err) - require.Equal(t, FinalizeCommitted, res.Outcome) - - repo, err := gogit.PlainOpen(worker.repoPath()) - require.NoError(t, err) - commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) - require.NoError(t, err) - assert.Equal(t, "jane@acme.com", commit.Author.Name, "the submitter authors the commit") - assert.Equal(t, "save: from the editor\n\nAuthor-Source: CommitRequest default/save; "+ - "no audit record names who made these changes", commit.Message) -} - -// TestAttach_SavedByGivesTheMessageAndNamesNoAuthor pins the milder claim: the request gives a window -// whose writes name no author its message, the commit stays authored as unresolved, and the message -// ends with a Saved-By trailer naming the submitter. -func TestAttach_SavedByGivesTheMessageAndNamesNoAuthor(t *testing.T) { - worker, _, _ := setupCommitPushSplitWorker(t) - createPlainGitTarget(t, worker, "team-a", "team-a") - - loop := newBranchWorkerEventLoop(worker, time.Hour) - defer loop.stopTimers() - - event := configMapTargetEvent("cm", "", "team-a") - event.Attribution = AttributionUnresolved - loop.handleQueueItem(WorkItem{Request: &WriteRequest{Events: []Event{event}}}) - require.NotNil(t, loop.openWindow) - - req := attachReq("jane@acme.com", 60*time.Second) - req.Attribution = AttributionResolved - req.Message = "save: from the editor" - req.Unattributed = UnattributedClaimSavedBy - serviceAttach(loop, req) - require.NotNil(t, loop.openWindow.pendingCR, "the request gives the window its message") - - forceDue(loop) - loop.endWake(0) - res, ok := outcome(t, worker) - require.True(t, ok) - require.NoError(t, res.Err) - require.Equal(t, FinalizeCommitted, res.Outcome) - - repo, err := gogit.PlainOpen(worker.repoPath()) - require.NoError(t, err) - commit, err := repo.CommitObject(plumbing.NewHash(res.Commit)) - require.NoError(t, err) - assert.Equal(t, UnresolvedAuthorDisplayName, commit.Author.Name, "nobody is named as the author") - assert.Equal(t, "save: from the editor\n\nSaved-By: jane@acme.com ", commit.Message) -} - // TestAttach_ForeignWindowIsNotStolen verifies an attach for a different author // parks (never finalizes another author's window) and resolves WindowMismatch once // its grace elapses, leaving the foreign window open. diff --git a/internal/git/open_window.go b/internal/git/open_window.go index a1fe5299..8679f247 100644 --- a/internal/git/open_window.go +++ b/internal/git/open_window.go @@ -46,11 +46,6 @@ type openWindow struct { // pendingCR identifies the CommitRequest attached to this window; at most one. On // finalize its outcome is resolved (Committed once the carrying write pushes). pendingCR *commitRequestID - // claim and claimedBy say what the attached CommitRequest claimed of the window, whose writes - // name no author, and who submitted it (pendingCommitRequest.claims): its message (SavedBy), or - // the window, which the submitter then authors (TakeOver). Empty when it claimed nothing. - claim UnattributedClaim - claimedBy UserInfo // timers decide when the window closes. They are the GitTarget's, snapshotted when the window // opened, until a CommitRequest attaches and replaces them with its own. diff --git a/internal/git/pending_writes.go b/internal/git/pending_writes.go index 71cebe44..f3abce4e 100644 --- a/internal/git/pending_writes.go +++ b/internal/git/pending_writes.go @@ -250,7 +250,7 @@ func (p PendingWrite) Author() string { // writes, including any OIDC display name and email. Empty writes have no // per-user author and return the zero value. func (p PendingWrite) AuthorUserInfo() UserInfo { - if p.Kind == PendingWriteRequestRecord || p.TakenOver { + if p.Kind == PendingWriteRequestRecord { return p.RequestAuthor } if len(p.Events) == 0 { @@ -303,7 +303,7 @@ const ( // mirrors AuthorUserInfo: the window is single-author, so the first event's outcome describes // the whole write. Empty writes never attempt attribution. func (p PendingWrite) AttributionOutcome() AttributionOutcome { - if p.Kind == PendingWriteRequestRecord || p.TakenOver { + if p.Kind == PendingWriteRequestRecord { return p.RequestAttribution } if len(p.Events) == 0 { diff --git a/internal/git/types.go b/internal/git/types.go index 0c2edde0..392fc6ad 100644 --- a/internal/git/types.go +++ b/internal/git/types.go @@ -384,14 +384,6 @@ type PendingWrite struct { // window's author is. RequestAuthor UserInfo RequestAttribution AttributionOutcome - // TakenOver says the CommitRequest attached to this grouped write took over a window whose - // writes named no author: RequestAuthor and RequestAttribution, the submitter's, author the - // commit, and its message ends with an Author-Source trailer (withTakeOverNote). - TakenOver bool - // SavedBy is the submitter of the CommitRequest that gave this grouped write its message without - // claiming its writes, which named no author: the commit stays unattributed, and its message ends - // with a Saved-By trailer naming them (withSavedByNote). Zero when no request did. - SavedBy UserInfo // emptyCommitted is stamped by executePendingWrites, the way CommitSHA is: the commit this // write made changed no file. A request riding it resolves with the cause (AlreadyPresent or // NoOpenWindow) and the commit, so a caller can tell "saved your changes" from "recorded your diff --git a/internal/queue/command_author_store.go b/internal/queue/command_author_store.go deleted file mode 100644 index ad012b48..00000000 --- a/internal/queue/command_author_store.go +++ /dev/null @@ -1,84 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package queue - -import ( - "context" - "encoding/json" - "fmt" - "time" - - "github.com/redis/go-redis/v9" - "k8s.io/apimachinery/pkg/types" -) - -// commandAuthorKeySuffix namespaces the keys this store owns. It shares the top-level -// author domain with audit-sourced resource facts, but the command subfamily has a -// different provenance and no grace-window join. -const commandAuthorKeySuffix = ":author:v1:command:" - -// commandAuthorRecordTTL bounds a captured authorship record. It is NOT tuned to cover -// any wait: by the authorship invariant the record is written before the object exists, -// and the controller reads it on the first reconcile after persist — typically -// sub-second. The TTL exists ONLY so an orphan record (a command object deleted before -// its reconcile) self-cleans. It is a fixed internal constant, deliberately not a flag: -// there is nothing to tune. An hour is generous headroom for a slow reconcile backlog or -// a leader failover. -const commandAuthorRecordTTL = time.Hour - -// CommandAuthor is the minimal authorship captured at admission for one command object. -// It carries only what a git commit author needs — no RV, no auditID, no conflict bit: -// this is a 1:1 command capture, not a post-persist join. -type CommandAuthor struct { - Author string `json:"author"` - DisplayName string `json:"displayName,omitempty"` - Email string `json:"email,omitempty"` - RequestedAt string `json:"requestedAt,omitempty"` // RFC3339Nano, for lag metrics/debug -} - -// CommandAuthorStore records and reads command authorship. It shares RedisStore's -// connection but is wired whenever the validate-operator-types webhook is enabled — -// independent of --author-attribution, which only governs mirrored-resource attribution. -type CommandAuthorStore struct { - client *redis.Client - ttl time.Duration - // keyPrefix is the root namespace shared with the RedisStore that built this store. - keyPrefix string -} - -// RecordCommandAuthor is the admission-side write: capture the authenticated submitter -// the instant a command CREATE is admitted, before it persists. Last-write-wins (a -// CREATE fires admission once; a retried admission re-asserts the same user). -func (s *CommandAuthorStore) RecordCommandAuthor( - ctx context.Context, uid types.UID, author CommandAuthor, -) error { - raw, err := json.Marshal(author) - if err != nil { - return fmt.Errorf("marshal command author: %w", err) - } - return s.client.Set(ctx, s.key(uid), raw, s.ttl).Err() -} - -// LookupCommandAuthor is the controller-side read, keyed by the persisted object's UID. -// ok=false means no record was captured — the validate-operator-types webhook is not -// configured (or a best-effort write missed) — and the controller finalizes as the -// committer, immediately. -func (s *CommandAuthorStore) LookupCommandAuthor( - ctx context.Context, uid types.UID, -) (CommandAuthor, bool) { - raw, err := s.client.Get(ctx, s.key(uid)).Bytes() - if err != nil { - return CommandAuthor{}, false - } - var a CommandAuthor - if json.Unmarshal(raw, &a) != nil || a.Author == "" { - return CommandAuthor{}, false - } - return a, true -} - -// key identifies the command by UID alone — globally unique, like the watch cursor key, -// so namespace/name/kind would be redundant (kept out for a tight key). -func (s *CommandAuthorStore) key(uid types.UID) string { - return resolveKeyPrefix(s.keyPrefix) + commandAuthorKeySuffix + escapeKeyField(string(uid)) -} diff --git a/internal/queue/command_author_store_test.go b/internal/queue/command_author_store_test.go deleted file mode 100644 index 289751ac..00000000 --- a/internal/queue/command_author_store_test.go +++ /dev/null @@ -1,83 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package queue - -import ( - "context" - "testing" - "time" - - "github.com/stretchr/testify/require" -) - -func newTestCommandAuthorStore(t *testing.T) *CommandAuthorStore { - t.Helper() - return newTestRedisStore(t).CommandAuthorStore() -} - -func TestCommandAuthorStore_RecordAndLookup(t *testing.T) { - store := newTestCommandAuthorStore(t) - ctx := context.Background() - - author := CommandAuthor{ - Author: "alice", - DisplayName: "Alice", - Email: "alice@example.com", - RequestedAt: "2026-06-29T00:00:00Z", - } - require.NoError(t, store.RecordCommandAuthor(ctx, "cr-uid", author)) - - got, ok := store.LookupCommandAuthor(ctx, "cr-uid") - require.True(t, ok) - require.Equal(t, author, got) -} - -func TestCommandAuthorStore_LookupMissIsImmediate(t *testing.T) { - store := newTestCommandAuthorStore(t) - - // Present-or-never: an unrecorded uid is an immediate miss, no wait. - _, ok := store.LookupCommandAuthor(context.Background(), "never-recorded") - require.False(t, ok) -} - -func TestCommandAuthorStore_LastWriteWins(t *testing.T) { - store := newTestCommandAuthorStore(t) - ctx := context.Background() - - require.NoError(t, store.RecordCommandAuthor(ctx, "cr-uid", CommandAuthor{Author: "alice"})) - require.NoError(t, store.RecordCommandAuthor(ctx, "cr-uid", CommandAuthor{Author: "bob"})) - - got, ok := store.LookupCommandAuthor(ctx, "cr-uid") - require.True(t, ok) - require.Equal(t, "bob", got.Author) -} - -func TestCommandAuthorStore_EmptyAuthorIsMiss(t *testing.T) { - store := newTestCommandAuthorStore(t) - ctx := context.Background() - - // A record with no author cannot name a commit author, so it reads as a miss. - require.NoError(t, store.RecordCommandAuthor(ctx, "cr-uid", CommandAuthor{})) - _, ok := store.LookupCommandAuthor(ctx, "cr-uid") - require.False(t, ok) -} - -func TestCommandAuthorStore_RecordCarriesCleanupTTL(t *testing.T) { - redisStore, mr := newTestRedisStoreWithRedis(t) - store := redisStore.CommandAuthorStore() - ctx := context.Background() - - require.NoError(t, store.RecordCommandAuthor(ctx, "cr-uid", CommandAuthor{Author: "alice"})) - - // The record carries a fixed cleanup TTL and self-cleans once it elapses (an orphan - // command deleted before its reconcile never blocks Redis). - require.Equal(t, commandAuthorRecordTTL, mr.TTL(store.key("cr-uid"))) - mr.FastForward(commandAuthorRecordTTL + time.Second) - _, ok := store.LookupCommandAuthor(ctx, "cr-uid") - require.False(t, ok) -} - -func TestCommandAuthorStore_KeyReadableFormat(t *testing.T) { - store := newTestCommandAuthorStore(t) - require.Equal(t, "gitops-reverser:author:v1:command:cr-uid", store.key("cr-uid")) -} diff --git a/internal/queue/key_prefix_test.go b/internal/queue/key_prefix_test.go index 18591579..4a8ba07d 100644 --- a/internal/queue/key_prefix_test.go +++ b/internal/queue/key_prefix_test.go @@ -91,8 +91,6 @@ func TestRedisStore_KeyPrefixReachesEveryKeyFamily(t *testing.T) { stream := store.FactStream(RedisFactStreamConfig{}) require.Equal(t, "cell-a:tenant-7:author:v2:audit:route:default:apps/deployments", stream.streamKey(FactStreamKeyFor("default", gvr.GroupResource()))) - - require.Equal(t, "cell-a:tenant-7:author:v1:command:cr-uid", store.CommandAuthorStore().key("cr-uid")) } func TestRedisStore_DefaultPrefixIsUnchanged(t *testing.T) { @@ -103,7 +101,6 @@ func TestRedisStore_DefaultPrefixIsUnchanged(t *testing.T) { store := newPrefixedRedisStore(t, DefaultKeyPrefix) require.Equal(t, "gitops-reverser:watch:v1:target:gtuid-3:configmaps:cluster:last-rv", store.watchCursorKey("gtuid-3", types.CollectionKeyFor(coreConfigmapsGVR(), ""))) - require.Equal(t, "gitops-reverser:author:v1:command:cr-uid", store.CommandAuthorStore().key("cr-uid")) } // A store built without NewRedisStore — a zero-value struct in a test, a future constructor @@ -116,7 +113,6 @@ func TestRedisStore_ZeroValueStoreStillWritesPrefixedKeys(t *testing.T) { var store RedisStore // keyPrefix == "" require.Equal(t, "gitops-reverser:watch:v1:target:gtuid-3:configmaps:cluster:last-rv", store.watchCursorKey("gtuid-3", types.CollectionKeyFor(coreConfigmapsGVR(), ""))) - require.Equal(t, "gitops-reverser:author:v1:command:cr-uid", store.CommandAuthorStore().key("cr-uid")) require.Equal(t, "gitops-reverser:author:v2:audit:route:default:configmaps", store.FactStream(RedisFactStreamConfig{}).streamKey( FactStreamKeyFor("default", coreConfigmapsGVR().GroupResource()))) diff --git a/internal/queue/redis_store.go b/internal/queue/redis_store.go index fce81530..2b9e33c4 100644 --- a/internal/queue/redis_store.go +++ b/internal/queue/redis_store.go @@ -81,14 +81,6 @@ func (s *RedisStore) KeyPrefix() string { return s.keyPrefix } -// CommandAuthorStore builds the command-authorship store on this connection. Wire it -// when the validate-operator-types webhook is enabled; it does not depend on attribution. The -// record lives in the same top-level author domain as audit facts but in the separate -// command subfamily, with its own fixed cleanup TTL. -func (s *RedisStore) CommandAuthorStore() *CommandAuthorStore { - return &CommandAuthorStore{client: s.client, ttl: commandAuthorRecordTTL, keyPrefix: s.keyPrefix} -} - // Ping checks liveness of the underlying Redis/Valkey connection. The readiness gate // uses it so the pod does not report ready before its resume-cursor store is reachable. func (s *RedisStore) Ping(ctx context.Context) error { diff --git a/internal/webhook/admission_allow_handler.go b/internal/webhook/admission_allow_handler.go deleted file mode 100644 index bb5f6893..00000000 --- a/internal/webhook/admission_allow_handler.go +++ /dev/null @@ -1,26 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "context" - - "sigs.k8s.io/controller-runtime/pkg/webhook/admission" -) - -const ( - // ValidateAllPath is the broad observe-all validating admission endpoint. It - // matches every resource and always allows — a test-only capture/observation - // surface today (the e2e SUT wires it; the Helm chart does not), kept as the - // stable extension point for a future cluster-wide policy. Per-our-type handling - // (authorship, config validation) lives on ValidateOperatorTypesPath instead. - ValidateAllPath = "/validate-all" -) - -// AdmissionAllowHandler is a validating admission handler that always allows requests. -type AdmissionAllowHandler struct{} - -// Handle returns an allow response for every admission request. -func (AdmissionAllowHandler) Handle(_ context.Context, _ admission.Request) admission.Response { - return admission.Allowed("allowed") -} diff --git a/internal/webhook/admission_allow_handler_test.go b/internal/webhook/admission_allow_handler_test.go deleted file mode 100644 index 40838c1f..00000000 --- a/internal/webhook/admission_allow_handler_test.go +++ /dev/null @@ -1,18 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "context" - "testing" - - "github.com/stretchr/testify/assert" - ctrladmission "sigs.k8s.io/controller-runtime/pkg/webhook/admission" -) - -func TestAdmissionAllowHandler_AllowsEveryRequest(t *testing.T) { - resp := AdmissionAllowHandler{}.Handle(context.Background(), ctrladmission.Request{}) - - assert.True(t, resp.Allowed) - assert.Equal(t, "allowed", resp.Result.Message) -} diff --git a/internal/webhook/fuzz_test.go b/internal/webhook/fuzz_test.go index 588c0e94..1ee1a5ee 100644 --- a/internal/webhook/fuzz_test.go +++ b/internal/webhook/fuzz_test.go @@ -10,11 +10,10 @@ import ( ) // FuzzDecodeEventList is the project's dynamic-analysis (fuzzing) gate for the -// network-facing admission/audit surface — the code that decodes untrusted JSON -// posted by the Kubernetes API server. It varies the request body and asserts -// that the whole audit ingress path (decode -> intrinsic accept gate -> record) -// never panics and always answers with a valid HTTP status, and that the sibling -// admission uid probe never panics on the same untrusted bytes. +// network-facing audit surface — the code that decodes untrusted JSON posted by +// the Kubernetes API server. It varies the request body and asserts that the +// whole audit ingress path (decode -> intrinsic accept gate -> record) never +// panics and always answers with a valid HTTP status. // // The handler carries no FactRecorder, so processing is side-effect-free: this // exercises decoding and classification without a live store. Seed inputs are @@ -30,7 +29,6 @@ func FuzzDecodeEventList(f *testing.F) { `{`, // truncated `not json`, ``, - `{"metadata":{"uid":"abc-123"}}`, // admission uid probe shape } for _, s := range seeds { f.Add([]byte(s)) @@ -52,9 +50,5 @@ func FuzzDecodeEventList(f *testing.F) { if w.Code < 100 || w.Code > 599 { t.Fatalf("invalid HTTP status %d for body %q", w.Code, body) } - - // The admission command-authorship path parses the same class of untrusted - // JSON with a lightweight probe; it must also never panic. - _ = commandObjectUID(body) }) } diff --git a/internal/webhook/validate_operator_types_handler.go b/internal/webhook/validate_operator_types_handler.go deleted file mode 100644 index ef09336b..00000000 --- a/internal/webhook/validate_operator_types_handler.go +++ /dev/null @@ -1,175 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "context" - "encoding/json" - "time" - - authnv1 "k8s.io/api/authentication/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/types" - "sigs.k8s.io/controller-runtime/pkg/client" - logf "sigs.k8s.io/controller-runtime/pkg/log" - "sigs.k8s.io/controller-runtime/pkg/webhook/admission" - - "github.com/ConfigButler/gitops-reverser/internal/queue" -) - -// ValidateOperatorTypesPath is the validating admission endpoint scoped to our own -// operator CRDs. Today its one job is command authorship — capturing the submitter of a -// command kind (a CommitRequest) into Redis and always allowing — but it is the intended -// home for per-our-type admission generally (e.g. config validation of WatchRule / -// GitProvider / GitTarget later), which would be added as additional webhook-config -// entries with their own rules and failurePolicy. Distinct from the broad observe-all -// ValidateAllPath. -const ValidateOperatorTypesPath = "/validate-operator-types" - -const ( - // displayNameExtraKey is the user.extra key carrying the OIDC "name" claim, when - // the API server is configured to map it. It mirrors the audit path's extra key - // (internal/queue/audit_event_parsing.go) so a command author and a mirrored- - // resource author resolve the same display name from the same claim. - displayNameExtraKey = "configbutler.ai/claims/display-name" - // emailExtraKey is the user.extra key carrying the OIDC "email" claim. - emailExtraKey = "configbutler.ai/claims/email" -) - -// isCommandKind reports whether gr is one of our own command CRDs whose submitter we -// capture at admission. It is the registry of command kinds: adding one is a single -// case here plus one rule in the webhook config — the handler body does not change. -func isCommandKind(gr metav1.GroupResource) bool { - switch gr { - case metav1.GroupResource{Group: "configbutler.ai", Resource: "commitrequests"}: - return true - // future: case metav1.GroupResource{Group: "configbutler.ai", Resource: ""}: - default: - return false - } -} - -// CommandAuthorRecorder records the authenticated submitter of one command object. -// *queue.CommandAuthorStore satisfies it; the interface keeps the handler unit-testable -// without a live Redis. -type CommandAuthorRecorder interface { - RecordCommandAuthor(ctx context.Context, uid types.UID, author queue.CommandAuthor) error -} - -// ValidateOperatorTypesHandler is the admission handler for our operator CRDs. It dispatches on the -// resource under review and does two unrelated things: -// -// - For a COMMAND kind (a CommitRequest) it captures the authenticated submitter into the -// CommandAuthorStore and always allows — pure observation with a single side effect (a Redis -// upsert), never a rejection, so a user's command never depends on it succeeding (a missed -// capture leaves the request without a claimed actor; see -// docs/spec/commitrequest-admission-authorship.md). -// - For a WatchRule it validates the one cross-object rule admission can usefully give feedback -// on: a second source namespace against a GitTarget that declared its folder namespace-free. -// That one CAN reject, and it is the only thing here that does. -// -// The two branches share an endpoint and nothing else. Validation runs on dry-run (it has no side -// effects, and a server-side dry-run must report the rejection it would get), while the capture -// branch honours NoneOnDryRun. -type ValidateOperatorTypesHandler struct { - Store CommandAuthorRecorder - // Client reads the GitTarget a WatchRule names, and that target's other rules. Nil leaves the - // WatchRule branch evaluating nothing and allowing, which is the same degradation the - // fail-open failurePolicy already provides. - Client client.Client -} - -// Handle dispatches on the resource under review: a WatchRule is validated (and may be rejected), -// and a command CREATE has its {uid → author} recorded before the object persists (the authorship -// invariant, §2) and is then allowed. Every early return on the command path still allows: an -// unrecognised kind, a dry-run, a missing uid, or an unauthenticated request simply records nothing -// and leaves the request without a claimed actor downstream. -func (h *ValidateOperatorTypesHandler) Handle(ctx context.Context, req admission.Request) admission.Response { - log := logf.FromContext(ctx).WithName("validate-operator-types") - - gr := metav1.GroupResource{Group: req.Resource.Group, Resource: req.Resource.Resource} - if isWatchRuleKind(gr) { - return h.validateWatchRuleSourceNamespaces(ctx, req) - } - if !isCommandKind(gr) { - // Belt-and-suspenders; the webhook rules already scope us to the kinds above. - return admission.Allowed("not a kind this endpoint judges") - } - // Dry-run never persists, so the controller will never read this record — and we - // declare sideEffects: NoneOnDryRun, so we must honor it. - if req.DryRun != nil && *req.DryRun { - return admission.Allowed("dry-run: not recorded") - } - - // No store means the webhook is running without a Redis backend (admission is on by - // default, but command-author capture is Redis-backed). Allow without recording; the - // controller then uses the no-actor path (AuthorAttributed=False). - if h.Store == nil { - return admission.Allowed("no author store: not recorded") - } - - // metadata.uid identifies the object (not req.UID, which identifies the review). The - // API server assigns it before validating admission runs, so it is present even for - // a generateName CREATE — no response-body name recovery is ever needed. - uid := commandObjectUID(req.Object.Raw) - if uid == "" { - log.Info("command object has no uid; not recording", "namespace", req.Namespace, "name", req.Name) - return admission.Allowed("no uid: not recorded") - } - - author := queue.CommandAuthor{ - Author: req.UserInfo.Username, // effective (impersonated) user — the apiserver resolved it - DisplayName: firstExtraValue(req.UserInfo.Extra, displayNameExtraKey), - Email: firstExtraValue(req.UserInfo.Extra, emailExtraKey), - RequestedAt: time.Now().UTC().Format(time.RFC3339Nano), - } - if author.Author == "" { - return admission.Allowed("no user to attribute") // anonymous never reaches a persisted CREATE - } - - // Synchronous, best-effort: the write completes before we return, so it is present - // before the object is visible. A failure leaves the request without a claimed actor — - // never block the command. - if err := h.Store.RecordCommandAuthor(ctx, uid, author); err != nil { - log.Error(err, "record command author failed; request will claim no actor", - "resource", gr.String(), "namespace", req.Namespace, "name", req.Name, "uid", uid) - return admission.Allowed("author record failed") - } - // Info (not V(1)) on purpose: command CREATEs are rare, and this line proves the - // webhook was actually invoked and the author captured — the first thing to check - // when a CommitRequest unexpectedly reports AuthorAttributed=False. - log.Info("recorded command author at admission", - "resource", gr.String(), "namespace", req.Namespace, "name", req.Name, - "uid", uid, "author", author.Author) - return admission.Allowed("author recorded") -} - -// commandObjectUID extracts metadata.uid from an admission request's raw object. It -// reads only the field it needs (like the audit handler's metadata probes), so it never -// depends on the command kind being registered in a decoder scheme. -func commandObjectUID(raw []byte) types.UID { - if len(raw) == 0 { - return "" - } - var probe struct { - Metadata struct { - UID types.UID `json:"uid"` - } `json:"metadata"` - } - if err := json.Unmarshal(raw, &probe); err != nil { - return "" - } - return probe.Metadata.UID -} - -// firstExtraValue returns the first value for key in an admission request's -// user.extra map, or "" when the key is absent or carries no values. It mirrors the -// audit path's helper (internal/queue/audit_event_parsing.go) for the same user.extra -// shape carried on an admission request. -func firstExtraValue(extra map[string]authnv1.ExtraValue, key string) string { - values := extra[key] - if len(values) == 0 { - return "" - } - return values[0] -} diff --git a/internal/webhook/validate_operator_types_handler_test.go b/internal/webhook/validate_operator_types_handler_test.go deleted file mode 100644 index 7d184517..00000000 --- a/internal/webhook/validate_operator_types_handler_test.go +++ /dev/null @@ -1,190 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "context" - "errors" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - admissionv1 "k8s.io/api/admission/v1" - authnv1 "k8s.io/api/authentication/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" - ctrladmission "sigs.k8s.io/controller-runtime/pkg/webhook/admission" - - "github.com/ConfigButler/gitops-reverser/internal/queue" -) - -type recordedAuthor struct { - uid types.UID - author queue.CommandAuthor -} - -// fakeCommandAuthorRecorder captures RecordCommandAuthor calls and can be made to fail. -type fakeCommandAuthorRecorder struct { - calls []recordedAuthor - err error -} - -func (f *fakeCommandAuthorRecorder) RecordCommandAuthor( - _ context.Context, uid types.UID, author queue.CommandAuthor, -) error { - f.calls = append(f.calls, recordedAuthor{uid: uid, author: author}) - return f.err -} - -func commitRequestResource() metav1.GroupVersionResource { - return metav1.GroupVersionResource{Group: "configbutler.ai", Version: "v1alpha3", Resource: "commitrequests"} -} - -// commandReview builds an admission.Request for a command CREATE with the given raw -// object body and submitter identity. -func commandReview( - resource metav1.GroupVersionResource, raw string, user authnv1.UserInfo, dryRun *bool, -) ctrladmission.Request { - return ctrladmission.Request{ - AdmissionRequest: admissionv1.AdmissionRequest{ - UID: "review-uid", - Resource: resource, - Name: "save-1", - Namespace: "team-a", - Operation: admissionv1.Create, - UserInfo: user, - DryRun: dryRun, - Object: runtime.RawExtension{Raw: []byte(raw)}, - }, - } -} - -func TestValidateOperatorTypesHandler_RecordsCommandAuthor(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - user := authnv1.UserInfo{ - Username: "alice", - Extra: map[string]authnv1.ExtraValue{ - displayNameExtraKey: {"Alice Liddell"}, - emailExtraKey: {"alice@example.com"}, - }, - } - resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, user, nil)) - - assert.True(t, resp.Allowed) - require.Len(t, rec.calls, 1) - assert.Equal(t, types.UID("cr-uid"), rec.calls[0].uid) - got := rec.calls[0].author - assert.Equal(t, "alice", got.Author) - assert.Equal(t, "Alice Liddell", got.DisplayName) - assert.Equal(t, "alice@example.com", got.Email) - assert.NotEmpty(t, got.RequestedAt, "RequestedAt is stamped for lag/debug") -} - -func TestValidateOperatorTypesHandler_NilStoreNoOp(t *testing.T) { - // Admission stays enabled without Redis: no Store, so the handler must allow the - // command without recording (and without a nil dereference). - h := &ValidateOperatorTypesHandler{} - - resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, - authnv1.UserInfo{Username: "alice"}, nil)) - - assert.True(t, resp.Allowed, "a missing author store never blocks a command") -} - -func TestValidateOperatorTypesHandler_ServiceAccountSubmitter(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - user := authnv1.UserInfo{Username: "system:serviceaccount:team-a:deployer"} - resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"sa-uid"}}`, user, nil)) - - assert.True(t, resp.Allowed) - require.Len(t, rec.calls, 1) - assert.Equal(t, "system:serviceaccount:team-a:deployer", rec.calls[0].author.Author) -} - -func TestValidateOperatorTypesHandler_NonCommandKindIsNotRecorded(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - configmaps := metav1.GroupVersionResource{Group: "", Version: "v1", Resource: "configmaps"} - resp := h.Handle(context.Background(), - commandReview(configmaps, `{"metadata":{"uid":"cm-uid"}}`, authnv1.UserInfo{Username: "alice"}, nil)) - - assert.True(t, resp.Allowed) - assert.Empty(t, rec.calls, "a non-command kind is never recorded") -} - -func TestValidateOperatorTypesHandler_DryRunIsNotRecorded(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - dryRun := true - resp := h.Handle( - context.Background(), - commandReview( - commitRequestResource(), - `{"metadata":{"uid":"cr-uid"}}`, - authnv1.UserInfo{Username: "alice"}, - &dryRun, - ), - ) - - assert.True(t, resp.Allowed) - assert.Empty(t, rec.calls, "dry-run never persists, so nothing is recorded") -} - -func TestValidateOperatorTypesHandler_MissingUIDIsNotRecorded(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{}}`, authnv1.UserInfo{Username: "alice"}, nil)) - - assert.True(t, resp.Allowed) - assert.Empty(t, rec.calls, "without a uid the record cannot be keyed") -} - -func TestValidateOperatorTypesHandler_NoUserIsNotRecorded(t *testing.T) { - rec := &fakeCommandAuthorRecorder{} - h := &ValidateOperatorTypesHandler{Store: rec} - - resp := h.Handle(context.Background(), - commandReview(commitRequestResource(), `{"metadata":{"uid":"cr-uid"}}`, authnv1.UserInfo{}, nil)) - - assert.True(t, resp.Allowed) - assert.Empty(t, rec.calls, "an unauthenticated request names no author") -} - -// A failed Redis write must never reject the command: the handler logs and still -// allows, and the controller degrades to the committer. -func TestValidateOperatorTypesHandler_RecordErrorStillAllows(t *testing.T) { - rec := &fakeCommandAuthorRecorder{err: errors.New("redis down")} - h := &ValidateOperatorTypesHandler{Store: rec} - - resp := h.Handle( - context.Background(), - commandReview( - commitRequestResource(), - `{"metadata":{"uid":"cr-uid"}}`, - authnv1.UserInfo{Username: "alice"}, - nil, - ), - ) - - assert.True(t, resp.Allowed, "a record failure must never block the command") - require.Len(t, rec.calls, 1, "the write was attempted") -} - -func TestCommandObjectUID(t *testing.T) { - assert.Equal(t, types.UID("cr-uid"), commandObjectUID([]byte(`{"metadata":{"uid":"cr-uid"}}`))) - assert.Empty(t, commandObjectUID([]byte(`{"metadata":{}}`))) - assert.Empty(t, commandObjectUID(nil)) - assert.Empty(t, commandObjectUID([]byte("not json"))) -} diff --git a/internal/webhook/watchrule_source_namespace_admission.go b/internal/webhook/watchrule_source_namespace_admission.go deleted file mode 100644 index 3b78f2f4..00000000 --- a/internal/webhook/watchrule_source_namespace_admission.go +++ /dev/null @@ -1,138 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "context" - "encoding/json" - "fmt" - "sort" - - apierrors "k8s.io/apimachinery/pkg/api/errors" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - k8stypes "k8s.io/apimachinery/pkg/types" - "sigs.k8s.io/controller-runtime/pkg/client" - logf "sigs.k8s.io/controller-runtime/pkg/log" - "sigs.k8s.io/controller-runtime/pkg/webhook/admission" - - v1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" -) - -// isWatchRuleKind reports whether gr is the namespaced WatchRule. ClusterWatchRule is deliberately -// absent: it watches cluster-scoped resources, which have no namespace to bring to a target. -func isWatchRuleKind(gr metav1.GroupResource) bool { - return gr == metav1.GroupResource{Group: "configbutler.ai", Resource: "watchrules"} -} - -// validateWatchRuleSourceNamespaces is the FEEDBACK half of the one-source-namespace rule -// (docs/layout/model.md § "The second guard"): a GitTarget that declared its folder namespace-free -// admits exactly one source namespace, and a rule bringing a second one is rejected here, at the -// moment the mistake is made, instead of leaving a target that refuses every write until someone -// reads its status. -// -// It is not the rule's enforcement. The writer's own precondition is, and it holds whatever this -// returns: this check is one-shot, so it cannot see a serializeNamespace flipped to false after the -// rules were created, the webhook is fail-open (failurePolicy: Ignore), and a cluster can run the -// operator without the admission server at all. Everything that must be true of the bytes is -// decided at the write. See docs/spec/where-validation-lives.md. -// -// Every failure to evaluate ALLOWS, for the same reason: a rejection this handler cannot justify -// is a rejection of the user's object on the strength of an unread GitTarget. -func (h *ValidateOperatorTypesHandler) validateWatchRuleSourceNamespaces( - ctx context.Context, - req admission.Request, -) admission.Response { - log := logf.FromContext(ctx).WithName("watchrule-source-namespaces") - if h.Client == nil { - return admission.Allowed("no client: source namespaces not evaluated") - } - - var rule v1alpha3.WatchRule - if err := json.Unmarshal(req.Object.Raw, &rule); err != nil { - log.Error(err, "could not decode the WatchRule under review", "namespace", req.Namespace, "name", req.Name) - return admission.Allowed("undecodable: source namespaces not evaluated") - } - // A CREATE carries no namespace on the object itself until the API server defaults it. - if rule.Namespace == "" { - rule.Namespace = req.Namespace - } - - var target v1alpha3.GitTarget - targetKey := k8stypes.NamespacedName{Namespace: rule.Namespace, Name: rule.Spec.GitTargetRef.Name} - if err := h.Client.Get(ctx, targetKey, &target); err != nil { - if !apierrors.IsNotFound(err) { - log.Error(err, "could not read the GitTarget this rule names", "gitTarget", targetKey) - } - // A rule naming a target that does not exist yet is ordinary: rules and targets are - // applied in whatever order the manifest happens to list them, and the reconciler holds - // such a rule unready. It is not this check's business. - return admission.Allowed("GitTarget unreadable: source namespaces not evaluated") - } - if target.Spec.SerializeNamespace == nil || *target.Spec.SerializeNamespace { - return admission.Allowed("the target declares no namespace-free folder") - } - - admitted, err := h.admittedSourceNamespaces(ctx, &target, rule.Name) - if err != nil { - log.Error(err, "could not list the sibling WatchRules", "gitTarget", targetKey) - return admission.Allowed("sibling rules unreadable: source namespaces not evaluated") - } - - for _, item := range rule.Spec.Rules { - if item.IsSourceNamespaceWildcard() { - return admission.Denied(fmt.Sprintf( - "GitTarget %s/%s sets spec.serializeNamespace: false, which admits exactly one source "+ - "namespace, and sourceNamespace %q cannot be shown to be one namespace", - target.Namespace, target.Name, v1alpha3.SourceNamespaceWildcard)) - } - admitted[item.EffectiveSourceNamespace(rule.Namespace)] = struct{}{} - } - if len(admitted) <= 1 { - return admission.Allowed("one source namespace") - } - - names := make([]string, 0, len(admitted)) - for ns := range admitted { - names = append(names, ns) - } - sort.Strings(names) - return admission.Denied(fmt.Sprintf( - "GitTarget %s/%s sets spec.serializeNamespace: false, which admits exactly one source namespace, "+ - "but this rule would make %v reach it; the documents it writes carry no metadata.namespace, so "+ - "two namespaces produce one document two objects overwrite in turn. Unset serializeNamespace on "+ - "the target so each document takes the namespace of the root governing it, or point this rule at "+ - "a target of its own", - target.Namespace, target.Name, names)) -} - -// admittedSourceNamespaces is the set already reaching the target, excluding the rule under review -// so an UPDATE is judged on what it would become rather than on what it is. It mirrors -// (internal/git).resolveSourceNamespaces, which computes the same set at the write. -// -// A wildcard among the OTHER rules is not this request's fault, so it adds nothing here: the write -// refuses that target anyway, and rejecting an unrelated edit for it would leave the user unable to -// fix the rule that caused it. -func (h *ValidateOperatorTypesHandler) admittedSourceNamespaces( - ctx context.Context, - target *v1alpha3.GitTarget, - excludeRule string, -) (map[string]struct{}, error) { - var rules v1alpha3.WatchRuleList - if err := h.Client.List(ctx, &rules, client.InNamespace(target.Namespace)); err != nil { - return nil, err - } - admitted := map[string]struct{}{target.Namespace: {}} - for i := range rules.Items { - sibling := &rules.Items[i] - if sibling.Name == excludeRule || sibling.Spec.GitTargetRef.Name != target.Name { - continue - } - for _, item := range sibling.Spec.Rules { - if item.IsSourceNamespaceWildcard() { - continue - } - admitted[item.EffectiveSourceNamespace(sibling.Namespace)] = struct{}{} - } - } - return admitted, nil -} diff --git a/internal/webhook/watchrule_source_namespace_admission_test.go b/internal/webhook/watchrule_source_namespace_admission_test.go deleted file mode 100644 index b0b83f69..00000000 --- a/internal/webhook/watchrule_source_namespace_admission_test.go +++ /dev/null @@ -1,182 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package webhook - -import ( - "encoding/json" - "testing" - - "github.com/fluxcd/pkg/apis/meta" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - admissionv1 "k8s.io/api/admission/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - clientgoscheme "k8s.io/client-go/kubernetes/scheme" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/client/fake" - ctrladmission "sigs.k8s.io/controller-runtime/pkg/webhook/admission" - - v1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" -) - -// The feedback half of the one-source-namespace rule. Everything asserted here is a rejection or a -// deliberate refusal to judge; the rule itself is enforced at the write, and -// internal/git asserts that half. - -func namespaceFreeTarget(serialize *bool) *v1alpha3.GitTarget { - return &v1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "checkout-artifact", Namespace: "shop"}, - Spec: v1alpha3.GitTargetSpec{ - Path: "apps/checkout", - SerializeNamespace: serialize, - }, - } -} - -func watchRuleFor(name, sourceNamespace string) *v1alpha3.WatchRule { - return &v1alpha3.WatchRule{ - ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "shop"}, - Spec: v1alpha3.WatchRuleSpec{ - GitTargetRef: meta.LocalObjectReference{Name: "checkout-artifact"}, - Rules: []v1alpha3.ResourceRule{{ - Resources: []string{"configmaps"}, - SourceNamespace: sourceNamespace, - }}, - }, - } -} - -func watchRuleReview(t *testing.T, rule *v1alpha3.WatchRule, operation admissionv1.Operation) ctrladmission.Request { - t.Helper() - raw, err := json.Marshal(rule) - require.NoError(t, err) - return ctrladmission.Request{ - AdmissionRequest: admissionv1.AdmissionRequest{ - Resource: metav1.GroupVersionResource{ - Group: "configbutler.ai", Version: "v1alpha3", Resource: "watchrules"}, - Operation: operation, - Namespace: rule.Namespace, - Name: rule.Name, - Object: runtime.RawExtension{Raw: raw}, - }, - } -} - -func watchRuleHandler(t *testing.T, objects ...client.Object) *ValidateOperatorTypesHandler { - t.Helper() - scheme := runtime.NewScheme() - require.NoError(t, clientgoscheme.AddToScheme(scheme)) - require.NoError(t, v1alpha3.AddToScheme(scheme)) - return &ValidateOperatorTypesHandler{ - Client: fake.NewClientBuilder().WithScheme(scheme).WithObjects(objects...).Build(), - } -} - -func TestWatchRuleAdmission_SecondSourceNamespaceIsRejected(t *testing.T) { - no := false - handler := watchRuleHandler(t, namespaceFreeTarget(&no), watchRuleFor("content", "")) - - response := handler.Handle(t.Context(), - watchRuleReview(t, watchRuleFor("content-billing", "billing"), admissionv1.Create)) - - require.False(t, response.Allowed) - assert.Contains(t, response.Result.Message, "admits exactly one source namespace") - assert.Contains(t, response.Result.Message, "[billing shop]") -} - -func TestWatchRuleAdmission_FirstSourceNamespaceIsAdmitted(t *testing.T) { - no := false - handler := watchRuleHandler(t, namespaceFreeTarget(&no)) - - response := handler.Handle(t.Context(), watchRuleReview(t, watchRuleFor("content", ""), admissionv1.Create)) - - assert.True(t, response.Allowed, "the target's own namespace is the one namespace it admits") -} - -// A wildcard is rejected statically: neither reading of "*" can be shown to be one namespace, so -// nothing is enumerated to find out. -func TestWatchRuleAdmission_WildcardIsRejectedWithoutEnumerating(t *testing.T) { - no := false - handler := watchRuleHandler(t, namespaceFreeTarget(&no)) - - response := handler.Handle(t.Context(), - watchRuleReview(t, watchRuleFor("everything", v1alpha3.SourceNamespaceWildcard), admissionv1.Create)) - - require.False(t, response.Allowed) - assert.Contains(t, response.Result.Message, "cannot be shown to be one namespace") -} - -// An UPDATE is judged on what the rule would BECOME. Re-submitting the rule that already holds the -// second namespace must not be rejected for its own existing value, or the only edit that could fix -// it would be blocked by it. -func TestWatchRuleAdmission_UpdateExcludesTheRuleUnderReview(t *testing.T) { - no := false - existing := watchRuleFor("content", "billing") - handler := watchRuleHandler(t, namespaceFreeTarget(&no), existing) - - narrowed := watchRuleFor("content", "shop") - response := handler.Handle(t.Context(), watchRuleReview(t, narrowed, admissionv1.Update)) - - assert.True(t, response.Allowed, "an edit that narrows the rule back to one namespace must land") -} - -func TestWatchRuleAdmission_UnfencedTargetsAreNotJudged(t *testing.T) { - yes := true - for _, tc := range []struct { - name string - serialize *bool - }{ - {"unset infers per document and is never fenced", nil}, - {"true means every document carries its own namespace", &yes}, - } { - t.Run(tc.name, func(t *testing.T) { - handler := watchRuleHandler(t, namespaceFreeTarget(tc.serialize), watchRuleFor("content", "")) - - response := handler.Handle(t.Context(), - watchRuleReview(t, watchRuleFor("content-billing", "billing"), admissionv1.Create)) - - assert.True(t, response.Allowed) - }) - } -} - -// Every way of failing to evaluate ALLOWS. A rejection this handler cannot justify is a rejection -// of the user's object on the strength of a GitTarget nobody read. -func TestWatchRuleAdmission_UnevaluatableRequestsAreAllowed(t *testing.T) { - no := false - - t.Run("no client", func(t *testing.T) { - handler := &ValidateOperatorTypesHandler{} - response := handler.Handle(t.Context(), - watchRuleReview(t, watchRuleFor("content-billing", "billing"), admissionv1.Create)) - assert.True(t, response.Allowed) - }) - - t.Run("the GitTarget does not exist yet", func(t *testing.T) { - handler := watchRuleHandler(t) - response := handler.Handle(t.Context(), - watchRuleReview(t, watchRuleFor("content-billing", "billing"), admissionv1.Create)) - assert.True(t, response.Allowed, "rules and targets are applied in whatever order the manifest lists them") - }) - - t.Run("an undecodable object", func(t *testing.T) { - handler := watchRuleHandler(t, namespaceFreeTarget(&no)) - review := watchRuleReview(t, watchRuleFor("content", ""), admissionv1.Create) - review.Object.Raw = []byte("{not json") - assert.True(t, handler.Handle(t.Context(), review).Allowed) - }) -} - -// A rule pointing at a different GitTarget contributes nothing, so a target fenced to one namespace -// is not rejected because of somebody else's rule. -func TestWatchRuleAdmission_OtherTargetsRulesAreNotCounted(t *testing.T) { - no := false - elsewhere := watchRuleFor("elsewhere", "billing") - elsewhere.Spec.GitTargetRef.Name = "other-artifact" - handler := watchRuleHandler(t, namespaceFreeTarget(&no), elsewhere) - - response := handler.Handle(t.Context(), watchRuleReview(t, watchRuleFor("content", ""), admissionv1.Create)) - - assert.True(t, response.Allowed) -} diff --git a/test/e2e/author_mode_test.go b/test/e2e/author_mode_test.go index 89ddf15e..70c41067 100644 --- a/test/e2e/author_mode_test.go +++ b/test/e2e/author_mode_test.go @@ -16,7 +16,7 @@ import "testing" // configured-author row below therefore turns attribution off explicitly. func TestConfiguredAuthorModeFromArgs_AllScenarios(t *testing.T) { // The real jsonpath output shape: a bracketed, quoted, comma-separated list. - const live = `["--metrics-bind-address=:8443","--admission-webhook",` + + const live = `["--metrics-bind-address=:8443",` + `"--redis-addr=valkey.valkey-e2e.svc.cluster.local:6379","--redis-insecure"]` for _, tc := range []struct { diff --git a/test/e2e/commit_request_e2e_test.go b/test/e2e/commit_request_e2e_test.go index 595dd969..d19d0e82 100644 --- a/test/e2e/commit_request_e2e_test.go +++ b/test/e2e/commit_request_e2e_test.go @@ -216,15 +216,9 @@ var _ = Describe("Save Request", Label("commit-request", "audit-consumer"), Orde _, _ = kubectlRunInNamespace(testNs, "delete", "commitrequest", commitRequestName, "--ignore-not-found=true") }) - // generateName authorship is now captured at admission, not from the audit - // response body: the API server assigns metadata.name and metadata.uid before - // validating admission runs, so the validate-operator-types webhook records the - // submitter keyed by uid with no response-body name recovery (the old audit - // generateName headache is gone, see - // docs/spec/commitrequest-admission-authorship.md). This spec proves a - // generateName CommitRequest still finalizes and becomes Ready. It is skipped in - // configured-author mode, where the edit's window is committer-authored and the - // named admission author would not match it end to end. + // A generateName CommitRequest is attributed from its audit fact, keyed by the uid and + // resourceVersion the API server assigned, so no name recovery from the response body is needed. + // This spec proves a generateName CommitRequest still finalizes and becomes Ready. It("finalizes a CommitRequest created with metadata.generateName", func() { basePath := "e2e/commit-request-test" seed := GinkgoRandomSeed() diff --git a/test/e2e/gittarget_overlap_e2e_test.go b/test/e2e/gittarget_overlap_e2e_test.go index 358e7874..36f66d4b 100644 --- a/test/e2e/gittarget_overlap_e2e_test.go +++ b/test/e2e/gittarget_overlap_e2e_test.go @@ -16,8 +16,7 @@ import ( // GitTarget path may be equal to, an ancestor of, or a descendant of another's. // Sibling folders are fine; nesting is rejected at the Validated gate so every // materialized folder has exactly one owner. The reject path surfaces as a -// reconcile-time status condition (Ready=False / ValidationFailed), not through -// the current allow-only validating admission webhook. +// reconcile-time status condition (Ready=False / ValidationFailed). var _ = Describe("Manager GitTarget Overlap Guard", Label("manager"), Ordered, func() { const providerName = "gitprovider-overlap" diff --git a/test/e2e/quickstart_framework_e2e_test.go b/test/e2e/quickstart_framework_e2e_test.go index f94ec1bb..2745336e 100644 --- a/test/e2e/quickstart_framework_e2e_test.go +++ b/test/e2e/quickstart_framework_e2e_test.go @@ -138,9 +138,9 @@ func (r *quickstartFrameworkRun) helmInstallReadmeQuickstart() { chart = "charts/gitops-reverser" } - // ONE apply, exactly as the README's step 4 does it. This works because no webhook this chart - // installs is failurePolicy: Fail — the starter GitTarget is gated by nothing, so it is admitted - // while the Deployment it rolls is still coming up. Namespace authorization is enforced at + // ONE apply, exactly as the README's step 4 does it. This works because the chart installs no + // admission webhook — the starter GitTarget is gated by nothing, so it is admitted while the + // Deployment it rolls is still coming up. Namespace authorization is enforced at // reconcile instead (checkSourceAuthorization, inside the Validated gate), which does not need // the manager to be reachable at admission time. args := []string{ @@ -177,7 +177,7 @@ func (r *quickstartFrameworkRun) helmInstallReadmeQuickstart() { // verifyNoRedisConfiguredAuthor asserts the controller logged the no-Redis // configured-author sentinel (main.go emits "configured-author mode: no Redis // configured" only when queue.redis.addr is empty) and that the pod is not -// crash-looping — proving admission-on-without-Redis is a healthy no-op. +// crash-looping — proving running without Redis is a healthy mode. func (r *quickstartFrameworkRun) verifyNoRedisConfiguredAuthor() { releaseNs := resolveE2ENamespace() release := resolveE2EInstallName(releaseNs) @@ -199,7 +199,7 @@ func (r *quickstartFrameworkRun) verifyNoRedisConfiguredAuthor() { for _, field := range strings.Fields(restarts) { count, convErr := strconv.Atoi(field) Expect(convErr).NotTo(HaveOccurred()) - Expect(count).To(Equal(0), "controller must not crash-loop with admission on and no Redis") + Expect(count).To(Equal(0), "controller must not crash-loop without Redis") } } diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index f124e212..32b53790 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -807,24 +807,10 @@ spec: // each request reports the apply: Applied=True, with the revision Flux applied. The second save // is made as soon as the first is pushed, so the two may share one apply. // - // No write in the workspace names its author, since kcp sends no audit facts, and the request's - // submitter is captured at admission in this cluster. The GitTarget's grant for its own namespace - // allows a takeover, each request asks for one, and the workspace's KubeProvider has never - // received a fact, so each request takes over the window of its edit: its submitter authors the commit, and the message - // ends with a trailer that says so. Every event from the workspace waits out the attribution - // grace first, so the requests wait longer than that for their window. - It("takes over the window of an edit in the workspace, and reports the apply on each of two saves", func() { - verifyResourceCondition("kubeprovider", ws, testNs, "AuditFactsReceived", "Unknown", "", "") - _, err := kubectlRunInNamespace( - testNs, - "patch", - "gittarget", - p.targetName, - "--type=merge", - "-p", - `{"spec":{"allowedCommitRequests":[{"consumerNamespaces":{"from":"Same"},"unattributedWindows":"TakeOver"}]}}`, - ) - Expect(err).NotTo(HaveOccurred()) + // No write in the workspace names its author, since kcp sends no audit facts, and neither does + // the request, so each request attaches to the window of its edit. Every event from the workspace + // waits out the attribution grace first, so the requests wait longer than that for their window. + It("saves an edit in the workspace, and reports the apply on each of two saves", func() { names := []string{"sc-flux-save-1", "sc-flux-save-2"} commits := map[string]string{} for i, name := range names { @@ -841,10 +827,6 @@ spec: committed, inGit := r.committedValue(Default, timingConfigMap) Expect(inGit).To(BeTrue()) Expect(committed).To(Equal(value), "the save's commit holds its edit") - message, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commits[name]) - Expect(err).NotTo(HaveOccurred()) - Expect(message).To(ContainSubstring("Author-Source: CommitRequest "+testNs+"/"+name), - "the commit says its author is the request's submitter") } last := commits[names[len(names)-1]] @@ -872,7 +854,7 @@ spec: // A save made in the workspace itself: a RemoteSaveRequest, served because the workspace's // KubeProvider enables them. It names the GitTarget, which must admit it: one is refused while // the GitTarget admits none. kcp sends no audit facts, so the request names nobody, like the - // edit, and attaches to the edit's window under the ordinary rule, with no takeover. Its own author is + // edit, and attaches to the edit's window under the ordinary rule. Its own author is // looked for first, for the whole attribution grace, so the edit is made once the request is // waiting for a window. It("saves an edit made in the workspace with a RemoteSaveRequest created there", func() { @@ -1048,7 +1030,6 @@ spec: message: "e2e: save %s and apply it" catchUp: AfterPush window: - author: TakeOver attachTimeout: "45s" maxDuration: "4s" `, name, namespace, gitTarget, name), "apply", "-f", "-") From a2023a9a05545e2d0a2ba137beede35d0d42bbb3 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 09:53:46 +0000 Subject: [PATCH 25/33] feat(commitrequest)!: serve requests through one core, with the author from audit A CommitRequest is served by commitRequestCore, which decides every step from the object as it is stored, so the control-cluster reconciler is a thin front end, and the source-cluster watch can become another: - The named GitTarget's KubeProvider alone picks the path that owns a request. On the control cluster that is an inCluster provider. One with a kubeconfig may point back here and serve the request itself, so it is left alone, unless it serves no CommitRequests: then the request is NotAllowed, and the message says to create it on that cluster. - The author comes from the request's own audit fact on the route of that KubeProvider, awaited with the attribution grace whenever attribution is on. The reconciler acquires the fact stream of commitrequests on the route, which it never did. A request no fact names, on a provider that has received audit facts, is AuthorUnresolved and saves nothing. - The owner claims a request by writing its first status, status.author and Ready=Unknown (Saving), before it attaches. Every status write is conditional on the version it was decided from; after the claim a conflict re-applies the worker's outcome and never restarts the save. - A GitTarget that does not exist is GitTargetNotFound once attachTimeout has passed, written only into an empty status. spec.window.attachTimeout defaults to 5s, since the request now waits for its author as its edits do. Steps of different requests run four at a time, so one request's author wait does not hold up the others. BREAKING CHANGE: a CommitRequest's author is found from its audit event, reported in status.author; in progress it is Ready=Unknown rather than Ready=False; spec.window.attachTimeout defaults to 5s. Co-Authored-By: Claude Opus 5.5 --- api/v1alpha3/commitrequest_types.go | 33 +- api/v1alpha3/kubeprovider_types.go | 6 + cmd/main.go | 15 +- .../bases/configbutler.ai_commitrequests.yaml | 30 +- docs/configuration.md | 2 +- internal/controller/commitrequest_apply.go | 34 +- .../controller/commitrequest_controller.go | 745 +++--------------- .../commitrequest_controller_test.go | 25 +- .../commitrequest_controller_unit_test.go | 73 +- internal/controller/commitrequest_core.go | 543 +++++++++++++ .../controller/commitrequest_core_test.go | 513 ++++++++++++ internal/controller/commitrequest_decide.go | 250 ++++++ internal/controller/commitrequest_finalize.go | 37 +- .../commitrequest_outcome_metric_test.go | 12 +- .../duration_fields_admission_test.go | 2 +- test/e2e/commit_request_e2e_test.go | 20 +- 16 files changed, 1605 insertions(+), 735 deletions(-) create mode 100644 internal/controller/commitrequest_core.go create mode 100644 internal/controller/commitrequest_core_test.go create mode 100644 internal/controller/commitrequest_decide.go diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 1630eb48..3480e7ea 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -58,9 +58,9 @@ type CommitRequestSpec struct { // for one, and the timers that close the window once it has. They replace the // GitTarget's timers for that window. A request never opens a window: only a // write does. Omitted, the request attaches to the author's current or next - // window within 2s and closes it 2s later. + // window within 5s and closes it 2s later. // +optional - // +kubebuilder:default={attach: "CurrentOrNext", attachTimeout: "2s", maxDuration: "2s"} + // +kubebuilder:default={attach: "CurrentOrNext", attachTimeout: "5s", maxDuration: "2s"} Window *CommitRequestWindow `json:"window,omitempty"` // WhenNothingToCommit decides what happens when the request ends with nothing @@ -146,14 +146,14 @@ type CommitRequestWindow struct { Attach AttachPolicy `json:"attach,omitempty"` // AttachTimeout is how long the request waits for a window, as a Go duration, - // from when GitOps Reverser picks the request up. "0s" attaches only to a - // window already open. A request whose wait ran out never takes a later - // window. At most "5m". Defaults to "2s", which covers a write's wait for its - // audit event. + // from when GitOps Reverser picks the request up, which is after its own + // author was found. "0s" attaches only to a window already open. A request + // whose wait ran out never takes a later window. At most "5m". Defaults to + // "5s", which covers a write's wait for its audit event. // +optional // +kubebuilder:validation:Type=string // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" - // +kubebuilder:default="2s" + // +kubebuilder:default="5s" // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.attachTimeout must not exceed 5m" AttachTimeout *metav1.Duration `json:"attachTimeout,omitempty"` @@ -183,8 +183,9 @@ type CommitRequestWindow struct { // // - Ready (summary): True once the request reached a terminal outcome that // is not an error — a pushed commit, or a benign no-commit (nothing to -// save, already present, or a foreign open window). False while in -// progress or when it failed. +// save, already present, or a foreign open window). Unknown while in +// progress, from the first status write, which records the author. False +// when it failed. // - Reconciling / Stalled: the kstatus progress / blocked pair. // Reconciling=True while finalizing; Stalled=True when the finalize failed // and needs attention. @@ -196,13 +197,22 @@ type CommitRequestStatus struct { // +optional ObservedGeneration int64 `json:"observedGeneration,omitempty"` + // Author is who created this request, as the audit events of the cluster + // it was created on name them. Empty when they name nobody, or when author + // attribution is off: then the request saves only writes that name nobody + // either. + // +optional + Author string `json:"author,omitempty"` + // Conditions are the latest observations of this CommitRequest's state. // Reconciling and Stalled follow kstatus. // // Three conditions say how far the save got: // - // - Ready (kstatus): True once the request ended without error, with a - // pushed commit or with nothing to save. It never waits for an apply. + // - Ready (kstatus): Unknown while the save runs. True once the request + // ended without error, with a pushed commit or with nothing to save; it + // never waits for an apply. False when it failed: reason NotAllowed, + // GitTargetNotFound, SaveInProgress or FinalizeFailed. // - Pushed: True once the commit is on the remote. // - Applied: set only with spec.catchUp "AfterPush", once Ready is True. // @@ -253,6 +263,7 @@ type CommitRequestStatus struct { // +kubebuilder:printcolumn:name="GitTarget",type=string,JSONPath=`.spec.gitTargetRef.name` // +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` // +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` +// +kubebuilder:printcolumn:name="Author",type=string,JSONPath=`.status.author` // +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.commit` // +kubebuilder:printcolumn:name="Pushed",type=string,JSONPath=`.status.conditions[?(@.type=="Pushed")].status`,priority=1 // +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 diff --git a/api/v1alpha3/kubeprovider_types.go b/api/v1alpha3/kubeprovider_types.go index 663bde97..5b575888 100644 --- a/api/v1alpha3/kubeprovider_types.go +++ b/api/v1alpha3/kubeprovider_types.go @@ -264,6 +264,12 @@ func (p *KubeProvider) Key() types.NamespacedName { return types.NamespacedName{Namespace: p.Namespace, Name: p.Name} } +// ServesCommitRequests reports whether GitOps Reverser serves the CommitRequests created on this +// provider's cluster, through its own watch of it. +func (p *KubeProvider) ServesCommitRequests() bool { + return p.Spec.RemoteSaveRequests == RemoteSaveRequestsEnabled +} + // IsInCluster reports whether this provider asks for the operator's own identity (spec.inCluster). // Whether it may have it is the installation's decision, not this object's. func (p *KubeProvider) IsInCluster() bool { diff --git a/cmd/main.go b/cmd/main.go index c78af423..0e771e3b 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -408,12 +408,17 @@ func main() { Follower: applyDispatcher, }).SetupWithManager(mgr), "unable to create controller", "controller", kind) } + // A CommitRequest's author comes from its audit fact on the route of its GitTarget's + // KubeProvider, as a watched write's does, so the reconciler shares the watch plane's resolver + // and fact streams; both stay nil with attribution off. if err := (&controller.CommitRequestReconciler{ - Client: mgr.GetClient(), - Scheme: mgr.GetScheme(), - APIReader: mgr.GetAPIReader(), - Finalizer: eventRouter, - TTL: cfg.commitRequestTTL, + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + APIReader: mgr.GetAPIReader(), + Finalizer: eventRouter, + Authors: watchMgr.AuthorResolver, + FactStreams: watchMgr.FactStreams, + TTL: cfg.commitRequestTTL, }).SetupWithManager(mgr); err != nil { setupLog.Error(err, "unable to create controller", "controller", "CommitRequest") os.Exit(1) diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 31f10e00..01f9d1bf 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -24,6 +24,9 @@ spec: - jsonPath: .status.conditions[?(@.type=="Ready")].reason name: Reason type: string + - jsonPath: .status.author + name: Author + type: string - jsonPath: .status.commit name: Commit type: string @@ -148,14 +151,14 @@ spec: window: default: attach: CurrentOrNext - attachTimeout: 2s + attachTimeout: 5s maxDuration: 2s description: |- Window says which commit window this request attaches to, how long it waits for one, and the timers that close the window once it has. They replace the GitTarget's timers for that window. A request never opens a window: only a write does. Omitted, the request attaches to the author's current or next - window within 2s and closes it 2s later. + window within 5s and closes it 2s later. properties: attach: default: CurrentOrNext @@ -170,13 +173,13 @@ spec: - Next type: string attachTimeout: - default: 2s + default: 5s description: |- AttachTimeout is how long the request waits for a window, as a Go duration, - from when GitOps Reverser picks the request up. "0s" attaches only to a - window already open. A request whose wait ran out never takes a later - window. At most "5m". Defaults to "2s", which covers a write's wait for its - audit event. + from when GitOps Reverser picks the request up, which is after its own + author was found. "0s" attaches only to a window already open. A request + whose wait ran out never takes a later window. At most "5m". Defaults to + "5s", which covers a write's wait for its audit event. pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ type: string x-kubernetes-validations: @@ -241,6 +244,13 @@ spec: holds the request's commit, and may be a later one. Set once Applied is True with reason Applied. type: string + author: + description: |- + Author is who created this request, as the audit events of the cluster + it was created on name them. Empty when they name nobody, or when author + attribution is off: then the request saves only writes that name nobody + either. + type: string branch: description: Branch is the branch the commit is made on. Set once the request ends. @@ -256,8 +266,10 @@ spec: Three conditions say how far the save got: - - Ready (kstatus): True once the request ended without error, with a - pushed commit or with nothing to save. It never waits for an apply. + - Ready (kstatus): Unknown while the save runs. True once the request + ended without error, with a pushed commit or with nothing to save; it + never waits for an apply. False when it failed: reason NotAllowed, + GitTargetNotFound, SaveInProgress or FinalizeFailed. - Pushed: True once the commit is on the remote. - Applied: set only with spec.catchUp "AfterPush", once Ready is True. diff --git a/docs/configuration.md b/docs/configuration.md index eab697ec..7b6ab3ea 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -181,7 +181,7 @@ The references are immutable: to hold another `Application`, delete the applier | `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedCommitRequests` must admit this one | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | -| `window.attachTimeout` | `2s` | How long to wait for a window to attach to | +| `window.attachTimeout` | `5s` | How long to wait for a window to attach to | | `window.idleTimeout` | no idle close | Close the attached window after this much silence | | `window.maxDuration` | `2s` | Close the attached window this long after the attach | | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | diff --git a/internal/controller/commitrequest_apply.go b/internal/controller/commitrequest_apply.go index deed6f43..d68f950c 100644 --- a/internal/controller/commitrequest_apply.go +++ b/internal/controller/commitrequest_apply.go @@ -7,11 +7,10 @@ import ( "time" "k8s.io/apimachinery/pkg/api/equality" + apierrors "k8s.io/apimachinery/pkg/api/errors" apimeta "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - logf "sigs.k8s.io/controller-runtime/pkg/log" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" "github.com/ConfigButler/gitops-reverser/internal/git" @@ -44,21 +43,20 @@ func applyReported(cr *configbutleraiv1alpha3.CommitRequest) bool { } // followApply reads where the request's apply stands, writes it to Applied, and comes back until it -// is settled; then the request expires as any finished one does. -func (r *CommitRequestReconciler) followApply( - ctx context.Context, - cr *configbutleraiv1alpha3.CommitRequest, -) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") +// is settled; then the request expires as any finished one does. A write that conflicts is decided +// again on the next step, from the request as it is stored. +func (c *commitRequestCore) followApply( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (time.Duration, error) { attach := git.AttachCommitRequest{ Namespace: cr.Namespace, Name: cr.Name, UID: string(cr.UID), GitTargetName: cr.GitTargetKey().Name, GitTargetNamespace: cr.GitTargetKey().Namespace, } - apply, known, err := r.Finalizer.LookupCommitRequestApply(ctx, attach) + apply, known, err := c.finalizer.LookupCommitRequestApply(ctx, attach) if err != nil { - log.V(1).Info("Cannot read the CommitRequest's apply yet; will retry", + p.log.V(1).Info("Cannot read the CommitRequest's apply yet; will retry", "name", client.ObjectKeyFromObject(cr), "err", err.Error()) - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + return commitRequestPollInterval, nil } if !known { apply = unknownApply(cr) @@ -66,17 +64,19 @@ func (r *CommitRequestReconciler) followApply( before := cr.Status.DeepCopy() setAppliedCondition(cr, apply) if !equality.Semantic.DeepEqual(*before, cr.Status) { - if err := r.Status().Update(ctx, cr); err != nil { - // A conflict reads the request again; anything else retries the same way. - return ctrl.Result{}, client.IgnoreNotFound(err) + if err := p.store.updateStatus(ctx, cr); err != nil { + if apierrors.IsConflict(err) { + return commitRequestRequeueNow, nil + } + return 0, client.IgnoreNotFound(err) } - log.Info("CommitRequest apply reported", "name", client.ObjectKeyFromObject(cr), + p.log.Info("CommitRequest apply reported", "name", client.ObjectKeyFromObject(cr), "reason", apply.Reason, "appliedRevision", cr.Status.AppliedRevision, "message", apply.Message) } if applyReported(cr) { - return r.expireFinished(ctx, cr) + return c.expireFinished(ctx, p, cr) } - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil + return commitRequestPollInterval, nil } // unknownApply is the apply of a request its worker knows nothing of: still being taken on, shortly diff --git a/internal/controller/commitrequest_controller.go b/internal/controller/commitrequest_controller.go index e7b766b5..a7676365 100644 --- a/internal/controller/commitrequest_controller.go +++ b/internal/controller/commitrequest_controller.go @@ -4,21 +4,20 @@ package controller import ( "context" - "errors" - "fmt" + "sync" "time" - "github.com/go-logr/logr" - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" logf "sigs.k8s.io/controller-runtime/pkg/log" configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/authz" "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/queue" + "github.com/ConfigButler/gitops-reverser/internal/watch" ) // CommitRequestFinalizer is the EventRouter seam the reconciler drives, using the @@ -64,14 +63,13 @@ const ( commitRequestPollInterval = 2 * time.Second // commitRequestPushAllowance is what the safety bound allows on top of a request's own - // timers, for the push cooldown and its retries. Authorship is settled synchronously at first - // sight, so no attribution wait belongs here. + // timers: the wait for its author, the push cooldown and its retries. commitRequestPushAllowance = 120 * time.Second // The request window defaults mirror the +kubebuilder:default markers on CommitRequestWindow. // The API server fills them in, so these apply only to a client that bypasses defaulting; // resolving to the same values keeps the two paths from disagreeing. - defaultAttachTimeout = 2 * time.Second + defaultAttachTimeout = 5 * time.Second defaultMaxDuration = 2 * time.Second ) @@ -120,696 +118,139 @@ func (w requestWindow) resolveTimeout() time.Duration { return w.attachTimeout + w.maxDuration + commitRequestPushAllowance } -// CommitRequestReconciler deliberately does NOT use reconcileStatus, which every other controller -// here shares. The difference is not stylistic, so it is worth saying once: -// -// - Its For() carries no GenerationChangedPredicate, so a status write re-enqueues the object. -// The lost-write hazard reconcileStatus exists to close — the winning status-only update being -// filtered out, leaving nothing to correct the stale answer — cannot arise. -// - It reads through APIReader (uncached) precisely so a cache echo cannot re-drive work, which -// is the same lag that makes reconcileStatus lose its optimistic lock in the first place. -// - The policies are opposites, and reconcileStatus's is the wrong one here. It drops a losing -// write and asks the caller to come back and recompute; a re-run of THIS reconcile would -// re-finalize an already-flushed window. So writeTerminalStatus retries in place, bounded, and -// gives up rather than requeue. (The worker now defends the same boundary from its side: a -// request whose window has committed stays identifiable until the push settles it, so a -// re-sent attach is recognized rather than registered afresh. This policy is still the right -// one, but a re-run is no longer the only thing standing between a flushed window and a -// NoOpenWindow that never happened.) +// progressRecorded reports whether the request's status already says this phase, and why it is +// held, so a poll writes only what changed. An empty phase says nothing new. +func progressRecorded(cr *configbutleraiv1alpha3.CommitRequest, phase git.CommitRequestPhase, held string) bool { + current := findCondition(cr.Status.Conditions, ConditionTypeReconciling) + if current == nil { + return false + } + if phase == "" { + return true + } + reason, message := progressFor(phase, held) + return current.Reason == reason && current.Message == message +} + +// CommitRequestReconciler is the control-cluster path of a CommitRequest: it picks up the requests +// on the cluster GitOps Reverser runs in, with the operator's own client, and hands each step to +// commitRequestCore, which serves both paths. // -// CommitRequestReconciler drives a CommitRequest through its state machine -// (docs/spec/commitrequest-design.md): it sends the attach to the GitTarget's worker, which -// attaches the request to the author's window with its message and timers, and polls the -// outcome. The attach wait is anchored at the worker's registration, so there is no -// controller-side delay. While it polls, the controller reports the phase the worker reports. No -// window before the attach deadline resolves NoWindow; a window belonging to someone else stays -// open for its own author. +// It reads through APIReader (uncached), so a cache echo of its own status write can never re-drive a +// step that already happened: every step decides from the object as it is stored. Its For() carries +// no GenerationChangedPredicate, so a status or metadata write, such as an edit of the delete-after +// annotation, reconciles the request again. type CommitRequestReconciler struct { client.Client Scheme *runtime.Scheme - // APIReader performs uncached reads so a stale cache echo of our own - // status stamp can never re-run a finalize that already reached a - // terminal phase. Nil falls back to the (cached) Client. + // APIReader performs uncached reads of the requests. Nil falls back to the (cached) Client. APIReader client.Reader - // Finalizer attaches the request to the author-bound open window and reports - // its outcome. The attached window determines the eventual Git author. + // Finalizer attaches the request to its author's window and reports its outcome. Nil leaves + // every request alone. Finalizer CommitRequestFinalizer + // Authors finds a request's creator from audit facts, as a watched write's author is. Nil means + // author attribution is off: a request then names no author, and nothing waits for one. + Authors watch.AuthorResolver + // FactStreams is the set the attribution fact follower reads; the reconciler adds the stream of + // commitrequests on each audit route a creator is looked up on. Nil with attribution off. + FactStreams *queue.FactStreamSet + // TTL is how long a request is kept once it finishes. It is written onto the request as // CommitRequestDeleteAfterAnnotation at that moment; zero writes nothing, so the request stays. TTL time.Duration + + streamsOnce sync.Once + streams *commitRequestStreams } -// CommitRequestDeleteAfterAnnotation carries the RFC 3339 time after which the controller deletes a -// finished CommitRequest. The controller writes it once, when the request finishes, and from then -// on the annotation alone decides: remove it to keep the request, or edit it to move the deletion. -// A request that finished before this annotation existed carries none and is never deleted. +// CommitRequestDeleteAfterAnnotation carries the RFC 3339 time after which a finished CommitRequest is +// deleted. It is written once, when the request finishes, and from then on the annotation alone +// decides: remove it to keep the request, or edit it to move the deletion. A request that finished +// before this annotation existed carries none and is never deleted. const CommitRequestDeleteAfterAnnotation = "configbutler.ai/delete-after" // +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests,verbs=get;list;watch;patch;delete // +kubebuilder:rbac:groups=configbutler.ai,resources=commitrequests/status,verbs=get;update;patch -// Reconcile advances one CommitRequest through attribute → attach + poll → -// terminal status. With MaxConcurrentReconciles=1 concurrent CommitRequests are -// serialized by construction, and the worker keys attaches by request identity so -// re-sends across poll requeues are idempotent. +// Reconcile takes one step of a CommitRequest on the control cluster. func (r *CommitRequestReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - - commitRequest, done, err := r.loadActionableCommitRequest(ctx, req) - if err != nil { - return ctrl.Result{}, err - } - if done { - if commitRequest != nil { - if followsApply(commitRequest) { - return r.followApply(ctx, commitRequest) - } - return r.expireFinished(ctx, commitRequest) - } - return ctrl.Result{}, nil - } - - if r.Finalizer == nil { - log.V(1).Info("CommitRequest finalize disabled: no Finalizer configured", - "name", req.NamespacedName) - return ctrl.Result{}, nil - } - - if handled, err := r.refusePrunedGitTargetRef(ctx, commitRequest); handled || err != nil { - return ctrl.Result{}, err - } - - admitted, refusal, err := r.admitAtFirstSight(ctx, log, commitRequest) - if err != nil || admitted == commitRequestRefused { - return ctrl.Result{}, err - } - - if err := r.stampFirstSightConditions(ctx, commitRequest); err != nil { - return ctrl.Result{}, err - } - - // The schema rejects an invalid literal message at admission, so a rejection here names an - // object the schema could not have seen: one stored before the rule landed. It is terminal - // rather than retried, because CommitRequest.spec is immutable and no apply can repair it. - if err := git.ValidateLiteralCommitMessage(commitRequest.Spec.Message); err != nil { - // The fallback identity: this fails before ServiceCommitRequest, so nothing has - // confirmed the named GitTarget exists. - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, - commitRequestTarget{}) - return ctrl.Result{}, nil - } - - // 2. ATTACH + POLL: register the attach idempotently the instant we attribute - // (no controller-side delay — the worker anchors the wait at its registration) - // and poll the outcome. - window := resolveRequestWindow(commitRequest.Spec) - attach := git.AttachCommitRequest{ - Namespace: commitRequest.Namespace, - Name: commitRequest.Name, - UID: string(commitRequest.UID), - Attribution: git.AttributionNotAttempted, - GitTargetName: commitRequest.GitTargetKey().Name, - GitTargetNamespace: commitRequest.GitTargetKey().Namespace, - GitTargetUID: commitRequest.Status.GitTargetUID, - Message: commitRequest.Spec.Message, - Attach: window.attach, - AttachTimeout: window.attachTimeout, - IdleTimeout: window.idleTimeout, - MaxDuration: window.maxDuration, - CommitEmpty: commitRequest.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, - CatchUpNow: commitRequest.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, - } - if admitted == commitRequestReplaced { - return r.refuseOnceUnheld(ctx, log, req, commitRequest, attach, refusal) - } - return r.sendCommitRequest(ctx, log, req, commitRequest, attach, - time.Since(commitRequest.CreationTimestamp.Time) >= window.resolveTimeout(), - admitted == commitRequestTargetMissing) -} - -// sendCommitRequest sends the attach, or the withdraw once the request is past its bound, or, while -// its GitTarget does not exist and never admitted it, neither: nothing is attached before a GitTarget -// admitted the request, so one created later is asked first, on a later poll. -func (r *CommitRequestReconciler) sendCommitRequest( - ctx context.Context, - log logr.Logger, - req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attach git.AttachCommitRequest, - pastTheBound, targetMissing bool, -) (ctrl.Result, error) { - switch { - case pastTheBound: - // Past the bound the controller only withdraws: an attach sent now would sit behind the - // withdraw on the worker's FIFO and could register the request after it was cancelled. - return r.withdrawCommitRequest(ctx, log, req, commitRequest, attach) - case targetMissing: - return r.pollCommitRequest(ctx, commitRequest, git.PhaseWaitingForWorker, "") - } - return r.attachCommitRequest(ctx, log, req, commitRequest, attach) -} - -// commitRequestAdmission is how admitAtFirstSight left a request. -type commitRequestAdmission int - -const ( - // commitRequestAdmitted is a request its GitTarget admitted, now or before. - commitRequestAdmitted commitRequestAdmission = iota - // commitRequestTargetMissing is a request whose GitTarget does not exist and never admitted it. - commitRequestTargetMissing - // commitRequestRefused is a request now failed. - commitRequestRefused - // commitRequestReplaced is a request a GitTarget refuses that replaced the one that admitted it. An - // attach may have been sent before, so it is refused only once no worker holds it. - commitRequestReplaced -) - -// admitAtFirstSight runs admitCommitRequest, records the GitTarget that admitted the request, and fails -// a request it refuses. A refusal is final unless that same GitTarget, by UID, admitted the request -// before: then its grants were lowered after an attach may have been sent, and the worker, which may -// hold the request, keeps what it was first sent. A GitTarget that did not exist when the request was -// first seen admitted nothing, so its refusal stands. One that was recreated since admitted nothing -// either, but the GitTarget it replaced did: its refusal is commitRequestReplaced, returned with the -// refusal, and waits for the worker that may hold the request. err is a read or write to retry. -func (r *CommitRequestReconciler) admitAtFirstSight( - ctx context.Context, - log logr.Logger, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (commitRequestAdmission, error, error) { - targetUID, err := r.admitCommitRequest(ctx, commitRequest) - recorded := commitRequest.Status.GitTargetUID - switch { - case err == nil && targetUID == "": - if recorded != "" { - // Its GitTarget is gone since it admitted the request: the attach path reports that. - return commitRequestAdmitted, nil, nil - } - return commitRequestTargetMissing, nil, r.stampFirstSightConditions(ctx, commitRequest) - case err == nil: - if recorded == targetUID { - return commitRequestAdmitted, nil, nil - } - // Recorded before any attach is sent for this GitTarget, so a later refusal can tell the two - // apart. At first sight the in-progress stamp that follows writes it. - commitRequest.Status.GitTargetUID = targetUID - if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) == nil { - return commitRequestAdmitted, nil, nil - } - return commitRequestAdmitted, nil, r.Status().Update(ctx, commitRequest) - case !errors.Is(err, errCommitRequestNotAllowed): - return commitRequestAdmitted, nil, err - case recorded == targetUID: - return commitRequestAdmitted, nil, nil - case recorded != "": - return commitRequestReplaced, err, nil - } - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, err, - resolvedCommitRequestTarget(commitRequest)) - return commitRequestRefused, nil, nil -} - -// refuseOnceUnheld fails a request that a replacement of its GitTarget refuses. The GitTarget it -// replaced admitted the request, so an attach may have been sent, and the worker holding it may have -// committed it. That worker answers first: its outcome is reported, and a request it holds is waited -// for. The refusal is written only once the request is withdrawn, or nothing held it. -func (r *CommitRequestReconciler) refuseOnceUnheld( - ctx context.Context, - log logr.Logger, - req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attach git.AttachCommitRequest, - refusal error, -) (ctrl.Result, error) { - result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) - if withdrawErr != nil || !resolved { - if withdrawErr != nil { - log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", - "name", req.NamespacedName, "err", withdrawErr.Error()) - } - return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) - } - finalizeErr := result.Err - if errors.Is(finalizeErr, git.ErrCommitRequestWithdrawn) { - result, finalizeErr = git.FinalizeResult{}, refusal - } - r.writeTerminalStatus(ctx, log, commitRequest, result, finalizeErr, - resolvedCommitRequestTarget(commitRequest)) - return ctrl.Result{}, nil + after, err := r.core().serve(ctx, r.path(ctx), req.NamespacedName) + return ctrl.Result{RequeueAfter: after}, err } -// errCommitRequestNotAllowed is a request whose GitTarget's spec.allowedCommitRequests admits no grant for -// its namespace. It is terminal: the spec is immutable, so a later grant reaches only a new request. -var errCommitRequestNotAllowed = errors.New("the GitTarget does not admit CommitRequests from this namespace") - -// admitCommitRequest decides whether the request may save its GitTarget's windows: a grant in the -// GitTarget's spec.allowedCommitRequests must admit the request's namespace, or the request is -// errCommitRequestNotAllowed. It returns the UID of the GitTarget that decided. -// -// A GitTarget that does not exist admits nothing yet, and is reported with an empty UID. Any other -// read error is returned as is, so the request is retried rather than refused. -func (r *CommitRequestReconciler) admitCommitRequest( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (string, error) { - var target configbutleraiv1alpha3.GitTarget - if err := r.Get(ctx, commitRequest.GitTargetKey(), &target); err != nil { - if apierrors.IsNotFound(err) { - return "", nil - } - return "", fmt.Errorf("read GitTarget %s: %w", commitRequest.GitTargetKey(), err) - } - decision, err := authz.CommitRequestAdmitted(ctx, r.Client, commitRequest, &target) - if err != nil { - return string(target.UID), err +// core is the shared core over this reconciler's clients. +func (r *CommitRequestReconciler) core() *commitRequestCore { + r.streamsOnce.Do(func() { r.streams = &commitRequestStreams{set: r.FactStreams} }) + return &commitRequestCore{ + reader: r.Client, finalizer: r.Finalizer, authors: r.Authors, streams: r.streams, ttl: r.TTL, } - if !decision.Allowed { - return string(target.UID), fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message) - } - return string(target.UID), nil } -// attachCommitRequest sends the attach and reports the outcome once the worker has one. Until then -// it records the worker's phase and polls; a service error is a not-yet-serviceable worker, so it -// polls on the same terms. -func (r *CommitRequestReconciler) attachCommitRequest( - ctx context.Context, - log logr.Logger, - req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attach git.AttachCommitRequest, -) (ctrl.Result, error) { - result, resolved, serviceErr := r.Finalizer.ServiceCommitRequest(ctx, attach) - if serviceErr != nil || !resolved { - if serviceErr != nil { - log.V(1).Info("CommitRequest attach not yet serviceable; will retry", - "name", req.NamespacedName, "err", serviceErr.Error()) - } - return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) - } - - if result.Err != nil { - log.Error(result.Err, "CommitRequest finalize failed", - "gitTarget", commitRequest.GitTargetKey().String(), "name", req.NamespacedName) - } - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, - resolvedCommitRequestTarget(commitRequest)) - return ctrl.Result{}, nil -} - -// withdrawCommitRequest handles a request past its safety window. A request the worker holds keeps -// polling with no bound of its own: only the worker can say how it ends. Any other request is -// withdrawn, and fails only on the worker's answer, so a request reported failed can never be -// committed afterwards. A withdraw that cannot be serviced yet polls on the same terms. -func (r *CommitRequestReconciler) withdrawCommitRequest( - ctx context.Context, - log logr.Logger, - req ctrl.Request, - commitRequest *configbutleraiv1alpha3.CommitRequest, - attach git.AttachCommitRequest, -) (ctrl.Result, error) { - result, resolved, withdrawErr := r.Finalizer.WithdrawCommitRequest(ctx, attach) - if withdrawErr != nil || !resolved { - if withdrawErr != nil { - log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", - "name", req.NamespacedName, "err", withdrawErr.Error()) - } - // Not answered yet, or the worker holds the request: keep polling. - return r.pollCommitRequest(ctx, commitRequest, result.Phase, result.Held) - } - - if !errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { - // The worker resolved it on its own before the withdraw: report that outcome, as the - // attach path would have. - r.writeTerminalStatus(ctx, log, commitRequest, result, result.Err, - resolvedCommitRequestTarget(commitRequest)) - return ctrl.Result{}, nil - } - - // Withdrawn. A worker's answer, or a GitTarget that was read, carries the branch, and only - // then is the request's identity known. - target := commitRequestTarget{} - if result.Branch != "" { - target = resolvedCommitRequestTarget(commitRequest) +// path is the control-cluster path: the operator's own client, and finished requests are deleted. +func (r *CommitRequestReconciler) path(ctx context.Context) commitRequestPath { + reader := r.APIReader + if reader == nil { + reader = r.Client } - - message := resolveTimeoutMessage - switch { - case errors.Is(result.Err, git.ErrBranchWorkerStopped): - message = git.ErrBranchWorkerStopped.Error() - case result.Phase == git.PhaseWaitingForWorker: - message = noWorkerTimeoutMessage + return commitRequestPath{ + store: typedCommitRequestStore{reader: reader, writer: r.Client}, + deletes: true, + log: logf.FromContext(ctx).WithName("CommitRequestReconciler"), } - log.Info("CommitRequest was not acted on within the safety window; withdrawn and failed closed", - "name", req.NamespacedName, "reason", message) - r.writeTerminalStatus(ctx, log, commitRequest, git.FinalizeResult{}, errors.New(message), target) - return ctrl.Result{}, nil } -// pollCommitRequest reports where the worker says the request stands and polls again. -func (r *CommitRequestReconciler) pollCommitRequest( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - phase git.CommitRequestPhase, - held string, -) (ctrl.Result, error) { - if err := r.recordWorkerPhase(ctx, commitRequest, phase, held); err != nil { - return ctrl.Result{}, err - } - return ctrl.Result{RequeueAfter: commitRequestPollInterval}, nil +// typedCommitRequestStore reads and writes CommitRequests through a controller-runtime client. An +// Update carries the object's resourceVersion, so every write it makes is conditional on it. +type typedCommitRequestStore struct { + reader client.Reader + writer client.Client } -// stampFirstSightConditions stamps the still-running conditions the first time a request is seen, -// so the object reports its progress (kstatus InProgress). A disabled controller returns before -// this, so it never stamps. -func (r *CommitRequestReconciler) stampFirstSightConditions( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) error { - if findCondition(commitRequest.Status.Conditions, ConditionTypeReady) != nil { - return nil - } - markCommitRequestProgressing(commitRequest, "", "") - if err := r.Status().Update(ctx, commitRequest); err != nil { - return err +func (s typedCommitRequestStore) get( + ctx context.Context, key types.NamespacedName, +) (*configbutleraiv1alpha3.CommitRequest, error) { + var cr configbutleraiv1alpha3.CommitRequest + if err := s.reader.Get(ctx, key, &cr); err != nil { + return nil, err } - logf.FromContext(ctx).V(1).Info("Stamped CommitRequest in-progress conditions", - "name", client.ObjectKeyFromObject(commitRequest)) - return nil + return &cr, nil } -// refusePrunedGitTargetRef fails a CommitRequest whose spec names no GitTarget, which can only be -// one thing: an object stored before spec.targetRef was renamed to spec.gitTargetRef, whose old -// value the apiserver stopped serving the moment the new CRDs landed. Admission refuses an empty -// name on every path, so nothing else can produce it. -// -// It is terminal rather than retried, and it has to be, because the object cannot be repaired: -// CommitRequest.spec is wholly immutable, so no apply can put the name back. Retrying would spend -// the controller's attention on an object that will never resolve, and would report a transient -// "get GitTarget" error for a permanent condition. The message names the only fix there is. -func (r *CommitRequestReconciler) refusePrunedGitTargetRef( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (bool, error) { - if commitRequest.Spec.GitTargetRef.Name != "" { - return false, nil - } - - // This path reaches a terminal state without going through writeTerminalStatus, so it carries - // its own increment. It is rare and migration-only, which is exactly the kind of terminal - // state that must not be missing from the counter. Note the status write below is returned for - // requeue, so a failing write re-decides and increments again: see the instrument's doc - // comment on what one increment means. - recordCommitRequestOutcome(ctx, crOutcomeFailed, commitRequestTarget{}) - failCommitRequest(commitRequest, crReasonGitTargetRefPruned, - "spec.gitTargetRef is empty: this request was created before spec.targetRef was renamed, "+ - "and its value was pruned by the upgrade. A CommitRequest spec is immutable, so this "+ - "one cannot be repaired — delete it and create a new one.") - if err := r.Status().Update(ctx, commitRequest); err != nil { - return true, err - } - r.stampDeleteAfter(ctx, commitRequest) - logf.FromContext(ctx).Info( - "CommitRequest names no GitTarget: created before the gitTargetRef rename", - "name", client.ObjectKeyFromObject(commitRequest)) - return true, nil +func (s typedCommitRequestStore) updateStatus(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error { + return s.writer.Status().Update(ctx, cr) } -// recordWorkerPhase reports the phase the worker says the request is in. It writes only on a -// change, so polling does not re-write status every interval. An empty phase — the worker has not -// registered the request yet — keeps whatever is shown, so the controller never reports a phase -// the worker has not confirmed. -func (r *CommitRequestReconciler) recordWorkerPhase( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, - phase git.CommitRequestPhase, - held string, +func (s typedCommitRequestStore) annotate( + ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, key, value string, ) error { - if progressRecorded(commitRequest, phase, held) { - return nil + patch := client.MergeFromWithOptions(cr.DeepCopy(), client.MergeFromWithOptimisticLock{}) + if cr.Annotations == nil { + cr.Annotations = map[string]string{} } - markCommitRequestProgressing(commitRequest, phase, held) - return r.Status().Update(ctx, commitRequest) + cr.Annotations[key] = value + return s.writer.Patch(ctx, cr, patch) } -// progressRecorded reports whether the request's status already says this phase, and why it is -// held, so a poll writes only what changed. An empty phase says nothing new. -func progressRecorded(cr *configbutleraiv1alpha3.CommitRequest, phase git.CommitRequestPhase, held string) bool { - current := findCondition(cr.Status.Conditions, ConditionTypeReconciling) - if current == nil { - return false - } - if phase == "" { - return true - } - reason, message := progressFor(phase, held) - return current.Reason == reason && current.Message == message +func (s typedCommitRequestStore) remove(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error { + uid, resourceVersion := cr.UID, cr.ResourceVersion + return s.writer.Delete(ctx, cr, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) } -// loadActionableCommitRequest fetches the CommitRequest and short-circuits -// everything that must not reach the finalize: a deleted object, a terminal -// outcome, and — via an uncached re-read — a stale cache echo of our own -// terminal write from a previous invocation (work past this point must happen -// at most once per CommitRequest). done=true means the finalize has nothing -// further to do; the request is returned with it when it is terminal, so its -// expiry can be decided, and nil when it is gone. -func (r *CommitRequestReconciler) loadActionableCommitRequest( - ctx context.Context, - req ctrl.Request, -) (*configbutleraiv1alpha3.CommitRequest, bool, error) { - var commitRequest configbutleraiv1alpha3.CommitRequest - if err := r.Get(ctx, req.NamespacedName, &commitRequest); err != nil { - return nil, true, client.IgnoreNotFound(err) - } - if commitRequestIsTerminal(&commitRequest) { - return &commitRequest, true, nil - } - - if r.APIReader != nil { - if err := r.APIReader.Get(ctx, req.NamespacedName, &commitRequest); err != nil { - return nil, true, client.IgnoreNotFound(err) - } - if commitRequestIsTerminal(&commitRequest) { - return &commitRequest, true, nil - } - } - - return &commitRequest, false, nil -} - -// expireFinished deletes a finished request once the time in its delete-after annotation has -// passed, and otherwise comes back when it will have. No annotation means keep: the TTL flag only -// decides what is WRITTEN at finish, so changing it never reaches back to a request already stamped, -// and a request whose annotation someone removed stays. -// -// The delete is preconditioned on the resourceVersion that was judged, so an edit to the annotation -// that races it fails the delete with a conflict, and the requeue reads the edit. The edit itself -// needs no extra wiring: this controller has no generation filter, so a metadata change reconciles -// the request again. -func (r *CommitRequestReconciler) expireFinished( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) (ctrl.Result, error) { - value, ok := commitRequest.Annotations[CommitRequestDeleteAfterAnnotation] - if !ok || !commitRequest.DeletionTimestamp.IsZero() { - return ctrl.Result{}, nil - } - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - deleteAfter, parsed := parseDeleteAfter(value) - if !parsed { - // Kept rather than guessed at: an annotation a person mistyped is not consent to delete. - log.Info("CommitRequest delete-after annotation is not an RFC 3339 time; keeping the request", - "name", client.ObjectKeyFromObject(commitRequest), "value", value) - return ctrl.Result{}, nil - } - - if remaining := time.Until(deleteAfter); remaining > 0 { - return ctrl.Result{RequeueAfter: remaining}, nil - } - - uid, resourceVersion := commitRequest.UID, commitRequest.ResourceVersion - err := r.Delete(ctx, commitRequest, client.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}) - if err != nil { - return ctrl.Result{}, client.IgnoreNotFound(err) - } - log.Info("Deleted finished CommitRequest past its delete-after time", - "name", client.ObjectKeyFromObject(commitRequest), "deleteAfter", value) - return ctrl.Result{}, nil -} - -// parseDeleteAfter reads the annotation's RFC 3339 time, reporting false for anything else. -func parseDeleteAfter(value string) (time.Time, bool) { - t, err := time.Parse(time.RFC3339, value) - return t, err == nil -} - -// stampDeleteAfter writes the delete-after annotation onto a request that has just finished. It -// runs only on the transition to terminal, never on a later reconcile of a terminal request, which -// is what keeps requests that finished before this existed, and requests someone un-stamped, alone. -// -// A value that is already there wins: a submitter may set its own delete-after at creation, and a -// person may extend it in the moment between the terminal status write and this stamp. So the patch -// carries the resourceVersion it was computed from, and a conflict re-reads the request rather than -// overwriting; if the annotation appeared meanwhile, it is left exactly as it is. -// -// A write that still fails is logged, not requeued: the request is then simply kept, the safe -// direction, and a requeue would re-enter the finalize this controller must run at most once. -func (r *CommitRequestReconciler) stampDeleteAfter( - ctx context.Context, - commitRequest *configbutleraiv1alpha3.CommitRequest, -) { - if r.TTL <= 0 { - return - } - log := logf.FromContext(ctx).WithName("CommitRequestReconciler") - reader := r.APIReader - if reader == nil { - reader = r.Client - } - deleteAfter := time.Now().Add(r.TTL).UTC().Format(time.RFC3339) - - current := commitRequest - for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { - if _, set := current.Annotations[CommitRequestDeleteAfterAnnotation]; set { - return - } - patch := client.MergeFromWithOptions(current.DeepCopy(), client.MergeFromWithOptimisticLock{}) - if current.Annotations == nil { - current.Annotations = map[string]string{} - } - current.Annotations[CommitRequestDeleteAfterAnnotation] = deleteAfter - err := r.Patch(ctx, current, patch) - if err == nil || apierrors.IsNotFound(err) { - return - } - if !apierrors.IsConflict(err) { - log.Error(err, "Failed to stamp delete-after on a finished CommitRequest; it will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) - return - } - var fresh configbutleraiv1alpha3.CommitRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { - if !apierrors.IsNotFound(getErr) { - log.Error(getErr, "Failed to re-read CommitRequest to stamp delete-after; it will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) - } - return - } - if fresh.UID != commitRequest.UID { - return - } - current = &fresh - } - log.Error(nil, "Gave up stamping delete-after after repeated conflicts; the request will be kept", - "name", client.ObjectKeyFromObject(commitRequest)) -} - -// commitRequestStatusUpdateAttempts bounds the terminal-status conflict retry. -const commitRequestStatusUpdateAttempts = 3 - -// writeTerminalStatus records the finalize outcome on the CommitRequest, -// retrying on optimistic-concurrency conflicts. Like the audit-consumer path -// it replaces, a permanently failing write is logged and given up on rather -// than returned for requeue: re-running the reconcile would re-finalize an -// already-flushed window and mis-report the outcome as NoOpenWindow. -func (r *CommitRequestReconciler) writeTerminalStatus( - ctx context.Context, - log logr.Logger, - commitRequest *configbutleraiv1alpha3.CommitRequest, - result git.FinalizeResult, - finalizeErr error, - target commitRequestTarget, -) { - expectedUID := commitRequest.UID - reader := r.APIReader - if reader == nil { - reader = r.Client - } - - // Once, here, and deliberately not inside the loop below: result and finalizeErr are inputs - // that do not change across attempts, so this is the point the outcome is settled. Recording - // per attempt would over-count every request that hit a conflict. - recordCommitRequestOutcome(ctx, commitRequestOutcome(result, finalizeErr), target) - - current := commitRequest - for attempt := 1; attempt <= commitRequestStatusUpdateAttempts; attempt++ { - applyFinalizeResultToStatus(current, result, finalizeErr) - - err := r.Status().Update(ctx, current) - if err == nil { - r.stampDeleteAfter(ctx, current) - finalizeError := "" - if finalizeErr != nil { - finalizeError = finalizeErr.Error() - } - readyReason, readyMessage := commitRequestReadyReason(current) - log.Info("CommitRequest finalized", - "name", client.ObjectKeyFromObject(current), - "ready", commitRequestConditionStatus(current, ConditionTypeReady), - "reason", readyReason, - "message", readyMessage, - "pushed", commitRequestConditionStatus(current, ConditionTypePushed), - "branch", current.Status.Branch, - "commit", current.Status.Commit, - "outcome", result.Outcome, - "finalizeError", finalizeError, - "gitTarget", current.GitTargetKey().String(), - "age", time.Since(current.CreationTimestamp.Time).String()) - return - } - if !apierrors.IsConflict(err) { - log.Error(err, "Failed to write CommitRequest status") - return - } - - log.V(1).Info("Conflict writing CommitRequest status; retrying", "attempt", attempt) - var fresh configbutleraiv1alpha3.CommitRequest - if getErr := reader.Get(ctx, client.ObjectKeyFromObject(commitRequest), &fresh); getErr != nil { - if apierrors.IsNotFound(getErr) { - log.Info("CommitRequest deleted before status could be written; skipping") - return - } - log.Error(getErr, "Failed to re-read CommitRequest for status update") - return - } - // Never stamp the outcome onto a different incarnation or over a - // terminal outcome another writer got in first. - if fresh.UID != expectedUID { - log.Info("CommitRequest UID changed before status could be written; skipping", - "expectedUID", expectedUID, "objectUID", fresh.UID) - return - } - if commitRequestIsTerminal(&fresh) { - return - } - current = &fresh - } - - log.Error(nil, "Gave up writing CommitRequest status after repeated conflicts") -} - -// SetupWithManager sets up the controller with the Manager. -// MaxConcurrentReconciles is pinned to 1 on purpose: the single worker IS the -// multi-CommitRequest ordering design — concurrent CommitRequests for the same -// GitTarget are serialized exactly as a dedicated finalize-coordinator -// goroutine would serialize them, without the extra moving parts (see -// docs/spec/commitrequest-design.md). -// -// Restart recovery is best-effort by design: the -// message is durable in spec.message, so on restart any non-terminal request is -// re-reconciled — author-resolved from the admission cache when present and -// re-attached — which heals the common cases. The one knowingly-accepted gap is a request whose -// commit was already pushed but whose terminal status was not yet written: the -// in-memory outcome is gone, the re-driven attach finds the work already mirrored, -// and it resolves Rejected/AlreadyPresent. We do not build a durable record to -// close that. +// SetupWithManager sets up the controller with the Manager. Each step decides from the stored object +// and the worker keys attaches by request identity, so steps of different requests may run at once; +// a step can wait out the attribution grace for its request's author, so they must, or one request's +// wait would hold up every other. func (r *CommitRequestReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&configbutleraiv1alpha3.CommitRequest{}). - WithOptions(controller.Options{MaxConcurrentReconciles: 1}). + WithOptions(controller.Options{MaxConcurrentReconciles: commitRequestConcurrency}). Named("commitrequest"). Complete(r) } + +// commitRequestConcurrency is how many CommitRequest steps run at once. +const commitRequestConcurrency = 4 diff --git a/internal/controller/commitrequest_controller_test.go b/internal/controller/commitrequest_controller_test.go index 6ee4e7ce..3100c017 100644 --- a/internal/controller/commitrequest_controller_test.go +++ b/internal/controller/commitrequest_controller_test.go @@ -17,6 +17,21 @@ import ( var _ = Describe("CommitRequest controller", func() { const namespace = "default" + const targetName = "commitrequest-target" + + // The GitTarget the requests name, mirroring through the suite's in-cluster KubeProvider, so the + // control-cluster path owns them. Its branch does not exist; the Finalizer never asks for one. + BeforeEach(func() { + target := &configbutleraiv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: targetName, Namespace: namespace}, + Spec: configbutleraiv1alpha3.GitTargetSpec{ + KubeProviderRef: testKubeProviderRef(), + GitBranchRef: meta.NamespacedObjectReference{Name: "missing-branch"}, + Path: "commitrequest-folder", + }, + } + Expect(client.IgnoreAlreadyExists(k8sClient.Create(ctx, target))).To(Succeed()) + }) // The suite registers the reconciler with a Finalizer that never resolves, so these specs // cover the in-progress stamp and the terminal short-circuit only; the full attach → @@ -29,7 +44,7 @@ var _ = Describe("CommitRequest controller", func() { }, Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{ - Name: "team-a-config", + Name: targetName, }, Message: "increase checkout API memory", }, @@ -45,7 +60,7 @@ var _ = Describe("CommitRequest controller", func() { g.Expect(reconciling.Status).To(Equal(metav1.ConditionTrue)) ready := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) - g.Expect(ready.Status).To(Equal(metav1.ConditionFalse)) + g.Expect(ready.Status).To(Equal(metav1.ConditionUnknown)) }, 10*time.Second, 200*time.Millisecond).Should(Succeed()) }) @@ -57,20 +72,20 @@ var _ = Describe("CommitRequest controller", func() { }, Spec: configbutleraiv1alpha3.CommitRequestSpec{ GitTargetRef: meta.NamespacedObjectReference{ - Name: "team-a-config", + Name: targetName, }, }, } Expect(k8sClient.Create(ctx, commitRequest)).To(Succeed()) key := client.ObjectKeyFromObject(commitRequest) - // Wait for the initial in-progress stamp. + // Wait for the claim. Eventually(func(g Gomega) { var fetched configbutleraiv1alpha3.CommitRequest g.Expect(k8sClient.Get(ctx, key, &fetched)).To(Succeed()) ready := apimeta.FindStatusCondition(fetched.Status.Conditions, ConditionTypeReady) g.Expect(ready).NotTo(BeNil()) - g.Expect(ready.Status).To(Equal(metav1.ConditionFalse)) + g.Expect(ready.Status).To(Equal(metav1.ConditionUnknown)) }, 10*time.Second, 200*time.Millisecond).Should(Succeed()) // Simulate a finalize having recorded the terminal outcome (Ready=True). diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index 9fbf94a4..bbd4713f 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -109,11 +109,11 @@ func withReadyCommitted(cr *configv1alpha3.CommitRequest) *configv1alpha3.Commit return cr } -// withInProgress stamps the still-running conditions a post-restart redelivery -// would already carry (Ready=False, Reconciling=True), which is non-terminal. +// withInProgress stamps the claim a post-restart redelivery would already carry (Ready=Unknown, +// Reconciling=True), which is non-terminal. func withInProgress(cr *configv1alpha3.CommitRequest) *configv1alpha3.CommitRequest { apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ - Type: ConditionTypeReady, Status: metav1.ConditionFalse, + Type: ConditionTypeReady, Status: metav1.ConditionUnknown, Reason: string(git.PhaseWaitingForWindow), Message: "in progress", }) apimeta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ @@ -142,9 +142,14 @@ func newCommitRequestClient(t *testing.T, fns *interceptor.Funcs, objects ...cli return builder.Build() } -// withGitTargets gives each GitTarget a UID, as the API server would, and, when objects hold no -// GitTarget at all, adds the one each CommitRequest names: a request is only attached once a GitTarget -// admitted it. A test that passes GitTargets of its own gets only those. +// localProvider is the KubeProvider every fixture GitTarget mirrors through: the operator's own +// identity, so the control-cluster path owns its requests. +const localProvider = "local" + +// withGitTargets gives each GitTarget a UID, as the API server would, and a KubeProvider, the in-cluster +// localProvider in its namespace unless it names one. When objects hold no GitTarget at all, it adds the +// one each CommitRequest names: a request is only attached once a GitTarget admitted it. A test that +// passes GitTargets of its own gets only those. func withGitTargets(objects []client.Object) []client.Object { present := map[client.ObjectKey]bool{} for _, obj := range objects { @@ -153,10 +158,13 @@ func withGitTargets(objects []client.Object) []client.Object { if target.UID == "" { target.UID = types.UID("uid-" + target.Namespace + "-" + target.Name) } + if target.Spec.KubeProviderRef.Name == "" { + target.Spec.KubeProviderRef.Name = localProvider + } } } if len(present) > 0 { - return objects + return withLocalProviders(objects) } for _, obj := range objects { request, ok := obj.(*configv1alpha3.CommitRequest) @@ -165,9 +173,35 @@ func withGitTargets(objects []client.Object) []client.Object { } key := request.GitTargetKey() present[key] = true - objects = append(objects, &configv1alpha3.GitTarget{ObjectMeta: metav1.ObjectMeta{ - Name: key.Name, Namespace: key.Namespace, UID: types.UID("uid-" + key.Namespace + "-" + key.Name), - }}) + objects = append(objects, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{ + Name: key.Name, Namespace: key.Namespace, UID: types.UID("uid-" + key.Namespace + "-" + key.Name), + }, + Spec: configv1alpha3.GitTargetSpec{KubeProviderRef: meta.NamespacedObjectReference{Name: localProvider}}, + }) + } + return withLocalProviders(objects) +} + +// withLocalProviders adds the in-cluster localProvider to each namespace a GitTarget names it in, unless +// objects hold a KubeProvider of that name already. +func withLocalProviders(objects []client.Object) []client.Object { + present := map[client.ObjectKey]bool{} + for _, obj := range objects { + if provider, ok := obj.(*configv1alpha3.KubeProvider); ok { + present[client.ObjectKeyFromObject(provider)] = true + } + } + for _, obj := range objects { + target, ok := obj.(*configv1alpha3.GitTarget) + if !ok || target.KubeProviderKey().Name != localProvider || present[target.KubeProviderKey()] { + continue + } + present[target.KubeProviderKey()] = true + objects = append(objects, &configv1alpha3.KubeProvider{ + ObjectMeta: metav1.ObjectMeta{Name: localProvider, Namespace: target.KubeProviderKey().Namespace}, + Spec: configv1alpha3.KubeProviderSpec{InCluster: &configv1alpha3.KubeProviderInCluster{}}, + }) } return objects } @@ -412,7 +446,7 @@ func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) "the window is passed to the worker, not consumed here") got := fetchCommitRequest(t, c, "save-linger") - requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) + requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, crReasonSaving) assert.False(t, commitRequestIsTerminal(&got), "an unresolved request is not terminal") for _, phase := range []git.CommitRequestPhase{ @@ -422,7 +456,7 @@ func TestCommitRequestReconcile_NotResolvedReportsTheWorkersPhase(t *testing.T) reconcileCommitRequest(t, r, "save-linger") got = fetchCommitRequest(t, c, "save-linger") requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, string(phase)) - requireCondition(t, got, ConditionTypeReady, metav1.ConditionFalse, string(phase)) + requireCondition(t, got, ConditionTypeReady, metav1.ConditionUnknown, string(phase)) } // A phase the worker has not (yet) reported keeps what is shown rather than guessing. @@ -445,7 +479,7 @@ func TestCommitRequestReconcile_ServiceErrorPolls(t *testing.T) { assert.Equal(t, commitRequestPollInterval, res.RequeueAfter) got := fetchCommitRequest(t, c, "save-transient") - requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, ReasonProgressing) + requireCondition(t, got, ConditionTypeReconciling, metav1.ConditionTrue, crReasonSaving) assert.False(t, commitRequestIsTerminal(&got)) } @@ -891,8 +925,8 @@ func TestCommitRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T } } -// A request whose GitTarget does not exist is not attached: it waits, and once a GitTarget by that -// name appears it must admit the request first. Its earlier in-progress conditions grant nothing. +// A request whose GitTarget does not exist is not attached: it waits, writing nothing, since no path +// knows it owns it, and once a GitTarget by that name appears it must admit the request first. func TestCommitRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testing.T) { cr := newCommitRequest("save") cr.Spec.GitTargetRef = meta.NamespacedObjectReference{Name: "later", Namespace: "platform"} @@ -906,12 +940,14 @@ func TestCommitRequestAttach_AGitTargetCreatedLaterMustAdmitTheRequest(t *testin assert.Empty(t, f.calls, "nothing is attached before a GitTarget admitted the request") assert.Positive(t, res.RequeueAfter, "it waits for the GitTarget") got := fetchCommitRequest(t, c, "save") - require.NotNil(t, apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady), "it reports progress") - assert.Empty(t, got.Status.GitTargetUID) + assert.Empty(t, got.Status.Conditions, "within attachTimeout nothing is written") require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "later", Namespace: "platform", UID: "uid-later"}, - Spec: configv1alpha3.GitTargetSpec{AllowedCommitRequests: []configv1alpha3.GitTargetCommitRequestGrant{}}, + Spec: configv1alpha3.GitTargetSpec{ + KubeProviderRef: meta.NamespacedObjectReference{Name: localProvider}, + AllowedCommitRequests: []configv1alpha3.GitTargetCommitRequestGrant{}, + }, })) reconcileCommitRequest(t, r, "save") @@ -996,6 +1032,7 @@ func TestCommitRequestAttach_ARefusingReplacementAsksTheWorkerFirst(t *testing.T refusing := configv1alpha3.GitTargetSpec{ AllowedCommitRequests: []configv1alpha3.GitTargetCommitRequestGrant{}, } + refusing.KubeProviderRef = meta.NamespacedObjectReference{Name: localProvider} require.NoError(t, c.Create(context.Background(), &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default", UID: "uid-second"}, Spec: refusing, diff --git a/internal/controller/commitrequest_core.go b/internal/controller/commitrequest_core.go new file mode 100644 index 00000000..133b3e46 --- /dev/null +++ b/internal/controller/commitrequest_core.go @@ -0,0 +1,543 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "errors" + "fmt" + "sync" + "time" + + "github.com/go-logr/logr" + apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/authz" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/queue" + "github.com/ConfigButler/gitops-reverser/internal/watch" +) + +// A CommitRequest is picked up in one of two ways (docs/spec/commitrequest-design.md): by the +// CommitRequestReconciler on the control cluster, or by the CommitRequestServer's watch of a source +// cluster through its KubeProvider. Both hand it to commitRequestCore, which decides everything else +// once, so the two paths cannot drift: who owns the request, whether its GitTarget admits it, who +// created it, the attach and its outcome, the apply it asked for, and how long it is kept. +// +// Three rules keep several observers of one request safe: +// +// 1. Only the owner, the path the named GitTarget's KubeProvider picks, serves the request. A path +// that is not the owner writes only GitTargetNotFound, after attachTimeout, into an empty status, +// because nothing else is true for every observer. +// 2. Every status write is conditional on the resourceVersion its decision was made from. A conflict +// reads the request again and decides again. +// 3. The owner claims a request by writing its first status, the author and Ready=Unknown, before it +// attaches. After the claim only the owner writes, so a later conflict comes from its own writes, +// and an outcome the worker produced is written onto the fresh object as it is. +// +// Every step is decided again from the object as it is stored, so the owner is structural rather than +// "the process that wrote the status": after a restart, the owner finds its own claim and resumes. + +// commitRequestGVR is the served resource of a CommitRequest, on whichever cluster it lives. +func commitRequestGVR() schema.GroupVersionResource { + return configbutleraiv1alpha3.GroupVersion.WithResource("commitrequests") +} + +const ( + // commitRequestStatusAttempts bounds a write's re-read on conflict before the step is decided again. + commitRequestStatusAttempts = 3 + // commitRequestUnownedRecheck is how soon a request left to another path is looked at again: its + // GitTarget's KubeProvider may start or stop serving it. + commitRequestUnownedRecheck = 5 * time.Minute +) + +// commitRequestStore is how one path reads and writes the CommitRequests it serves. +type commitRequestStore interface { + // get reads the request as it is stored. + get(ctx context.Context, key types.NamespacedName) (*configbutleraiv1alpha3.CommitRequest, error) + // updateStatus writes cr's status, conditional on cr.ResourceVersion. On success cr carries the + // version it wrote. + updateStatus(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error + // annotate sets one annotation, conditional on cr.ResourceVersion. + annotate(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, key, value string) error + // remove deletes the request, conditional on its UID and resourceVersion. + remove(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error +} + +// commitRequestPath is one way of picking a CommitRequest up. +type commitRequestPath struct { + // provider is the KubeProvider whose watch of a source cluster this is; empty on the control + // cluster. + provider types.NamespacedName + store commitRequestStore + // deletes says whether this path stamps configbutler.ai/delete-after on a request that finishes, + // and deletes a finished request once that time has passed. + deletes bool + log logr.Logger +} + +// controlCluster reports whether this is the control-cluster path. +func (p commitRequestPath) controlCluster() bool { return p.provider.Name == "" } + +// gitTargetKey is the GitTarget a request names. An omitted namespace is the request's own on the +// control cluster; a request on a source cluster must name it, so without one it is not picked up. +func (p commitRequestPath) gitTargetKey(cr *configbutleraiv1alpha3.CommitRequest) (types.NamespacedName, bool) { + if !p.controlCluster() && cr.Spec.GitTargetRef.Namespace == "" { + return types.NamespacedName{}, false + } + return cr.GitTargetKey(), true +} + +// commitRequestCore serves a CommitRequest for either path, one step at a time. +type commitRequestCore struct { + // reader reads GitTargets, KubeProviders and Namespaces on the control cluster. + reader client.Reader + finalizer CommitRequestFinalizer + // authors finds a request's creator from audit facts; nil when author attribution is off. + authors watch.AuthorResolver + // streams holds the fact stream of commitrequests on each audit route a creator is looked up on. + streams *commitRequestStreams + // ttl is how long a finished request is kept, written as its delete-after time; zero writes none. + ttl time.Duration +} + +// serve takes one step for the request key on path p, and says how soon to take the next. Zero means +// nothing is due until the request changes. +func (c *commitRequestCore) serve( + ctx context.Context, p commitRequestPath, key types.NamespacedName, +) (time.Duration, error) { + cr, err := p.store.get(ctx, key) + if err != nil { + return 0, client.IgnoreNotFound(err) + } + if !cr.DeletionTimestamp.IsZero() { + return 0, nil + } + if commitRequestIsTerminal(cr) { + return c.serveFinished(ctx, p, cr) + } + if c.finalizer == nil { + p.log.V(1).Info("CommitRequest finalize disabled: no Finalizer configured", "name", key) + return 0, nil + } + return c.serveOpen(ctx, p, cr) +} + +// serveOpen is the step of a request whose save has not ended: it waits for its GitTarget, leaves it +// to the path that owns it, or claims and drives it as its owner. +func (c *commitRequestCore) serveOpen( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (time.Duration, error) { + if cr.Spec.GitTargetRef.Name == "" { + return 0, c.refusePrunedGitTargetRef(ctx, p, cr) + } + targetKey, named := p.gitTargetKey(cr) + if !named { + return 0, nil + } + target, found, err := c.gitTarget(ctx, targetKey) + if err != nil { + return 0, err + } + if !found { + if commitRequestClaimed(cr) { + // Its GitTarget went away after the claim. Only a worker that holds the request can still + // say how it ends, and that answer is true for every observer. + return c.drive(ctx, p, cr, nil) + } + return c.awaitGitTarget(ctx, p, cr, targetKey) + } + provider, owner, refusal, err := c.owns(ctx, p, target) + if err != nil { + return 0, err + } + if !owner { + if refusal != "" && len(cr.Status.Conditions) == 0 { + return c.decide(ctx, p, cr, commitRequestOutcome{err: fmt.Errorf("%w: %s", errCommitRequestNotAllowed, + refusal)}, false) + } + return commitRequestUnownedRecheck, nil + } + if commitRequestClaimed(cr) { + return c.drive(ctx, p, cr, target) + } + return c.claim(ctx, p, cr, target, provider) +} + +// gitTarget reads the GitTarget key names; found is false when there is none, or it is being deleted. +func (c *commitRequestCore) gitTarget( + ctx context.Context, key types.NamespacedName, +) (*configbutleraiv1alpha3.GitTarget, bool, error) { + var target configbutleraiv1alpha3.GitTarget + if err := c.reader.Get(ctx, key, &target); err != nil { + if apierrors.IsNotFound(err) { + return nil, false, nil + } + return nil, false, fmt.Errorf("read GitTarget %s: %w", key, err) + } + return &target, target.DeletionTimestamp.IsZero(), nil +} + +// owns decides whether path p serves requests for target: the target's KubeProvider alone picks the +// path. On the control cluster that is a provider with inCluster. A provider with a kubeconfig may point +// back at the control cluster, and then its own watch serves the request, so it is left alone, unless +// that provider serves no CommitRequests: then refusal says why, and where to create the request +// instead. A provider that cannot be found picks no path yet. +func (c *commitRequestCore) owns( + ctx context.Context, p commitRequestPath, target *configbutleraiv1alpha3.GitTarget, +) (*configbutleraiv1alpha3.KubeProvider, bool, string, error) { + key := target.KubeProviderKey() + var provider configbutleraiv1alpha3.KubeProvider + if err := c.reader.Get(ctx, key, &provider); err != nil { + if apierrors.IsNotFound(err) { + return nil, false, "", nil + } + return nil, false, "", fmt.Errorf("read KubeProvider %s: %w", key, err) + } + if !p.controlCluster() { + return &provider, key == p.provider, "", nil + } + if provider.IsInCluster() { + return &provider, true, "", nil + } + if provider.ServesCommitRequests() { + return &provider, false, "", nil + } + return &provider, false, fmt.Sprintf("GitTarget %s mirrors another cluster, through KubeProvider %s; create "+ + "the CommitRequest there, and set spec.commitRequests: Reconcile on KubeProvider %s", + client.ObjectKeyFromObject(target), key, key), nil +} + +// awaitGitTarget waits attachTimeout from the request's creation for the GitTarget it names, which may +// be created alongside it, and then reports GitTargetNotFound. That outcome is true for every observer, +// so any path may write it, but only into an empty status: a path that claimed the request in the +// meantime has started, and is never overwritten. +func (c *commitRequestCore) awaitGitTarget( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, key types.NamespacedName, +) (time.Duration, error) { + if len(cr.Status.Conditions) > 0 { + return commitRequestPollInterval, nil + } + wait := resolveRequestWindow(cr.Spec).attachTimeout - time.Since(cr.CreationTimestamp.Time) + if wait > 0 { + return min(wait, commitRequestPollInterval), nil + } + return c.decide(ctx, p, cr, commitRequestOutcome{err: fmt.Errorf("%w: GitTarget %s does not exist", + errGitTargetNotFound, key), fallbackTarget: true}, false) +} + +// admit decides whether target admits the request in spec.allowedCommitRequests: admitted, or the +// refusal, which wraps errCommitRequestNotAllowed. err is a read to retry. +func (c *commitRequestCore) admit( + ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, target *configbutleraiv1alpha3.GitTarget, +) (bool, error, error) { + decision, err := authz.CommitRequestAdmitted(ctx, c.reader, cr, target) + if err != nil { + return false, nil, err + } + if !decision.Allowed { + return false, fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message), nil + } + return true, nil, nil +} + +// claim takes an unclaimed request that p owns: it checks the message and the GitTarget's grants, +// finds the request's creator, and writes the first status, which records the author. Only once that +// write landed is the request attached. +func (c *commitRequestCore) claim( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + target *configbutleraiv1alpha3.GitTarget, provider *configbutleraiv1alpha3.KubeProvider, +) (time.Duration, error) { + // The schema rejects an invalid literal message at admission, so a rejection here names an object + // the schema could not have seen: one stored before the rule landed. It is terminal rather than + // retried, because the spec is immutable and no apply can repair it. + if err := git.ValidateLiteralCommitMessage(cr.Spec.Message); err != nil { + return c.decide(ctx, p, cr, commitRequestOutcome{err: err, fallbackTarget: true}, false) + } + admitted, refusal, err := c.admit(ctx, cr, target) + if err != nil { + return 0, err + } + if !admitted { + return c.decide(ctx, p, cr, commitRequestOutcome{err: refusal}, false) + } + author, refused := c.creator(ctx, cr, provider) + if refused { + return c.decide(ctx, p, cr, commitRequestOutcome{authorUnresolved: true}, false) + } + + claimed, err := c.writeClaim(ctx, p, cr, author, string(target.UID)) + if err != nil || !claimed { + return commitRequestRequeueNow, err + } + p.log.Info("CommitRequest claimed", "name", client.ObjectKeyFromObject(cr), + "gitTarget", client.ObjectKeyFromObject(target), "author", author) + return c.drive(ctx, p, cr, target) +} + +// creator is the request's author, found from its audit fact on the route of the GitTarget's +// KubeProvider, as a watched write's author is, and within the same grace. The wait runs whenever +// attribution is on. refused reports a request no fact names on a provider that has received audit +// facts: its fact went missing, and it must not take the window that stays unattributed. +// +// The lookup matches the request's resourceVersion, which every status write moves, so it is made +// only before the claim, and the claim records the answer. +func (c *commitRequestCore) creator( + ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, provider *configbutleraiv1alpha3.KubeProvider, +) (string, bool) { + if c.authors == nil { + return "", false + } + c.streams.ensure(provider.AuditRoute()) + user, outcome := c.authors.ResolveAuthor(ctx, watch.AuthorQuery{ + AuditRoute: provider.AuditRoute(), + GVR: commitRequestGVR(), + UID: cr.UID, + ResourceVersion: cr.ResourceVersion, + Namespace: cr.Namespace, + Labels: cr.Labels, + Name: cr.Name, + ExactCapable: true, + }) + if outcome.NamesActor() { + return user.Username, false + } + refused := outcome != git.AttributionNotAttempted && + apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) + return "", refused +} + +// writeClaim writes the first status of a request: its author, the GitTarget that admitted it, and +// Ready=Unknown. A conflict reads the request again, and the claim is written onto it only while it is +// still unclaimed; otherwise claimed=false, and the next step decides from what is there. +func (c *commitRequestCore) writeClaim( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, author, targetUID string, +) (bool, error) { + for attempt := 1; ; attempt++ { + cr.Status.Author = author + cr.Status.GitTargetUID = targetUID + markCommitRequestProgressing(cr, "", "") + err := p.store.updateStatus(ctx, cr) + if err == nil { + return true, nil + } + if !apierrors.IsConflict(err) || attempt == commitRequestStatusAttempts { + return false, err + } + fresh, same, getErr := c.reread(ctx, p, cr) + if getErr != nil || !same || len(fresh.Status.Conditions) > 0 { + return false, getErr + } + *cr = *fresh + } +} + +// reread reads cr again; same is false when it is gone or was replaced by another request of its name. +func (c *commitRequestCore) reread( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (*configbutleraiv1alpha3.CommitRequest, bool, error) { + fresh, err := p.store.get(ctx, client.ObjectKeyFromObject(cr)) + if err != nil { + return nil, false, client.IgnoreNotFound(err) + } + return fresh, fresh.UID == cr.UID, nil +} + +// attachFor is the attach of a claimed request to its GitTarget's window. +func (c *commitRequestCore) attachFor(cr *configbutleraiv1alpha3.CommitRequest) git.AttachCommitRequest { + window := resolveRequestWindow(cr.Spec) + attribution := git.AttributionNotAttempted + switch { + case cr.Status.Author != "": + attribution = git.AttributionResolved + case c.authors != nil: + attribution = git.AttributionUnresolved + } + key := cr.GitTargetKey() + return git.AttachCommitRequest{ + Namespace: cr.Namespace, + Name: cr.Name, + UID: string(cr.UID), + Author: cr.Status.Author, + Attribution: attribution, + GitTargetName: key.Name, + GitTargetNamespace: key.Namespace, + GitTargetUID: cr.Status.GitTargetUID, + Message: cr.Spec.Message, + Attach: window.attach, + AttachTimeout: window.attachTimeout, + IdleTimeout: window.idleTimeout, + MaxDuration: window.maxDuration, + CommitEmpty: cr.Spec.WhenNothingToCommit == configbutleraiv1alpha3.NothingToCommitCommitEmpty, + CatchUpNow: cr.Spec.CatchUp == configbutleraiv1alpha3.CatchUpAfterPush, + } +} + +// drive advances a claimed request: it sends the attach, or the withdraw once the request is past its +// bound, and records the worker's phase until the worker has an outcome. target is nil when the +// GitTarget is gone. +// +// A GitTarget with another UID than the one that admitted the request replaced it, and admitted +// nothing: it is asked again. Its admission is recorded before anything is attached to it; its +// refusal is written only once no worker holds the request, because the GitTarget it replaced may +// have had the request committed. +func (c *commitRequestCore) drive( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + target *configbutleraiv1alpha3.GitTarget, +) (time.Duration, error) { + attach := c.attachFor(cr) + if target != nil && string(target.UID) != cr.Status.GitTargetUID { + admitted, refusal, err := c.admit(ctx, cr, target) + if err != nil { + return 0, err + } + if !admitted { + return c.refuseOnceUnheld(ctx, p, cr, attach, refusal) + } + cr.Status.GitTargetUID = string(target.UID) + if err := p.store.updateStatus(ctx, cr); err != nil { + return commitRequestRequeueNow, ignoreConflict(err) + } + attach.GitTargetUID = cr.Status.GitTargetUID + } + if time.Since(cr.CreationTimestamp.Time) >= resolveRequestWindow(cr.Spec).resolveTimeout() { + // Past the bound only the withdraw is sent: an attach sent now would sit behind it on the + // worker's FIFO and could register the request after it was cancelled. + return c.withdraw(ctx, p, cr, attach) + } + result, resolved, err := c.finalizer.ServiceCommitRequest(ctx, attach) + if err != nil || !resolved { + if err != nil { + p.log.V(1).Info("CommitRequest attach not yet serviceable; will retry", + "name", client.ObjectKeyFromObject(cr), "err", err.Error()) + } + return c.recordPhase(ctx, p, cr, result.Phase, result.Held) + } + if result.Err != nil { + p.log.Error(result.Err, "CommitRequest finalize failed", "name", client.ObjectKeyFromObject(cr), + "gitTarget", cr.GitTargetKey().String()) + } + return c.decide(ctx, p, cr, commitRequestOutcome{result: result, err: result.Err}, true) +} + +// withdraw handles a claimed request past its safety window. A request the worker holds keeps polling +// with no bound of its own: only the worker can say how it ends. Any other request is withdrawn, and +// fails only on the worker's answer, so a request reported failed can never be committed afterwards. +func (c *commitRequestCore) withdraw( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + attach git.AttachCommitRequest, +) (time.Duration, error) { + result, resolved, err := c.finalizer.WithdrawCommitRequest(ctx, attach) + if err != nil || !resolved { + if err != nil { + p.log.V(1).Info("CommitRequest withdraw not yet serviceable; will retry", + "name", client.ObjectKeyFromObject(cr), "err", err.Error()) + } + return c.recordPhase(ctx, p, cr, result.Phase, result.Held) + } + if !errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { + // The worker resolved it on its own before the withdraw: report that outcome. + return c.decide(ctx, p, cr, commitRequestOutcome{result: result, err: result.Err}, true) + } + message := resolveTimeoutMessage + switch { + case errors.Is(result.Err, git.ErrBranchWorkerStopped): + message = git.ErrBranchWorkerStopped.Error() + case result.Phase == git.PhaseWaitingForWorker: + message = noWorkerTimeoutMessage + } + p.log.Info("CommitRequest was not acted on within the safety window; withdrawn and failed closed", + "name", client.ObjectKeyFromObject(cr), "reason", message) + // A worker's answer carries the branch, and only then is the request's GitTarget known. + return c.decide(ctx, p, cr, commitRequestOutcome{err: errors.New(message), + fallbackTarget: result.Branch == ""}, true) +} + +// refuseOnceUnheld fails a request that a replacement of its GitTarget refuses. The worker that may +// hold it answers first: its outcome is reported, and a request it holds is waited for. The refusal is +// written only once the request is withdrawn, or nothing held it. +func (c *commitRequestCore) refuseOnceUnheld( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + attach git.AttachCommitRequest, refusal error, +) (time.Duration, error) { + result, resolved, err := c.finalizer.WithdrawCommitRequest(ctx, attach) + if err != nil || !resolved { + return c.recordPhase(ctx, p, cr, result.Phase, result.Held) + } + outcome := commitRequestOutcome{result: result, err: result.Err} + if errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { + outcome = commitRequestOutcome{err: refusal} + } + return c.decide(ctx, p, cr, outcome, true) +} + +// recordPhase reports the phase the worker says the request is in, and polls again. It writes only on +// a change; a conflict is left to the next poll, which reads the request again. +func (c *commitRequestCore) recordPhase( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + phase git.CommitRequestPhase, held string, +) (time.Duration, error) { + if progressRecorded(cr, phase, held) { + return commitRequestPollInterval, nil + } + markCommitRequestProgressing(cr, phase, held) + if err := p.store.updateStatus(ctx, cr); err != nil { + return commitRequestPollInterval, ignoreConflict(err) + } + return commitRequestPollInterval, nil +} + +// refusePrunedGitTargetRef fails a request whose spec names no GitTarget, which can only be one +// thing: an object stored before spec.targetRef was renamed to spec.gitTargetRef, whose old value the +// apiserver stopped serving the moment the new CRDs landed. Admission refuses an empty name on every +// path, so nothing else can produce it. It is terminal, because the spec is immutable: the message +// names the only fix there is. +func (c *commitRequestCore) refusePrunedGitTargetRef( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) error { + if len(cr.Status.Conditions) > 0 { + return nil + } + _, err := c.decide(ctx, p, cr, commitRequestOutcome{pruned: true, fallbackTarget: true}, false) + return err +} + +// commitRequestStreams holds, for each audit route a CommitRequest's creator is looked up on, the fact +// stream of commitrequests there, so the fact follower reads those audit facts at all. Without it the +// lookup would wait out the grace and name nobody. A stream is acquired the first time its route is +// needed and held from then on: a stream followed late is replayed from the TTL horizon, so the first +// request on a route still finds its fact. +type commitRequestStreams struct { + set *queue.FactStreamSet + mu sync.Mutex + held map[string]func() +} + +// ensure acquires the stream of commitrequests on route, once. +func (s *commitRequestStreams) ensure(route string) { + if s == nil || s.set == nil { + return + } + s.mu.Lock() + defer s.mu.Unlock() + if _, held := s.held[route]; held { + return + } + if s.held == nil { + s.held = map[string]func(){} + } + s.held[route] = s.set.Acquire(queue.FactStreamKeyFor(route, commitRequestGVR().GroupResource())) +} + +// ignoreConflict drops a conflict: the next step reads the request again and decides from that. +func ignoreConflict(err error) error { + if apierrors.IsConflict(err) { + return nil + } + return err +} diff --git a/internal/controller/commitrequest_core_test.go b/internal/controller/commitrequest_core_test.go new file mode 100644 index 00000000..666c96b9 --- /dev/null +++ b/internal/controller/commitrequest_core_test.go @@ -0,0 +1,513 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "sync" + "testing" + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + apierrors "k8s.io/apimachinery/pkg/api/errors" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/interceptor" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" + "github.com/ConfigButler/gitops-reverser/internal/queue" + "github.com/ConfigButler/gitops-reverser/internal/watch" +) + +// fakeAuthors answers every author lookup alike, after delay, and records the queries. +type fakeAuthors struct { + user git.UserInfo + outcome git.AttributionOutcome + delay time.Duration + + mu sync.Mutex + queries []watch.AuthorQuery +} + +func (f *fakeAuthors) ResolveAuthor(_ context.Context, q watch.AuthorQuery) (git.UserInfo, git.AttributionOutcome) { + f.mu.Lock() + f.queries = append(f.queries, q) + f.mu.Unlock() + time.Sleep(f.delay) + return f.user, f.outcome +} + +func (f *fakeAuthors) lookups() int { + f.mu.Lock() + defer f.mu.Unlock() + return len(f.queries) +} + +// jane is an author the audit facts name. +func jane() *fakeAuthors { + return &fakeAuthors{user: git.UserInfo{Username: "jane@example.com"}, outcome: git.AttributionResolved} +} + +// controlPath is the control-cluster path over c. +func controlPath(c client.Client) commitRequestPath { + return commitRequestPath{store: typedCommitRequestStore{reader: c, writer: c}, deletes: true, log: logr.Discard()} +} + +// watcherPath is the watch of KubeProvider default/name, over c: a kubeconfig that points back at the +// control cluster, so it sees the same requests the control-cluster path does. +func watcherPath(c client.Client, name string, deletes bool) commitRequestPath { + return commitRequestPath{ + provider: types.NamespacedName{Namespace: "default", Name: name}, + store: typedCommitRequestStore{reader: c, writer: c}, + deletes: deletes, + log: logr.Discard(), + } +} + +// kubeconfigProvider is a KubeProvider with a kubeconfig, which serves CommitRequests when serving is +// true. +func kubeconfigProvider(name string, serving bool) *configv1alpha3.KubeProvider { + provider := &configv1alpha3.KubeProvider{ + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: "default"}, + Spec: configv1alpha3.KubeProviderSpec{SecretRef: &meta.SecretKeyReference{Name: name}}, + } + if serving { + provider.Spec.RemoteSaveRequests = configv1alpha3.RemoteSaveRequestsEnabled + } + return provider +} + +// mirroringTarget is the GitTarget team-a-config, mirroring through provider. +func mirroringTarget(provider string) *configv1alpha3.GitTarget { + return &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, + Spec: configv1alpha3.GitTargetSpec{KubeProviderRef: meta.NamespacedObjectReference{Name: provider}}, + } +} + +func committed() *fakeFinalizer { + return &fakeFinalizer{ + resolved: true, result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, + } +} + +func serveStep(t *testing.T, core *commitRequestCore, p commitRequestPath, name string) time.Duration { + t.Helper() + after, err := core.serve(context.Background(), p, types.NamespacedName{Namespace: "default", Name: name}) + require.NoError(t, err) + return after +} + +func commitRequestReady(t *testing.T, c client.Client, name string) *metav1.Condition { + t.Helper() + cr := fetchCommitRequest(t, c, name) + return apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady) +} + +// A request on the control cluster may be seen by the control-cluster path and by a kubeconfig watcher +// pointing back at the same cluster. The GitTarget's KubeProvider alone picks which one serves it; the +// other writes nothing. A request that leaves the namespace out is the control cluster's form, and a +// watcher never picks it up. +func TestCommitRequestCore_OverlappingObserversOfOneCluster(t *testing.T) { + for _, tc := range []struct { + name string + provider string + namespaceSet bool + wantServedBy string + wantNoService bool + }{ + {name: "in-cluster GitTarget, namespace omitted", provider: localProvider, wantServedBy: "control"}, + {name: "in-cluster GitTarget, namespace set", provider: localProvider, namespaceSet: true, + wantServedBy: "control"}, + {name: "watched GitTarget, namespace set", provider: "edge", namespaceSet: true, wantServedBy: "edge"}, + {name: "watched GitTarget, namespace omitted", provider: "edge", wantNoService: true}, + } { + t.Run(tc.name, func(t *testing.T) { + cr := newCommitRequest("save") + if tc.namespaceSet { + cr.Spec.GitTargetRef.Namespace = "default" + } + c := newCommitRequestClient(t, nil, cr, mirroringTarget(tc.provider), kubeconfigProvider("edge", true)) + f := committed() + core := &commitRequestCore{reader: c, finalizer: f} + + // The watcher steps first, so a wrong owner would get there before the right one. + serveStep(t, core, watcherPath(c, "edge", false), "save") + if tc.wantServedBy == "control" { + assert.Empty(t, fetchCommitRequest(t, c, "save").Status.Conditions, "the watcher writes nothing") + } + serveStep(t, core, controlPath(c), "save") + + ready := commitRequestReady(t, c, "save") + if tc.wantNoService { + assert.Nil(t, ready, "nobody serves it: the known gap of a kubeconfig to the control cluster") + assert.Empty(t, f.calls) + return + } + require.NotNil(t, ready) + assert.Equal(t, crReasonCommitted, ready.Reason, "served by %s", tc.wantServedBy) + assert.Len(t, f.calls, 1, "served once") + }) + } +} + +// Two kubeconfig providers on one source cluster both watch a request; only the GitTarget's own one +// serves it, whichever sees it first. The other never refuses it. +func TestCommitRequestCore_TwoProvidersOnOneSourceCluster(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + c := newCommitRequestClient(t, nil, cr, mirroringTarget("team-a"), + kubeconfigProvider("team-a", true), kubeconfigProvider("team-b", true)) + f := committed() + core := &commitRequestCore{reader: c, finalizer: f} + + serveStep(t, core, watcherPath(c, "team-b", true), "save") + assert.Empty(t, fetchCommitRequest(t, c, "save").Status.Conditions, "the other provider leaves it alone") + serveStep(t, core, watcherPath(c, "team-a", true), "save") + + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonCommitted, ready.Reason) + assert.Len(t, f.calls, 1) +} + +// A control-cluster request for a GitTarget that mirrors another cluster is refused when that +// KubeProvider serves no CommitRequests: the request's author would come from the control cluster's +// audit and its edits' from the other one's. One that serves them may point back here, so the request +// is left to its watch. +func TestCommitRequestCore_AGitTargetMirroringAnotherCluster(t *testing.T) { + t.Run("its provider serves no CommitRequests", func(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save"), mirroringTarget("edge"), + kubeconfigProvider("edge", false)) + f := committed() + serveStep(t, &commitRequestCore{reader: c, finalizer: f}, controlPath(c), "save") + + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionFalse, ready.Status) + assert.Equal(t, crReasonNotAllowed, ready.Reason) + assert.Contains(t, ready.Message, "create the CommitRequest there") + assert.Empty(t, f.calls) + }) + t.Run("its provider serves them", func(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save"), mirroringTarget("edge"), + kubeconfigProvider("edge", true)) + f := committed() + after := serveStep(t, &commitRequestCore{reader: c, finalizer: f}, controlPath(c), "save") + + assert.Nil(t, commitRequestReady(t, c, "save"), "left alone") + assert.Empty(t, f.calls) + assert.Equal(t, commitRequestUnownedRecheck, after) + }) +} + +// staleStore serves its first read from a snapshot taken earlier, so a path decides from a version +// another path has since written over, as two processes racing would. +type staleStore struct { + typedCommitRequestStore + + snapshot *configv1alpha3.CommitRequest +} + +func (s *staleStore) get(ctx context.Context, key types.NamespacedName) (*configv1alpha3.CommitRequest, error) { + if snapshot := s.snapshot; snapshot != nil { + s.snapshot = nil + return snapshot.DeepCopy(), nil + } + return s.typedCommitRequestStore.get(ctx, key) +} + +// A GitTarget created during the attach wait races the GitTargetNotFound an observer decided before it +// existed. When the owner's claim lands first, the observer's write conflicts and it stops. +func TestCommitRequestCore_TheClaimWinsOverAStaleGitTargetNotFound(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-time.Minute)) + c := newCommitRequestClient(t, nil, cr) + stale := fetchCommitRequest(t, c, "save") + f := &fakeFinalizer{} // the save goes on + serveStep(t, &commitRequestCore{reader: c, finalizer: f}, controlPath(c), "save") + require.Len(t, f.calls, 1, "the owner claimed the request and attached it") + + // The observer read the request, and found no GitTarget, before the owner's claim. + noTarget := newCommitRequestClient(t, nil, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "elsewhere", Namespace: "default"}, + }) + observer := watcherPath(c, "edge", false) + observer.store = &staleStore{typedCommitRequestStore: typedCommitRequestStore{reader: c, writer: c}, + snapshot: &stale} + serveStep(t, &commitRequestCore{reader: noTarget, finalizer: &fakeFinalizer{}}, observer, "save") + + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionUnknown, ready.Status, "the claim stands") + assert.Equal(t, crReasonSaving, ready.Reason) +} + +// When the GitTargetNotFound lands first, the owner's claim, decided from the version before it, +// conflicts: the owner reads the outcome and stops, and neither attaches nor commits. +func TestCommitRequestCore_AGitTargetNotFoundWinsOverALateClaim(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + cr.CreationTimestamp = metav1.NewTime(time.Now().Add(-time.Minute)) + noTarget := newCommitRequestClient(t, nil, cr, &configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "elsewhere", Namespace: "default"}, + }) + stale := fetchCommitRequest(t, noTarget, "save") + serveStep(t, &commitRequestCore{reader: noTarget, finalizer: &fakeFinalizer{}}, + watcherPath(noTarget, "edge", false), "save") + ready := commitRequestReady(t, noTarget, "save") + require.NotNil(t, ready) + require.Equal(t, crReasonGitTargetNotFound, ready.Reason) + + // The GitTarget appears; the owner read the request before the outcome was written. + withTarget := newCommitRequestClient(t, nil, mirroringTarget(localProvider)) + owner := controlPath(noTarget) + owner.store = &staleStore{typedCommitRequestStore: typedCommitRequestStore{reader: noTarget, writer: noTarget}, + snapshot: &stale} + f := committed() + serveStep(t, &commitRequestCore{reader: withTarget, finalizer: f}, owner, "save") + + assert.Empty(t, f.calls, "the owner never attaches a request another path finished") + ready = commitRequestReady(t, noTarget, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonGitTargetNotFound, ready.Reason) +} + +// GitTargetNotFound is written only after attachTimeout, and only into an empty status. +func TestCommitRequestCore_GitTargetNotFoundWaitsAndNeverOverwritesAClaim(t *testing.T) { + fresh := newCommitRequest("fresh") + fresh.Spec.GitTargetRef.Name = "missing" + claimed := withInProgress(newCommitRequest("claimed")) + claimed.Spec.GitTargetRef.Name = "missing" + claimed.CreationTimestamp = metav1.NewTime(time.Now().Add(-time.Minute)) + c := newCommitRequestClient(t, nil, fresh, claimed, mirroringTarget(localProvider)) + core := &commitRequestCore{reader: c, finalizer: &fakeFinalizer{}} + + after := serveStep(t, core, controlPath(c), "fresh") + assert.Positive(t, after) + assert.Empty(t, fetchCommitRequest(t, c, "fresh").Status.Conditions, "within attachTimeout nothing is written") + + serveStep(t, core, controlPath(c), "claimed") + ready := commitRequestReady(t, c, "claimed") + require.NotNil(t, ready) + assert.NotEqual(t, crReasonGitTargetNotFound, ready.Reason, "a claimed request is never refused for it") +} + +// A status conflict after the worker produced an outcome re-applies the outcome onto the fresh object; +// the save is not restarted. +func TestCommitRequestCore_AConflictAfterTheOutcomeReappliesIt(t *testing.T) { + conflicts := 1 + fns := interceptor.Funcs{ + SubResourceUpdate: func(ctx context.Context, c client.Client, sub string, obj client.Object, + opts ...client.SubResourceUpdateOption) error { + cr, ok := obj.(*configv1alpha3.CommitRequest) + if ok && conflicts > 0 && commitRequestIsTerminal(cr) { + conflicts-- + return apierrors.NewConflict(commitRequestGVR().GroupResource(), cr.Name, nil) + } + return c.SubResource(sub).Update(ctx, obj, opts...) + }, + } + c := newCommitRequestClient(t, &fns, withInProgress(newCommitRequest("save"))) + f := committed() + serveStep(t, &commitRequestCore{reader: c, finalizer: f}, controlPath(c), "save") + + assert.Len(t, f.calls, 1, "the worker is asked once") + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonCommitted, ready.Reason) + assert.Equal(t, "c0ffee", fetchCommitRequest(t, c, "save").Status.Commit) +} + +// After a restart, the owner finds its own claim and resumes the save with the author it recorded, +// without looking the author up again: the claim moved the resourceVersion the fact was keyed by. +func TestCommitRequestCore_ARestartResumesAClaimedSave(t *testing.T) { + cr := withInProgress(newCommitRequest("save")) + cr.Status.Author = "jane@example.com" + c := newCommitRequestClient(t, nil, cr) + authors := jane() + f := committed() + serveStep(t, &commitRequestCore{reader: c, finalizer: f, authors: authors}, controlPath(c), "save") + + require.Len(t, f.calls, 1) + assert.Equal(t, "jane@example.com", f.calls[0].Author) + assert.Equal(t, git.AttributionResolved, f.calls[0].Attribution) + assert.Zero(t, authors.lookups(), "the recorded author is used") + assert.Equal(t, crReasonCommitted, commitRequestReady(t, c, "save").Reason) +} + +// A request's author comes from its own audit fact, on the route of its GitTarget's KubeProvider, and +// is written in the claim, before anything is attached. The lookup is made with the fact stream of +// commitrequests acquired on that route. +func TestCommitRequestCore_TheAuthorComesFromAuditAndIsClaimedFirst(t *testing.T) { + cr := newCommitRequest("save") + cr.ResourceVersion = "" + c := newCommitRequestClient(t, nil, cr) + stored := fetchCommitRequest(t, c, "save") + authors := jane() + streams := queue.NewFactStreamSet() + var claimedFirst bool + f := &fakeFinalizer{} + core := &commitRequestCore{reader: c, finalizer: f, authors: authors, + streams: &commitRequestStreams{set: streams}} + path := controlPath(c) + path.store = authorCheckingStore{typedCommitRequestStore: typedCommitRequestStore{reader: c, writer: c}, + onWrite: func(cr *configv1alpha3.CommitRequest) { + if len(f.calls) == 0 && cr.Status.Author == "jane@example.com" { + claimedFirst = true + } + }} + serveStep(t, core, path, "save") + + require.Len(t, authors.queries, 1) + query := authors.queries[0] + assert.Equal(t, "default.local", query.AuditRoute, "the route of the GitTarget's KubeProvider") + assert.Equal(t, commitRequestGVR(), query.GVR) + assert.Equal(t, stored.ResourceVersion, query.ResourceVersion, "the version the request was created with") + assert.True(t, query.ExactCapable) + assert.Equal(t, 1, streams.Len(), "the stream of commitrequests on the route is followed") + assert.True(t, claimedFirst, "the author is written before the attach") + require.Len(t, f.calls, 1) + assert.Equal(t, "jane@example.com", f.calls[0].Author) + assert.Equal(t, "jane@example.com", fetchCommitRequest(t, c, "save").Status.Author) +} + +// authorCheckingStore calls onWrite with each status it writes. +type authorCheckingStore struct { + typedCommitRequestStore + + onWrite func(*configv1alpha3.CommitRequest) +} + +func (s authorCheckingStore) updateStatus(ctx context.Context, cr *configv1alpha3.CommitRequest) error { + s.onWrite(cr) + return s.typedCommitRequestStore.updateStatus(ctx, cr) +} + +// The request's audit fact may arrive late, within the grace, on a provider whose AuditFactsReceived is +// not True yet: the core waits for it whenever attribution is on, so the request is attributed. +func TestCommitRequestCore_ADelayedAuditFactStillNamesTheAuthor(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save")) + authors := jane() + authors.delay = 50 * time.Millisecond + f := &fakeFinalizer{} + serveStep(t, &commitRequestCore{reader: c, finalizer: f, authors: authors}, controlPath(c), "save") + + require.Len(t, f.calls, 1) + assert.Equal(t, "jane@example.com", f.calls[0].Author) + assert.Equal(t, "jane@example.com", fetchCommitRequest(t, c, "save").Status.Author) +} + +// A request no audit fact names is refused on a provider that has received facts, since its fact went +// missing; on one that has not, it names nobody and saves writes that name nobody either. +func TestCommitRequestCore_ARequestNoFactNames(t *testing.T) { + unnamed := &fakeAuthors{outcome: git.AttributionUnresolved} + t.Run("on a provider that received facts", func(t *testing.T) { + provider := &configv1alpha3.KubeProvider{ + ObjectMeta: metav1.ObjectMeta{Name: localProvider, Namespace: "default"}, + Spec: configv1alpha3.KubeProviderSpec{InCluster: &configv1alpha3.KubeProviderInCluster{}}, + Status: configv1alpha3.KubeProviderStatus{Conditions: []metav1.Condition{{ + Type: KubeProviderConditionAuditFactsReceived, Status: metav1.ConditionTrue, Reason: "Received", + }}}, + } + c := newCommitRequestClient(t, nil, newCommitRequest("save"), mirroringTarget(localProvider), provider) + f := committed() + serveStep(t, &commitRequestCore{reader: c, finalizer: f, authors: unnamed}, controlPath(c), "save") + + assert.Empty(t, f.calls) + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, metav1.ConditionTrue, ready.Status) + assert.Equal(t, crReasonAuthorUnresolved, ready.Reason) + }) + t.Run("on a provider that did not", func(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save")) + f := &fakeFinalizer{} + serveStep(t, &commitRequestCore{reader: c, finalizer: f, authors: unnamed}, controlPath(c), "save") + + require.Len(t, f.calls, 1) + assert.Empty(t, f.calls[0].Author) + assert.Equal(t, git.AttributionUnresolved, f.calls[0].Attribution) + }) +} + +// A finished request is stamped with delete-after and deleted once it has passed on a path that +// deletes finished requests, and left as it is on one that does not. Removing the annotation keeps it. +func TestCommitRequestCore_Retention(t *testing.T) { + for _, tc := range []struct { + name string + path func(client.Client) commitRequestPath + deletes bool + }{ + {name: "control cluster", path: controlPath, deletes: true}, + {name: "a watcher that deletes", path: func(c client.Client) commitRequestPath { + return watcherPath(c, "edge", true) + }, deletes: true}, + {name: "a watcher that does not", path: func(c client.Client) commitRequestPath { + return watcherPath(c, "edge", false) + }}, + } { + t.Run(tc.name, func(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + target := mirroringTarget("edge") + if tc.name == "control cluster" { + target = mirroringTarget(localProvider) + } + c := newCommitRequestClient(t, nil, cr, target, kubeconfigProvider("edge", true)) + core := &commitRequestCore{reader: c, finalizer: committed(), ttl: time.Hour} + serveStep(t, core, tc.path(c), "save") + + got := fetchCommitRequest(t, c, "save") + require.True(t, commitRequestIsTerminal(&got)) + _, stamped := got.Annotations[CommitRequestDeleteAfterAnnotation] + assert.Equal(t, tc.deletes, stamped) + + got.Annotations = map[string]string{ + CommitRequestDeleteAfterAnnotation: time.Now().Add(-time.Minute).UTC().Format(time.RFC3339), + } + require.NoError(t, c.Update(context.Background(), &got)) + serveStep(t, core, tc.path(c), "save") + var gone configv1alpha3.CommitRequest + err := c.Get(context.Background(), client.ObjectKeyFromObject(&got), &gone) + assert.Equal(t, tc.deletes, apierrors.IsNotFound(err), "deleted once delete-after passed") + }) + } + + t.Run("removing the annotation keeps the request", func(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save")) + core := &commitRequestCore{reader: c, finalizer: committed(), ttl: time.Hour} + serveStep(t, core, controlPath(c), "save") + got := fetchCommitRequest(t, c, "save") + delete(got.Annotations, CommitRequestDeleteAfterAnnotation) + require.NoError(t, c.Update(context.Background(), &got)) + + assert.Zero(t, serveStep(t, core, controlPath(c), "save")) + fetchCommitRequest(t, c, "save") + }) +} + +// A request served by a watcher carries attach: Next and whenNothingToCommit: CommitEmpty to the +// worker, as one on the control cluster does. +func TestCommitRequestCore_AWatcherCarriesNextAndCommitEmpty(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + cr.Spec.Window = &configv1alpha3.CommitRequestWindow{ + Attach: configv1alpha3.AttachNext, AttachTimeout: &metav1.Duration{Duration: time.Second}, + } + cr.Spec.WhenNothingToCommit = configv1alpha3.NothingToCommitCommitEmpty + c := newCommitRequestClient(t, nil, cr, mirroringTarget("edge"), kubeconfigProvider("edge", true)) + f := &fakeFinalizer{} + serveStep(t, &commitRequestCore{reader: c, finalizer: f}, watcherPath(c, "edge", false), "save") + + require.Len(t, f.calls, 1) + assert.Equal(t, configv1alpha3.AttachNext, f.calls[0].Attach) + assert.True(t, f.calls[0].CommitEmpty) +} diff --git a/internal/controller/commitrequest_decide.go b/internal/controller/commitrequest_decide.go new file mode 100644 index 00000000..4bc3fd10 --- /dev/null +++ b/internal/controller/commitrequest_decide.go @@ -0,0 +1,250 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "errors" + "time" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "sigs.k8s.io/controller-runtime/pkg/client" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +// commitRequestRequeueNow is the next step of a request whose write conflicted: it reads the request +// again and decides from that. +const commitRequestRequeueNow = 100 * time.Millisecond + +// errCommitRequestNotAllowed is a request its GitTarget does not admit, or one this path refuses. It is +// terminal: the spec is immutable, so a later grant reaches only a new request. +var errCommitRequestNotAllowed = errors.New("the CommitRequest is not allowed") + +// errGitTargetNotFound is a request whose GitTarget did not exist within its attachTimeout. +var errGitTargetNotFound = errors.New("the GitTarget does not exist") + +// authorUnresolvedMessage explains a request whose own audit event went missing on a cluster that sends +// them. +const authorUnresolvedMessage = "no audit event names who created this request, though its cluster sends " + + "audit events: its event went missing, so it saves nothing. Save again" + +// commitRequestOutcome is how one request ended, before it is written. +type commitRequestOutcome struct { + result git.FinalizeResult + err error + // authorUnresolved is a request no audit fact names on a cluster that sends them. + authorUnresolved bool + // pruned is a request whose spec names no GitTarget. + pruned bool + // fallbackTarget counts the outcome under empty GitTarget labels: the GitTarget was never resolved. + fallbackTarget bool +} + +// metricOutcome is the outcome's value on gitopsreverser_commit_requests_total. +func (o commitRequestOutcome) metricOutcome() string { + switch { + case o.pruned: + return crOutcomeFailed + case o.authorUnresolved: + return crOutcomeAuthorUnresolved + } + return commitRequestOutcomeValue(o.result, o.err) +} + +// apply writes the outcome onto cr's conditions. +func (o commitRequestOutcome) apply(cr *configbutleraiv1alpha3.CommitRequest) { + switch { + case o.pruned: + cr.Status.ObservedGeneration = cr.Generation + failCommitRequest(cr, crReasonGitTargetRefPruned, + "spec.gitTargetRef is empty: this request was created before spec.targetRef was renamed, "+ + "and its value was pruned by the upgrade. A CommitRequest spec is immutable, so this "+ + "one cannot be repaired — delete it and create a new one.") + case o.authorUnresolved: + cr.Status.ObservedGeneration = cr.Generation + rejectCommitRequest(cr, crReasonAuthorUnresolved, authorUnresolvedMessage) + default: + applyFinalizeResultToStatus(cr, o.result, o.err) + } +} + +// decide writes a request's outcome. afterClaim says the owner claimed the request: then a conflict +// re-applies the outcome onto the fresh object, unless that already holds one, and the save is never +// restarted. Before the claim a conflict on anything already written means another path got there +// first, and the next step decides again from what it wrote. +// +// An outcome after the claim is counted once, where it is settled, rather than per write attempt: the +// counter measures what the worker decided. One decided before the claim is counted only once written, +// since a path that loses the race decided nothing. +func (c *commitRequestCore) decide( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + o commitRequestOutcome, afterClaim bool, +) (time.Duration, error) { + target := resolvedCommitRequestTarget(cr) + if o.fallbackTarget { + target = commitRequestTarget{} + } + if afterClaim { + recordCommitRequestOutcome(ctx, o.metricOutcome(), target) + } + current, written, err := c.writeOutcome(ctx, p, cr, o, afterClaim) + if err != nil || !written { + return commitRequestRequeueNow, err + } + if !afterClaim { + recordCommitRequestOutcome(ctx, o.metricOutcome(), target) + } + c.stampDeleteAfter(ctx, p, current) + readyReason, readyMessage := commitRequestReadyReason(current) + p.log.Info("CommitRequest finalized", + "name", client.ObjectKeyFromObject(current), + "ready", commitRequestConditionStatus(current, ConditionTypeReady), + "reason", readyReason, + "message", readyMessage, + "author", current.Status.Author, + "branch", current.Status.Branch, + "commit", current.Status.Commit, + "gitTarget", current.GitTargetKey().String(), + "age", time.Since(current.CreationTimestamp.Time).String()) + if followsApply(current) { + return commitRequestPollInterval, nil + } + return c.expireFinished(ctx, p, current) +} + +// writeOutcome writes o onto cr, re-reading it on a conflict. written is false when the fresh object +// already holds an outcome, or, before the claim, anything another path wrote; then the next step +// decides from that. +func (c *commitRequestCore) writeOutcome( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + o commitRequestOutcome, afterClaim bool, +) (*configbutleraiv1alpha3.CommitRequest, bool, error) { + current := cr + for attempt := 1; ; attempt++ { + o.apply(current) + err := p.store.updateStatus(ctx, current) + if err == nil { + return current, true, nil + } + if !apierrors.IsConflict(err) || attempt == commitRequestStatusAttempts { + p.log.Error(err, "Failed to write CommitRequest status", "name", client.ObjectKeyFromObject(cr)) + return current, false, err + } + fresh, same, getErr := c.reread(ctx, p, current) + if getErr != nil || !same { + return current, false, getErr + } + if commitRequestIsTerminal(fresh) || (!afterClaim && len(fresh.Status.Conditions) > 0) { + return current, false, nil + } + current = fresh + } +} + +// serveFinished is the step of a request whose save ended: following the apply it asked for, as its +// owner, and then its retention. +func (c *commitRequestCore) serveFinished( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (time.Duration, error) { + if !followsApply(cr) { + return c.expireFinished(ctx, p, cr) + } + // A GitTarget that is gone is asked for nothing, which is true for every observer. + target, found, err := c.gitTarget(ctx, cr.GitTargetKey()) + if err != nil { + return 0, err + } + if found { + _, owner, _, err := c.owns(ctx, p, target) + if err != nil { + return 0, err + } + if !owner { + return commitRequestUnownedRecheck, nil + } + } + return c.followApply(ctx, p, cr) +} + +// stampDeleteAfter writes the delete-after annotation onto a request that has just finished, on a path +// that deletes finished requests. It runs only on the transition to an outcome, never on a later step +// of a finished request, which is what keeps requests someone un-stamped alone. +// +// A value that is already there wins: a creator may set its own delete-after, and a person may extend +// it in the moment between the outcome and this stamp. So the write is conditional on the version it +// was computed from, and a conflict reads the request again rather than overwriting. +// +// A write that still fails is logged, not retried: the request is then kept, the safe direction. +func (c *commitRequestCore) stampDeleteAfter( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) { + if !p.deletes || c.ttl <= 0 { + return + } + deleteAfter := time.Now().Add(c.ttl).UTC().Format(time.RFC3339) + current := cr + for attempt := 1; attempt <= commitRequestStatusAttempts; attempt++ { + if _, set := current.Annotations[CommitRequestDeleteAfterAnnotation]; set { + return + } + err := p.store.annotate(ctx, current, CommitRequestDeleteAfterAnnotation, deleteAfter) + if err == nil || apierrors.IsNotFound(err) { + return + } + if !apierrors.IsConflict(err) { + p.log.Error(err, "Failed to stamp delete-after on a finished CommitRequest; it will be kept", + "name", client.ObjectKeyFromObject(cr)) + return + } + fresh, same, getErr := c.reread(ctx, p, current) + if getErr != nil || !same { + return + } + current = fresh + } + p.log.Error(nil, "Gave up stamping delete-after after repeated conflicts; the request will be kept", + "name", client.ObjectKeyFromObject(cr)) +} + +// expireFinished deletes a finished request once the time in its delete-after annotation has passed, +// on a path that deletes finished requests, and otherwise comes back when it will have. No annotation +// means keep: the TTL only decides what is written at finish, so changing it never reaches back to a +// request already stamped, and a request whose annotation someone removed stays. +// +// The delete is conditional on the resourceVersion that was judged, so an edit to the annotation that +// races it fails the delete with a conflict, and the next step reads the edit. +func (c *commitRequestCore) expireFinished( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (time.Duration, error) { + value, ok := cr.Annotations[CommitRequestDeleteAfterAnnotation] + if !p.deletes || !ok || !cr.DeletionTimestamp.IsZero() { + return 0, nil + } + deleteAfter, parsed := parseDeleteAfter(value) + if !parsed { + // Kept rather than guessed at: an annotation a person mistyped is not consent to delete. + p.log.Info("CommitRequest delete-after annotation is not an RFC 3339 time; keeping the request", + "name", client.ObjectKeyFromObject(cr), "value", value) + return 0, nil + } + if remaining := time.Until(deleteAfter); remaining > 0 { + return remaining, nil + } + if err := p.store.remove(ctx, cr); err != nil { + if apierrors.IsConflict(err) { + return commitRequestRequeueNow, nil + } + return 0, client.IgnoreNotFound(err) + } + p.log.Info("Deleted finished CommitRequest past its delete-after time", + "name", client.ObjectKeyFromObject(cr), "deleteAfter", value) + return 0, nil +} + +// parseDeleteAfter reads the annotation's RFC 3339 time, reporting false for anything else. +func parseDeleteAfter(value string) (time.Time, bool) { + t, err := time.Parse(time.RFC3339, value) + return t, err == nil +} diff --git a/internal/controller/commitrequest_finalize.go b/internal/controller/commitrequest_finalize.go index fbbf6e67..a7588897 100644 --- a/internal/controller/commitrequest_finalize.go +++ b/internal/controller/commitrequest_finalize.go @@ -27,6 +27,11 @@ const ( crReasonGitTargetRefPruned = "GitTargetRefPruned" crReasonNotAllowed = "NotAllowed" crReasonSaveInProgress = "SaveInProgress" + crReasonGitTargetNotFound = "GitTargetNotFound" + crReasonAuthorUnresolved = "AuthorUnresolved" + // crReasonSaving is the progress reason of a request just claimed, before the worker reports a + // phase. + crReasonSaving = "Saving" ) // The bounded `outcome` values on gitopsreverser_commit_requests_total. @@ -42,15 +47,17 @@ const ( crOutcomeAlreadyPresent = "already_present" crOutcomeSaveInProgress = "save_in_progress" crOutcomeFailed = "failed" + // crOutcomeAuthorUnresolved is a request no audit fact names, on a cluster that sends them. + crOutcomeAuthorUnresolved = "author_unresolved" ) -// commitRequestOutcome maps one finalize result to its counter value. +// commitRequestOutcomeValue maps one finalize result to its counter value. // // A finalize error outranks the outcome because FinalizeResult.Outcome is only set when Err is nil, // and an unrecognized outcome counts as failed for the same reason applyFinalizeResultToStatus // fails it: an empty or unknown outcome with no error is a bug, and a bug that increments nothing // is a bug nobody sees. -func commitRequestOutcome(result git.FinalizeResult, finalizeErr error) string { +func commitRequestOutcomeValue(result git.FinalizeResult, finalizeErr error) string { if finalizeErr != nil { return crOutcomeFailed } @@ -141,11 +148,11 @@ func setCommitRequestCondition( cr.Status.Conditions = upsertCondition(cr.Status.Conditions, conditionType, status, reason, message, cr.Generation) } -// Progress-condition messages, one per phase the worker reports. The author is settled -// synchronously at first sight (present-or-never, §2), so there is no "waiting for the author" -// phase. Until the worker has registered the request the reason is the generic Progressing. +// Progress-condition messages, one per phase the worker reports. The author is found before the +// claim, so there is no "waiting for the author" phase. From the claim until the worker has registered +// the request the reason is Saving. const ( - progressingMessage = "sent to the branch worker; waiting for it to register the request" + progressingMessage = "claimed; sent to the branch worker, waiting for it to register the request" waitingForWorkerMessage = "the GitTarget has no branch worker yet; waiting for its reconcile to start one" waitingForWindowMessage = "registered with the worker; waiting for a matching commit window to attach to" collectingWindowMessage = "attached to a commit window; collecting writes until its timers close it" @@ -170,15 +177,15 @@ func progressFor(phase git.CommitRequestPhase, held string) (string, string) { } return string(phase), waitingForPushMessage default: - return ReasonProgressing, progressingMessage + return crReasonSaving, progressingMessage } } -// setCommitRequestProgress stamps the four progress conditions (Ready=False, +// setCommitRequestProgress stamps the four progress conditions (Ready=Unknown, // Reconciling=True, Stalled=False, Pushed=Unknown) with one unifying reason, so a // single wait reads consistently across them. func setCommitRequestProgress(cr *configv1alpha3.CommitRequest, reason, message string) { - setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionFalse, reason, message) + setCommitRequestCondition(cr, ConditionTypeReady, metav1.ConditionUnknown, reason, message) setCommitRequestCondition(cr, ConditionTypeReconciling, metav1.ConditionTrue, reason, message) setCommitRequestCondition(cr, ConditionTypeStalled, metav1.ConditionFalse, reason, notStalledMessage) setCommitRequestCondition(cr, ConditionTypePushed, metav1.ConditionUnknown, reason, pushPendingMessage) @@ -209,8 +216,11 @@ func applyFinalizeResultToStatus( if finalizeErr != nil { reason := crReasonFinalizeFailed - if errors.Is(finalizeErr, errCommitRequestNotAllowed) { + switch { + case errors.Is(finalizeErr, errCommitRequestNotAllowed): reason = crReasonNotAllowed + case errors.Is(finalizeErr, errGitTargetNotFound): + reason = crReasonGitTargetNotFound } failCommitRequest(cr, reason, finalizeErr.Error()) return @@ -293,6 +303,13 @@ func commitRequestIsTerminal(cr *configv1alpha3.CommitRequest) bool { conditionIsTrue(cr.Status.Conditions, ConditionTypeStalled) } +// commitRequestClaimed reports whether a request's owner claimed it and its save has not ended: +// Ready=Unknown. +func commitRequestClaimed(cr *configv1alpha3.CommitRequest) bool { + ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady) + return ready != nil && ready.Status == metav1.ConditionUnknown && !commitRequestIsTerminal(cr) +} + // commitRequestConditionStatus returns the status of one condition as a loggable // string, or "" when the condition is absent. func commitRequestConditionStatus(cr *configv1alpha3.CommitRequest, conditionType string) string { diff --git a/internal/controller/commitrequest_outcome_metric_test.go b/internal/controller/commitrequest_outcome_metric_test.go index d25df234..5133f57d 100644 --- a/internal/controller/commitrequest_outcome_metric_test.go +++ b/internal/controller/commitrequest_outcome_metric_test.go @@ -19,6 +19,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime/schema" "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/client/interceptor" @@ -159,7 +160,7 @@ func TestCommitRequestOutcome_MapsEveryFinalizeResult(t *testing.T) { } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { - assert.Equal(t, tc.want, commitRequestOutcome(tc.result, tc.err)) + assert.Equal(t, tc.want, commitRequestOutcomeValue(tc.result, tc.err)) }) } } @@ -277,8 +278,13 @@ func TestCommitRequestMetric_FailedStatusWriteThenRedeliveryCountsTwice(t *testi } r := &CommitRequestReconciler{Client: c, APIReader: c, Finalizer: f} - reconcileCommitRequest(t, r, "save-lost") - reconcileCommitRequest(t, r, "save-lost") + // The failed write is returned, so the request is reconciled again. + for range 2 { + _, err := r.Reconcile(context.Background(), ctrl.Request{ + NamespacedName: client.ObjectKey{Namespace: "default", Name: "save-lost"}, + }) + require.Error(t, err) + } value, ok := outcomeCount(t, reader, crOutcomeCommitted) require.True(t, ok) diff --git a/internal/controller/duration_fields_admission_test.go b/internal/controller/duration_fields_admission_test.go index 2a2b450a..37100057 100644 --- a/internal/controller/duration_fields_admission_test.go +++ b/internal/controller/duration_fields_admission_test.go @@ -205,7 +205,7 @@ var _ = Describe("Duration fields", func() { DeferCleanup(func() { _ = k8sClient.Delete(ctx, omitted) }) Expect(omitted.Spec.Window).NotTo(BeNil(), "the block default must be filled in") Expect(omitted.Spec.Window.Attach).To(Equal(configbutleraiv1alpha3.AttachCurrentOrNext)) - Expect(omitted.Spec.Window.AttachTimeout.Duration).To(Equal(2 * time.Second)) + Expect(omitted.Spec.Window.AttachTimeout.Duration).To(Equal(5 * time.Second)) Expect(omitted.Spec.Window.MaxDuration.Duration).To(Equal(2 * time.Second)) Expect(omitted.Spec.Window.IdleTimeout).To(BeNil(), "no idle close unless one is asked for") Expect(omitted.Spec.WhenNothingToCommit).To(Equal(configbutleraiv1alpha3.NothingToCommitResolve)) diff --git a/test/e2e/commit_request_e2e_test.go b/test/e2e/commit_request_e2e_test.go index d19d0e82..f6bb6259 100644 --- a/test/e2e/commit_request_e2e_test.go +++ b/test/e2e/commit_request_e2e_test.go @@ -112,6 +112,20 @@ var _ = Describe("Save Request", Label("commit-request", "audit-consumer"), Orde recentCommitDiagnostics(repo.CheckoutDir, basePath)) }, 2*time.Minute, 3*time.Second).Should(Succeed()) + if !configuredAuthorModeEnabled() { + // The round trip admission used to stand in for: the request's author comes from its own + // audit event, and it is the author of the edit's window, which it saved. + By("verifying the CommitRequest's author came from audit and authored the commit") + Eventually(func(g Gomega) { + author := commitRequestField(g, testNs, commitRequestName, "{.status.author}") + g.Expect(author).NotTo(BeEmpty(), "the request's author is found from its audit event") + commitAuthor, logErr := gitRun(repo.CheckoutDir, "log", "-1", "--pretty=%an") + g.Expect(logErr).NotTo(HaveOccurred()) + g.Expect(strings.TrimSpace(commitAuthor)).To(Equal(author), + "the request saved its own author's window") + }, 30*time.Second, 2*time.Second).Should(Succeed()) + } + // The wiring check for the metric instruments, made here because this is where the evidence // is: the commit above is verified in Git, so a missing series is a broken exporter rather // than a spec that did not run. Both queries are scoped to this suite's own GitTarget, so @@ -420,9 +434,9 @@ spec: // applyCommitRequestWithGenerateName creates a CommitRequest using // metadata.generateName and returns the server-allocated name. The spec creates its work and this -// CommitRequest back-to-back: authorship is settled synchronously at admission (no controller-side -// wait), so a zero wait would race the work's watch event and could resolve NoWindow before the -// window opens. Waiting to attach is the documented mechanism for a CommitRequest issued +// CommitRequest back-to-back: the request's author and its work's arrive through the same audit join, +// so a zero wait would race the work's watch event and could resolve NoWindow before the window +// opens. Waiting to attach is the documented mechanism for a CommitRequest issued // concurrently with its work (UC2). // // The wait is deliberately tight: it covers audit delivery (--audit-webhook-batch-max-wait=1s) and From cc4d2217d5410be72ddf08f03ff008cce56b0b26 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:10:14 +0000 Subject: [PATCH 26/33] feat(commitrequest): serve the CommitRequests of a source cluster through the shared core A CommitRequest can be created on the cluster a GitTarget mirrors, next to the edits it saves. The source.configbutler.ai group and its RemoteSaveRequest are gone: CommitRequestServer watches the CommitRequests of every KubeProvider whose new spec.commitRequests is Reconcile or ReconcileAndDeleteFinished, through that provider's identity, and hands each one to the same core as the control-cluster reconciler, so attach: Next, whenNothingToCommit: CommitEmpty, the phase reasons and the metric reach it too. - The GitTarget's KubeProvider alone picks the path that serves a request, so two providers watching one cluster no longer race to refuse each other's requests. A request on a source cluster must name its GitTarget's namespace, or it is not picked up. - GitTarget.spec.allowedCommitRequests grants set exactly one of consumerNamespaces (the control cluster) and sourceNamespaces (the cluster the GitTarget mirrors, bounded by what the KubeProvider lends it). allowedRemoteSaveRequests and requestSelector are gone. - spec.commitRequests defaults to Ignore, and takes only Ignore beside inCluster. Only ReconcileAndDeleteFinished stamps delete-after and deletes finished requests on the source cluster. - A provider watches only the namespaces it lends its GitTargets, one watch across the cluster when a grant lends "*". The KubeProvider reports it in status.commitRequests.namespaces and the CommitRequestsServed condition (Watching, NothingToWatch, CRDNotInstalled, VersionNotServed, Forbidden, Unreachable), outside Ready, shown at priority 1 as Saves. - The default watch policy excludes configbutler.ai/commitrequests. - The CommitRequest CRD carries its schema revision as the last line of the kind's description, guarded by a hash test, and ships on its own as the release asset commitrequest-crd.yaml. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 1 + .github/workflows/release.yml | 17 + .golangci.yml | 4 +- .vale/styles/HouseStyle/Headings.yml | 1 - Taskfile-build.yml | 5 +- api/source/v1alpha1/groupversion_info.go | 25 - .../v1alpha1/remotesaverequest_types.go | 227 ------ api/source/v1alpha1/zz_generated.deepcopy.go | 159 ---- api/v1alpha3/access_grants.go | 116 ++- api/v1alpha3/access_grants_test.go | 58 +- api/v1alpha3/commitrequest_schema_test.go | 52 ++ api/v1alpha3/commitrequest_types.go | 24 +- api/v1alpha3/gittarget_types.go | 40 +- api/v1alpha3/kubeprovider_types.go | 97 ++- api/v1alpha3/zz_generated.deepcopy.go | 68 +- cmd/main.go | 22 +- .../bases/configbutler.ai_commitrequests.yaml | 8 +- .../crd/bases/configbutler.ai_gittargets.yaml | 119 +-- .../bases/configbutler.ai_kubeproviders.yaml | 83 ++- ...ce.configbutler.ai_remotesaverequests.yaml | 299 -------- config/crd/kustomization.yaml | 1 - config/rbac/role.yaml | 16 - docs/configuration.md | 386 +++++----- docs/rbac.md | 31 +- hack/crdfields/fields.yaml | 41 +- .../authz/commitrequest_admission_test.go | 4 +- .../access_grants_admission_test.go | 44 +- .../commitrequest_controller_unit_test.go | 4 +- internal/controller/commitrequest_core.go | 52 +- .../controller/commitrequest_core_test.go | 13 +- internal/controller/commitrequest_server.go | 691 +++++++++++++++++ .../controller/commitrequest_server_test.go | 311 ++++++++ .../controller/kubeprovider_commitrequests.go | 46 ++ .../controller/kubeprovider_controller.go | 11 + internal/watch/cluster_context.go | 18 + internal/watch/remote_save_requests.go | 702 ------------------ internal/watch/remote_save_requests_test.go | 667 ----------------- internal/watch/resource_policy.go | 5 +- test/e2e/source_cluster_e2e_test.go | 250 +++---- 39 files changed, 1895 insertions(+), 2823 deletions(-) delete mode 100644 api/source/v1alpha1/groupversion_info.go delete mode 100644 api/source/v1alpha1/remotesaverequest_types.go delete mode 100644 api/source/v1alpha1/zz_generated.deepcopy.go create mode 100644 api/v1alpha3/commitrequest_schema_test.go delete mode 100644 config/crd/bases/source.configbutler.ai_remotesaverequests.yaml create mode 100644 internal/controller/commitrequest_server.go create mode 100644 internal/controller/commitrequest_server_test.go create mode 100644 internal/controller/kubeprovider_commitrequests.go delete mode 100644 internal/watch/remote_save_requests.go delete mode 100644 internal/watch/remote_save_requests_test.go diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c65832e3..64ceaac0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -458,6 +458,7 @@ jobs: name: release-bundle path: | dist/crds.yaml + dist/commitrequest-crd.yaml dist/install.yaml gitops-reverser.tgz if-no-files-found: error diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a60adb27..cd10cff0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -354,6 +354,11 @@ jobs: - name: Sign crds.yaml (cosign keyless) run: cosign sign-blob --bundle crds.yaml.sigstore.json --yes dist/crds.yaml + # The CommitRequest CRD alone, for the source clusters that serve CommitRequests and need + # none of the other kinds. It is part of crds.yaml too; this copy is signed on its own. + - name: Sign commitrequest-crd.yaml (cosign keyless) + run: cosign sign-blob --bundle commitrequest-crd.yaml.sigstore.json --yes dist/commitrequest-crd.yaml + - name: Attest SLSA provenance for install.yaml id: attest-install-yaml uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2 @@ -372,6 +377,15 @@ jobs: - name: Rename crds provenance bundle to the Scorecard-recognized suffix run: cp "${{ steps.attest-crds-yaml.outputs.bundle-path }}" crds.yaml.intoto.jsonl + - name: Attest SLSA provenance for commitrequest-crd.yaml + id: attest-commitrequest-crd + uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2 + with: + subject-path: dist/commitrequest-crd.yaml + + - name: Rename commitrequest-crd provenance bundle to the Scorecard-recognized suffix + run: cp "${{ steps.attest-commitrequest-crd.outputs.bundle-path }}" commitrequest-crd.yaml.intoto.jsonl + - name: Upload installer bundle as release assets uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3 with: @@ -384,6 +398,9 @@ jobs: dist/crds.yaml crds.yaml.sigstore.json crds.yaml.intoto.jsonl + dist/commitrequest-crd.yaml + commitrequest-crd.yaml.sigstore.json + commitrequest-crd.yaml.intoto.jsonl dist/install.yaml install.yaml.sigstore.json install.yaml.intoto.jsonl diff --git a/.golangci.yml b/.golangci.yml index 405888fd..ca80b8a7 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -328,7 +328,7 @@ linters: # Allow global variables in specific patterns (Kubernetes operator patterns) - text: "(scheme|setupLog|GroupVersion|SchemeBuilder|AddToScheme) is a global variable" linters: [gochecknoglobals] - - path: 'api/(v1alpha3|source/v1alpha1)/.*\.go' + - path: 'api/v1alpha3/.*\.go' linters: [gochecknoglobals, gochecknoinits] - path: 'cmd/main\.go' linters: [gochecknoglobals, cyclop, gochecknoinits, gocognit, funlen] @@ -361,7 +361,7 @@ linters: linters: [staticcheck] # Kubernetes controller-runtime still uses this pattern in API packages. - text: "SA1019:.*scheme\\.Builder is deprecated" - path: 'api/(v1alpha3|source/v1alpha1)/groupversion_info\.go' + path: 'api/v1alpha3/groupversion_info\.go' linters: [staticcheck] # mgr.GetEventRecorderFor returns the record.EventRecorder our shared status writer emits on # (a single Event per persisted Ready transition). It is the ubiquitous kubebuilder idiom; the diff --git a/.vale/styles/HouseStyle/Headings.yml b/.vale/styles/HouseStyle/Headings.yml index 7279f21d..a42f4758 100644 --- a/.vale/styles/HouseStyle/Headings.yml +++ b/.vale/styles/HouseStyle/Headings.yml @@ -55,7 +55,6 @@ exceptions: - ClusterWatchRule - KubeProvider - CommitRequest - - RemoteSaveRequest - APIResourceCatalog - HelmRelease - Kustomization diff --git a/Taskfile-build.yml b/Taskfile-build.yml index 44353062..5e122e2d 100644 --- a/Taskfile-build.yml +++ b/Taskfile-build.yml @@ -55,7 +55,6 @@ tasks: - exclude: cmd/**/zz_generated.deepcopy.go generates: - config/crd/bases/configbutler.ai*.yaml - - config/crd/bases/source.configbutler.ai*.yaml - config/rbac/role.yaml cmds: - | @@ -112,7 +111,6 @@ tasks: - exclude: api/**/zz_generated.deepcopy.go generates: - api/v1alpha3/zz_generated.deepcopy.go - - api/source/v1alpha1/zz_generated.deepcopy.go cmds: - '{{.CONTROLLER_GEN}} object:headerFile="hack/boilerplate.go.txt" paths=./api/...' @@ -774,6 +772,7 @@ tasks: - exclude: cmd/**/zz_generated.deepcopy.go generates: - dist/crds.yaml + - dist/commitrequest-crd.yaml - dist/install.yaml cmds: - | @@ -787,6 +786,8 @@ tasks: # here. Each chart CRD file already begins with its own `---`, so a plain cat is valid # multi-doc YAML. cat charts/gitops-reverser/crds/*.yaml > dist/crds.yaml + # A source cluster needs only the CommitRequest CRD, so it ships on its own too. + cp charts/gitops-reverser/crds/configbutler.ai_commitrequests.yaml dist/commitrequest-crd.yaml {{.HELM}} template {{.INSTALL_NAME}} charts/gitops-reverser \ --namespace {{.NAMESPACE}} \ --set labels.managedBy=kubectl \ diff --git a/api/source/v1alpha1/groupversion_info.go b/api/source/v1alpha1/groupversion_info.go deleted file mode 100644 index 227a1dfc..00000000 --- a/api/source/v1alpha1/groupversion_info.go +++ /dev/null @@ -1,25 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -// Package v1alpha1 contains API Schema definitions for the source.configbutler.ai v1alpha1 API group: -// the kinds GitOps Reverser serves on a source cluster, where the edits it mirrors are made. It is -// installed there on its own, apart from the configbutler.ai kinds of the cluster the operator runs -// in. -// +kubebuilder:object:generate=true -// +groupName=source.configbutler.ai -package v1alpha1 - -import ( - "k8s.io/apimachinery/pkg/runtime/schema" - "sigs.k8s.io/controller-runtime/pkg/scheme" -) - -var ( - // GroupVersion is group version used to register these objects. - GroupVersion = schema.GroupVersion{Group: "source.configbutler.ai", Version: "v1alpha1"} - - // SchemeBuilder is used to add go types to the GroupVersionKind scheme. - SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} - - // AddToScheme adds the types in this group-version to the given scheme. - AddToScheme = SchemeBuilder.AddToScheme -) diff --git a/api/source/v1alpha1/remotesaverequest_types.go b/api/source/v1alpha1/remotesaverequest_types.go deleted file mode 100644 index 266153f6..00000000 --- a/api/source/v1alpha1/remotesaverequest_types.go +++ /dev/null @@ -1,227 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package v1alpha1 - -import ( - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" -) - -// A RemoteSaveRequest is the source-cluster twin of a configbutler.ai CommitRequest, for the people who edit -// a source cluster and cannot reach the cluster GitOps Reverser runs in. It is its own kind, in its own -// group, rather than a CommitRequest with another schema: its GitTarget is on another cluster than the -// request, its author comes from the source cluster's audit events rather than from admission, and it -// is installed alone. docs/future/source-side-save-plan.md has the reasoning. -// -// The spec is immutable, compared as CommitRequestSpec compares it: durations as durations, so a typed -// client's re-spelling of "1m" as "1m0s" is not a change. - -// RemoteSaveRequestSpec defines the desired state of RemoteSaveRequest. It -// is immutable. -// +kubebuilder:validation:XValidation:rule="self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) == has(oldSelf.message) && (!has(self.message) || self.message == oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == oldSelf.catchUp)",message="RemoteSaveRequest spec is immutable after creation" -type RemoteSaveRequestSpec struct { - // GitTargetRef names the GitTarget whose commit window this request - // attaches to, on the cluster GitOps Reverser runs in. Its namespace is a - // namespace of that cluster, not of this one. The GitTarget must mirror - // this cluster, and admit this request in spec.allowedRemoteSaveRequests, - // or the request fails with reason "NotAllowed". - // +required - GitTargetRef GitTargetReference `json:"gitTargetRef"` - - // Message is the commit message this request asks for. Omitted, the - // GitTarget's spec.commit.message.liveTemplate words the commit. - // - // The text is never evaluated as a template. If the GitTarget sets - // spec.commit.message.requestTemplate, that template wraps this text; - // otherwise the text is committed as written. - // - // It is 1–1024 characters and must contain a non-whitespace character. - // Newlines are allowed, for a subject and body; tabs and other control - // characters are rejected. - // +optional - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=1024 - // +kubebuilder:validation:Pattern=`^[^\x00-\x09\x0B-\x1F\x7F]*$` - // +kubebuilder:validation:XValidation:rule="self.matches(r'[^\\s\\x{0085}\\x{00A0}\\x{1680}\\x{2000}-\\x{200A}\\x{2028}\\x{2029}\\x{202F}\\x{205F}\\x{3000}]')",message="message must contain a non-whitespace character" - Message string `json:"message,omitempty"` - - // The default is spelled out rather than {} because the API server checks a default against - // the block's CEL rules without applying the nested field defaults first. - - // Window says how long the request waits for a commit window with the - // edits it saves, and when that window closes once it has. Omitted, the - // request waits 5s and closes the window 2s after it attached. - // +optional - // +kubebuilder:default={attachTimeout: "5s", maxDuration: "2s"} - Window *RemoteSaveRequestWindow `json:"window,omitempty"` - - // CatchUp decides when the GitTarget's applier is asked to apply the - // branch after this request pushes a commit. "AsDefinedOnClaim" leaves it - // to the applier claim's spec.catchUpAfter. "AfterPush" asks right after - // the push, and the Applied condition reports how it went. The cluster - // already holds the saved objects. Defaults to "AsDefinedOnClaim". - // +optional - // +kubebuilder:validation:Enum=AsDefinedOnClaim;AfterPush - // +kubebuilder:default=AsDefinedOnClaim - CatchUp CatchUpPolicy `json:"catchUp,omitempty"` -} - -// GitTargetReference names a GitTarget on the cluster GitOps Reverser runs -// in. -type GitTargetReference struct { - // Namespace is the GitTarget's namespace, on the cluster GitOps Reverser - // runs in. - // +required - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=63 - Namespace string `json:"namespace"` - - // Name is the GitTarget's name. - // +required - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - Name string `json:"name"` -} - -// CatchUpPolicy is when a RemoteSaveRequest's commit is applied by a -// GitTarget's applier. -type CatchUpPolicy string - -const ( - // CatchUpAsDefinedOnClaim leaves the apply to the applier claim's spec.catchUpAfter. - CatchUpAsDefinedOnClaim CatchUpPolicy = "AsDefinedOnClaim" - // CatchUpAfterPush asks the applier to apply the tip right after the commit is pushed. - CatchUpAfterPush CatchUpPolicy = "AfterPush" -) - -// RemoteSaveRequestWindow is how long a RemoteSaveRequest waits for a commit -// window, and when that window closes. A window is only ever opened by an -// edit; a request waits for one and never opens one. -// +kubebuilder:validation:XValidation:rule="duration(self.attachTimeout) == duration(oldSelf.attachTimeout) && duration(self.maxDuration) == duration(oldSelf.maxDuration) && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) || duration(self.idleTimeout) == duration(oldSelf.idleTimeout))",message="RemoteSaveRequest spec is immutable after creation" -// +kubebuilder:validation:XValidation:rule="!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)",message="spec.window.idleTimeout must not exceed maxDuration" -type RemoteSaveRequestWindow struct { - // AttachTimeout is how long the request waits for a commit window, as a - // Go duration, from when GitOps Reverser picks the request up, which is - // after its own author was found. A request whose wait ran out never - // takes a later window. At most "5m". Defaults to "5s". - // +optional - // +kubebuilder:validation:Type=string - // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" - // +kubebuilder:default="5s" - // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.attachTimeout must not exceed 5m" - AttachTimeout *metav1.Duration `json:"attachTimeout,omitempty"` - - // IdleTimeout closes the attached window this long after its last edit, - // as a Go duration. Omitted, only maxDuration closes it. At most "5m", - // and not more than maxDuration. - // +optional - // +kubebuilder:validation:Type=string - // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" - // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.idleTimeout must not exceed 5m" - IdleTimeout *metav1.Duration `json:"idleTimeout,omitempty"` - - // MaxDuration closes the attached window this long after the request - // attached, however many edits keep arriving, as a Go duration. At most - // "5m". Defaults to "2s". - // +optional - // +kubebuilder:validation:Type=string - // +kubebuilder:validation:Pattern="^([0-9]+(\\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$" - // +kubebuilder:default="2s" - // +kubebuilder:validation:XValidation:rule="duration(self) <= duration('5m')",message="spec.window.maxDuration must not exceed 5m" - MaxDuration *metav1.Duration `json:"maxDuration,omitempty"` -} - -// RemoteSaveRequestStatus defines the observed state of RemoteSaveRequest. -type RemoteSaveRequestStatus struct { - // ObservedGeneration is the latest generation observed by GitOps Reverser. - // +optional - ObservedGeneration int64 `json:"observedGeneration,omitempty"` - - // Author is who created this request, as this cluster's audit events name - // them. Empty when they name nobody: then the request saves only edits - // that name nobody either. - // +optional - Author string `json:"author,omitempty"` - - // Conditions are the latest observations of this RemoteSaveRequest's state. - // - // Ready is True once the request ended without error: reason "Committed", - // or, with nothing committed, "NoWindow", "AlreadyPresent", - // "WindowMismatch", or "AuthorUnresolved" when this cluster sends audit - // events and none named the request's creator. Ready=False with - // Stalled=True is a failure: "NotAllowed" when the GitTarget does not - // admit the request or mirrors another cluster, "GitTargetNotFound", - // "SaveInProgress" when an earlier save of the same author on the - // GitTarget was still in progress, or "Failed". - // - // Applied, with spec.catchUp "AfterPush", is True once the applier applied - // a revision that holds the commit, or needed none, and False only when - // the branch lost the commit. It never changes Ready. - // +optional - // +listType=map - // +listMapKey=type - // +patchMergeKey=type - // +patchStrategy=merge - Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` - - // Branch is the branch the save was committed on. - // +optional - Branch string `json:"branch,omitempty"` - - // Commit is the commit the save made, once it is on the remote. - // +optional - Commit string `json:"commit,omitempty"` - - // AppliedRevision is the revision the applier applied, with spec.catchUp - // "AfterPush". It holds the commit, and may be a later one. - // +optional - AppliedRevision string `json:"appliedRevision,omitempty"` -} - -// +kubebuilder:object:root=true -// +kubebuilder:subresource:status -// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` -// +kubebuilder:printcolumn:name="Reason",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].reason` -// +kubebuilder:printcolumn:name="GitTarget",type=string,JSONPath=`.spec.gitTargetRef.name` -// +kubebuilder:printcolumn:name="Author",type=string,JSONPath=`.status.author` -// +kubebuilder:printcolumn:name="Commit",type=string,JSONPath=`.status.commit`,priority=1 -// +kubebuilder:printcolumn:name="Applied",type=string,JSONPath=`.status.conditions[?(@.type=="Applied")].status`,priority=1 -// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` - -// RemoteSaveRequest asks GitOps Reverser to save its creator's edits in the -// GitTarget spec.gitTargetRef names: the creator's commit window there gets -// its message and closes on its timers. The window holds the creator's edits -// anywhere that GitTarget mirrors. Create it on the cluster the edits were -// made on, as JSON. Its creator is found from this cluster's audit events, as -// the edits' authors are, so it saves only its creator's edits; on a cluster -// with no audit it names nobody, and saves edits that name nobody. Its spec is -// immutable, and status reports the commit. -type RemoteSaveRequest struct { - metav1.TypeMeta `json:",inline"` - - // metadata is a standard object metadata - // +optional - metav1.ObjectMeta `json:"metadata,omitempty,omitzero"` - - // spec defines the desired state of RemoteSaveRequest. It is immutable after - // creation. - // +required - Spec RemoteSaveRequestSpec `json:"spec"` - - // status defines the observed state of RemoteSaveRequest. - // +optional - Status RemoteSaveRequestStatus `json:"status,omitempty,omitzero"` -} - -// +kubebuilder:object:root=true - -// RemoteSaveRequestList contains a list of RemoteSaveRequest. -type RemoteSaveRequestList struct { - metav1.TypeMeta `json:",inline"` - metav1.ListMeta `json:"metadata,omitempty"` - - Items []RemoteSaveRequest `json:"items"` -} - -func init() { - SchemeBuilder.Register(&RemoteSaveRequest{}, &RemoteSaveRequestList{}) -} diff --git a/api/source/v1alpha1/zz_generated.deepcopy.go b/api/source/v1alpha1/zz_generated.deepcopy.go deleted file mode 100644 index 6e0cfc5f..00000000 --- a/api/source/v1alpha1/zz_generated.deepcopy.go +++ /dev/null @@ -1,159 +0,0 @@ -//go:build !ignore_autogenerated - -// SPDX-License-Identifier: Apache-2.0 - -// Code generated by controller-gen. DO NOT EDIT. - -package v1alpha1 - -import ( - "k8s.io/apimachinery/pkg/apis/meta/v1" - runtime "k8s.io/apimachinery/pkg/runtime" -) - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitTargetReference) DeepCopyInto(out *GitTargetReference) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetReference. -func (in *GitTargetReference) DeepCopy() *GitTargetReference { - if in == nil { - return nil - } - out := new(GitTargetReference) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequest) DeepCopyInto(out *RemoteSaveRequest) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequest. -func (in *RemoteSaveRequest) DeepCopy() *RemoteSaveRequest { - if in == nil { - return nil - } - out := new(RemoteSaveRequest) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *RemoteSaveRequest) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestList) DeepCopyInto(out *RemoteSaveRequestList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]RemoteSaveRequest, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestList. -func (in *RemoteSaveRequestList) DeepCopy() *RemoteSaveRequestList { - if in == nil { - return nil - } - out := new(RemoteSaveRequestList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *RemoteSaveRequestList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestSpec) DeepCopyInto(out *RemoteSaveRequestSpec) { - *out = *in - out.GitTargetRef = in.GitTargetRef - if in.Window != nil { - in, out := &in.Window, &out.Window - *out = new(RemoteSaveRequestWindow) - (*in).DeepCopyInto(*out) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestSpec. -func (in *RemoteSaveRequestSpec) DeepCopy() *RemoteSaveRequestSpec { - if in == nil { - return nil - } - out := new(RemoteSaveRequestSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestStatus) DeepCopyInto(out *RemoteSaveRequestStatus) { - *out = *in - if in.Conditions != nil { - in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestStatus. -func (in *RemoteSaveRequestStatus) DeepCopy() *RemoteSaveRequestStatus { - if in == nil { - return nil - } - out := new(RemoteSaveRequestStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *RemoteSaveRequestWindow) DeepCopyInto(out *RemoteSaveRequestWindow) { - *out = *in - if in.AttachTimeout != nil { - in, out := &in.AttachTimeout, &out.AttachTimeout - *out = new(v1.Duration) - **out = **in - } - if in.IdleTimeout != nil { - in, out := &in.IdleTimeout, &out.IdleTimeout - *out = new(v1.Duration) - **out = **in - } - if in.MaxDuration != nil { - in, out := &in.MaxDuration, &out.MaxDuration - *out = new(v1.Duration) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new RemoteSaveRequestWindow. -func (in *RemoteSaveRequestWindow) DeepCopy() *RemoteSaveRequestWindow { - if in == nil { - return nil - } - out := new(RemoteSaveRequestWindow) - in.DeepCopyInto(out) - return out -} diff --git a/api/v1alpha3/access_grants.go b/api/v1alpha3/access_grants.go index ab7d749b..c0629c0a 100644 --- a/api/v1alpha3/access_grants.go +++ b/api/v1alpha3/access_grants.go @@ -220,80 +220,70 @@ type KubeProviderApplierClaimGrant struct { } // GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to -// save a GitTarget's commit windows. +// save a GitTarget's commit windows: on the cluster GitOps Reverser runs in +// (consumerNamespaces), or on the cluster the GitTarget mirrors +// (sourceNamespaces). It sets exactly one of the two. +// +kubebuilder:validation:XValidation:rule="has(self.consumerNamespaces) != has(self.sourceNamespaces)",message="a grant in allowedCommitRequests sets exactly one of consumerNamespaces and sourceNamespaces" type GitTargetCommitRequestGrant struct { - // ConsumerNamespaces are the namespaces whose CommitRequests this grant - // admits. from: "Same" is this GitTarget's namespace. - // +required - ConsumerNamespaces ConsumerNamespaces `json:"consumerNamespaces"` + // ConsumerNamespaces are the namespaces, on the cluster GitOps Reverser + // runs in, whose CommitRequests this grant admits. from: "Same" is this + // GitTarget's namespace. It admits requests only for a GitTarget whose + // KubeProvider sets spec.inCluster. + // +optional + ConsumerNamespaces *ConsumerNamespaces `json:"consumerNamespaces,omitempty"` + + // SourceNamespaces are the namespaces, on the cluster spec.kubeProviderRef + // names, whose CommitRequests this grant admits: exact names, or "*" for + // every namespace this GitTarget may mirror. Variables are refused. A + // namespace the KubeProvider does not lend this GitTarget in + // spec.allowedGitTargets is never admitted. It admits requests only for a + // GitTarget whose KubeProvider names a kubeconfig. + // +optional + // +listType=set + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=64 + SourceNamespaces []SourceNamespace `json:"sourceNamespaces,omitempty"` } -// AdmitsCommitRequest reports whether spec.allowedCommitRequests admits a CommitRequest in namespace, -// by the labels that namespace carries in the control cluster. +// AdmitsCommitRequest reports whether spec.allowedCommitRequests admits a CommitRequest in namespace +// of the control cluster, by the labels that namespace carries there: its consumerNamespaces grants. func (t *GitTarget) AdmitsCommitRequest(namespace string, nsLabels map[string]string) (bool, error) { admitting, err := admitAll(consumersOf(t.commitRequestGrants()), t.Namespace, namespace, nsLabels) return len(admitting) > 0, err } -// DescribeAllowedCommitRequests names the namespaces spec.allowedCommitRequests admits. +// DescribeAllowedCommitRequests names the namespaces spec.allowedCommitRequests admits on the control +// cluster. func (t *GitTarget) DescribeAllowedCommitRequests() string { - return describeAdmitted(consumersOf(t.commitRequestGrants()), t.Namespace) -} - -// SourceNamespace is a namespace of a source cluster, by exact name, or "*" -// for every namespace. -// +kubebuilder:validation:MaxLength=63 -// +kubebuilder:validation:XValidation:rule="self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$')",message="a source namespace is a namespace name, or '*' alone; variables are refused, because a source cluster's namespaces are not the control cluster's" -type SourceNamespace string - -// GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some -// namespaces of the source cluster to save a GitTarget's commit windows. -type GitTargetRemoteSaveRequestGrant struct { - // SourceNamespaces are the namespaces, on the cluster the GitTarget's - // spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: - // exact names, or "*" for every namespace the GitTarget mirrors. - // +required - // +listType=set - // +kubebuilder:validation:MinItems=1 - // +kubebuilder:validation:MaxItems=64 - SourceNamespaces []SourceNamespace `json:"sourceNamespaces"` - - // RequestSelector narrows this grant to the RemoteSaveRequests whose labels - // match. Omitted, it admits every request from sourceNamespaces. Whoever - // creates a request picks its labels, so this picks requests and proves - // nothing about who made them. - // +optional - RequestSelector *metav1.LabelSelector `json:"requestSelector,omitempty"` + grants := t.commitRequestGrants() + consumers := make([]ConsumerNamespaces, 0, len(grants)) + for i := range grants { + if grants[i].ConsumerNamespaces != nil { + consumers = append(consumers, *grants[i].ConsumerNamespaces) + } + } + return describeAdmitted(consumers, t.Namespace) } -// AdmitsRemoteSaveRequest reports whether spec.allowedRemoteSaveRequests admits a RemoteSaveRequest -// in namespace, on the source cluster, carrying requestLabels. A selector that does not convert is an -// error, never a silent allow; it is returned only when no grant admits. -func (t *GitTarget) AdmitsRemoteSaveRequest(namespace string, requestLabels map[string]string) (bool, error) { - var errs []error - for i := range t.Spec.AllowedRemoteSaveRequests { - grant := &t.Spec.AllowedRemoteSaveRequests[i] - if !slices.ContainsFunc(grant.SourceNamespaces, func(ns SourceNamespace) bool { +// AdmitsSourceCommitRequest reports whether spec.allowedCommitRequests admits a CommitRequest in +// namespace of the cluster this GitTarget mirrors: its sourceNamespaces grants, "*" for any. +func (t *GitTarget) AdmitsSourceCommitRequest(namespace string) bool { + for _, grant := range t.commitRequestGrants() { + if slices.ContainsFunc(grant.SourceNamespaces, func(ns SourceNamespace) bool { return ns == "*" || string(ns) == namespace }) { - continue - } - if grant.RequestSelector == nil { - return true, nil - } - selector, err := metav1.LabelSelectorAsSelector(grant.RequestSelector) - if err != nil { - errs = append(errs, fmt.Errorf("grant %d: requestSelector %s: %w", i, - metav1.FormatLabelSelector(grant.RequestSelector), err)) - continue - } - if selector.Matches(labels.Set(requestLabels)) { - return true, nil + return true } } - return false, errors.Join(errs...) + return false } +// SourceNamespace is a namespace of a source cluster, by exact name, or "*" +// for every namespace. +// +kubebuilder:validation:MaxLength=63 +// +kubebuilder:validation:XValidation:rule="self == '*' || self.matches('^[a-z0-9]([-a-z0-9]*[a-z0-9])?$')",message="a source namespace is a namespace name, or '*' alone; variables are refused, because a source cluster's namespaces are not the control cluster's" +type SourceNamespace string + // The API server defaults an omitted grant list on GitBranch, KubeProvider and GitTarget (the // +kubebuilder:default markers on their fields), so a list read from it is never nil. These mirror // those markers for an object that did not come from the API server, such as one built in memory or @@ -327,7 +317,7 @@ func DefaultKubeProviderApplierClaimGrants() []KubeProviderApplierClaimGrant { // DefaultGitTargetCommitRequestGrants is GitTarget.spec.allowedCommitRequests when it is omitted: // CommitRequests in the GitTarget's own namespace. func DefaultGitTargetCommitRequestGrants() []GitTargetCommitRequestGrant { - return []GitTargetCommitRequestGrant{{ConsumerNamespaces: ConsumerNamespaces{From: GrantFromSame}}} + return []GitTargetCommitRequestGrant{{ConsumerNamespaces: &ConsumerNamespaces{From: GrantFromSame}}} } // orDefault is grants, or def when grants is nil (omitted rather than empty). @@ -358,8 +348,14 @@ func (t *GitTarget) commitRequestGrants() []GitTargetCommitRequestGrant { return orDefault(t.Spec.AllowedCommitRequests, DefaultGitTargetCommitRequestGrants) } -// consumers is the admitting half of a GitTargetCommitRequestGrant. -func (g *GitTargetCommitRequestGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } +// consumers is the admitting half of a GitTargetCommitRequestGrant on the control cluster. A grant of +// source namespaces admits nobody there: its zero ConsumerNamespaces has no from. +func (g *GitTargetCommitRequestGrant) consumers() ConsumerNamespaces { + if g.ConsumerNamespaces == nil { + return ConsumerNamespaces{} + } + return *g.ConsumerNamespaces +} // consumers is the admitting half of a GitProviderGitBranchGrant. func (g *GitProviderGitBranchGrant) consumers() ConsumerNamespaces { return g.ConsumerNamespaces } diff --git a/api/v1alpha3/access_grants_test.go b/api/v1alpha3/access_grants_test.go index 1519c2c6..b921ec50 100644 --- a/api/v1alpha3/access_grants_test.go +++ b/api/v1alpha3/access_grants_test.go @@ -350,46 +350,32 @@ func TestGitBranch_GitProviderKey(t *testing.T) { assert.Equal(t, "platform/shared-repo", branch.GitProviderKey().String()) } -// A GitTarget admits a RemoteSaveRequest only when one grant admits both its namespace and its -// labels; a grant whose selector does not convert admits nothing, and says why only when no other -// grant admits. -func TestGitTarget_AdmitsRemoteSaveRequest(t *testing.T) { - broken := &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ - {Key: "app", Operator: "Near"}, - }} - cart := &metav1.LabelSelector{MatchLabels: map[string]string{"app": "cart"}} - target := func(grants ...GitTargetRemoteSaveRequestGrant) *GitTarget { - return &GitTarget{Spec: GitTargetSpec{AllowedRemoteSaveRequests: grants}} +// A GitTarget admits a CommitRequest by the side its grant names: consumerNamespaces admit a +// namespace of the control cluster, sourceNamespaces a namespace of the cluster the GitTarget mirrors. +// A grant for one side admits nothing on the other. +func TestGitTarget_AdmitsCommitRequestBySide(t *testing.T) { + target := func(grants ...GitTargetCommitRequestGrant) *GitTarget { + return &GitTarget{ObjectMeta: metav1.ObjectMeta{Namespace: "team-a"}, Spec: GitTargetSpec{ + AllowedCommitRequests: grants, + }} + } + source := func(namespaces ...SourceNamespace) GitTargetCommitRequestGrant { + return GitTargetCommitRequestGrant{SourceNamespaces: namespaces} } - labels := map[string]string{"app": "cart"} - admitted, err := target().AdmitsRemoteSaveRequest("shop", labels) + defaulted := &GitTarget{ObjectMeta: metav1.ObjectMeta{Namespace: "team-a"}} + admitted, err := defaulted.AdmitsCommitRequest("team-a", nil) require.NoError(t, err) - assert.False(t, admitted, "nothing is admitted by default") + assert.True(t, admitted, "the default admits the GitTarget's own namespace") + assert.False(t, defaulted.AdmitsSourceCommitRequest("team-a"), "the default admits nothing on a source cluster") - admitted, err = target(GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"*"}}). - AdmitsRemoteSaveRequest("shop", labels) - require.NoError(t, err) - assert.True(t, admitted) + assert.False(t, target().AdmitsSourceCommitRequest("shop"), "an empty list admits nobody") + assert.True(t, target(source("*")).AdmitsSourceCommitRequest("shop")) + assert.True(t, target(source("shop")).AdmitsSourceCommitRequest("shop")) + assert.False(t, target(source("cart")).AdmitsSourceCommitRequest("shop")) - admitted, err = target( - GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: cart}, - ). - AdmitsRemoteSaveRequest("shop", map[string]string{"app": "admin"}) + admitted, err = target(source("*")).AdmitsCommitRequest("team-a", nil) require.NoError(t, err) - assert.False(t, admitted, "its labels must match too") - - admitted, err = target( - GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: broken}, - ). - AdmitsRemoteSaveRequest("shop", labels) - require.Error(t, err) - assert.False(t, admitted) - - admitted, err = target( - GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: broken}, - GitTargetRemoteSaveRequestGrant{SourceNamespaces: []SourceNamespace{"shop"}, RequestSelector: cart}, - ).AdmitsRemoteSaveRequest("shop", labels) - require.NoError(t, err, "another grant admits") - assert.True(t, admitted) + assert.False(t, admitted, "a source grant admits nothing on the control cluster") + assert.Equal(t, "admits no namespace", target(source("*")).DescribeAllowedCommitRequests()) } diff --git a/api/v1alpha3/commitrequest_schema_test.go b/api/v1alpha3/commitrequest_schema_test.go new file mode 100644 index 00000000..f6e92dd1 --- /dev/null +++ b/api/v1alpha3/commitrequest_schema_test.go @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: Apache-2.0 + +package v1alpha3 + +import ( + "crypto/sha256" + "encoding/hex" + "os" + "regexp" + "strconv" + "strings" + "testing" + + "github.com/stretchr/testify/require" +) + +// commitRequestSchemaHashes is the hash of the generated CommitRequest CRD at each schema revision, +// with the revision line itself left out. A change to the CRD that keeps the revision fails +// TestCommitRequestSchemaRevision: raise CommitRequestSchemaRevision, and the revision line, and add +// the new hash here. +var commitRequestSchemaHashes = map[int]string{ + 1: "ea99cba6116e0f562c0be931c13d6580eabdc2e7ba32d486d20a0a5029163e99", +} + +// commitRequestRevisionLine is the last line of the kind's description. +var commitRequestRevisionLine = regexp.MustCompile( + `Schema revision (\d+), compatible from revision (\d+)\.`) + +// The CommitRequest CRD carries its schema revision as the last line of the kind's description, and +// the revision moves with every change to the generated CRD, so a cluster that installed it on its own +// can be judged by it. +func TestCommitRequestSchemaRevision(t *testing.T) { + raw, err := os.ReadFile("../../config/crd/bases/configbutler.ai_commitrequests.yaml") + require.NoError(t, err) + crd := string(raw) + + match := commitRequestRevisionLine.FindStringSubmatch(crd) + require.NotNil(t, match, "the CRD's description must end with the schema revision line") + revision, _ := strconv.Atoi(match[1]) + floor, _ := strconv.Atoi(match[2]) + require.Equal(t, CommitRequestSchemaRevision, revision, "the revision line and CommitRequestSchemaRevision") + require.Equal(t, CommitRequestSchemaFloor, floor, "the revision line and CommitRequestSchemaFloor") + require.LessOrEqual(t, floor, revision) + + sum := sha256.Sum256([]byte(strings.Replace(crd, match[0], "", 1))) + hash := hex.EncodeToString(sum[:]) + want, recorded := commitRequestSchemaHashes[revision] + require.True(t, recorded, "revision %d has no hash recorded in commitRequestSchemaHashes", revision) + require.Equal(t, want, hash, "the generated CommitRequest CRD changed at revision %d: raise the schema "+ + "revision in api/v1alpha3/commitrequest_types.go, and record the new CRD's hash for it in "+ + "commitRequestSchemaHashes", revision) +} diff --git a/api/v1alpha3/commitrequest_types.go b/api/v1alpha3/commitrequest_types.go index 3480e7ea..4bfc6b3c 100644 --- a/api/v1alpha3/commitrequest_types.go +++ b/api/v1alpha3/commitrequest_types.go @@ -258,6 +258,14 @@ type CommitRequestStatus struct { AppliedRevision string `json:"appliedRevision,omitempty"` } +// The last line of the kind's description is its schema revision, which a cluster that installed the +// CRD on its own, as a source cluster does, can be judged by. It ships in the CRD, so the same +// published schema reads it back: CommitRequestSchemaRevision and CommitRequestSchemaFloor below +// must say the same, and TestCommitRequestSchemaRevision fails when the generated CRD changes and +// they do not move. Raise the revision with every change to the generated CRD, and the floor ("compatible +// from") with a change that breaks an older reader or writer: a new required field, a removed field, +// or a changed meaning. + // +kubebuilder:object:root=true // +kubebuilder:subresource:status // +kubebuilder:printcolumn:name="GitTarget",type=string,JSONPath=`.spec.gitTargetRef.name` @@ -273,8 +281,12 @@ type CommitRequestStatus struct { // CommitRequest asks for a save: it attaches to its author's current or next // commit window on a GitTarget, closes that window on its own timers, and -// gives the commit its message. Its spec is immutable, and status.commit -// reports the commit once it is on the remote. +// gives the commit its message. Create it on the cluster GitOps Reverser runs +// in, or on a source cluster whose KubeProvider sets spec.commitRequests. Its +// spec is immutable, and status.commit reports the commit once it is on the +// remote. +// +// Schema revision 1, compatible from revision 1. type CommitRequest struct { metav1.TypeMeta `json:",inline"` @@ -312,6 +324,14 @@ func (s *CommitRequest) GitTargetKey() types.NamespacedName { return types.NamespacedName{Namespace: namespace, Name: s.Spec.GitTargetRef.Name} } +// CommitRequestSchemaRevision and CommitRequestSchemaFloor are the revision of the CommitRequest +// schema this operator was built with, and the oldest revision it can serve: the last line of the +// kind's description. +const ( + CommitRequestSchemaRevision = 1 + CommitRequestSchemaFloor = 1 +) + func init() { SchemeBuilder.Register(&CommitRequest{}, &CommitRequestList{}) } diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index f8d0b0c9..6f9d8f11 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -185,42 +185,28 @@ type GitTargetSpec struct { Commit *GitTargetCommitSpec `json:"commit,omitempty"` // AllowedCommitRequests says which CommitRequests may save this GitTarget's - // commit windows, by the namespace they are in: + // commit windows. Each grant sets one of: // - // - Omitted: the CommitRequests in this GitTarget's namespace, as one grant - // with from: "Same". - // - Empty list ([]): no CommitRequest. - // - A list of grants: the CommitRequests that at least one grant allows. + // - consumerNamespaces: namespaces of the cluster GitOps Reverser runs + // in. They admit requests only when spec.kubeProviderRef names an + // inCluster KubeProvider. + // - sourceNamespaces: namespaces of the cluster spec.kubeProviderRef + // names, which must set spec.commitRequests. They admit requests only + // when that KubeProvider names a kubeconfig. // - // A list replaces the default: the CommitRequests in this namespace keep - // access only if at least one grant allows them. A request no grant allows + // Omitted, the CommitRequests in this GitTarget's namespace are admitted, + // as one grant with consumerNamespaces from: "Same". An empty list ([]) + // admits none. A list replaces the default. A request no grant admits // fails with reason "NotAllowed". + // + // An admitted request saves its creator's whole commit window on this + // GitTarget, writes outside the request's namespace included. // +optional // +listType=atomic // +kubebuilder:validation:MaxItems=64 // +kubebuilder:default={{consumerNamespaces: {from: Same}}} AllowedCommitRequests []GitTargetCommitRequestGrant `json:"allowedCommitRequests"` - // Consent lives on the GitTarget, not on the KubeProvider that serves the requests: the provider's - // owner lends a cluster, and whether a save made on it may steer this folder's commits, with its - // message, its timers and an apply, is the folder's owner's decision. Nothing is admitted by - // default, because a source cluster's namespaces are not the control cluster's, so there is no - // "Same" to default to. - - // AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster - // spec.kubeProviderRef names, that may save this GitTarget's commit - // windows. A request is admitted when one grant admits both its namespace - // and its labels. Omitted or empty, none is. A request that GitTargets - // mirror but none admits reports "NotAllowed". The KubeProvider must also - // set spec.remoteSaveRequests "Enabled". - // - // An admitted request saves its creator's whole commit window on this - // GitTarget, edits outside the request's namespace included. - // +optional - // +listType=atomic - // +kubebuilder:validation:MaxItems=16 - AllowedRemoteSaveRequests []GitTargetRemoteSaveRequestGrant `json:"allowedRemoteSaveRequests,omitempty"` - // The scan keeps running while writes are suspended: freezing status.placement too would leave // a stale answer at the moment it costs most. A suspended target is Ready=True reason // Suspended, because not writing is the configured outcome rather than ill health. A local diff --git a/api/v1alpha3/kubeprovider_types.go b/api/v1alpha3/kubeprovider_types.go index 5b575888..ab5c6172 100644 --- a/api/v1alpha3/kubeprovider_types.go +++ b/api/v1alpha3/kubeprovider_types.go @@ -28,6 +28,10 @@ import ( // mirroring through it, so the credential is immutable, including switching // arms. A rotated kubeconfig is a change to the Secret, which stays allowed. // +kubebuilder:validation:XValidation:rule="has(self.inCluster) == has(oldSelf.inCluster) && (!has(self.secretRef) || self.secretRef == oldSelf.secretRef)",message="spec.secretRef and spec.inCluster are immutable; delete and recreate the KubeProvider to point it at a different cluster" +// +// The cluster GitOps Reverser runs in has no switch: its CommitRequests are +// always served, by the operator itself. +// +kubebuilder:validation:XValidation:rule="!has(self.inCluster) || !has(self.commitRequests) || self.commitRequests == 'Ignore'",message="spec.commitRequests must be Ignore beside spec.inCluster: the CommitRequests of the cluster GitOps Reverser runs in are always served" type KubeProviderSpec struct { // SecretRef names the Secret, in this namespace, that holds a kubeconfig for // the cluster. With no key, the kubeconfig is read from "value", then @@ -100,32 +104,43 @@ type KubeProviderSpec struct { // +optional Attribution *KubeProviderAttribution `json:"attribution,omitempty"` - // RemoteSaveRequests says whether GitOps Reverser serves the - // RemoteSaveRequests (source.configbutler.ai) created on this cluster. - // "Enabled" watches them, saves the commit windows of the GitTargets that - // mirror each one's namespace through this KubeProvider and admit it in - // spec.allowedRemoteSaveRequests, and writes each one's status back. - // Defaults to "Disabled". + // CommitRequests says whether GitOps Reverser serves the CommitRequests + // created on this cluster, for the GitTargets that mirror through this + // KubeProvider: + // + // - "Ignore": they are left alone. + // - "Reconcile": the requests in the namespaces this KubeProvider lends + // its GitTargets in spec.allowedGitTargets are served, and their status + // is written. A finished request stays. + // - "ReconcileAndDeleteFinished": as "Reconcile", and a finished request + // is annotated configbutler.ai/delete-after and deleted once that time + // has passed. + // + // Defaults to "Ignore". Only "Ignore" goes with inCluster: the + // CommitRequests of the cluster GitOps Reverser runs in are always served. // - // "Enabled" writes to this cluster: the credential needs get, list, watch - // and delete on remotesaverequests, and update on - // remotesaverequests/status. The RemoteSaveRequest CRD must be installed - // on the cluster. + // The credential needs get, list and watch on commitrequests, and update + // on commitrequests/status, in the configbutler.ai group, in each lent + // namespace; "ReconcileAndDeleteFinished" also needs patch and delete on + // commitrequests. The CommitRequest CRD must be installed on the cluster. + // The CommitRequestsServed condition reports how it goes. // +optional - // +kubebuilder:validation:Enum=Disabled;Enabled - // +kubebuilder:default=Disabled - RemoteSaveRequests RemoteSaveRequestsMode `json:"remoteSaveRequests,omitempty"` + // +kubebuilder:validation:Enum=Ignore;Reconcile;ReconcileAndDeleteFinished + // +kubebuilder:default=Ignore + CommitRequests CommitRequestsPolicy `json:"commitRequests,omitempty"` } -// RemoteSaveRequestsMode is whether a KubeProvider's RemoteSaveRequests are -// served. -type RemoteSaveRequestsMode string +// CommitRequestsPolicy is whether a KubeProvider's CommitRequests are served, +// and whether finished ones are deleted. +type CommitRequestsPolicy string const ( - // RemoteSaveRequestsDisabled leaves the cluster's RemoteSaveRequests alone. - RemoteSaveRequestsDisabled RemoteSaveRequestsMode = "Disabled" - // RemoteSaveRequestsEnabled serves the cluster's RemoteSaveRequests. - RemoteSaveRequestsEnabled RemoteSaveRequestsMode = "Enabled" + // CommitRequestsIgnore leaves the cluster's CommitRequests alone. + CommitRequestsIgnore CommitRequestsPolicy = "Ignore" + // CommitRequestsReconcile serves the cluster's CommitRequests and keeps finished ones. + CommitRequestsReconcile CommitRequestsPolicy = "Reconcile" + // CommitRequestsReconcileAndDeleteFinished serves them and deletes finished ones. + CommitRequestsReconcileAndDeleteFinished CommitRequestsPolicy = "ReconcileAndDeleteFinished" ) // KubeProviderInCluster selects the operator's own identity. It has no fields: @@ -196,6 +211,11 @@ type KubeProviderStatus struct { // +optional AuditRoute string `json:"auditRoute,omitempty"` + // CommitRequests is what this KubeProvider serves of its cluster's + // CommitRequests. Absent while spec.commitRequests is "Ignore". + // +optional + CommitRequests *KubeProviderCommitRequestsStatus `json:"commitRequests,omitempty"` + // Conditions are the latest observations of this KubeProvider's state. Ready, // Reconciling and Stalled follow kstatus; Stalled=True with reason // InClusterNotAuthorized or AuditRouteConflict means the KubeProvider is not @@ -209,6 +229,19 @@ type KubeProviderStatus struct { // NoAuditDelivery (no audit events arrive at all), or RouteUnused (audit // events arrive, but never on this route). It is absent when author // attribution is off. + // + // CommitRequestsServed, which is not part of Ready either, says whether + // this cluster's CommitRequests are served. It is absent while + // spec.commitRequests is "Ignore". Each reason that is not True ends with + // its fix: + // + // - True, Watching: watched in status.commitRequests.namespaces. + // - True, NothingToWatch: no GitTarget mirroring through this KubeProvider + // is lent a namespace. + // - False, CRDNotInstalled: the cluster serves no CommitRequest CRD. + // - False, VersionNotServed: the CRD does not serve v1alpha3. + // - False, Forbidden: the credential may not list or watch them. + // - Unknown, Unreachable: the cluster's API could not be read. // +optional // +listType=map // +listMapKey=type @@ -217,6 +250,17 @@ type KubeProviderStatus struct { Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` } +// KubeProviderCommitRequestsStatus is what a KubeProvider serves of its +// cluster's CommitRequests. +type KubeProviderCommitRequestsStatus struct { + // Namespaces are the namespaces whose CommitRequests are watched: the ones + // this KubeProvider lends the GitTargets that mirror through it. "*" is + // every namespace, in one watch. + // +optional + // +listType=set + Namespaces []string `json:"namespaces,omitempty"` +} + // +kubebuilder:object:root=true // +kubebuilder:subresource:status // +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].status` @@ -224,6 +268,7 @@ type KubeProviderStatus struct { // +kubebuilder:printcolumn:name="FactsReceived",type=string,JSONPath=`.status.conditions[?(@.type=="AuditFactsReceived")].status` // +kubebuilder:printcolumn:name="Route",type=string,JSONPath=`.status.auditRoute`,priority=1 // +kubebuilder:printcolumn:name="Validated",type=string,JSONPath=`.status.conditions[?(@.type=="Validated")].status`,priority=1 +// +kubebuilder:printcolumn:name="Saves",type=string,JSONPath=`.status.conditions[?(@.type=="CommitRequestsServed")].reason`,priority=1 // +kubebuilder:printcolumn:name="Message",type=string,JSONPath=`.status.conditions[?(@.type=="Ready")].message`,priority=1 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` @@ -265,9 +310,17 @@ func (p *KubeProvider) Key() types.NamespacedName { } // ServesCommitRequests reports whether GitOps Reverser serves the CommitRequests created on this -// provider's cluster, through its own watch of it. +// provider's cluster, through its own watch of it: spec.commitRequests is one of the two Reconcile +// values, and the provider names a kubeconfig. func (p *KubeProvider) ServesCommitRequests() bool { - return p.Spec.RemoteSaveRequests == RemoteSaveRequestsEnabled + return !p.IsInCluster() && (p.Spec.CommitRequests == CommitRequestsReconcile || + p.Spec.CommitRequests == CommitRequestsReconcileAndDeleteFinished) +} + +// DeletesFinishedCommitRequests reports whether finished CommitRequests on this provider's cluster +// are deleted. +func (p *KubeProvider) DeletesFinishedCommitRequests() bool { + return p.ServesCommitRequests() && p.Spec.CommitRequests == CommitRequestsReconcileAndDeleteFinished } // IsInCluster reports whether this provider asks for the operator's own identity (spec.inCluster). diff --git a/api/v1alpha3/zz_generated.deepcopy.go b/api/v1alpha3/zz_generated.deepcopy.go index e22ff622..e666b9ee 100644 --- a/api/v1alpha3/zz_generated.deepcopy.go +++ b/api/v1alpha3/zz_generated.deepcopy.go @@ -1201,7 +1201,16 @@ func (in *GitTarget) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetCommitRequestGrant) DeepCopyInto(out *GitTargetCommitRequestGrant) { *out = *in - in.ConsumerNamespaces.DeepCopyInto(&out.ConsumerNamespaces) + if in.ConsumerNamespaces != nil { + in, out := &in.ConsumerNamespaces, &out.ConsumerNamespaces + *out = new(ConsumerNamespaces) + (*in).DeepCopyInto(*out) + } + if in.SourceNamespaces != nil { + in, out := &in.SourceNamespaces, &out.SourceNamespaces + *out = make([]SourceNamespace, len(*in)) + copy(*out, *in) + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetCommitRequestGrant. @@ -1317,31 +1326,6 @@ func (in *GitTargetPlacementStatus) DeepCopy() *GitTargetPlacementStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *GitTargetRemoteSaveRequestGrant) DeepCopyInto(out *GitTargetRemoteSaveRequestGrant) { - *out = *in - if in.SourceNamespaces != nil { - in, out := &in.SourceNamespaces, &out.SourceNamespaces - *out = make([]SourceNamespace, len(*in)) - copy(*out, *in) - } - if in.RequestSelector != nil { - in, out := &in.RequestSelector, &out.RequestSelector - *out = new(v1.LabelSelector) - (*in).DeepCopyInto(*out) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetRemoteSaveRequestGrant. -func (in *GitTargetRemoteSaveRequestGrant) DeepCopy() *GitTargetRemoteSaveRequestGrant { - if in == nil { - return nil - } - out := new(GitTargetRemoteSaveRequestGrant) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GitTargetRetentionStatus) DeepCopyInto(out *GitTargetRetentionStatus) { *out = *in @@ -1403,13 +1387,6 @@ func (in *GitTargetSpec) DeepCopyInto(out *GitTargetSpec) { (*in)[i].DeepCopyInto(&(*out)[i]) } } - if in.AllowedRemoteSaveRequests != nil { - in, out := &in.AllowedRemoteSaveRequests, &out.AllowedRemoteSaveRequests - *out = make([]GitTargetRemoteSaveRequestGrant, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GitTargetSpec. @@ -1562,6 +1539,26 @@ func (in *KubeProviderClient) DeepCopy() *KubeProviderClient { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *KubeProviderCommitRequestsStatus) DeepCopyInto(out *KubeProviderCommitRequestsStatus) { + *out = *in + if in.Namespaces != nil { + in, out := &in.Namespaces, &out.Namespaces + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KubeProviderCommitRequestsStatus. +func (in *KubeProviderCommitRequestsStatus) DeepCopy() *KubeProviderCommitRequestsStatus { + if in == nil { + return nil + } + out := new(KubeProviderCommitRequestsStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *KubeProviderGitTargetGrant) DeepCopyInto(out *KubeProviderGitTargetGrant) { *out = *in @@ -1687,6 +1684,11 @@ func (in *KubeProviderSpec) DeepCopy() *KubeProviderSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *KubeProviderStatus) DeepCopyInto(out *KubeProviderStatus) { *out = *in + if in.CommitRequests != nil { + in, out := &in.CommitRequests, &out.CommitRequests + *out = new(KubeProviderCommitRequestsStatus) + (*in).DeepCopyInto(*out) + } if in.Conditions != nil { in, out := &in.Conditions, &out.Conditions *out = make([]v1.Condition, len(*in)) diff --git a/cmd/main.go b/cmd/main.go index 0e771e3b..7da0cc03 100644 --- a/cmd/main.go +++ b/cmd/main.go @@ -326,15 +326,18 @@ func main() { // Setup watch manager (must be after controllers are set up) fatalIfErr(watchMgr.SetupWithManager(mgr), "unable to setup watch ingestion manager") fatalIfErr(mgr.Add(watchMgr), "unable to add watch ingestion manager") - // RemoteSaveRequests are saves made on a source cluster, next to the edits they save: served on every - // KubeProvider whose spec.remoteSaveRequests is Enabled, attributed from that cluster's audit facts. - fatalIfErr(mgr.Add(&watch.RemoteSaveRequestServer{ - Manager: watchMgr, - Client: mgr.GetClient(), - Finalizer: eventRouter, - TTL: cfg.commitRequestTTL, - Log: ctrl.Log.WithName("remotesaverequests"), - }), "unable to add the RemoteSaveRequest server") + // CommitRequests created on a source cluster, next to the edits they save: served through every + // KubeProvider whose spec.commitRequests reconciles them, attributed from that cluster's audit facts. + commitRequestServer := &controller.CommitRequestServer{ + Client: mgr.GetClient(), + Clusters: watchMgr, + Finalizer: eventRouter, + Authors: watchMgr.AuthorResolver, + FactStreams: watchMgr.FactStreams, + TTL: cfg.commitRequestTTL, + Log: ctrl.Log.WithName("commitrequests"), + } + fatalIfErr(mgr.Add(commitRequestServer), "unable to add the CommitRequest server") // The GitBranch, GitTarget and applier claim controllers find a GitProvider's GitBranches, in any // namespace, through this index. @@ -364,6 +367,7 @@ func main() { Client: mgr.GetClient(), Scheme: mgr.GetScheme(), KubeConfigSafety: cfg.kubeConfigSafety, + CommitRequests: commitRequestServer, Recorder: mgr.GetEventRecorderFor("kubeprovider"), } // A nil registry must stay a nil INTERFACE, not one wrapping a nil pointer: the reconciler diff --git a/config/crd/bases/configbutler.ai_commitrequests.yaml b/config/crd/bases/configbutler.ai_commitrequests.yaml index 01f9d1bf..64e4b8eb 100644 --- a/config/crd/bases/configbutler.ai_commitrequests.yaml +++ b/config/crd/bases/configbutler.ai_commitrequests.yaml @@ -55,8 +55,12 @@ spec: description: |- CommitRequest asks for a save: it attaches to its author's current or next commit window on a GitTarget, closes that window on its own timers, and - gives the commit its message. Its spec is immutable, and status.commit - reports the commit once it is on the remote. + gives the commit its message. Create it on the cluster GitOps Reverser runs + in, or on a source cluster whose KubeProvider sets spec.commitRequests. Its + spec is immutable, and status.commit reports the commit once it is on the + remote. + + Schema revision 1, compatible from revision 1. properties: apiVersion: description: |- diff --git a/config/crd/bases/configbutler.ai_gittargets.yaml b/config/crd/bases/configbutler.ai_gittargets.yaml index f5a0521e..84e81baa 100644 --- a/config/crd/bases/configbutler.ai_gittargets.yaml +++ b/config/crd/bases/configbutler.ai_gittargets.yaml @@ -113,25 +113,35 @@ spec: from: Same description: |- AllowedCommitRequests says which CommitRequests may save this GitTarget's - commit windows, by the namespace they are in: + commit windows. Each grant sets one of: - - Omitted: the CommitRequests in this GitTarget's namespace, as one grant - with from: "Same". - - Empty list ([]): no CommitRequest. - - A list of grants: the CommitRequests that at least one grant allows. + - consumerNamespaces: namespaces of the cluster GitOps Reverser runs + in. They admit requests only when spec.kubeProviderRef names an + inCluster KubeProvider. + - sourceNamespaces: namespaces of the cluster spec.kubeProviderRef + names, which must set spec.commitRequests. They admit requests only + when that KubeProvider names a kubeconfig. - A list replaces the default: the CommitRequests in this namespace keep - access only if at least one grant allows them. A request no grant allows + Omitted, the CommitRequests in this GitTarget's namespace are admitted, + as one grant with consumerNamespaces from: "Same". An empty list ([]) + admits none. A list replaces the default. A request no grant admits fails with reason "NotAllowed". + + An admitted request saves its creator's whole commit window on this + GitTarget, writes outside the request's namespace included. items: description: |- GitTargetCommitRequestGrant admits the CommitRequests of some namespaces to - save a GitTarget's commit windows. + save a GitTarget's commit windows: on the cluster GitOps Reverser runs in + (consumerNamespaces), or on the cluster the GitTarget mirrors + (sourceNamespaces). It sets exactly one of the two. properties: consumerNamespaces: description: |- - ConsumerNamespaces are the namespaces whose CommitRequests this grant - admits. from: "Same" is this GitTarget's namespace. + ConsumerNamespaces are the namespaces, on the cluster GitOps Reverser + runs in, whose CommitRequests this grant admits. from: "Same" is this + GitTarget's namespace. It admits requests only for a GitTarget whose + KubeProvider sets spec.inCluster. properties: from: description: |- @@ -240,83 +250,14 @@ spec: rule: '!has(self.selector) || (has(self.selector.matchLabels) && size(self.selector.matchLabels) > 0) || (has(self.selector.matchExpressions) && size(self.selector.matchExpressions) > 0)' - required: - - consumerNamespaces - type: object - maxItems: 64 - type: array - x-kubernetes-list-type: atomic - allowedRemoteSaveRequests: - description: |- - AllowedRemoteSaveRequests are the RemoteSaveRequests, made on the cluster - spec.kubeProviderRef names, that may save this GitTarget's commit - windows. A request is admitted when one grant admits both its namespace - and its labels. Omitted or empty, none is. A request that GitTargets - mirror but none admits reports "NotAllowed". The KubeProvider must also - set spec.remoteSaveRequests "Enabled". - - An admitted request saves its creator's whole commit window on this - GitTarget, edits outside the request's namespace included. - items: - description: |- - GitTargetRemoteSaveRequestGrant admits the RemoteSaveRequests of some - namespaces of the source cluster to save a GitTarget's commit windows. - properties: - requestSelector: - description: |- - RequestSelector narrows this grant to the RemoteSaveRequests whose labels - match. Omitted, it admits every request from sourceNamespaces. Whoever - creates a request picks its labels, so this picks requests and proves - nothing about who made them. - properties: - matchExpressions: - description: matchExpressions is a list of label selector - requirements. The requirements are ANDed. - items: - description: |- - A label selector requirement is a selector that contains values, a key, and an operator that - relates the key and values. - properties: - key: - description: key is the label key that the selector - applies to. - type: string - operator: - description: |- - operator represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists and DoesNotExist. - type: string - values: - description: |- - values is an array of string values. If the operator is In or NotIn, - the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. This array is replaced during a strategic - merge patch. - items: - type: string - type: array - x-kubernetes-list-type: atomic - required: - - key - - operator - type: object - type: array - x-kubernetes-list-type: atomic - matchLabels: - additionalProperties: - type: string - description: |- - matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels - map is equivalent to an element of matchExpressions, whose key field is "key", the - operator is "In", and the values array contains only "value". The requirements are ANDed. - type: object - type: object - x-kubernetes-map-type: atomic sourceNamespaces: description: |- - SourceNamespaces are the namespaces, on the cluster the GitTarget's - spec.kubeProviderRef names, whose RemoteSaveRequests this grant admits: - exact names, or "*" for every namespace the GitTarget mirrors. + SourceNamespaces are the namespaces, on the cluster spec.kubeProviderRef + names, whose CommitRequests this grant admits: exact names, or "*" for + every namespace this GitTarget may mirror. Variables are refused. A + namespace the KubeProvider does not lend this GitTarget in + spec.allowedGitTargets is never admitted. It admits requests only for a + GitTarget whose KubeProvider names a kubeconfig. items: description: |- SourceNamespace is a namespace of a source cluster, by exact name, or "*" @@ -332,10 +273,12 @@ spec: minItems: 1 type: array x-kubernetes-list-type: set - required: - - sourceNamespaces type: object - maxItems: 16 + x-kubernetes-validations: + - message: a grant in allowedCommitRequests sets exactly one of + consumerNamespaces and sourceNamespaces + rule: has(self.consumerNamespaces) != has(self.sourceNamespaces) + maxItems: 64 type: array x-kubernetes-list-type: atomic applierClaimRef: diff --git a/config/crd/bases/configbutler.ai_kubeproviders.yaml b/config/crd/bases/configbutler.ai_kubeproviders.yaml index 8d9d3c01..1487bc31 100644 --- a/config/crd/bases/configbutler.ai_kubeproviders.yaml +++ b/config/crd/bases/configbutler.ai_kubeproviders.yaml @@ -32,6 +32,10 @@ spec: name: Validated priority: 1 type: string + - jsonPath: .status.conditions[?(@.type=="CommitRequestsServed")].reason + name: Saves + priority: 1 + type: string - jsonPath: .status.conditions[?(@.type=="Ready")].message name: Message priority: 1 @@ -501,6 +505,34 @@ spec: minimum: 1 type: integer type: object + commitRequests: + default: Ignore + description: |- + CommitRequests says whether GitOps Reverser serves the CommitRequests + created on this cluster, for the GitTargets that mirror through this + KubeProvider: + + - "Ignore": they are left alone. + - "Reconcile": the requests in the namespaces this KubeProvider lends + its GitTargets in spec.allowedGitTargets are served, and their status + is written. A finished request stays. + - "ReconcileAndDeleteFinished": as "Reconcile", and a finished request + is annotated configbutler.ai/delete-after and deleted once that time + has passed. + + Defaults to "Ignore". Only "Ignore" goes with inCluster: the + CommitRequests of the cluster GitOps Reverser runs in are always served. + + The credential needs get, list and watch on commitrequests, and update + on commitrequests/status, in the configbutler.ai group, in each lent + namespace; "ReconcileAndDeleteFinished" also needs patch and delete on + commitrequests. The CommitRequest CRD must be installed on the cluster. + The CommitRequestsServed condition reports how it goes. + enum: + - Ignore + - Reconcile + - ReconcileAndDeleteFinished + type: string inCluster: description: |- InCluster makes this KubeProvider use the operator's own identity in the @@ -509,24 +541,6 @@ spec: InClusterNotAuthorized, and is never used. Set exactly one of secretRef and inCluster. Immutable: delete and recreate the KubeProvider to change it. type: object - remoteSaveRequests: - default: Disabled - description: |- - RemoteSaveRequests says whether GitOps Reverser serves the - RemoteSaveRequests (source.configbutler.ai) created on this cluster. - "Enabled" watches them, saves the commit windows of the GitTargets that - mirror each one's namespace through this KubeProvider and admit it in - spec.allowedRemoteSaveRequests, and writes each one's status back. - Defaults to "Disabled". - - "Enabled" writes to this cluster: the credential needs get, list, watch - and delete on remotesaverequests, and update on - remotesaverequests/status. The RemoteSaveRequest CRD must be installed - on the cluster. - enum: - - Disabled - - Enabled - type: string secretRef: description: |- SecretRef names the Secret, in this namespace, that holds a kubeconfig for @@ -560,6 +574,11 @@ spec: recreate the KubeProvider to point it at a different cluster rule: has(self.inCluster) == has(oldSelf.inCluster) && (!has(self.secretRef) || self.secretRef == oldSelf.secretRef) + - message: 'spec.commitRequests must be Ignore beside spec.inCluster: + the CommitRequests of the cluster GitOps Reverser runs in are always + served' + rule: '!has(self.inCluster) || !has(self.commitRequests) || self.commitRequests + == ''Ignore''' status: description: status defines the observed state of KubeProvider. properties: @@ -569,6 +588,21 @@ spec: . when that is empty. The AuditFactsReceived condition describes this route, and starts over when it changes. type: string + commitRequests: + description: |- + CommitRequests is what this KubeProvider serves of its cluster's + CommitRequests. Absent while spec.commitRequests is "Ignore". + properties: + namespaces: + description: |- + Namespaces are the namespaces whose CommitRequests are watched: the ones + this KubeProvider lends the GitTargets that mirror through it. "*" is + every namespace, in one watch. + items: + type: string + type: array + x-kubernetes-list-type: set + type: object conditions: description: |- Conditions are the latest observations of this KubeProvider's state. Ready, @@ -584,6 +618,19 @@ spec: NoAuditDelivery (no audit events arrive at all), or RouteUnused (audit events arrive, but never on this route). It is absent when author attribution is off. + + CommitRequestsServed, which is not part of Ready either, says whether + this cluster's CommitRequests are served. It is absent while + spec.commitRequests is "Ignore". Each reason that is not True ends with + its fix: + + - True, Watching: watched in status.commitRequests.namespaces. + - True, NothingToWatch: no GitTarget mirroring through this KubeProvider + is lent a namespace. + - False, CRDNotInstalled: the cluster serves no CommitRequest CRD. + - False, VersionNotServed: the CRD does not serve v1alpha3. + - False, Forbidden: the credential may not list or watch them. + - Unknown, Unreachable: the cluster's API could not be read. items: description: Condition contains details for one aspect of the current state of this API Resource. diff --git a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml b/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml deleted file mode 100644 index be2bdf6e..00000000 --- a/config/crd/bases/source.configbutler.ai_remotesaverequests.yaml +++ /dev/null @@ -1,299 +0,0 @@ ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.23.0 - name: remotesaverequests.source.configbutler.ai -spec: - group: source.configbutler.ai - names: - kind: RemoteSaveRequest - listKind: RemoteSaveRequestList - plural: remotesaverequests - singular: remotesaverequest - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.conditions[?(@.type=="Ready")].status - name: Ready - type: string - - jsonPath: .status.conditions[?(@.type=="Ready")].reason - name: Reason - type: string - - jsonPath: .spec.gitTargetRef.name - name: GitTarget - type: string - - jsonPath: .status.author - name: Author - type: string - - jsonPath: .status.commit - name: Commit - priority: 1 - type: string - - jsonPath: .status.conditions[?(@.type=="Applied")].status - name: Applied - priority: 1 - type: string - - jsonPath: .metadata.creationTimestamp - name: Age - type: date - name: v1alpha1 - schema: - openAPIV3Schema: - description: |- - RemoteSaveRequest asks GitOps Reverser to save its creator's edits in the - GitTarget spec.gitTargetRef names: the creator's commit window there gets - its message and closes on its timers. The window holds the creator's edits - anywhere that GitTarget mirrors. Create it on the cluster the edits were - made on, as JSON. Its creator is found from this cluster's audit events, as - the edits' authors are, so it saves only its creator's edits; on a cluster - with no audit it names nobody, and saves edits that name nobody. Its spec is - immutable, and status reports the commit. - properties: - apiVersion: - description: |- - APIVersion defines the versioned schema of this representation of an object. - Servers should convert recognized schemas to the latest internal value, and - may reject unrecognized values. - More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources - type: string - kind: - description: |- - Kind is a string value representing the REST resource this object represents. - Servers may infer this from the endpoint the client submits requests to. - Cannot be updated. - In CamelCase. - More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds - type: string - metadata: - type: object - spec: - description: |- - spec defines the desired state of RemoteSaveRequest. It is immutable after - creation. - properties: - catchUp: - default: AsDefinedOnClaim - description: |- - CatchUp decides when the GitTarget's applier is asked to apply the - branch after this request pushes a commit. "AsDefinedOnClaim" leaves it - to the applier claim's spec.catchUpAfter. "AfterPush" asks right after - the push, and the Applied condition reports how it went. The cluster - already holds the saved objects. Defaults to "AsDefinedOnClaim". - enum: - - AsDefinedOnClaim - - AfterPush - type: string - gitTargetRef: - description: |- - GitTargetRef names the GitTarget whose commit window this request - attaches to, on the cluster GitOps Reverser runs in. Its namespace is a - namespace of that cluster, not of this one. The GitTarget must mirror - this cluster, and admit this request in spec.allowedRemoteSaveRequests, - or the request fails with reason "NotAllowed". - properties: - name: - description: Name is the GitTarget's name. - maxLength: 253 - minLength: 1 - type: string - namespace: - description: |- - Namespace is the GitTarget's namespace, on the cluster GitOps Reverser - runs in. - maxLength: 63 - minLength: 1 - type: string - required: - - name - - namespace - type: object - message: - description: |- - Message is the commit message this request asks for. Omitted, the - GitTarget's spec.commit.message.liveTemplate words the commit. - - The text is never evaluated as a template. If the GitTarget sets - spec.commit.message.requestTemplate, that template wraps this text; - otherwise the text is committed as written. - - It is 1–1024 characters and must contain a non-whitespace character. - Newlines are allowed, for a subject and body; tabs and other control - characters are rejected. - maxLength: 1024 - minLength: 1 - pattern: ^[^\x00-\x09\x0B-\x1F\x7F]*$ - type: string - x-kubernetes-validations: - - message: message must contain a non-whitespace character - rule: self.matches(r'[^\s\x{0085}\x{00A0}\x{1680}\x{2000}-\x{200A}\x{2028}\x{2029}\x{202F}\x{205F}\x{3000}]') - window: - default: - attachTimeout: 5s - maxDuration: 2s - description: |- - Window says how long the request waits for a commit window with the - edits it saves, and when that window closes once it has. Omitted, the - request waits 5s and closes the window 2s after it attached. - properties: - attachTimeout: - default: 5s - description: |- - AttachTimeout is how long the request waits for a commit window, as a - Go duration, from when GitOps Reverser picks the request up, which is - after its own author was found. A request whose wait ran out never - takes a later window. At most "5m". Defaults to "5s". - pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ - type: string - x-kubernetes-validations: - - message: spec.window.attachTimeout must not exceed 5m - rule: duration(self) <= duration('5m') - idleTimeout: - description: |- - IdleTimeout closes the attached window this long after its last edit, - as a Go duration. Omitted, only maxDuration closes it. At most "5m", - and not more than maxDuration. - pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ - type: string - x-kubernetes-validations: - - message: spec.window.idleTimeout must not exceed 5m - rule: duration(self) <= duration('5m') - maxDuration: - default: 2s - description: |- - MaxDuration closes the attached window this long after the request - attached, however many edits keep arriving, as a Go duration. At most - "5m". Defaults to "2s". - pattern: ^([0-9]+(\.[0-9]+)?(ns|us|µs|μs|ms|s|m|h))+$ - type: string - x-kubernetes-validations: - - message: spec.window.maxDuration must not exceed 5m - rule: duration(self) <= duration('5m') - type: object - x-kubernetes-validations: - - message: RemoteSaveRequest spec is immutable after creation - rule: duration(self.attachTimeout) == duration(oldSelf.attachTimeout) - && duration(self.maxDuration) == duration(oldSelf.maxDuration) - && has(self.idleTimeout) == has(oldSelf.idleTimeout) && (!has(self.idleTimeout) - || duration(self.idleTimeout) == duration(oldSelf.idleTimeout)) - - message: spec.window.idleTimeout must not exceed maxDuration - rule: '!has(self.idleTimeout) || duration(self.idleTimeout) <= duration(self.maxDuration)' - required: - - gitTargetRef - type: object - x-kubernetes-validations: - - message: RemoteSaveRequest spec is immutable after creation - rule: self.gitTargetRef == oldSelf.gitTargetRef && has(self.message) - == has(oldSelf.message) && (!has(self.message) || self.message == - oldSelf.message) && has(self.window) == has(oldSelf.window) && has(self.catchUp) - == has(oldSelf.catchUp) && (!has(self.catchUp) || self.catchUp == - oldSelf.catchUp) - status: - description: status defines the observed state of RemoteSaveRequest. - properties: - appliedRevision: - description: |- - AppliedRevision is the revision the applier applied, with spec.catchUp - "AfterPush". It holds the commit, and may be a later one. - type: string - author: - description: |- - Author is who created this request, as this cluster's audit events name - them. Empty when they name nobody: then the request saves only edits - that name nobody either. - type: string - branch: - description: Branch is the branch the save was committed on. - type: string - commit: - description: Commit is the commit the save made, once it is on the - remote. - type: string - conditions: - description: |- - Conditions are the latest observations of this RemoteSaveRequest's state. - - Ready is True once the request ended without error: reason "Committed", - or, with nothing committed, "NoWindow", "AlreadyPresent", - "WindowMismatch", or "AuthorUnresolved" when this cluster sends audit - events and none named the request's creator. Ready=False with - Stalled=True is a failure: "NotAllowed" when the GitTarget does not - admit the request or mirrors another cluster, "GitTargetNotFound", - "SaveInProgress" when an earlier save of the same author on the - GitTarget was still in progress, or "Failed". - - Applied, with spec.catchUp "AfterPush", is True once the applier applied - a revision that holds the commit, or needed none, and False only when - the branch lost the commit. It never changes Ready. - items: - description: Condition contains details for one aspect of the current - state of this API Resource. - properties: - lastTransitionTime: - description: |- - lastTransitionTime is the last time the condition transitioned from one status to another. - This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: |- - message is a human readable message indicating details about the transition. - This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: |- - observedGeneration represents the .metadata.generation that the condition was set based upon. - For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date - with respect to the current state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: |- - reason contains a programmatic identifier indicating the reason for the condition's last transition. - Producers of specific condition types may define expected values and meanings for this field, - and whether the values are considered a guaranteed API. - The value should be a CamelCase string. - This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - x-kubernetes-list-map-keys: - - type - x-kubernetes-list-type: map - observedGeneration: - description: ObservedGeneration is the latest generation observed - by GitOps Reverser. - format: int64 - type: integer - type: object - required: - - spec - type: object - served: true - storage: true - subresources: - status: {} diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index d805d968..330ed986 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -11,4 +11,3 @@ resources: - bases/configbutler.ai_kubeproviders.yaml - bases/configbutler.ai_commitrequests.yaml - bases/configbutler.ai_watchrules.yaml - - bases/source.configbutler.ai_remotesaverequests.yaml diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 2e4f0972..5baf7a21 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -115,19 +115,3 @@ rules: - get - list - watch -- apiGroups: - - source.configbutler.ai - resources: - - remotesaverequests - verbs: - - delete - - get - - list - - watch -- apiGroups: - - source.configbutler.ai - resources: - - remotesaverequests/status - verbs: - - get - - update diff --git a/docs/configuration.md b/docs/configuration.md index 7b6ab3ea..4b2b2e5d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -105,7 +105,7 @@ Set exactly one of `secretRef` and `inCluster`. Both are immutable: delete and r | `client.qps` | `20` | Client rate limit for this source cluster. Refused beside `inCluster` | | `client.burst` | `30` | Client burst for this source cluster. Refused beside `inCluster` | | `attribution.auditRoute` | `.` | Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) | -| `remoteSaveRequests` | `Disabled` | `Enabled` serves the `RemoteSaveRequest`s created on this cluster, which writes to it. See [`RemoteSaveRequest`](#remotesaverequest) | +| `commitRequests` | `Ignore` | Whether the `CommitRequest`s created on this cluster are served, and finished ones deleted: `Ignore`, `Reconcile`, `ReconcileAndDeleteFinished`. See [serving them](#serving-the-commitrequests-of-a-source-cluster) | ### `GitTarget` (namespaced): which branch and folder @@ -119,8 +119,7 @@ The destination fields are immutable: to move a target, delete it and create a n | `commit.window.idleTimeout` | `5s` | Close a commit window after this much silence. `0s` commits every write on its own. See [the commit window](#the-commit-window-speccommitwindow) | | `commit.window.maxDuration` | `1m` | Close a commit window this long after it opened, however much keeps arriving | | `commit.message` | the built-in templates | How commits are phrased. See [commit messages](commit-messages.md) | -| `allowedCommitRequests` | `[{"consumerNamespaces":{"from":"Same"}}]` | The namespaces whose `CommitRequest`s may save this target's windows. See [who may save](#who-may-save-specallowedcommitrequests) | -| `allowedRemoteSaveRequests` | none admitted | The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and save its windows. See [`RemoteSaveRequest`](#remotesaverequest) | +| `allowedCommitRequests` | `[{"consumerNamespaces":{"from":"Same"}}]` | The `CommitRequest`s that may save this target's windows: `consumerNamespaces` on the cluster GitOps Reverser runs in, `sourceNamespaces` on the one it mirrors. See [who may save](#who-may-save-specallowedcommitrequests) | | `placement.byType` | the built-in path | Per-type path templates for new documents. See [where new resources are written](#where-new-resources-are-written-specplacement) | | `placement.default` | the built-in path | Catch-all path template for types with no `byType` entry | | `placement.useKustomize` | off | Keep the folder a kustomize folder. See [keeping the folder a kustomize folder](#keeping-the-folder-a-kustomize-folder-specplacementusekustomize) | @@ -174,11 +173,11 @@ The references are immutable: to hold another `Application`, delete the applier | `applicationRef` | **required** | The `Application`, with automated sync off, that syncs the folders of the `GitTarget`s naming this applier claim. See [who applies this folder](#who-applies-this-folder-specapplierclaimref) | | `catchUpAfter` | `0s` | As a `FluxClaim`'s: how long the `Application`'s last sync stays behind the branch after a push of GitOps Reverser's own before a sync of the tip is asked for. Off by default, since a sync runs the `Application`'s hooks and Argo CD's refresh already compares the live state against the tip. Otherwise `10s` to `24h` | -### `CommitRequest` (namespaced): save now +### `CommitRequest` (namespaced, here or on a source cluster): save now | Field | Default | What it does | |---|---|---| -| `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedCommitRequests` must admit this one | +| `gitTargetRef` | **required** | The target whose commit window the request attaches to: in this namespace, or `namespace` names another, whose `allowedCommitRequests` must admit this one. On a source cluster `namespace` is required. See [where a request is created](#where-a-request-is-created) | | `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | | `window.attach` | `CurrentOrNext` | Which window to attach to: the author's current or next one, or only the next. See [the request window](#the-request-window-specwindow) | | `window.attachTimeout` | `5s` | How long to wait for a window to attach to | @@ -187,17 +186,6 @@ The references are immutable: to hold another `Application`, delete the applier | `whenNothingToCommit` | `Resolve` | `CommitEmpty` records the message in an empty commit when nothing changed | | `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it | -### `RemoteSaveRequest` (namespaced, on a source cluster): save the edits made here - -| Field | Default | What it does | -|---|---|---| -| `gitTargetRef` | **required** | The `GitTarget` to save, by `namespace` and `name` on the cluster GitOps Reverser runs in. Its `allowedRemoteSaveRequests` must admit this request | -| `message` | the target's templates | Commit message, committed verbatim unless `requestTemplate` frames it | -| `window.attachTimeout` | `5s` | How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) | -| `window.idleTimeout` | no idle close | Close the attached window after this much silence | -| `window.maxDuration` | `2s` | Close the attached window this long after the attach | -| `catchUp` | `AsDefinedOnClaim` | `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit is pushed; the `Applied` condition reports it | - ## Sharing connections across namespaces @@ -2968,6 +2956,11 @@ Use it sparingly. rules. It attaches to its author's commit window, gives that window its message, and decides when the window closes, instead of the target's timers. +It is the one kind of GitOps Reverser that can also live on a source cluster. A request for a +`GitTarget` that mirrors the cluster GitOps Reverser runs in is created there; a request for a +`GitTarget` that mirrors another cluster is created on that cluster, next to the edits it saves. See +[where a request is created](#where-a-request-is-created). + The important fields are: - `spec.gitTargetRef`: the target whose window to attach to, by `name`, in the request's own @@ -2977,6 +2970,8 @@ The important fields are: [the request window](#the-request-window-specwindow) - `spec.whenNothingToCommit`: what to do when the request ends with nothing to commit; see [when there is nothing to commit](#when-there-is-nothing-to-commit) +- `spec.catchUp`: whether the applier is asked to apply the save at once; see + [applying a save at once](#applying-a-save-at-once) Example: @@ -2995,7 +2990,7 @@ spec: Route traffic to the port exposed by the API container. window: attach: CurrentOrNext - attachTimeout: "2s" + attachTimeout: "5s" maxDuration: "2s" ``` @@ -3009,6 +3004,30 @@ configure [`requestTemplate`](commit-messages.md#framing-a-save-message) on the parsed as a template. A rejected request leaves automatic mirroring available. The submitter chooses any semantic prefix; free-form messages are accepted. +### Where a request is created + +The `GitTarget` a request names decides where it belongs, through the `KubeProvider` its +`spec.kubeProviderRef` names: + +| The `GitTarget`'s `KubeProvider` | Create the request on | `gitTargetRef.namespace` | +| --- | --- | --- | +| `inCluster` | the cluster GitOps Reverser runs in | optional: omitted, the request's own namespace | +| a kubeconfig, with `spec.commitRequests` set to a `Reconcile` value | the cluster that kubeconfig connects to | required: a namespace of the cluster GitOps Reverser runs in | + +- **A request on a source cluster must name the `GitTarget`'s namespace.** That namespace belongs to + the cluster GitOps Reverser runs in, so there is nothing on the source cluster to default to. A + request without it is never picked up there, and gets no status. +- **A request on the cluster GitOps Reverser runs in, for a `GitTarget` that mirrors another + cluster, is refused** with reason `NotAllowed`, when that `KubeProvider` serves no + `CommitRequest`s: its author would come from this cluster's audit and its edits' authors from the + other one's. Create it on the other cluster instead. When that `KubeProvider` does serve them, the + request is left alone, since the kubeconfig may point back at this cluster and then its own watch + serves the request. If it points elsewhere, nothing serves the request and it gets no status. + +Only the path the `GitTarget`'s `KubeProvider` picks writes a request's progress, so several +`KubeProvider`s that watch one cluster never fight over a request. Any of them may report +`GitTargetNotFound` for a request whose `GitTarget` does not exist once `attachTimeout` has passed. + ### The request window (`spec.window`) A request never opens a window; only writes do. It waits for one, attaches to at most one, and @@ -3017,8 +3036,7 @@ replaces that window's timers with its own, shorter or longer: | Field | Meaning | Default | |---|---|---| | `attach` | `CurrentOrNext` attaches to the author's window if one is open when the worker registers the request, and otherwise to the next one a write opens. `Next` first closes the author's open window, under the message it already carries, and attaches to the next one | `CurrentOrNext` | -| `author` | `Own` attaches only to the author's window. `SavedBy` and `TakeOver` may also claim a window whose writes name no author; see [saving where writes name no author](#saving-where-writes-name-no-author) | `Own` | -| `attachTimeout` | stop waiting for a window this long after the worker registers the request | `2s` | +| `attachTimeout` | stop waiting for a window this long after the worker registers the request | `5s` | | `idleTimeout` | close the attached window after this much silence; every write restarts it | no idle close | | `maxDuration` | close the attached window this long after the attach, however much keeps arriving | `2s` | @@ -3047,12 +3065,12 @@ and restarts nothing: its deadlines and, for `Next`, its one close are fixed at #### Sizing the request window -`attachTimeout` covers the gap between creating the request and its first write reaching the +`attachTimeout` counts from when GitOps Reverser picks the request up, which is after the request's +own audit event arrived. It covers the gap between that and the request's first write reaching the branch worker. A write is held in the watch path until its audit fact arrives, so the floor is the -API server's `--audit-webhook-batch-max-wait` plus the attribution join (roughly 1 to 1.5 seconds at -the reference configuration of `1s`). The default of `"2s"` clears that; a loaded or distant cluster -may want `"4s"` to `"5s"`, and a save created before its writes needs the time to its first write -on top. +spread between the request's audit event and its writes'. The default of `"5s"` clears that on a +loaded or distant cluster; a save created before its writes needs the time to its first write on +top. The floor assumes the stream is not busy. A watch stream attributes its events in order, so a save's edit can also queue behind an earlier event of the same type that is still waiting for its fact, for @@ -3064,22 +3082,40 @@ Widening `attachTimeout` hides that delay rather than removing it. attach. The worst case from registration to the finalize is `attachTimeout` plus `maxDuration`; the push follows, with its cooldown and any retries. -Do not size either against `--author-attribution-grace`. When that grace expires with no fact the -write still ships, as a window that names no actor, and a request naming a submitter can never -attach to it: the outcome is `WindowMismatch` no matter how long the request waits. On a cluster -that sends no audit events at all, see -[saving where writes name no author](#saving-where-writes-name-no-author). +The two directions are not symmetric. Overshooting costs a few seconds of latency on the commit; +undershooting `attachTimeout` resolves the request `Ready=True` with reason `NoWindow` while the +edit commits seconds later under the target's `liveTemplate`, which reads as a save button that did +nothing. + +A request cannot rename a finalized commit, including a local commit waiting for push. Between the +finalize and the push the request is still in flight, and GitOps Reverser keeps re-sending its +attach until it reads an outcome; the worker recognizes the re-send as the same request, so a +request never resolves `NoWindow` while its commit waits out the push cooldown. Applying resources +and a request together gives no ordering guarantee between controllers and watch streams. See the +[request contract](spec/commitrequest-design.md). ### Who may save: `spec.allowedCommitRequests` -A `GitTarget` says which namespaces' `CommitRequest`s may save its windows in -`spec.allowedCommitRequests`, a grant list like the others (see [the grants](#the-grants)): +A `GitTarget` says which `CommitRequest`s may save its windows in `spec.allowedCommitRequests`, a +grant list like the others (see [the grants](#the-grants)). Each grant sets exactly one of two +fields, one for each place a request can be created: + +- **`consumerNamespaces`** admits requests on the cluster GitOps Reverser runs in, by namespace, + with `from: Same | Names | Selector | All`, as in every other grant list. It admits requests only + for a `GitTarget` whose `KubeProvider` sets `inCluster`. +- **`sourceNamespaces`** admits requests on the cluster the `GitTarget` mirrors, by exact namespace + name, or `"*"` for every namespace the `GitTarget` may mirror. Variables are refused. A namespace + the `KubeProvider` does not lend this `GitTarget` in `spec.allowedGitTargets` is never admitted, + the same bound its `WatchRule`s have. It admits requests only for a `GitTarget` whose + `KubeProvider` names a kubeconfig. + +The list follows the pattern of the other grant lists: - **Omitted:** the `CommitRequest`s in the `GitTarget`'s own namespace, as one grant with - `from: Same` that claims nothing of a window whose writes name no author. + `consumerNamespaces: {from: Same}`. - **`[]`:** no `CommitRequest`. -- **A list:** the `CommitRequest`s at least one grant admits. The list replaces the default, so the - own namespace keeps access only if a grant admits it. +- **A list:** the `CommitRequest`s at least one grant admits. The list replaces the default, so a + `GitTarget` that mirrors another cluster writes a `sourceNamespaces` grant. ```yaml apiVersion: configbutler.ai/v1alpha3 @@ -3095,55 +3131,48 @@ spec: selector: matchLabels: platform.example.com/tenant: "true" - unattributedWindows: SavedBy ``` -A request no grant admits fails when it is first seen, before it attaches: `Ready=False`, -`Stalled=True`, reason `NotAllowed`. Each grant also sets `unattributedWindows` for the requests it -admits, described below; a request admitted by several grants may claim the most any of them allows. - -### Saving where writes name no author - -On a source cluster that sends no audit events, no write names its author, so every write lands in -a window that names no actor. A request whose submitter was captured at admission cannot attach to -such a window, and its save ends `WindowMismatch`. Two settings let it claim one, and both default to -claiming nothing: +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: shop + namespace: team-shop +spec: + kubeProviderRef: {name: shop-workspace} + allowedCommitRequests: + - sourceNamespaces: ["*"] +``` -- the `unattributedWindows` of the grant in the `GitTarget`'s `spec.allowedCommitRequests` that admits - the request's namespace says how much it may claim: `Keep` (the default), `SavedBy`, or `TakeOver`; -- the request's `spec.window.author` says what this save asks for: `Own` (the default), `SavedBy`, - or `TakeOver`. +A request no grant admits fails before it attaches: `Ready=False`, `Stalled=True`, reason +`NotAllowed`. The message says which field applies on the cluster the request was made on. -| `spec.window.author` | What the request does with a window whose writes name no author | -|---|---| -| `Own` | nothing: it attaches only to its submitter's own window | -| `SavedBy` | gives the window its message; the commit names no author, and its message ends with `Saved-By: ` | -| `TakeOver` | takes the window over; the submitter authors the commit, and its message ends with `Author-Source: CommitRequest /; no audit record names who made these changes` | +### Who the save is from -A grant's `TakeOver` allows both claims, and `SavedBy` allows only `SavedBy`. A request that asks for -more than the grants admitting it allow fails when it is first seen, before it attaches: -`Ready=False`, `Stalled=True`, reason `TakeOverNotAllowed`. +A request's author is found from its own audit event, on the cluster it was created on, the same +way an edit's author is: from the same index, with the same `--author-attribution-grace`. So a save +attaches only to its creator's windows, by the ordinary author rule, and `status.author` names the +creator. The author is recorded in the request's first status write, before it attaches. Create a +request on a source cluster as JSON: `kubectl` and browsers send JSON, and an API server that drops +the audit event of a YAML body loses the save's author without an error. -Both claims act only while the `GitTarget`'s `KubeProvider` has never received an audit fact: its -`AuditFactsReceived` condition is not `True`. On a cluster that has sent one, a write that names no -author means its fact went missing, so no request claims it, and a request that asked attaches to -its own windows only. +| The cluster the request was made on | The save | +| --- | --- | +| sends audit events, and one names the creator | attaches to the creator's windows; `status.author` names them | +| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window whose edits name nobody; the commit's author stays unresolved | +| sends audit events, but none names the creator | is refused, `Ready=True` with reason `AuthorUnresolved`: its event went missing, and attaching it would claim the window that should stay unattributed. Save again | +| any, with author attribution off | names nobody, and attaches to the `GitTarget`'s one unnamed window; the committer authors the commit | -Every write that names no author shares one window, whoever made it. A takeover therefore names the -submitter as the author of all of them, and `SavedBy` gives all of them the submitter's message. -Allow `TakeOver` only where one person edits the `GitTarget`'s namespaces at a time. +Who may save is decided by `allowedCommitRequests`; who authored the commit is decided by audit, or +by nobody. -The two directions are not symmetric. Overshooting costs a few seconds of latency on the commit; -undershooting `attachTimeout` resolves the request `Ready=True` with reason `NoWindow` while the -edit commits seconds later under the target's `liveTemplate`, which reads as a save button that did -nothing. +### What a save holds -A request cannot rename a finalized commit, including a local commit waiting for push. Between the -finalize and the push the request is still in flight, and the controller keeps re-sending its -attach until it reads an outcome; the worker recognizes the re-send as the same request, so a -request never resolves `NoWindow` while its commit waits out the push cooldown. Applying resources -and a request together gives no ordering guarantee between controllers and watch streams. See the -[request contract](spec/commitrequest-design.md). +A commit window holds one author's edits to one `GitTarget`, wherever that `GitTarget` mirrors them. +An admitted request therefore saves its creator's whole window, edits outside the request's own +namespace included: a grant in `allowedCommitRequests` authorizes exactly that. The request's +namespace decides only whether a grant admits it. ### When there is nothing to commit @@ -3174,26 +3203,32 @@ author's window never records one, because the history would then claim more tha A crash between the push and the status write can record the message twice: the worker recognizes a repeated request only while it runs. +### Its status + Progress and outcome are reported through kstatus-compatible **conditions** (no `phase` string). -`kubectl get commitrequest` shows `Ready`, its `Reason` and the `Commit`; `-o wide` adds -`AuthorAttributed` and `Pushed`. Automation must stop on either `Ready=True` or `Stalled=True`; +`kubectl get commitrequest` shows `Ready`, its `Reason`, the `Author` and the `Commit`; `-o wide` +adds `Pushed` and `Applied`. Automation must stop on either `Ready=True` or `Stalled=True`; `kubectl wait --for=condition=Ready` alone keeps waiting on terminal failures. `Ready=True` includes successful no-commit outcomes. Require `Pushed=True` and `status.commit` for evidence that the request produced a pushed commit: -- **Ready** (summary): `True` once the request reached a non-error terminal outcome. The `Ready` - condition's `reason` says which: `Committed` (a commit was pushed; `status.branch`/`status.commit` set), - or an outcome with nothing to commit: `NoWindow`, `WindowMismatch`, or `AlreadyPresent`. With - [`CommitEmpty`](#when-there-is-nothing-to-commit), `NoWindow` and `AlreadyPresent` can come with a - recorded empty commit. A failed finalize is `Ready=False` with reason `FinalizeFailed`. - - `Ready=False` with reason `SaveInProgress` means an earlier request of the same author on the same - `GitTarget` was still waiting for a commit window, or collecting one, when this one arrived. A - window carries one message, so the two would race for it and one message would be lost without a - word; the later request is refused at once instead. Save again with a new request once the earlier - one is `Ready`. A request for the `Next` window is not refused by one that is collecting: it closes - that window under the earlier request's message first. Saves by different authors never race, - since each author has windows of its own. +- **Ready** (summary): `Unknown` while the save runs, from the first status write, which records the + author. `True` once the request reached a non-error terminal outcome. The `Ready` condition's + `reason` says which: `Committed` (a commit was pushed; `status.branch`/`status.commit` set), or an + outcome with nothing to commit: `NoWindow`, `WindowMismatch`, `AlreadyPresent`, or + `AuthorUnresolved`. With [`CommitEmpty`](#when-there-is-nothing-to-commit), `NoWindow` and + `AlreadyPresent` can come with a recorded empty commit. `Ready=False` with `Stalled=True` is a + failure: `NotAllowed` when the `GitTarget` does not admit the request, or the request was made on + the wrong cluster; `GitTargetNotFound` when the `GitTarget` did not exist within `attachTimeout`; + `SaveInProgress`; or `FinalizeFailed`. + + `SaveInProgress` means an earlier request of the same author on the same `GitTarget` was still + waiting for a commit window, or collecting one, when this one arrived. A window carries one + message, so the two would race for it and one message would be lost without a word; the later + request is refused at once instead. Save again with a new request once the earlier one is `Ready`. + A request for the `Next` window is not refused by one that is collecting: it closes that window + under the earlier request's message first. Saves by different authors never race, since each + author has windows of its own. `AlreadyPresent` means **the remote confirmed there was nothing to add**, so it is reported after the push rather than when the local plan found no difference. That costs a few seconds and buys a @@ -3201,21 +3236,18 @@ request produced a pushed commit: the worker replays onto the moved branch, and such a request resolves `Committed` instead. A worker that stops before the push fails the request rather than leaving it to time out. - **Reconciling** / **Stalled**: the kstatus progress/blocked pair. `Reconciling=True` while the - request is in progress, with the phase the branch worker reports as its reason: `Progressing` - (sent, not yet registered), `WaitingForWorker` (the GitTarget has no branch worker yet), + request is in progress, with the phase the branch worker reports as its reason: `Saving` + (claimed, not yet registered), `WaitingForWorker` (the GitTarget has no branch worker yet), `WaitingForWindow`, `CollectingWindow`, or `WaitingForPush`. - `Stalled=True` when the finalize failed and needs attention (kstatus reports the object Failed). -- **AuthorAttributed**: `True` with reason `AttributedFromAdmission` when the internal commands - admission webhook captured the request submitter. `False` with reason `CommitterFallback` means capture - ran but no admission record exists; `False` with reason `AuthorCaptureDisabled` means capture is not - configured. Neither is a failure. The request then claims no actor and can attach only to an unnamed - watch window, whose Git author remains either the configured committer or the explicit unresolved author - according to the watch attribution outcome. + `Stalled=True` when the request failed and needs attention (kstatus reports the object Failed). - **Pushed**: `True` once the commit, empty or not, is in the remote repository. - **Applied**: only with [`catchUp: AfterPush`](#applying-a-save-at-once), once `Ready=True`: whether the applier caught up with the commit. It never changes `Ready`, so `Ready=True` alone does not mean the commit was applied. +A request is served again after GitOps Reverser restarts: the path that owns it finds its own +claim, and resumes with the author it recorded. + ### Applying a save at once A save is in Git once it is pushed, and the cluster already runs it, so by default nothing applies @@ -3241,7 +3273,7 @@ deleted only after its `Applied` condition is settled. ### Finished requests are deleted after 48 hours Every save leaves one `CommitRequest` behind. When a request finishes (`Ready=True` or -`Stalled=True`), the controller writes the time it will be deleted onto it: +`Stalled=True`), GitOps Reverser writes the time it will be deleted onto it: ```yaml metadata: @@ -3249,13 +3281,13 @@ metadata: configbutler.ai/delete-after: "2026-10-01T09:30:00Z" ``` -Once that time has passed, the controller deletes the request. The annotation is the whole -contract, so it is also how you change the outcome for one request: +Once that time has passed, the request is deleted. The annotation is the whole contract, so it is +also how you change the outcome for one request: - Remove the annotation to keep the request. - Edit the time, in RFC 3339 format, to delete it earlier or later. A value that does not parse keeps the request. -- Set it yourself when you create the request to choose the time up front. The controller never +- Set it yourself when you create the request to choose the time up front. GitOps Reverser never replaces a value that is already there. The period comes from `--commit-request-ttl` (Helm: `controllerManager.commitRequestTTL`, default @@ -3263,122 +3295,68 @@ The period comes from `--commit-request-ttl` (Helm: `controllerManager.commitReq requests are kept. A request that finished before this feature existed has no annotation and is never deleted. -## `RemoteSaveRequest` +On the cluster GitOps Reverser runs in this always applies. On a source cluster it applies only +when the `KubeProvider` sets `spec.commitRequests: ReconcileAndDeleteFinished`; with `Reconcile`, +nothing is stamped and nothing is deleted, and annotations stamped before a switch back to +`Reconcile` stay without effect. -A `RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`) is a save made on a **source cluster**, next -to the edits it saves, by someone who may have no access to the cluster GitOps Reverser runs in. It -names one `GitTarget` on the cluster GitOps Reverser runs in, and saves its creator's commit window -there: the window gets its message and closes on its timers, as with a [`CommitRequest`](#commitrequest). - -```yaml -apiVersion: source.configbutler.ai/v1alpha1 -kind: RemoteSaveRequest -metadata: - name: save-cart - namespace: shop -spec: - gitTargetRef: - namespace: platform - name: shop - message: "fix(shop): raise the cart limit" - catchUp: AfterPush -``` - -`spec.gitTargetRef` is required, and both its fields are: `namespace` is a namespace of the cluster -GitOps Reverser runs in, not of the source cluster. The people creating requests need no access to -`GitTarget`s; their tooling supplies the address. - -### Serving the `RemoteSaveRequest`s of a cluster +### Serving the `CommitRequest`s of a source cluster Three things turn it on, each on purpose: -1. **Install the CRD on the source cluster.** It is one small CRD, - `config/crd/bases/source.configbutler.ai_remotesaverequests.yaml`, apart from the - `configbutler.ai` kinds of the cluster GitOps Reverser runs in. The Helm chart and the installer - install it there too, for the clusters the operator mirrors with its own identity. -2. **Set `spec.remoteSaveRequests: Enabled` on the `KubeProvider`.** GitOps Reverser then watches the - cluster's `RemoteSaveRequest`s through that provider, and writes each one's status back. It is the - one write GitOps Reverser makes on a source cluster. A `RemoteSaveRequest` is never mirrored, - whatever a `WatchRule` selects. -3. **Grant it on the `GitTarget`, in `spec.allowedRemoteSaveRequests`.** The provider switch makes - the requests reachable; the `GitTarget`'s owner decides which of them may save its windows. An - omitted or empty list admits none. The `GitTarget` must also mirror the cluster the request was - made on: its `spec.kubeProviderRef` names the `KubeProvider` that served the request. - -What each value of `spec.remoteSaveRequests` does on the source cluster, and what the provider's -credential needs there, in the `source.configbutler.ai` group: +1. **Install the CRD on the source cluster.** It is one CRD, the release asset + `commitrequest-crd.yaml`, apart from the other `configbutler.ai` kinds, which a source cluster + does not need. Install it with `kubectl apply --server-side -f commitrequest-crd.yaml`. GitOps + Reverser never creates or updates it there. +2. **Set `spec.commitRequests` on the `KubeProvider`.** GitOps Reverser then watches the cluster's + `CommitRequest`s through that provider, in the namespaces it lends its `GitTarget`s in + `spec.allowedGitTargets` (one watch across the cluster when a grant lends `"*"`), and writes each + one's status back through the same credential. A `CommitRequest` is never mirrored, whatever a + `WatchRule` selects. +3. **Grant it on the `GitTarget`, in `spec.allowedCommitRequests`,** with `sourceNamespaces`. The + provider switch makes the requests reachable; the `GitTarget`'s owner decides which of them may + save its windows. + +Each value of `spec.commitRequests` makes the cluster a bigger source of `CommitRequest`s, and needs +more of the credential on `commitrequests` in the `configbutler.ai` group, in each lent namespace: | Value | What GitOps Reverser does | The credential needs | | --- | --- | --- | -| `Disabled` (default) | nothing | nothing | -| `Enabled` | serves the requests and writes their status, and deletes a finished request `--commit-request-ttl` after it finished (`0` keeps it) | `get`, `list`, `watch` and `delete` on `remotesaverequests`; `update` on `remotesaverequests/status` | +| `Ignore` (default) | nothing | nothing | +| `Reconcile` | serves the requests in the namespaces this `KubeProvider` lends its `GitTarget`s, and writes their status. A finished request stays, and no `delete-after` is stamped on it, because nothing would act on it | `get`, `list`, `watch`; `update` on `commitrequests/status` | +| `ReconcileAndDeleteFinished` | as `Reconcile`, and also stamps `configbutler.ai/delete-after` when a request finishes, and deletes the request once that time has passed | as `Reconcile`, plus `patch` and `delete` | + +A `KubeProvider` with `inCluster` takes only `Ignore`: the `CommitRequest`s of the cluster GitOps +Reverser runs in are always served, with the operator's own identity. ```yaml apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget +kind: KubeProvider +metadata: + name: shop-workspace + namespace: team-shop spec: - kubeProviderRef: {name: workspace} - allowedRemoteSaveRequests: - - sourceNamespaces: [shop] - requestSelector: - matchLabels: - app.kubernetes.io/part-of: shop + secretRef: {name: shop-workspace} + commitRequests: ReconcileAndDeleteFinished ``` -A request is admitted when one grant admits both its namespace and its labels. `sourceNamespaces` are -namespaces of the cluster `spec.kubeProviderRef` names, by exact name, or `"*"` for every namespace. -`requestSelector` is optional, and narrows the grant to the requests whose labels match. Whoever -creates a request picks its labels, so a selector proves nothing about who made it. - -### What a save holds - -A commit window holds one author's edits to one `GitTarget`, wherever that `GitTarget` mirrors them. -An admitted `RemoteSaveRequest` therefore saves its creator's whole window, edits outside the -request's own namespace included: a grant in `allowedRemoteSaveRequests` authorizes exactly that. The -request's namespace decides only whether a grant admits it. - -As with a `CommitRequest`, a second request of the same author on the same `GitTarget`, made while the -first still waits for a window or collects one, is refused at once with `SaveInProgress`: a window -carries one message, and the two would race for it. +The `KubeProvider` reports how serving goes in its `CommitRequestsServed` condition, which is not +part of `Ready`, and the namespaces it watches in `status.commitRequests.namespaces`. Both are absent +while `spec.commitRequests` is `Ignore`. `kubectl get kubeprovider -o wide` shows the reason in the +`Saves` column: -### Who the save is from - -A `RemoteSaveRequest`'s creator is found from the source cluster's audit events, as the edits' authors -are, so a save attaches only to its creator's windows, by the ordinary author rule. Create it as -JSON: `kubectl` and browsers send JSON, and an API server that drops the audit event of a YAML body -loses the save's author without an error. - -| The source cluster | The save | -| --- | --- | -| sends audit events, and one names the creator | attaches to the creator's windows; `status.author` names them | -| sends no audit events at all (its `KubeProvider` never received one) | names nobody, and attaches to a window whose edits name nobody; the commit's author stays unresolved | -| sends audit events, but none names the creator | is refused, `Ready=True` with reason `AuthorUnresolved`: its event went missing, and attaching it would claim the window that should stay unattributed. Save again | - -`spec.window.attachTimeout` (default `5s`) counts from when GitOps Reverser picks the request up, -which is after the request's own audit event arrived, so it only covers the spread between the -request's event and its edits'. `maxDuration` (default `2s`) and `idleTimeout` close the attached -window as for a `CommitRequest`. - -### Its status - -`status.author`, `status.branch` and `status.commit` say who saved and what was pushed, and, with -`catchUp: AfterPush`, `status.appliedRevision` what the applier applied. - -- **Ready** is `True` once the save ended without error: reason `Committed`, or, with nothing - committed, `NoWindow`, `AlreadyPresent`, `WindowMismatch` or `AuthorUnresolved`. `Ready=False` with - `Stalled=True` is a failure: `NotAllowed` when the `GitTarget` does not admit the request or - mirrors another cluster, `GitTargetNotFound` when it did not exist within `attachTimeout`, - `SaveInProgress`, or `Failed`. -- **Applied**, with `catchUp: AfterPush`, is `True` once the applier applied a revision that holds - the commit, or needed none, `False` only when the branch lost the commit, and `Unknown` with reason - `Lost` when GitOps Reverser restarted before it learned how the apply went. - -A request is served again after GitOps Reverser restarts. It keeps what its status already says: the -author found when it was first served, and an outcome already decided, in which case only its apply, -if it asked for one, is followed. - -A finished `RemoteSaveRequest` is deleted `--commit-request-ttl` (default `48h`) after it finished; -`0` keeps it. +| Status | Reason | When | +| --- | --- | --- | +| `True` | `Watching` | the message says how many namespaces, or "every namespace" | +| `True` | `NothingToWatch` | no `GitTarget` mirroring through this `KubeProvider` is lent a namespace | +| `False` | `CRDNotInstalled` | the cluster serves no `commitrequests.configbutler.ai`; apply `commitrequest-crd.yaml` | +| `False` | `VersionNotServed` | the CRD is there, but does not serve `v1alpha3`; apply `commitrequest-crd.yaml` of this release | +| `False` | `Forbidden` | `list` or `watch` was denied; the message names the namespaces and the verbs to grant | +| `Unknown` | `Unreachable` | the cluster's API could not be read; the message carries the error | + +It is checked on every resync, so a CRD applied by hand clears it without a restart. A finished +request the credential may not delete stays in place; the message says so, and the condition stays +`True`, because serving works. ## Audit ingestion settings diff --git a/docs/rbac.md b/docs/rbac.md index 2daae367..bee52cff 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -86,21 +86,22 @@ evaluated against `Namespace`s in the *source* cluster, so a remote provider's i grant it exactly the namespaces and types the rules name — a `sourceNamespace: "*"` item asks for a cluster-wide `list`/`watch` of its types, and gets a clean 403 if the credential cannot serve it. -### `remotesaverequests` on a source cluster - -A source credential only reads, with one opt-in exception. A `KubeProvider` with -`spec.remoteSaveRequests: Enabled` serves the -[`RemoteSaveRequest`s](configuration.md#remotesaverequest) created on its cluster, and writes their -status back there. Its credential then needs, in the `source.configbutler.ai` group, `get`, `list`, -`watch` and `delete` on `remotesaverequests` (`delete` for the finished-request TTL), and `update` on -`remotesaverequests/status`. Nothing else on a source cluster is written. The manager role carries -the same rules for the cluster the operator runs in, for an `inCluster` provider. - -Whoever may create a `RemoteSaveRequest` in a namespace may save their own commit window on any -`GitTarget` whose `spec.allowedRemoteSaveRequests` admits it. That grant, not the provider's switch, -is the `GitTarget` owner's consent. On the cluster GitOps Reverser runs in, whoever may create a -`CommitRequest` in a namespace may save on every `GitTarget` whose `spec.allowedCommitRequests` admits -that namespace. +### `commitrequests` on a source cluster + +A source credential only reads, with one opt-in exception. A `KubeProvider` whose +`spec.commitRequests` is `Reconcile` serves the +[`CommitRequest`s](configuration.md#serving-the-commitrequests-of-a-source-cluster) created on its +cluster, in the namespaces it lends its `GitTarget`s, and writes their status back there. Its +credential then needs, in the `configbutler.ai` group and in each of those namespaces, `get`, `list` +and `watch` on `commitrequests`, and `update` on `commitrequests/status`. +`ReconcileAndDeleteFinished` also deletes finished requests, so it needs `patch` and `delete` on +`commitrequests` too. Nothing else on a source cluster is written, the CRD included. The manager +role carries the same rules for the cluster the operator runs in. + +Whoever may create a `CommitRequest` in a namespace may save their own commit window on any +`GitTarget` whose `spec.allowedCommitRequests` admits it: by `sourceNamespaces` on a source cluster, +by `consumerNamespaces` on the cluster GitOps Reverser runs in. That grant, not the provider's +switch, is the `GitTarget` owner's consent. ## The operator does not read Secrets wholesale diff --git a/hack/crdfields/fields.yaml b/hack/crdfields/fields.yaml index 35ad4031..eb72b9d5 100644 --- a/hack/crdfields/fields.yaml +++ b/hack/crdfields/fields.yaml @@ -128,10 +128,11 @@ kinds: doc: >- Audit route this provider joins. A route is held by one namespace. See [audit route](#audit-route-and-auditfactsreceived) - - path: remoteSaveRequests + - path: commitRequests doc: >- - `Enabled` serves the `RemoteSaveRequest`s created on this cluster, which writes to it. See - [`RemoteSaveRequest`](#remotesaverequest) + Whether the `CommitRequest`s created on this cluster are served, and finished ones deleted: + `Ignore`, `Reconcile`, `ReconcileAndDeleteFinished`. See + [serving them](#serving-the-commitrequests-of-a-source-cluster) - kind: GitTarget title: "`GitTarget` (namespaced): which branch and folder" @@ -160,13 +161,9 @@ kinds: doc: How commits are phrased. See [commit messages](commit-messages.md) - path: allowedCommitRequests doc: >- - The namespaces whose `CommitRequest`s may save this target's windows. See + The `CommitRequest`s that may save this target's windows: `consumerNamespaces` on the + cluster GitOps Reverser runs in, `sourceNamespaces` on the one it mirrors. See [who may save](#who-may-save-specallowedcommitrequests) - - path: allowedRemoteSaveRequests - default: none admitted - doc: >- - The `RemoteSaveRequest`s, by source namespace and labels, that may name this target and - save its windows. See [`RemoteSaveRequest`](#remotesaverequest) - path: placement.byType default: the built-in path doc: >- @@ -301,12 +298,13 @@ kinds: Otherwise `10s` to `24h` - kind: CommitRequest - title: "`CommitRequest` (namespaced): save now" + title: "`CommitRequest` (namespaced, here or on a source cluster): save now" fields: - path: gitTargetRef doc: >- The target whose commit window the request attaches to: in this namespace, or `namespace` - names another, whose `allowedCommitRequests` must admit this one + names another, whose `allowedCommitRequests` must admit this one. On a source cluster + `namespace` is required. See [where a request is created](#where-a-request-is-created) - path: message default: the target's templates doc: Commit message, committed verbatim unless `requestTemplate` frames it @@ -329,24 +327,3 @@ kinds: pushed, instead of after the applier claim's `catchUpAfter`. The tip may hold later commits, and saves pushed close together can share one apply; the `Applied` condition reports it - - kind: RemoteSaveRequest - title: "`RemoteSaveRequest` (namespaced, on a source cluster): save the edits made here" - fields: - - path: gitTargetRef - doc: >- - The `GitTarget` to save, by `namespace` and `name` on the cluster GitOps Reverser runs in. - Its `allowedRemoteSaveRequests` must admit this request - - path: message - default: the target's templates - doc: Commit message, committed verbatim unless `requestTemplate` frames it - - path: window.attachTimeout - doc: How long to wait for a window with the edits it saves. See [`RemoteSaveRequest`](#remotesaverequest) - - path: window.idleTimeout - default: no idle close - doc: Close the attached window after this much silence - - path: window.maxDuration - doc: Close the attached window this long after the attach - - path: catchUp - doc: >- - `AfterPush` asks the `GitTarget`'s applier to apply the branch tip right after the commit - is pushed; the `Applied` condition reports it diff --git a/internal/authz/commitrequest_admission_test.go b/internal/authz/commitrequest_admission_test.go index 2bd4a1e1..1cd9cc4f 100644 --- a/internal/authz/commitrequest_admission_test.go +++ b/internal/authz/commitrequest_admission_test.go @@ -31,13 +31,13 @@ func TestCommitRequestAdmitted(t *testing.T) { } } bySelector := configv1alpha3.GitTargetCommitRequestGrant{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{ + ConsumerNamespaces: &configv1alpha3.ConsumerNamespaces{ From: configv1alpha3.GrantFromSelector, Selector: &metav1.LabelSelector{MatchLabels: map[string]string{"platform.example.com/tenant": "true"}}, }, } all := configv1alpha3.GitTargetCommitRequestGrant{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, + ConsumerNamespaces: &configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, } reader := fake.NewClientBuilder().WithScheme(admissionScheme(t)).WithObjects(tenant).Build() diff --git a/internal/controller/access_grants_admission_test.go b/internal/controller/access_grants_admission_test.go index cdf85963..7317fe2d 100644 --- a/internal/controller/access_grants_admission_test.go +++ b/internal/controller/access_grants_admission_test.go @@ -120,28 +120,46 @@ var _ = Describe("Access grants at admission", func() { refused(kubeProvider("allowedApplierClaims", consumers(same)), "applierNamespaces: Required value") }) - It("takes a GitTarget's RemoteSaveRequest grants by source namespace name or *, and nothing else", func() { + It("takes a GitTarget's CommitRequest grants with exactly one of consumerNamespaces and sourceNamespaces", func() { gitTarget := func(grants ...any) *unstructured.Unstructured { return object("GitTarget", map[string]any{ "gitBranchRef": map[string]any{"name": "b"}, "path": "apps", - "kubeProviderRef": map[string]any{"name": "k"}, "allowedRemoteSaveRequests": grants, + "kubeProviderRef": map[string]any{"name": "k"}, "allowedCommitRequests": grants, }) } from := func(namespaces ...any) map[string]any { return map[string]any{"sourceNamespaces": namespaces} } accepted(gitTarget(from("shop", "billing"))) accepted(gitTarget(from("*"))) - labelled := from("shop") - labelled["requestSelector"] = map[string]any{"matchLabels": map[string]any{"app": "cart"}} - accepted(gitTarget(labelled)) + accepted(gitTarget(map[string]any{"consumerNamespaces": same}, from("shop"))) refused(gitTarget(from("{consumer.namespace}")), "variables are refused") refused(gitTarget(from("Shop")), "a source namespace is a namespace name") refused(gitTarget(from()), "sourceNamespaces") - refused(gitTarget(map[string]any{}), "sourceNamespaces: Required value") + refused(gitTarget(map[string]any{}), "exactly one of consumerNamespaces and sourceNamespaces") + refused(gitTarget(map[string]any{"consumerNamespaces": map[string]any{}}), "from: Required value") + both := from("shop") + both["consumerNamespaces"] = same + refused(gitTarget(both), "exactly one of consumerNamespaces and sourceNamespaces") }) - // consumerNamespaces is required on every grant type and from is required inside it, with no default - // for either, so a grant that leaves the "who" out admits nobody instead of its holder's namespace. - // The block's own rules cannot make it required, so each list is checked here. + It("takes only Ignore for spec.commitRequests beside spec.inCluster", func() { + inCluster := func(commitRequests string) *unstructured.Unstructured { + return object( + "KubeProvider", + map[string]any{"inCluster": map[string]any{}, "commitRequests": commitRequests}, + ) + } + accepted(inCluster("Ignore")) + refused(inCluster("Reconcile"), "spec.commitRequests must be Ignore beside spec.inCluster") + refused(inCluster("ReconcileAndDeleteFinished"), "spec.commitRequests must be Ignore beside spec.inCluster") + accepted(object("KubeProvider", map[string]any{ + "secretRef": map[string]any{"name": "kubeconfig"}, "commitRequests": "ReconcileAndDeleteFinished", + })) + }) + + // consumerNamespaces is required on every grant type but a GitTarget's CommitRequest grant, which + // sets it or sourceNamespaces, and from is required inside it, with no default for either, so a + // grant that leaves the "who" out admits nobody instead of its holder's namespace. The block's own + // rules cannot make it required, so each list is checked here. for _, list := range []struct { name string build func(grant map[string]any) *unstructured.Unstructured @@ -159,14 +177,6 @@ var _ = Describe("Access grants at admission", func() { {"KubeProvider allowedApplierClaims", func(g map[string]any) *unstructured.Unstructured { return kubeProvider("allowedApplierClaims", g) }, map[string]any{"applierNamespaces": []any{"flux-system"}}}, - {"GitTarget allowedCommitRequests", - func(g map[string]any) *unstructured.Unstructured { - return object("GitTarget", map[string]any{ - "gitBranchRef": map[string]any{"name": "b"}, "path": "apps", - "kubeProviderRef": map[string]any{"name": "k"}, "allowedCommitRequests": []any{g}, - }) - }, - map[string]any{"unattributedWindows": "SavedBy"}}, } { It("refuses a "+list.name+" grant without a complete consumerNamespaces block", func() { refused(list.build(maps.Clone(list.scope)), "consumerNamespaces: Required value") diff --git a/internal/controller/commitrequest_controller_unit_test.go b/internal/controller/commitrequest_controller_unit_test.go index bbd4713f..5129b8cb 100644 --- a/internal/controller/commitrequest_controller_unit_test.go +++ b/internal/controller/commitrequest_controller_unit_test.go @@ -880,7 +880,7 @@ func TestCommitRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T } fromNames := func(names ...configv1alpha3.GrantNamespace) []configv1alpha3.GitTargetCommitRequestGrant { return []configv1alpha3.GitTargetCommitRequestGrant{{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromNames, Names: names}, + ConsumerNamespaces: &configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromNames, Names: names}, }} } cases := []struct { @@ -892,7 +892,7 @@ func TestCommitRequestAttach_AGitTargetElsewhereMustAdmitTheRequest(t *testing.T {"a grant for another namespace", fromNames("team-b"), false}, {"a grant for the request's namespace", fromNames("default"), true}, {"every namespace", []configv1alpha3.GitTargetCommitRequestGrant{{ - ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, + ConsumerNamespaces: &configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, }}, true}, {"an empty list admits nobody", []configv1alpha3.GitTargetCommitRequestGrant{}, false}, } diff --git a/internal/controller/commitrequest_core.go b/internal/controller/commitrequest_core.go index 133b3e46..c9ace55a 100644 --- a/internal/controller/commitrequest_core.go +++ b/internal/controller/commitrequest_core.go @@ -148,7 +148,7 @@ func (c *commitRequestCore) serveOpen( if commitRequestClaimed(cr) { // Its GitTarget went away after the claim. Only a worker that holds the request can still // say how it ends, and that answer is true for every observer. - return c.drive(ctx, p, cr, nil) + return c.drive(ctx, p, cr, nil, nil) } return c.awaitGitTarget(ctx, p, cr, targetKey) } @@ -164,7 +164,7 @@ func (c *commitRequestCore) serveOpen( return commitRequestUnownedRecheck, nil } if commitRequestClaimed(cr) { - return c.drive(ctx, p, cr, target) + return c.drive(ctx, p, cr, target, provider) } return c.claim(ctx, p, cr, target, provider) } @@ -233,15 +233,43 @@ func (c *commitRequestCore) awaitGitTarget( // admit decides whether target admits the request in spec.allowedCommitRequests: admitted, or the // refusal, which wraps errCommitRequestNotAllowed. err is a read to retry. +// +// On the control cluster, a grant's consumerNamespaces admit the request's namespace, by its labels +// there. On a source cluster, a grant's sourceNamespaces name the request's namespace, and the +// KubeProvider must lend that namespace to the GitTarget in spec.allowedGitTargets, the bound its +// WatchRules have too. func (c *commitRequestCore) admit( - ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, target *configbutleraiv1alpha3.GitTarget, + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, + target *configbutleraiv1alpha3.GitTarget, provider *configbutleraiv1alpha3.KubeProvider, ) (bool, error, error) { - decision, err := authz.CommitRequestAdmitted(ctx, c.reader, cr, target) + key := client.ObjectKeyFromObject(target) + if p.controlCluster() { + decision, err := authz.CommitRequestAdmitted(ctx, c.reader, cr, target) + if err != nil { + return false, nil, err + } + if !decision.Allowed { + return false, fmt.Errorf("%w: %s; on the cluster GitOps Reverser runs in, a grant admits by "+ + "consumerNamespaces", errCommitRequestNotAllowed, decision.Message), nil + } + return true, nil, nil + } + if !target.AdmitsSourceCommitRequest(cr.Namespace) { + return false, fmt.Errorf("%w: GitTarget %s does not admit CommitRequests from namespace %q of the "+ + "cluster it mirrors; on that cluster, a grant in spec.allowedCommitRequests admits by sourceNamespaces", + errCommitRequestNotAllowed, key, cr.Namespace), nil + } + labels, err := namespaceLabelsOf(ctx, c.reader, target.Namespace) + if err != nil { + return false, nil, err + } + scope, err := provider.AdmitsGitTarget(key, labels) if err != nil { return false, nil, err } - if !decision.Allowed { - return false, fmt.Errorf("%w: %s", errCommitRequestNotAllowed, decision.Message), nil + if !scope.AllowsSourceNamespace(cr.Namespace) { + return false, fmt.Errorf("%w: KubeProvider %s does not lend namespace %q to GitTarget %s in "+ + "spec.allowedGitTargets", errCommitRequestNotAllowed, provider.Key(), cr.Namespace, key), nil } return true, nil, nil } @@ -259,7 +287,7 @@ func (c *commitRequestCore) claim( if err := git.ValidateLiteralCommitMessage(cr.Spec.Message); err != nil { return c.decide(ctx, p, cr, commitRequestOutcome{err: err, fallbackTarget: true}, false) } - admitted, refusal, err := c.admit(ctx, cr, target) + admitted, refusal, err := c.admit(ctx, p, cr, target, provider) if err != nil { return 0, err } @@ -277,7 +305,7 @@ func (c *commitRequestCore) claim( } p.log.Info("CommitRequest claimed", "name", client.ObjectKeyFromObject(cr), "gitTarget", client.ObjectKeyFromObject(target), "author", author) - return c.drive(ctx, p, cr, target) + return c.drive(ctx, p, cr, target, provider) } // creator is the request's author, found from its audit fact on the route of the GitTarget's @@ -379,8 +407,8 @@ func (c *commitRequestCore) attachFor(cr *configbutleraiv1alpha3.CommitRequest) } // drive advances a claimed request: it sends the attach, or the withdraw once the request is past its -// bound, and records the worker's phase until the worker has an outcome. target is nil when the -// GitTarget is gone. +// bound, and records the worker's phase until the worker has an outcome. target and its provider are +// nil when the GitTarget is gone. // // A GitTarget with another UID than the one that admitted the request replaced it, and admitted // nothing: it is asked again. Its admission is recorded before anything is attached to it; its @@ -388,11 +416,11 @@ func (c *commitRequestCore) attachFor(cr *configbutleraiv1alpha3.CommitRequest) // have had the request committed. func (c *commitRequestCore) drive( ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, - target *configbutleraiv1alpha3.GitTarget, + target *configbutleraiv1alpha3.GitTarget, provider *configbutleraiv1alpha3.KubeProvider, ) (time.Duration, error) { attach := c.attachFor(cr) if target != nil && string(target.UID) != cr.Status.GitTargetUID { - admitted, refusal, err := c.admit(ctx, cr, target) + admitted, refusal, err := c.admit(ctx, p, cr, target, provider) if err != nil { return 0, err } diff --git a/internal/controller/commitrequest_core_test.go b/internal/controller/commitrequest_core_test.go index 666c96b9..c93ae03a 100644 --- a/internal/controller/commitrequest_core_test.go +++ b/internal/controller/commitrequest_core_test.go @@ -78,17 +78,24 @@ func kubeconfigProvider(name string, serving bool) *configv1alpha3.KubeProvider Spec: configv1alpha3.KubeProviderSpec{SecretRef: &meta.SecretKeyReference{Name: name}}, } if serving { - provider.Spec.RemoteSaveRequests = configv1alpha3.RemoteSaveRequestsEnabled + provider.Spec.CommitRequests = configv1alpha3.CommitRequestsReconcile } return provider } -// mirroringTarget is the GitTarget team-a-config, mirroring through provider. +// mirroringTarget is the GitTarget team-a-config, mirroring through provider. Through a provider +// other than the in-cluster one, it admits the requests of every namespace of the cluster it mirrors. func mirroringTarget(provider string) *configv1alpha3.GitTarget { - return &configv1alpha3.GitTarget{ + target := &configv1alpha3.GitTarget{ ObjectMeta: metav1.ObjectMeta{Name: "team-a-config", Namespace: "default"}, Spec: configv1alpha3.GitTargetSpec{KubeProviderRef: meta.NamespacedObjectReference{Name: provider}}, } + if provider != localProvider { + target.Spec.AllowedCommitRequests = []configv1alpha3.GitTargetCommitRequestGrant{ + {SourceNamespaces: []configv1alpha3.SourceNamespace{"*"}}, + } + } + return target } func committed() *fakeFinalizer { diff --git a/internal/controller/commitrequest_server.go b/internal/controller/commitrequest_server.go new file mode 100644 index 00000000..20797552 --- /dev/null +++ b/internal/controller/commitrequest_server.go @@ -0,0 +1,691 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "encoding/json" + "fmt" + "slices" + "sort" + "strings" + "sync" + "time" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + k8stypes "k8s.io/apimachinery/pkg/types" + k8swatch "k8s.io/apimachinery/pkg/watch" + "k8s.io/client-go/discovery" + "k8s.io/client-go/dynamic" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/event" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/queue" + "github.com/ConfigButler/gitops-reverser/internal/watch" +) + +// The source-cluster path of a CommitRequest: a request created on a cluster a KubeProvider with a +// kubeconfig connects to, next to the edits it saves. CommitRequestServer watches the CommitRequests of +// every KubeProvider whose spec.commitRequests is one of the Reconcile values, through that provider's +// own identity, and hands each one to commitRequestCore, as the CommitRequestReconciler does on the +// control cluster. It writes status, and with ReconcileAndDeleteFinished deletes finished requests, +// through the same identity; it never writes the CRD. +// +// A provider watches only the namespaces it lends the GitTargets that mirror through it, in +// spec.allowedGitTargets, so a credential scoped to those namespaces is enough. What it watches, and +// whether the cluster serves the CRD at all, is reported to the KubeProvider reconciler, which writes +// it as the CommitRequestsServed condition and status.commitRequests. + +// +kubebuilder:rbac:groups="",resources=namespaces,verbs=get;list;watch + +const ( + // commitRequestProviderResync is how often the server looks again at which KubeProviders to serve, + // what they lend, and whether their clusters serve the CRD. + commitRequestProviderResync = 10 * time.Second + // commitRequestReconnect is the wait before a watch of a cluster's CommitRequests opens again. + commitRequestReconnect = 10 * time.Second + // commitRequestEventBuffer is how many report changes wait for the KubeProvider reconciler. + commitRequestEventBuffer = 256 + // commitRequestRelist ends a watch this often, so every request is looked at again: one waiting for + // its delete-after time, or left to another path. + commitRequestRelist = 5 * time.Minute +) + +// CommitRequestsServed reasons, on a KubeProvider. +const ( + ReasonCommitRequestsWatching = "Watching" + ReasonCommitRequestsNothingToWatch = "NothingToWatch" + ReasonCommitRequestsCRDNotInstalled = "CRDNotInstalled" + ReasonCommitRequestsVersionNotServed = "VersionNotServed" + ReasonCommitRequestsForbidden = "Forbidden" + ReasonCommitRequestsUnreachable = "Unreachable" +) + +// commitRequestCRDFix is the fix for a cluster that serves no usable CommitRequest CRD. +const commitRequestCRDFix = "apply it with kubectl apply --server-side -f commitrequest-crd.yaml, from the " + + "release of GitOps Reverser this cluster is served by" + +// CommitRequestClusters reaches a source cluster through a KubeProvider's identity. *watch.Manager +// satisfies it. +type CommitRequestClusters interface { + ApplierDynamicClient(ctx context.Context, clusterID string) (dynamic.Interface, error) + SourceDiscovery(ctx context.Context, clusterID string) (discovery.DiscoveryInterface, error) +} + +// CommitRequestsReport is what the server last found of one KubeProvider's CommitRequests: the +// CommitRequestsServed condition, and the namespaces it watches. +type CommitRequestsReport struct { + Status metav1.ConditionStatus + Reason string + Message string + Namespaces []string +} + +// CommitRequestServer serves the CommitRequests of every KubeProvider that reconciles them. It runs on +// the leader only, as every status writer does. +type CommitRequestServer struct { + // Client reads KubeProviders, GitTargets and Namespaces on the control cluster. + Client client.Client + Clusters CommitRequestClusters + Finalizer CommitRequestFinalizer + // Authors finds a request's creator from the source cluster's audit facts; nil with attribution + // off. FactStreams is the set the fact follower reads; each watcher acquires the stream of + // commitrequests on its provider's route. + Authors watch.AuthorResolver + FactStreams *queue.FactStreamSet + // TTL is how long a finished request is kept on a provider that deletes them. + TTL time.Duration + Log logr.Logger + + mu sync.Mutex + watchers map[k8stypes.NamespacedName]*commitRequestWatcher + reports map[k8stypes.NamespacedName]CommitRequestsReport + events chan event.GenericEvent + streams commitRequestStreams +} + +// commitRequestWatcher is the watch of one KubeProvider's CommitRequests, in the namespaces it lends. +type commitRequestWatcher struct { + provider k8stypes.NamespacedName + config string + namespaces []string + path commitRequestPath + cancel context.CancelFunc + // serving holds the UIDs of the requests a goroutine serves now. + serving sync.Map + + mu sync.Mutex + // forbidden are the namespaces whose list or watch was denied, and undeletable those where a + // finished request could not be deleted. + forbidden map[string]bool + undeletable map[string]bool +} + +// CommitRequestsEvents names a KubeProvider whose report changed, so its reconcile writes it at once. +func (s *CommitRequestServer) CommitRequestsEvents() <-chan event.GenericEvent { + s.mu.Lock() + defer s.mu.Unlock() + return s.eventsLocked() +} + +func (s *CommitRequestServer) eventsLocked() chan event.GenericEvent { + if s.events == nil { + s.events = make(chan event.GenericEvent, commitRequestEventBuffer) + } + return s.events +} + +// CommitRequestsReport is the last report for provider, and false before the first one, or while the +// provider serves no CommitRequests. +func (s *CommitRequestServer) CommitRequestsReport(provider k8stypes.NamespacedName) (CommitRequestsReport, bool) { + s.mu.Lock() + defer s.mu.Unlock() + report, ok := s.reports[provider] + return report, ok +} + +// Start runs the server until ctx ends. +func (s *CommitRequestServer) Start(ctx context.Context) error { + ticker := time.NewTicker(commitRequestProviderResync) + defer ticker.Stop() + for { + s.syncProviders(ctx) + select { + case <-ctx.Done(): + s.stopAll() + return nil + case <-ticker.C: + } + } +} + +// syncProviders runs a watcher for every KubeProvider that reconciles its CommitRequests, in the +// namespaces it lends now, and stops the rest; and reports each one. +func (s *CommitRequestServer) syncProviders(ctx context.Context) { + var providers configbutleraiv1alpha3.KubeProviderList + if err := s.Client.List(ctx, &providers); err != nil { + s.Log.V(1).Info("Cannot list KubeProviders for CommitRequests; will retry", "err", err.Error()) + return + } + var targets configbutleraiv1alpha3.GitTargetList + if err := s.Client.List(ctx, &targets); err != nil { + s.Log.V(1).Info("Cannot list GitTargets for CommitRequests; will retry", "err", err.Error()) + return + } + serving := map[k8stypes.NamespacedName]bool{} + for i := range providers.Items { + provider := &providers.Items[i] + if provider.DeletionTimestamp != nil || !provider.ServesCommitRequests() { + continue + } + serving[provider.Key()] = true + s.syncProvider(ctx, provider, targets.Items) + } + s.mu.Lock() + defer s.mu.Unlock() + for key, watcher := range s.watchers { + if !serving[key] { + watcher.cancel() + delete(s.watchers, key) + } + } + for key := range s.reports { + if !serving[key] { + delete(s.reports, key) + s.notifyLocked(key) + } + } +} + +// syncProvider brings one provider's watcher in line with what it lends and with its cluster's CRD, +// and reports it. +func (s *CommitRequestServer) syncProvider( + ctx context.Context, provider *configbutleraiv1alpha3.KubeProvider, targets []configbutleraiv1alpha3.GitTarget, +) { + key := provider.Key() + namespaces, err := s.lentNamespaces(ctx, provider, targets) + if err != nil { + s.Log.V(1).Info("Cannot read what a KubeProvider lends; will retry", "kubeProvider", key, + "err", err.Error()) + return + } + report := s.probeCRD(ctx, key) + if report.Reason == "" { + report = s.runWatcher(ctx, provider, namespaces) + } else { + s.stopWatcher(key) + } + report.Namespaces = namespaces + s.mu.Lock() + defer s.mu.Unlock() + if s.reports == nil { + s.reports = map[k8stypes.NamespacedName]CommitRequestsReport{} + } + if previous, ok := s.reports[key]; ok && commitRequestsReportsEqual(previous, report) { + return + } + s.reports[key] = report + s.notifyLocked(key) +} + +// lentNamespaces are the namespaces provider lends, in spec.allowedGitTargets, to the GitTargets that +// mirror through it: sorted, and "*" alone when any grant lends every namespace. +func (s *CommitRequestServer) lentNamespaces( + ctx context.Context, provider *configbutleraiv1alpha3.KubeProvider, targets []configbutleraiv1alpha3.GitTarget, +) ([]string, error) { + var lent []string + for i := range targets { + target := &targets[i] + if target.DeletionTimestamp != nil || target.KubeProviderKey() != provider.Key() { + continue + } + labels, err := namespaceLabelsOf(ctx, s.Client, target.Namespace) + if err != nil { + return nil, err + } + scope, err := provider.AdmitsGitTarget(client.ObjectKeyFromObject(target), labels) + if err != nil || !scope.Admitted { + continue + } + lent = append(lent, scope.SourceNamespaces...) + } + if slices.Contains(lent, "*") { + return []string{"*"}, nil + } + sort.Strings(lent) + return slices.Compact(lent), nil +} + +// probeCRD reports, by discovery, a cluster that serves no CommitRequest CRD in v1alpha3, or that +// could not be read; an empty reason means the CRD is there. +func (s *CommitRequestServer) probeCRD(ctx context.Context, provider k8stypes.NamespacedName) CommitRequestsReport { + dc, err := s.Clusters.SourceDiscovery(ctx, watch.SourceClusterID(provider)) + if err != nil { + return unreachableCommitRequests(err) + } + return commitRequestCRDReport(dc) +} + +// commitRequestCRDReport reads, through dc, whether its cluster serves commitrequests in +// configbutler.ai/v1alpha3. +func commitRequestCRDReport(dc discovery.DiscoveryInterface) CommitRequestsReport { + groups, err := dc.ServerGroups() + if err != nil { + return unreachableCommitRequests(err) + } + gv := commitRequestGVR().GroupVersion() + var group *metav1.APIGroup + for i := range groups.Groups { + if groups.Groups[i].Name == gv.Group { + group = &groups.Groups[i] + } + } + notInstalled := CommitRequestsReport{Status: metav1.ConditionFalse, Reason: ReasonCommitRequestsCRDNotInstalled, + Message: "the cluster serves no commitrequests.configbutler.ai CRD: " + commitRequestCRDFix} + if group == nil { + return notInstalled + } + if !slices.ContainsFunc(group.Versions, func(v metav1.GroupVersionForDiscovery) bool { + return v.Version == gv.Version + }) { + return CommitRequestsReport{Status: metav1.ConditionFalse, Reason: ReasonCommitRequestsVersionNotServed, + Message: fmt.Sprintf("the commitrequests.configbutler.ai CRD does not serve %s: %s", + gv.Version, commitRequestCRDFix)} + } + resources, err := dc.ServerResourcesForGroupVersion(gv.String()) + if err != nil { + if apierrors.IsNotFound(err) { + return notInstalled + } + return unreachableCommitRequests(err) + } + if !slices.ContainsFunc(resources.APIResources, func(r metav1.APIResource) bool { + return r.Name == commitRequestGVR().Resource + }) { + return notInstalled + } + return CommitRequestsReport{} +} + +func unreachableCommitRequests(err error) CommitRequestsReport { + return CommitRequestsReport{Status: metav1.ConditionUnknown, Reason: ReasonCommitRequestsUnreachable, + Message: "the cluster's API could not be read: " + err.Error()} +} + +// runWatcher starts, restarts or keeps provider's watcher for namespaces, and reports how it goes. +func (s *CommitRequestServer) runWatcher( + ctx context.Context, provider *configbutleraiv1alpha3.KubeProvider, namespaces []string, +) CommitRequestsReport { + key := provider.Key() + if len(namespaces) == 0 { + s.stopWatcher(key) + return CommitRequestsReport{Status: metav1.ConditionTrue, Reason: ReasonCommitRequestsNothingToWatch, + Message: "no GitTarget mirroring through this KubeProvider is lent a namespace in spec.allowedGitTargets"} + } + deletes := provider.DeletesFinishedCommitRequests() + config := fmt.Sprintf("%s|%t|%s", provider.AuditRoute(), deletes, strings.Join(namespaces, ",")) + s.mu.Lock() + if s.watchers == nil { + s.watchers = map[k8stypes.NamespacedName]*commitRequestWatcher{} + } + watcher, running := s.watchers[key] + if running && watcher.config != config { + watcher.cancel() + running = false + } + if !running { + watchCtx, cancel := context.WithCancel(ctx) + watcher = &commitRequestWatcher{provider: key, config: config, namespaces: namespaces, cancel: cancel, + forbidden: map[string]bool{}, undeletable: map[string]bool{}} + s.watchers[key] = watcher + go s.run(watchCtx, watcher, provider.AuditRoute(), deletes) + } + s.mu.Unlock() + return watcher.report(deletes) +} + +func (s *CommitRequestServer) stopWatcher(key k8stypes.NamespacedName) { + s.mu.Lock() + defer s.mu.Unlock() + if watcher, ok := s.watchers[key]; ok { + watcher.cancel() + delete(s.watchers, key) + } +} + +func (s *CommitRequestServer) stopAll() { + s.mu.Lock() + defer s.mu.Unlock() + for key, watcher := range s.watchers { + watcher.cancel() + delete(s.watchers, key) + } +} + +// notifyLocked names provider on the events channel, without blocking: a full channel is caught up by +// the provider's steady requeue. +func (s *CommitRequestServer) notifyLocked(provider k8stypes.NamespacedName) { + select { + case s.eventsLocked() <- event.GenericEvent{Object: &configbutleraiv1alpha3.KubeProvider{ + ObjectMeta: metav1.ObjectMeta{Namespace: provider.Namespace, Name: provider.Name}, + }}: + default: + } +} + +// report is the CommitRequestsServed condition of a running watcher. +func (w *commitRequestWatcher) report(deletes bool) CommitRequestsReport { + w.mu.Lock() + defer w.mu.Unlock() + verbs := "get, list and watch on commitrequests, and update on commitrequests/status" + if deletes { + verbs += ", and patch and delete on commitrequests" + } + if len(w.forbidden) > 0 { + return CommitRequestsReport{Status: metav1.ConditionFalse, Reason: ReasonCommitRequestsForbidden, + Message: fmt.Sprintf("the credential may not list or watch CommitRequests in %s: grant it %s, "+ + "in the configbutler.ai group", describeNamespaces(sortedKeys(w.forbidden)), verbs)} + } + message := "watching CommitRequests in every namespace" + if !slices.Equal(w.namespaces, []string{"*"}) { + message = fmt.Sprintf("watching CommitRequests in %d namespaces: %s", len(w.namespaces), + strings.Join(w.namespaces, ", ")) + } + if len(w.undeletable) > 0 { + message += fmt.Sprintf("; finished requests in %s cannot be deleted and stay: grant delete on "+ + "commitrequests there", describeNamespaces(sortedKeys(w.undeletable))) + } + return CommitRequestsReport{Status: metav1.ConditionTrue, Reason: ReasonCommitRequestsWatching, Message: message} +} + +func (w *commitRequestWatcher) mark(set map[string]bool, namespace string, on bool) { + w.mu.Lock() + defer w.mu.Unlock() + if on { + set[namespace] = true + } else { + delete(set, namespace) + } +} + +// run watches one provider's CommitRequests until its context ends: one watch per lent namespace, or one +// across the cluster. +func (s *CommitRequestServer) run( + ctx context.Context, watcher *commitRequestWatcher, route string, deletes bool, +) { + log := s.Log.WithValues("kubeProvider", watcher.provider.String()) + log.Info("Serving the CommitRequests of a KubeProvider", "namespaces", watcher.namespaces) + if s.FactStreams != nil { + release := s.FactStreams.Acquire(queue.FactStreamKeyFor(route, commitRequestGVR().GroupResource())) + defer release() + } + dc, err := s.Clusters.ApplierDynamicClient(ctx, watch.SourceClusterID(watcher.provider)) + if err != nil { + log.Info("Cannot reach the KubeProvider's cluster for CommitRequests; will retry", "err", err.Error()) + return + } + resource := dc.Resource(commitRequestGVR()) + watcher.path = commitRequestPath{ + provider: watcher.provider, + store: dynamicCommitRequestStore{resource: resource, undeletable: watcher.markUndeletable}, + deletes: deletes, + log: log, + } + var wg sync.WaitGroup + for _, namespace := range watcher.namespaces { + if namespace == "*" { + namespace = metav1.NamespaceAll + } + wg.Add(1) + go func() { + defer wg.Done() + s.watchNamespace(ctx, log, watcher, resource, namespace) + }() + } + wg.Wait() +} + +func (w *commitRequestWatcher) markUndeletable(namespace string) { + w.mark(w.undeletable, namespace, true) +} + +// watchNamespace lists and watches one namespace's CommitRequests, or every namespace's, reconnecting +// after a failure, until ctx ends. +func (s *CommitRequestServer) watchNamespace( + ctx context.Context, log logr.Logger, watcher *commitRequestWatcher, + resource dynamic.NamespaceableResourceInterface, namespace string, +) { + for ctx.Err() == nil { + err := s.session(ctx, watcher, resource.Namespace(namespace)) + watcher.mark(watcher.forbidden, namespaceLabel(namespace), apierrors.IsForbidden(err)) + if err != nil && ctx.Err() == nil { + log.Info("The watch of CommitRequests ended; reconnecting", "namespace", namespaceLabel(namespace), + "err", err.Error()) + } + select { + case <-ctx.Done(): + case <-time.After(commitRequestReconnect): + } + } +} + +// session lists the requests and watches them, until the watch ends. +func (s *CommitRequestServer) session( + ctx context.Context, watcher *commitRequestWatcher, resource dynamic.ResourceInterface, +) error { + list, err := resource.List(ctx, metav1.ListOptions{}) + if err != nil { + return err + } + for i := range list.Items { + s.consider(ctx, watcher, &list.Items[i]) + } + timeout := int64(commitRequestRelist / time.Second) + w, err := resource.Watch(ctx, metav1.ListOptions{ + ResourceVersion: list.GetResourceVersion(), AllowWatchBookmarks: true, TimeoutSeconds: &timeout, + }) + if err != nil { + return err + } + defer w.Stop() + for { + select { + case <-ctx.Done(): + return nil + case ev, ok := <-w.ResultChan(): + if !ok { + return nil + } + switch ev.Type { + case k8swatch.Added, k8swatch.Modified: + if u, isObject := ev.Object.(*unstructured.Unstructured); isObject { + s.consider(ctx, watcher, u) + } + case k8swatch.Error: + return apierrors.FromObject(ev.Object) + case k8swatch.Deleted, k8swatch.Bookmark: + } + } + } +} + +// consider serves a request no goroutine serves yet: it takes steps until the core has nothing due +// before the next relist looks at the request again. +func (s *CommitRequestServer) consider( + ctx context.Context, + watcher *commitRequestWatcher, + u *unstructured.Unstructured, +) { + if u.GetDeletionTimestamp() != nil { + return + } + if _, serving := watcher.serving.LoadOrStore(u.GetUID(), struct{}{}); serving { + return + } + key := k8stypes.NamespacedName{Namespace: u.GetNamespace(), Name: u.GetName()} + core := s.core() + go func() { + defer watcher.serving.Delete(u.GetUID()) + for ctx.Err() == nil { + after, err := core.serve(ctx, watcher.path, key) + if err != nil { + watcher.path.log.V(1).Info("A CommitRequest step failed; will retry", "commitRequest", key, + "err", err.Error()) + after = commitRequestPollInterval + } + if after == 0 || after >= commitRequestRelist { + return + } + select { + case <-ctx.Done(): + case <-time.After(after): + } + } + }() +} + +func (s *CommitRequestServer) core() *commitRequestCore { + s.mu.Lock() + s.streams.set = s.FactStreams + s.mu.Unlock() + return &commitRequestCore{ + reader: s.Client, finalizer: s.Finalizer, authors: s.Authors, streams: &s.streams, ttl: s.TTL, + } +} + +// dynamicCommitRequestStore reads and writes a source cluster's CommitRequests through a dynamic +// client. Every write carries the object's resourceVersion, so it is conditional on it. +type dynamicCommitRequestStore struct { + resource dynamic.NamespaceableResourceInterface + // undeletable is told the namespace of a finished request the credential may not delete. + undeletable func(namespace string) +} + +func (s dynamicCommitRequestStore) get( + ctx context.Context, key k8stypes.NamespacedName, +) (*configbutleraiv1alpha3.CommitRequest, error) { + u, err := s.resource.Namespace(key.Namespace).Get(ctx, key.Name, metav1.GetOptions{}) + if err != nil { + return nil, err + } + return commitRequestFromUnstructured(u) +} + +func (s dynamicCommitRequestStore) updateStatus(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error { + u, err := commitRequestToUnstructured(cr) + if err != nil { + return err + } + written, err := s.resource.Namespace(cr.Namespace).UpdateStatus(ctx, u, metav1.UpdateOptions{}) + if err != nil { + return err + } + return commitRequestInto(written, cr) +} + +func (s dynamicCommitRequestStore) annotate( + ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, key, value string, +) error { + patch, err := json.Marshal(map[string]any{"metadata": map[string]any{ + "resourceVersion": cr.ResourceVersion, "annotations": map[string]string{key: value}, + }}) + if err != nil { + return err + } + written, err := s.resource.Namespace(cr.Namespace).Patch(ctx, cr.Name, k8stypes.MergePatchType, patch, + metav1.PatchOptions{}) + if err != nil { + return err + } + return commitRequestInto(written, cr) +} + +func (s dynamicCommitRequestStore) remove(ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest) error { + uid, resourceVersion := cr.UID, cr.ResourceVersion + err := s.resource.Namespace(cr.Namespace).Delete(ctx, cr.Name, metav1.DeleteOptions{ + Preconditions: &metav1.Preconditions{UID: &uid, ResourceVersion: &resourceVersion}, + }) + if apierrors.IsForbidden(err) && s.undeletable != nil { + s.undeletable(cr.Namespace) + return nil + } + return err +} + +func commitRequestFromUnstructured(u *unstructured.Unstructured) (*configbutleraiv1alpha3.CommitRequest, error) { + var cr configbutleraiv1alpha3.CommitRequest + if err := commitRequestInto(u, &cr); err != nil { + return nil, err + } + return &cr, nil +} + +func commitRequestInto(u *unstructured.Unstructured, cr *configbutleraiv1alpha3.CommitRequest) error { + *cr = configbutleraiv1alpha3.CommitRequest{} + if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, cr); err != nil { + return fmt.Errorf("read CommitRequest %s/%s: %w", u.GetNamespace(), u.GetName(), err) + } + return nil +} + +func commitRequestToUnstructured(cr *configbutleraiv1alpha3.CommitRequest) (*unstructured.Unstructured, error) { + object, err := runtime.DefaultUnstructuredConverter.ToUnstructured(cr) + if err != nil { + return nil, err + } + u := &unstructured.Unstructured{Object: object} + u.SetGroupVersionKind(configbutleraiv1alpha3.GroupVersion.WithKind("CommitRequest")) + return u, nil +} + +// namespaceLabelsOf reads the labels of a control-cluster namespace; one that does not exist has none. +func namespaceLabelsOf(ctx context.Context, reader client.Reader, namespace string) (map[string]string, error) { + var ns corev1.Namespace + if err := reader.Get(ctx, client.ObjectKey{Name: namespace}, &ns); err != nil { + if apierrors.IsNotFound(err) { + return map[string]string{}, nil + } + return nil, fmt.Errorf("read Namespace %s: %w", namespace, err) + } + return ns.Labels, nil +} + +// namespaceLabel names a watched namespace in a message, "*" for every namespace. +func namespaceLabel(namespace string) string { + if namespace == metav1.NamespaceAll { + return "*" + } + return namespace +} + +// describeNamespaces names namespaces in a message: "namespace a" or "namespaces a, b", and "every +// namespace" for "*". +func describeNamespaces(namespaces []string) string { + switch { + case slices.Contains(namespaces, "*"): + return "every namespace" + case len(namespaces) == 1: + return "namespace " + namespaces[0] + } + return "namespaces " + strings.Join(namespaces, ", ") +} + +func sortedKeys(set map[string]bool) []string { + keys := make([]string, 0, len(set)) + for key := range set { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + +func commitRequestsReportsEqual(a, b CommitRequestsReport) bool { + return a.Status == b.Status && a.Reason == b.Reason && a.Message == b.Message && + slices.Equal(a.Namespaces, b.Namespaces) +} diff --git a/internal/controller/commitrequest_server_test.go b/internal/controller/commitrequest_server_test.go new file mode 100644 index 00000000..12fea160 --- /dev/null +++ b/internal/controller/commitrequest_server_test.go @@ -0,0 +1,311 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + "context" + "errors" + "testing" + "time" + + meta "github.com/fluxcd/pkg/apis/meta" + "github.com/go-logr/logr" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + apimeta "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/discovery" + fakediscovery "k8s.io/client-go/discovery/fake" + "k8s.io/client-go/dynamic" + dynamicfake "k8s.io/client-go/dynamic/fake" + clienttesting "k8s.io/client-go/testing" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/event" + + configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/git" +) + +// fakeDiscovery serves resources, and fails with err when it is set. +func fakeDiscovery(err error, resources ...*metav1.APIResourceList) discovery.DiscoveryInterface { + fake := &clienttesting.Fake{Resources: resources} + if err != nil { + fake.AddReactor("get", "group", func(clienttesting.Action) (bool, runtime.Object, error) { + return true, nil, err + }) + } + return &fakediscovery.FakeDiscovery{Fake: fake} +} + +func commitRequestResources(groupVersion string, names ...string) *metav1.APIResourceList { + list := &metav1.APIResourceList{GroupVersion: groupVersion} + for _, name := range names { + list.APIResources = append(list.APIResources, metav1.APIResource{Name: name, Namespaced: true}) + } + return list +} + +// The CRD's state is read by discovery alone: no permission on CRDs is needed, so a credential scoped to +// namespaces reports it too. Each reason that is not True says how to fix it. +func TestCommitRequestCRDReport(t *testing.T) { + for _, tc := range []struct { + name string + discovery discovery.DiscoveryInterface + wantReason string + wantFix string + }{ + {name: "no configbutler.ai group", discovery: fakeDiscovery(nil), + wantReason: ReasonCommitRequestsCRDNotInstalled, wantFix: "kubectl apply --server-side"}, + {name: "the group without commitrequests", + discovery: fakeDiscovery(nil, commitRequestResources("configbutler.ai/v1alpha3", "gittargets")), + wantReason: ReasonCommitRequestsCRDNotInstalled, wantFix: "commitrequest-crd.yaml"}, + {name: "another version only", + discovery: fakeDiscovery(nil, commitRequestResources("configbutler.ai/v1alpha2", "commitrequests")), + wantReason: ReasonCommitRequestsVersionNotServed, wantFix: "commitrequest-crd.yaml"}, + {name: "served", discovery: fakeDiscovery(nil, + commitRequestResources("configbutler.ai/v1alpha3", "commitrequests"))}, + {name: "unreadable", discovery: fakeDiscovery(errors.New("connection refused")), + wantReason: ReasonCommitRequestsUnreachable, wantFix: "connection refused"}, + } { + t.Run(tc.name, func(t *testing.T) { + report := commitRequestCRDReport(tc.discovery) + assert.Equal(t, tc.wantReason, report.Reason) + assert.Contains(t, report.Message, tc.wantFix) + }) + } +} + +// A provider watches the namespaces it lends the GitTargets that mirror through it, "*" alone once any +// grant lends every namespace. +func TestCommitRequestServer_LentNamespaces(t *testing.T) { + provider := kubeconfigProvider("edge", true) + provider.Spec.AllowedGitTargets = []configv1alpha3.KubeProviderGitTargetGrant{{ + ConsumerNamespaces: configv1alpha3.ConsumerNamespaces{From: configv1alpha3.GrantFromAll}, + SourceNamespaces: []configv1alpha3.ScopeNamespace{configv1alpha3.ConsumerNamespaceVariable, "shared"}, + }} + target := func(namespace, provider string) configv1alpha3.GitTarget { + return configv1alpha3.GitTarget{ + ObjectMeta: metav1.ObjectMeta{Name: "t", Namespace: namespace}, + Spec: configv1alpha3.GitTargetSpec{KubeProviderRef: meta.NamespacedObjectReference{ + Name: provider, Namespace: "default", + }}, + } + } + c := newCommitRequestClient(t, nil) + s := &CommitRequestServer{Client: c} + + lent, err := s.lentNamespaces(context.Background(), provider, []configv1alpha3.GitTarget{ + target("team-b", "edge"), target("team-a", "edge"), target("team-c", "other"), + }) + require.NoError(t, err) + assert.Equal(t, []string{"shared", "team-a", "team-b"}, lent, "only its own GitTargets, sorted, once each") + + provider.Spec.AllowedGitTargets[0].SourceNamespaces = []configv1alpha3.ScopeNamespace{"*"} + lent, err = s.lentNamespaces(context.Background(), provider, []configv1alpha3.GitTarget{target("team-a", "edge")}) + require.NoError(t, err) + assert.Equal(t, []string{"*"}, lent) +} + +// A watcher reports what it watches, the namespaces it may not list or watch, and the ones where a +// finished request could not be deleted, which leaves serving True. +func TestCommitRequestWatcher_Report(t *testing.T) { + watcher := &commitRequestWatcher{namespaces: []string{"shop"}, forbidden: map[string]bool{}, + undeletable: map[string]bool{}} + report := watcher.report(true) + assert.Equal(t, metav1.ConditionTrue, report.Status) + assert.Equal(t, ReasonCommitRequestsWatching, report.Reason) + assert.Contains(t, report.Message, "1 namespaces: shop") + + watcher.markUndeletable("shop") + report = watcher.report(true) + assert.Equal(t, metav1.ConditionTrue, report.Status, "serving still works") + assert.Contains(t, report.Message, "cannot be deleted") + + watcher.mark(watcher.forbidden, "shop", true) + report = watcher.report(true) + assert.Equal(t, metav1.ConditionFalse, report.Status) + assert.Equal(t, ReasonCommitRequestsForbidden, report.Reason) + assert.Contains(t, report.Message, "namespace shop") + assert.Contains(t, report.Message, "patch and delete", "the verbs ReconcileAndDeleteFinished needs") + + every := &commitRequestWatcher{namespaces: []string{"*"}, forbidden: map[string]bool{}, + undeletable: map[string]bool{}} + assert.Contains(t, every.report(false).Message, "every namespace") +} + +// On a source cluster a request is admitted by a grant's sourceNamespaces, and only in a namespace the +// KubeProvider lends the GitTarget; on the control cluster by consumerNamespaces. Each refusal says +// which field applies. +func TestCommitRequestCore_AdmissionBySide(t *testing.T) { + t.Run("a source cluster request with no sourceNamespaces grant", func(t *testing.T) { + cr := newCommitRequest("save") + cr.Spec.GitTargetRef.Namespace = "default" + target := mirroringTarget("edge") + target.Spec.AllowedCommitRequests = nil + c := newCommitRequestClient(t, nil, cr, target, kubeconfigProvider("edge", true)) + f := committed() + serveStep(t, &commitRequestCore{reader: c, finalizer: f}, watcherPath(c, "edge", false), "save") + + assert.Empty(t, f.calls) + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonNotAllowed, ready.Reason) + assert.Contains(t, ready.Message, "sourceNamespaces") + }) + t.Run("a source cluster request in a namespace that is not lent", func(t *testing.T) { + cr := newCommitRequest("save") + cr.Namespace = "elsewhere" + cr.Spec.GitTargetRef.Namespace = "default" + c := newCommitRequestClient(t, nil, cr, mirroringTarget("edge"), kubeconfigProvider("edge", true)) + f := committed() + _, err := (&commitRequestCore{reader: c, finalizer: f}).serve(context.Background(), + watcherPath(c, "edge", false), types.NamespacedName{Namespace: "elsewhere", Name: "save"}) + require.NoError(t, err) + + assert.Empty(t, f.calls) + var got configv1alpha3.CommitRequest + require.NoError(t, c.Get(context.Background(), client.ObjectKey{Namespace: "elsewhere", Name: "save"}, &got)) + ready := apimeta.FindStatusCondition(got.Status.Conditions, ConditionTypeReady) + require.NotNil(t, ready) + assert.Equal(t, crReasonNotAllowed, ready.Reason) + assert.Contains(t, ready.Message, "does not lend namespace") + }) + t.Run("a control-cluster request no consumerNamespaces grant admits", func(t *testing.T) { + target := mirroringTarget(localProvider) + target.Spec.AllowedCommitRequests = []configv1alpha3.GitTargetCommitRequestGrant{ + {SourceNamespaces: []configv1alpha3.SourceNamespace{"*"}}, + } + c := newCommitRequestClient(t, nil, newCommitRequest("save"), target) + serveStep(t, &commitRequestCore{reader: c, finalizer: committed()}, controlPath(c), "save") + + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonNotAllowed, ready.Reason) + assert.Contains(t, ready.Message, "consumerNamespaces") + }) +} + +// fakeClusters serves one source cluster: its dynamic client, and its discovery. +type fakeClusters struct { + dynamic dynamic.Interface + discovery discovery.DiscoveryInterface +} + +func (f fakeClusters) ApplierDynamicClient(context.Context, string) (dynamic.Interface, error) { + return f.dynamic, nil +} + +func (f fakeClusters) SourceDiscovery(context.Context, string) (discovery.DiscoveryInterface, error) { + return f.discovery, nil +} + +// sourceCommitRequest is a CommitRequest as it is stored on a source cluster. +func sourceCommitRequest(namespace, name string) *unstructured.Unstructured { + u := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "configbutler.ai/v1alpha3", + "kind": "CommitRequest", + "metadata": map[string]any{ + "namespace": namespace, "name": name, "uid": "uid-" + name, "resourceVersion": "1", + "creationTimestamp": time.Now().UTC().Format(time.RFC3339), + }, + "spec": map[string]any{ + "gitTargetRef": map[string]any{"namespace": "default", "name": "team-a-config"}, + "message": "save: from the workspace", + }, + }} + return u +} + +// End to end through the server: a KubeProvider that reconciles its CommitRequests watches the namespace +// it lends, serves a request created there through the core, writes its status back on that cluster, +// and reports what it watches. With ReconcileAndDeleteFinished the finished request is stamped for +// deletion. +func TestCommitRequestServer_ServesARequestOnASourceCluster(t *testing.T) { + provider := kubeconfigProvider("edge", true) + provider.Spec.CommitRequests = configv1alpha3.CommitRequestsReconcileAndDeleteFinished + c := newCommitRequestClient(t, nil, mirroringTarget("edge"), provider) + gvr := commitRequestGVR() + source := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), + map[schema.GroupVersionResource]string{gvr: "CommitRequestList"}, sourceCommitRequest("default", "save")) + f := committed() + s := &CommitRequestServer{ + Client: c, Finalizer: f, TTL: time.Hour, Log: logr.Discard(), + Clusters: fakeClusters{dynamic: source, discovery: fakeDiscovery(nil, + commitRequestResources("configbutler.ai/v1alpha3", "commitrequests"))}, + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + s.syncProviders(ctx) + + report, ok := s.CommitRequestsReport(provider.Key()) + require.True(t, ok) + assert.Equal(t, ReasonCommitRequestsWatching, report.Reason) + assert.Equal(t, []string{"default"}, report.Namespaces, "the namespace the provider lends its GitTarget") + select { + case ev := <-s.CommitRequestsEvents(): + assert.Equal(t, provider.Name, ev.Object.GetName(), "the KubeProvider is told its report changed") + default: + t.Fatal("no event for the changed report") + } + + require.Eventually(t, func() bool { + u, err := source.Resource(gvr).Namespace("default").Get(ctx, "save", metav1.GetOptions{}) + if err != nil { + return false + } + cr, convErr := commitRequestFromUnstructured(u) + if convErr != nil { + return false + } + ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady) + _, stamped := cr.Annotations[CommitRequestDeleteAfterAnnotation] + return ready != nil && ready.Reason == crReasonCommitted && stamped + }, 10*time.Second, 50*time.Millisecond, "the request is served on its own cluster, and stamped for deletion") + require.Len(t, f.calls, 1) + assert.Equal(t, git.AttributionNotAttempted, f.calls[0].Attribution, "attribution is off") + + provider.Spec.CommitRequests = configv1alpha3.CommitRequestsIgnore + require.NoError(t, c.Update(ctx, provider)) + s.syncProviders(ctx) + _, ok = s.CommitRequestsReport(provider.Key()) + assert.False(t, ok, "a provider that ignores CommitRequests reports nothing") +} + +// A KubeProvider carries CommitRequestsServed and status.commitRequests only while it serves its +// cluster's CommitRequests. +func TestKubeProviderReconciler_CommitRequestsServed(t *testing.T) { + reporter := &fixedCommitRequestsReporter{report: CommitRequestsReport{ + Status: metav1.ConditionTrue, Reason: ReasonCommitRequestsWatching, Message: "watching", + Namespaces: []string{"shop"}, + }} + r := &KubeProviderReconciler{CommitRequests: reporter} + + provider := kubeconfigProvider("edge", true) + st := beginStatus(nil, nil, provider) + r.applyCommitRequestsServed(st, provider) + served := apimeta.FindStatusCondition(provider.Status.Conditions, KubeProviderConditionCommitRequestsServed) + require.NotNil(t, served) + assert.Equal(t, ReasonCommitRequestsWatching, served.Reason) + require.NotNil(t, provider.Status.CommitRequests) + assert.Equal(t, []string{"shop"}, provider.Status.CommitRequests.Namespaces) + + provider.Spec.CommitRequests = configv1alpha3.CommitRequestsIgnore + r.applyCommitRequestsServed(st, provider) + assert.Nil(t, apimeta.FindStatusCondition(provider.Status.Conditions, KubeProviderConditionCommitRequestsServed), + "removed while the provider ignores CommitRequests") + assert.Nil(t, provider.Status.CommitRequests) +} + +type fixedCommitRequestsReporter struct { + report CommitRequestsReport +} + +func (f *fixedCommitRequestsReporter) CommitRequestsReport(types.NamespacedName) (CommitRequestsReport, bool) { + return f.report, true +} + +func (f *fixedCommitRequestsReporter) CommitRequestsEvents() <-chan event.GenericEvent { return nil } diff --git a/internal/controller/kubeprovider_commitrequests.go b/internal/controller/kubeprovider_commitrequests.go new file mode 100644 index 00000000..f31dd159 --- /dev/null +++ b/internal/controller/kubeprovider_commitrequests.go @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: Apache-2.0 + +package controller + +import ( + k8stypes "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/event" + + configbutleraiv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" +) + +// KubeProviderConditionCommitRequestsServed reports whether a KubeProvider's cluster's CommitRequests +// are served. It is absent while spec.commitRequests is Ignore, and, like AuditFactsReceived, not part +// of Ready: mirroring works without it. +const KubeProviderConditionCommitRequestsServed = "CommitRequestsServed" + +// CommitRequestsReporter is what the KubeProvider reconciler reads to report a provider's CommitRequests. +// *CommitRequestServer satisfies it. +type CommitRequestsReporter interface { + // CommitRequestsReport is the last report for provider, and false before the first one. + CommitRequestsReport(provider k8stypes.NamespacedName) (CommitRequestsReport, bool) + // CommitRequestsEvents names a provider whose report changed. + CommitRequestsEvents() <-chan event.GenericEvent +} + +// applyCommitRequestsServed writes the CommitRequestsServed condition and status.commitRequests. A +// provider that serves no CommitRequests carries neither: a condition that always said "not applicable" +// would be noise, and a True left from before the switch would be wrong. Until the server has looked at +// a provider that serves them, what is there stays. +func (r *KubeProviderReconciler) applyCommitRequestsServed( + st *reconcileStatus, provider *configbutleraiv1alpha3.KubeProvider, +) { + if !provider.ServesCommitRequests() || r.CommitRequests == nil { + st.remove(KubeProviderConditionCommitRequestsServed) + provider.Status.CommitRequests = nil + return + } + report, ok := r.CommitRequests.CommitRequestsReport(provider.Key()) + if !ok { + return + } + st.set(KubeProviderConditionCommitRequestsServed, report.Status, report.Reason, report.Message) + provider.Status.CommitRequests = &configbutleraiv1alpha3.KubeProviderCommitRequestsStatus{ + Namespaces: report.Namespaces, + } +} diff --git a/internal/controller/kubeprovider_controller.go b/internal/controller/kubeprovider_controller.go index e6b8bc0c..9879a2f9 100644 --- a/internal/controller/kubeprovider_controller.go +++ b/internal/controller/kubeprovider_controller.go @@ -55,6 +55,10 @@ type KubeProviderReconciler struct { // absent rather than Unknown. AuditFacts AuditRouteFacts + // CommitRequests reports what each KubeProvider serves of its cluster's CommitRequests, for the + // CommitRequestsServed condition. Nil reports nothing. + CommitRequests CommitRequestsReporter + firsts kubeProviderLogFirsts // Recorder emits a Kubernetes Event on every persisted Ready transition; nil disables Events. @@ -152,6 +156,7 @@ func (r *KubeProviderReconciler) reconcileKubeProvider( // nobody posts under costs the commit AUTHOR, never the mirror, so it must not make a working // provider look broken to a kstatus reader. r.applyAuditFactsCondition(st, provider) + r.applyCommitRequestsServed(st, provider) st.applyReadiness(rd) // A failed status write is a real failure, not a verdict: propagate it so the provider is @@ -255,6 +260,12 @@ func (r *KubeProviderReconciler) SetupWithManager(mgr ctrl.Manager) error { )) } } + // A changed CommitRequest report, such as a CRD applied by hand, is written at once. + if r.CommitRequests != nil { + b = b.WatchesRawSource( + source.Channel(r.CommitRequests.CommitRequestsEvents(), &handler.EnqueueRequestForObject{}), + ) + } return b.Complete(r) } diff --git a/internal/watch/cluster_context.go b/internal/watch/cluster_context.go index 2274a801..4cb5b405 100644 --- a/internal/watch/cluster_context.go +++ b/internal/watch/cluster_context.go @@ -756,6 +756,24 @@ func (m *Manager) ApplierDynamicClient(ctx context.Context, clusterID string) (d return dc, nil } +// SourceDiscovery returns a discovery client for the cluster clusterID names, with that KubeProvider's +// identity, built for this call: what the CommitRequest server asks whether the cluster serves the +// CommitRequest CRD. It carries the dial timeout every discovery client here does. +func (m *Manager) SourceDiscovery(ctx context.Context, clusterID string) (discovery.DiscoveryInterface, error) { + if clusterID == configPlaneClusterID { + return nil, errors.New("no cluster named: the config plane is not a KubeProvider") + } + cfg, _, _, err := m.resolveSourceConfig(ctx, clusterID) + if err != nil { + return nil, err + } + dc, err := discovery.NewDiscoveryClientForConfig(discoveryRESTConfig(cfg)) + if err != nil { + return nil, fmt.Errorf("build discovery client for cluster %q: %w", describeCluster(clusterID), err) + } + return dc, nil +} + // sourceClusterDeclared reports whether any GitTarget's Declare captured this cluster id. func (m *Manager) sourceClusterDeclared(clusterID string) bool { for _, id := range m.watchPlane().clusters { diff --git a/internal/watch/remote_save_requests.go b/internal/watch/remote_save_requests.go deleted file mode 100644 index 46f7e36a..00000000 --- a/internal/watch/remote_save_requests.go +++ /dev/null @@ -1,702 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package watch - -import ( - "context" - "errors" - "fmt" - "sync" - "time" - - "github.com/go-logr/logr" - apierrors "k8s.io/apimachinery/pkg/api/errors" - apimeta "k8s.io/apimachinery/pkg/api/meta" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/runtime/schema" - k8stypes "k8s.io/apimachinery/pkg/types" - k8swatch "k8s.io/apimachinery/pkg/watch" - "k8s.io/client-go/dynamic" - "sigs.k8s.io/controller-runtime/pkg/client" - - sourcev1alpha1 "github.com/ConfigButler/gitops-reverser/api/source/v1alpha1" - configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/git" - "github.com/ConfigButler/gitops-reverser/internal/queue" -) - -// A RemoteSaveRequest is a save made on a source cluster, next to the edits it saves -// (docs/future/source-side-save-plan.md). The server watches them on every KubeProvider whose -// spec.remoteSaveRequests is "Enabled", through that provider's identity, and serves each one the way -// the CommitRequest controller serves a CommitRequest: it attaches the request to its creator's commit -// window on the GitTarget its spec.gitTargetRef names, polls the branch worker for the outcome, and -// writes it back to the request's status on the source cluster. -// -// Each party decides one thing. The request names the GitTarget; the KubeProvider it was served -// through says which cluster it came from, and the GitTarget must mirror that one; the GitTarget's -// spec.allowedRemoteSaveRequests admits the request's namespace and labels; and the window, which -// groups by author and GitTarget, decides which edits are saved, wherever that GitTarget mirrors them. -// -// The request's author is found as the edits' authors are, from the source cluster's audit facts, so -// a save attaches only to its creator's windows by the ordinary author rule. On a cluster that sends -// no audit events it names nobody, and attaches to a window whose edits name nobody. On a cluster -// whose KubeProvider has received an audit fact, a request that names nobody means its fact went -// missing; it is refused rather than attached, so it cannot claim the window that should stay -// unattributed. -// -// A request is served again after a restart or a dropped watch. Its status then carries what the -// first serving decided: its author, found once against the resourceVersion it was created with, -// which its status writes have moved since; and its outcome, which the worker may no longer know. - -// +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests,verbs=get;list;watch;delete -// +kubebuilder:rbac:groups=source.configbutler.ai,resources=remotesaverequests/status,verbs=get;update - -// remoteSaveRequestGVR() is the served resource of a RemoteSaveRequest. -func remoteSaveRequestGVR() schema.GroupVersionResource { - return sourcev1alpha1.GroupVersion.WithResource("remotesaverequests") -} - -const ( - // remoteSaveRequestProviderResync is how often the server looks for KubeProviders to serve. - remoteSaveRequestProviderResync = 10 * time.Second - // remoteSaveRequestReconnect is the wait before a watch of a cluster's RemoteSaveRequests opens again. - remoteSaveRequestReconnect = 10 * time.Second - // remoteSaveRequestRelist ends a watch this often, so finished requests past their TTL are deleted. - remoteSaveRequestRelist = 5 * time.Minute - // remoteSaveRequestPoll is how often a request's outcome is asked for. - remoteSaveRequestPoll = time.Second - // remoteSaveRequestPushAllowance is what a request may take beyond its own waits: the push, its - // cooldown and a retry. Past it, a request no worker holds is withdrawn. - remoteSaveRequestPushAllowance = 2 * time.Minute - // remoteSaveRequestApplyLost is how long a save's apply may stay unknown to its worker before it is - // reported lost: GitOps Reverser restarted, or the record aged out. - remoteSaveRequestApplyLost = time.Minute - // defaultSaveAttachTimeout and defaultSaveMaxDuration are the window when a request sets none. - defaultSaveAttachTimeout = 5 * time.Second - defaultSaveMaxDuration = 2 * time.Second -) - -// RemoteSaveRequest reasons, on the Ready and Applied conditions. -const ( - saveReasonCommitted = "Committed" - saveReasonNoWindow = "NoWindow" - saveReasonWindowMismatch = "WindowMismatch" - saveReasonAlreadyPresent = "AlreadyPresent" - saveReasonSaveInProgress = "SaveInProgress" - saveReasonFailed = "Failed" - saveReasonNotAllowed = "NotAllowed" - saveReasonGitTargetNotFound = "GitTargetNotFound" - saveReasonAuthorUnresolved = "AuthorUnresolved" - saveReasonSaving = "Saving" - saveReasonApplying = "Applying" - saveReasonLost = "Lost" -) - -// SaveFinalizer is what serves a save once it is attached: the CommitRequest service of the event -// router, which owns the branch workers. -type SaveFinalizer interface { - ServiceCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - WithdrawCommitRequest(ctx context.Context, attach git.AttachCommitRequest) (git.FinalizeResult, bool, error) - LookupCommitRequestApply(ctx context.Context, attach git.AttachCommitRequest) (git.SaveApply, bool, error) -} - -// RemoteSaveRequestServer serves the RemoteSaveRequests of every KubeProvider that enables them. It runs on the -// leader only, as every status writer does. -type RemoteSaveRequestServer struct { - // Manager is the watch manager: the source clusters' clients, their attribution, and which - // GitTargets mirror which namespace. - Manager *Manager - // Client reads KubeProviders and GitTargets on the cluster the operator runs in. - Client client.Client - // Finalizer serves an attached save. - Finalizer SaveFinalizer - // TTL is how long a finished request stays before it is deleted; zero keeps it. - TTL time.Duration - Log logr.Logger - - // dynamicClient overrides the source cluster's client, in tests. - dynamicClient func(ctx context.Context, provider k8stypes.NamespacedName) (dynamic.Interface, error) - - mu sync.Mutex - watchers map[k8stypes.NamespacedName]*remoteSaveRequestWatcher -} - -// remoteSaveRequestWatcher is the watch of one KubeProvider's RemoteSaveRequests. -type remoteSaveRequestWatcher struct { - provider k8stypes.NamespacedName - route string - cancel context.CancelFunc - // serving holds the UIDs of the requests being served now. - serving sync.Map -} - -// Start runs the server until ctx ends. -func (s *RemoteSaveRequestServer) Start(ctx context.Context) error { - ticker := time.NewTicker(remoteSaveRequestProviderResync) - defer ticker.Stop() - for { - s.syncProviders(ctx) - select { - case <-ctx.Done(): - s.stopAll() - return nil - case <-ticker.C: - } - } -} - -// syncProviders runs a watcher for every KubeProvider that enables RemoteSaveRequests, and stops the rest. -func (s *RemoteSaveRequestServer) syncProviders(ctx context.Context) { - var providers configv1alpha3.KubeProviderList - if err := s.Client.List(ctx, &providers); err != nil { - s.Log.V(1).Info("Cannot list KubeProviders for RemoteSaveRequests; will retry", "err", err.Error()) - return - } - want := map[k8stypes.NamespacedName]string{} - for i := range providers.Items { - provider := &providers.Items[i] - if provider.DeletionTimestamp == nil && - provider.Spec.RemoteSaveRequests == configv1alpha3.RemoteSaveRequestsEnabled { - want[provider.Key()] = provider.AuditRoute() - } - } - s.mu.Lock() - defer s.mu.Unlock() - if s.watchers == nil { - s.watchers = map[k8stypes.NamespacedName]*remoteSaveRequestWatcher{} - } - for key, watcher := range s.watchers { - if route, ok := want[key]; !ok || route != watcher.route { - watcher.cancel() - delete(s.watchers, key) - } - } - for key, route := range want { - if _, running := s.watchers[key]; running { - continue - } - watchCtx, cancel := context.WithCancel(ctx) - watcher := &remoteSaveRequestWatcher{provider: key, route: route, cancel: cancel} - s.watchers[key] = watcher - go s.run(watchCtx, watcher) - } -} - -func (s *RemoteSaveRequestServer) stopAll() { - s.mu.Lock() - defer s.mu.Unlock() - for key, watcher := range s.watchers { - watcher.cancel() - delete(s.watchers, key) - } -} - -// run watches one KubeProvider's RemoteSaveRequests until its context ends, reconnecting after a failure. -func (s *RemoteSaveRequestServer) run(ctx context.Context, watcher *remoteSaveRequestWatcher) { - log := s.Log.WithValues("kubeProvider", watcher.provider.String()) - log.Info("Serving the RemoteSaveRequests of a KubeProvider") - if s.Manager != nil && s.Manager.FactStreams != nil { - release := s.Manager.FactStreams.Acquire( - queue.FactStreamKeyFor(watcher.route, remoteSaveRequestGVR().GroupResource()), - ) - defer release() - } - for ctx.Err() == nil { - if err := s.session(ctx, log, watcher); err != nil && ctx.Err() == nil { - log.Info("The watch of RemoteSaveRequests ended; reconnecting", "err", err.Error()) - } - select { - case <-ctx.Done(): - case <-time.After(remoteSaveRequestReconnect): - } - } -} - -// session lists the cluster's RemoteSaveRequests and watches them, until the watch ends. -func (s *RemoteSaveRequestServer) session( - ctx context.Context, - log logr.Logger, - watcher *remoteSaveRequestWatcher, -) error { - dc, err := s.clusterClient(ctx, watcher.provider) - if err != nil { - return err - } - resource := dc.Resource(remoteSaveRequestGVR()) - list, err := resource.List(ctx, metav1.ListOptions{}) - if err != nil { - return fmt.Errorf("list RemoteSaveRequests: %w", err) - } - for i := range list.Items { - s.consider(ctx, log, watcher, resource, &list.Items[i]) - } - timeout := int64(remoteSaveRequestRelist / time.Second) - w, err := resource.Watch(ctx, metav1.ListOptions{ - ResourceVersion: list.GetResourceVersion(), AllowWatchBookmarks: true, TimeoutSeconds: &timeout, - }) - if err != nil { - return fmt.Errorf("watch RemoteSaveRequests: %w", err) - } - defer w.Stop() - for { - select { - case <-ctx.Done(): - return nil - case ev, ok := <-w.ResultChan(): - if !ok { - return nil - } - switch ev.Type { - case k8swatch.Added, k8swatch.Modified: - if u, isObject := ev.Object.(*unstructured.Unstructured); isObject { - s.consider(ctx, log, watcher, resource, u) - } - case k8swatch.Error: - return fmt.Errorf("watch RemoteSaveRequests: %w", apierrors.FromObject(ev.Object)) - case k8swatch.Deleted, k8swatch.Bookmark: - } - } - } -} - -func (s *RemoteSaveRequestServer) clusterClient( - ctx context.Context, - provider k8stypes.NamespacedName, -) (dynamic.Interface, error) { - if s.dynamicClient != nil { - return s.dynamicClient(ctx, provider) - } - return s.Manager.ApplierDynamicClient(ctx, SourceClusterID(provider)) -} - -// consider serves a request that has not finished and is not served yet, and deletes one that -// finished longer than the TTL ago. -func (s *RemoteSaveRequestServer) consider( - ctx context.Context, - log logr.Logger, - watcher *remoteSaveRequestWatcher, - resource dynamic.NamespaceableResourceInterface, - u *unstructured.Unstructured, -) { - var request sourcev1alpha1.RemoteSaveRequest - if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, &request); err != nil { - log.Info( - "Cannot read a RemoteSaveRequest", - "remoteSaveRequest", - u.GetNamespace()+"/"+u.GetName(), - "err", - err.Error(), - ) - return - } - if finishedAt, finished := remoteSaveRequestFinished(&request); finished { - if s.TTL > 0 && time.Since(finishedAt) > s.TTL && request.DeletionTimestamp == nil { - if err := resource.Namespace(request.Namespace).Delete(ctx, request.Name, metav1.DeleteOptions{ - Preconditions: &metav1.Preconditions{UID: &request.UID}, - }); err != nil && !apierrors.IsNotFound(err) { - log.V(1). - Info("Cannot delete a finished RemoteSaveRequest", "remoteSaveRequest", request.Namespace+"/"+request.Name, - "err", err.Error()) - } - } - return - } - if request.DeletionTimestamp != nil { - return - } - if _, serving := watcher.serving.LoadOrStore(request.UID, struct{}{}); serving { - return - } - go func() { - defer watcher.serving.Delete(request.UID) - s.serve(ctx, log, watcher, resource.Namespace(request.Namespace), &request) - }() -} - -// remoteSaveRequestFinished reports whether a request has an outcome that changes no more, and since when: -// Ready decided, and, for one that asked to catch up and committed, Applied decided too. An apply -// reported Lost is decided: no worker will ever say more of it. -func remoteSaveRequestFinished(request *sourcev1alpha1.RemoteSaveRequest) (time.Time, bool) { - ready := apimeta.FindStatusCondition(request.Status.Conditions, "Ready") - if ready == nil || ready.Status == metav1.ConditionUnknown { - return time.Time{}, false - } - finishedAt := ready.LastTransitionTime.Time - if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && ready.Reason == saveReasonCommitted { - applied := apimeta.FindStatusCondition(request.Status.Conditions, "Applied") - if applied == nil || (applied.Status == metav1.ConditionUnknown && applied.Reason != saveReasonLost) { - return time.Time{}, false - } - finishedAt = applied.LastTransitionTime.Time - } - return finishedAt, true -} - -// serve takes one request from attribution to its outcome, and to its apply when it asked to catch -// up, writing each step to its status. A request whose Ready is already decided, served again after a -// restart, only has its apply left to follow. -func (s *RemoteSaveRequestServer) serve( - ctx context.Context, log logr.Logger, watcher *remoteSaveRequestWatcher, resource dynamic.ResourceInterface, - request *sourcev1alpha1.RemoteSaveRequest, -) { - ref := request.Spec.GitTargetRef - log = log.WithValues("remoteSaveRequest", request.Namespace+"/"+request.Name, - "gitTarget", ref.Namespace+"/"+ref.Name) - status := request.Status.DeepCopy() - write := func() { - if err := writeRemoteSaveRequestStatus(ctx, resource, request, status); err != nil { - log.Info("Cannot write the RemoteSaveRequest's status", "err", err.Error()) - } - } - if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil && - ready.Status != metav1.ConditionUnknown { - if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && ready.Reason == saveReasonCommitted { - s.followApply(ctx, request, status, remoteSaveRequestAttach(request, status.Author, ""), write) - } - return - } - if err := git.ValidateLiteralCommitMessage(request.Spec.Message); err != nil { - failSave(request, status, saveReasonFailed, err.Error()) - write() - return - } - - author, attribution, refused := s.saveAuthor(ctx, watcher, request, status) - if refused { - setSaveCondition(request, status, "Ready", metav1.ConditionTrue, saveReasonAuthorUnresolved, - "no audit event of this cluster names who created this request, though the cluster sends audit "+ - "events: its event went missing, so it saves nothing. Save again") - write() - log.Info("Refused a RemoteSaveRequest whose author no audit event names, on a cluster that sends them") - return - } - status.Author = author - reason, message, targetUID, err := s.admit(ctx, watcher, request) - if err != nil { - log.Info("Cannot read the RemoteSaveRequest's GitTarget; it is served again on the next listing", - "err", err.Error()) - return - } - if reason != "" { - failSave(request, status, reason, message) - write() - log.Info("Refused a RemoteSaveRequest", "reason", reason) - return - } - - attach := remoteSaveRequestAttach(request, author, attribution) - attach.GitTargetUID = targetUID - setSaveCondition(request, status, "Ready", metav1.ConditionUnknown, saveReasonSaving, - "waiting for the edits to reach a commit window") - write() - - readmit := func() (string, string, string, error) { return s.admit(ctx, watcher, request) } - if !s.finalize(ctx, request, status, attach, readmit) { - return - } - write() - log.Info("RemoteSaveRequest saved", "reason", readyReason(status), "author", status.Author) - - if request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush && readyReason(status) == saveReasonCommitted { - s.followApply(ctx, request, status, attach, write) - } -} - -// saveAuthor is the request's creator, found from the source cluster's audit facts as a watched edit's -// author is. refused reports a request no fact names on a provider that has received audit facts. -// -// A request already being saved was attributed when it was first served, and its status records the -// answer. It is not looked up again: the status writes since have moved its resourceVersion, so its -// audit fact would no longer match, and on an audited cluster it would be refused for that. -func (s *RemoteSaveRequestServer) saveAuthor( - ctx context.Context, watcher *remoteSaveRequestWatcher, request *sourcev1alpha1.RemoteSaveRequest, - status *sourcev1alpha1.RemoteSaveRequestStatus, -) (string, git.AttributionOutcome, bool) { - if apimeta.FindStatusCondition(status.Conditions, "Ready") != nil { - if status.Author != "" { - return status.Author, git.AttributionResolved, false - } - return "", git.AttributionUnresolved, false - } - if s.Manager == nil || s.Manager.AuthorResolver == nil { - return "", git.AttributionNotAttempted, false - } - user, attribution := s.Manager.AuthorResolver.ResolveAuthor(ctx, AuthorQuery{ - AuditRoute: watcher.route, - GVR: remoteSaveRequestGVR(), - UID: request.UID, - ResourceVersion: request.ResourceVersion, - Namespace: request.Namespace, - Labels: request.Labels, - Name: request.Name, - ExactCapable: true, - }) - refused := !attribution.NamesActor() && attribution != git.AttributionNotAttempted && - s.sentAuditFacts(ctx, watcher.provider) - return user.Username, attribution, refused -} - -// sentAuditFacts reports whether the KubeProvider has ever received an audit fact: its -// AuditFactsReceived condition is True. A provider that cannot be read says no. -func (s *RemoteSaveRequestServer) sentAuditFacts(ctx context.Context, provider k8stypes.NamespacedName) bool { - var kp configv1alpha3.KubeProvider - if err := s.Client.Get(ctx, provider, &kp); err != nil { - return false - } - return apimeta.IsStatusConditionTrue(kp.Status.Conditions, "AuditFactsReceived") -} - -// admit decides whether the GitTarget the request names takes it: it exists, mirrors the cluster the -// request was served from, and admits the request's namespace and labels in -// spec.allowedRemoteSaveRequests. A refusal is a reason and a message; otherwise it returns the UID of -// the GitTarget that admitted the request, and err is a read to retry. A GitTarget that does not exist -// is waited for up to the request's attachTimeout, since it may be created alongside the request. -func (s *RemoteSaveRequestServer) admit( - ctx context.Context, watcher *remoteSaveRequestWatcher, request *sourcev1alpha1.RemoteSaveRequest, -) (string, string, string, error) { - ref := request.Spec.GitTargetRef - key := k8stypes.NamespacedName{Namespace: ref.Namespace, Name: ref.Name} - deadline := time.Now().Add(remoteSaveRequestAttach(request, "", "").AttachTimeout) - var target configv1alpha3.GitTarget - for { - err := s.Client.Get(ctx, key, &target) - if err == nil && target.DeletionTimestamp == nil { - break - } - if err != nil && !apierrors.IsNotFound(err) { - return "", "", "", err - } - if !time.Now().Before(deadline) { - return saveReasonGitTargetNotFound, fmt.Sprintf("GitTarget %s does not exist", key), "", nil - } - select { - case <-ctx.Done(): - return "", "", "", ctx.Err() - case <-time.After(remoteSaveRequestPoll): - } - } - if target.KubeProviderKey() != watcher.provider { - return saveReasonNotAllowed, fmt.Sprintf("GitTarget %s mirrors another cluster than this one", key), "", nil - } - admitted, err := target.AdmitsRemoteSaveRequest(request.Namespace, request.Labels) - if !admitted { - message := fmt.Sprintf("GitTarget %s does not admit RemoteSaveRequests from namespace %q, with these "+ - "labels, in spec.allowedRemoteSaveRequests", key, request.Namespace) - if err != nil { - message += fmt.Sprintf(" (a grant's requestSelector is invalid: %v)", err) - } - return saveReasonNotAllowed, message, "", nil - } - return "", "", string(target.UID), nil -} - -// remoteSaveRequestAttach is the attach of a request to its GitTarget's window. -func remoteSaveRequestAttach( - request *sourcev1alpha1.RemoteSaveRequest, author string, attribution git.AttributionOutcome, -) git.AttachCommitRequest { - attach := git.AttachCommitRequest{ - Namespace: request.Namespace, - Name: request.Name, - UID: string(request.UID), - Author: author, - Attribution: attribution, - GitTargetName: request.Spec.GitTargetRef.Name, - GitTargetNamespace: request.Spec.GitTargetRef.Namespace, - Message: request.Spec.Message, - Attach: configv1alpha3.AttachCurrentOrNext, - AttachTimeout: defaultSaveAttachTimeout, - MaxDuration: defaultSaveMaxDuration, - CatchUpNow: request.Spec.CatchUp == sourcev1alpha1.CatchUpAfterPush, - } - if window := request.Spec.Window; window != nil { - if window.AttachTimeout != nil { - attach.AttachTimeout = window.AttachTimeout.Duration - } - if window.MaxDuration != nil { - attach.MaxDuration = window.MaxDuration.Duration - } - if window.IdleTimeout != nil { - idle := window.IdleTimeout.Duration - attach.IdleTimeout = &idle - } - } - return attach -} - -// finalize polls the outcome until the worker resolved the request, withdrawing it once it is past its -// bound and still unheld, and records it on status. It reports false when ctx ended first. -// -// The GitTarget that admitted the request may be deleted while no worker holds it yet. Whatever has -// its name then is asked by readmit, as admit does, before the request is attached to it: a -// replacement admitted nothing, and its grants may not admit the request. -func (s *RemoteSaveRequestServer) finalize( - ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attach git.AttachCommitRequest, readmit func() (string, string, string, error), -) bool { - bound := time.Now().Add(attach.AttachTimeout + attach.MaxDuration + remoteSaveRequestPushAllowance) - for { - ask := s.Finalizer.ServiceCommitRequest - if time.Now().After(bound) { - ask = s.Finalizer.WithdrawCommitRequest - } - result, done, err := ask(ctx, attach) - if err == nil && done { - recordSaveOutcome(request, status, result) - return true - } - if errors.Is(err, errGitTargetGone) { - reason, message, uid, admitErr := readmit() - if reason != "" { - failSave(request, status, reason, message) - return true - } - if admitErr == nil { - attach.GitTargetUID = uid - } - } - select { - case <-ctx.Done(): - return false - case <-time.After(remoteSaveRequestPoll): - } - } -} - -// recordSaveOutcome records the worker's outcome: Ready, the branch and the commit. -func recordSaveOutcome( - request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - result git.FinalizeResult, -) { - status.Branch = result.Branch - if result.Err != nil { - message := result.Err.Error() - if errors.Is(result.Err, git.ErrCommitRequestWithdrawn) { - message = "the save was not served in time and was withdrawn" - } - failSave(request, status, saveReasonFailed, message) - return - } - ready := func(reason, message string) { - setSaveCondition(request, status, "Ready", metav1.ConditionTrue, reason, message) - } - switch result.Outcome { - case git.FinalizeCommitted: - status.Commit = result.Commit - ready(saveReasonCommitted, "the edits were committed and pushed") - case git.FinalizeNoOpenWindow: - ready(saveReasonNoWindow, "no edit of this request's author reached a commit window before attachTimeout") - case git.FinalizeWindowMismatch: - ready(saveReasonWindowMismatch, "only another author's commit window was open") - case git.FinalizeAlreadyPresent: - ready(saveReasonAlreadyPresent, "the edits were in Git already") - case git.FinalizeSaveInProgress: - failSave(request, status, saveReasonSaveInProgress, "an earlier save of the same author on this GitTarget "+ - "was still waiting for or collecting a commit window, which carries one message; save again once it "+ - "resolved") - default: - failSave(request, status, saveReasonFailed, "unexpected outcome: "+string(result.Outcome)) - } -} - -// failSave records a request that ends without saving and cannot recover: Ready=False, Stalled=True. -func failSave( - request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, reason, message string, -) { - setSaveCondition(request, status, "Ready", metav1.ConditionFalse, reason, message) - setSaveCondition(request, status, "Stalled", metav1.ConditionTrue, reason, message) -} - -// followApply polls the apply the committed save asked for until it is settled, and records it on the -// Applied condition; a save its worker knows nothing of for remoteSaveRequestApplyLost is Lost. -func (s *RemoteSaveRequestServer) followApply( - ctx context.Context, request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, - attach git.AttachCommitRequest, write func(), -) { - if apimeta.FindStatusCondition(status.Conditions, "Applied") == nil { - setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonApplying, - "waiting for the applier to apply the commit") - write() - } - started := time.Now() - for { - apply, known, err := s.Finalizer.LookupCommitRequestApply(ctx, attach) - switch { - case err == nil && known && (apply.Done || apply.Applied): - status.AppliedRevision = apply.AppliedRevision - setSaveApplied(request, status, apply.Reason) - write() - return - case err == nil && !known && time.Since(started) > remoteSaveRequestApplyLost: - setSaveCondition(request, status, "Applied", metav1.ConditionUnknown, saveReasonLost, - "GitOps Reverser does not know how the apply went") - write() - return - } - select { - case <-ctx.Done(): - return - case <-time.After(remoteSaveRequestPoll): - } - } -} - -// setSaveApplied records a settled apply: False only for a commit the branch lost, True otherwise. -func setSaveApplied( - request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, reason string, -) { - if reason == git.SaveApplyCommitDropped { - setSaveCondition(request, status, "Applied", metav1.ConditionFalse, reason, - "the branch no longer holds the commit") - return - } - setSaveCondition(request, status, "Applied", metav1.ConditionTrue, reason, - "the applier applied a revision that holds the commit, or needed no apply") -} - -func readyReason(status *sourcev1alpha1.RemoteSaveRequestStatus) string { - if ready := apimeta.FindStatusCondition(status.Conditions, "Ready"); ready != nil { - return ready.Reason - } - return "" -} - -func setSaveCondition( - request *sourcev1alpha1.RemoteSaveRequest, status *sourcev1alpha1.RemoteSaveRequestStatus, conditionType string, - condition metav1.ConditionStatus, reason, message string, -) { - status.ObservedGeneration = request.Generation - apimeta.SetStatusCondition(&status.Conditions, metav1.Condition{ - Type: conditionType, Status: condition, Reason: reason, Message: message, - ObservedGeneration: request.Generation, - }) -} - -// writeRemoteSaveRequestStatus writes status onto the request's latest version, retrying a conflict. -func writeRemoteSaveRequestStatus( - ctx context.Context, resource dynamic.ResourceInterface, request *sourcev1alpha1.RemoteSaveRequest, - status *sourcev1alpha1.RemoteSaveRequestStatus, -) error { - fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(status) - if err != nil { - return err - } - const attempts = 3 - for attempt := 1; ; attempt++ { - current, err := resource.Get(ctx, request.Name, metav1.GetOptions{}) - if err != nil { - return err - } - if current.GetUID() != request.UID { - return fmt.Errorf("RemoteSaveRequest %s/%s was replaced", request.Namespace, request.Name) - } - if err := unstructured.SetNestedMap(current.Object, fields, "status"); err != nil { - return err - } - _, err = resource.UpdateStatus(ctx, current, metav1.UpdateOptions{}) - if err == nil || !apierrors.IsConflict(err) || attempt == attempts { - return err - } - } -} diff --git a/internal/watch/remote_save_requests_test.go b/internal/watch/remote_save_requests_test.go deleted file mode 100644 index b2b48fd6..00000000 --- a/internal/watch/remote_save_requests_test.go +++ /dev/null @@ -1,667 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package watch - -import ( - "context" - "errors" - "sync" - "testing" - "time" - - meta "github.com/fluxcd/pkg/apis/meta" - "github.com/go-logr/logr" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - apierrors "k8s.io/apimachinery/pkg/api/errors" - apimeta "k8s.io/apimachinery/pkg/api/meta" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/runtime/schema" - k8stypes "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/dynamic" - dynamicfake "k8s.io/client-go/dynamic/fake" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/client/fake" - - sourcev1alpha1 "github.com/ConfigButler/gitops-reverser/api/source/v1alpha1" - configv1alpha3 "github.com/ConfigButler/gitops-reverser/api/v1alpha3" - "github.com/ConfigButler/gitops-reverser/internal/git" -) - -// saveFinalizer answers every attach and apply lookup the way the test sets, and records them. -type saveFinalizer struct { - mu sync.Mutex - result git.FinalizeResult - apply git.SaveApply - attaches []git.AttachCommitRequest -} - -func (f *saveFinalizer) ServiceCommitRequest( - _ context.Context, attach git.AttachCommitRequest, -) (git.FinalizeResult, bool, error) { - f.mu.Lock() - defer f.mu.Unlock() - f.attaches = append(f.attaches, attach) - return f.result, true, nil -} - -func (f *saveFinalizer) WithdrawCommitRequest( - _ context.Context, _ git.AttachCommitRequest, -) (git.FinalizeResult, bool, error) { - return git.FinalizeResult{Err: git.ErrCommitRequestWithdrawn}, true, nil -} - -func (f *saveFinalizer) LookupCommitRequestApply( - _ context.Context, _ git.AttachCommitRequest, -) (git.SaveApply, bool, error) { - return f.apply, true, nil -} - -// fixedAuthor resolves every query to one user, or to none. -type fixedAuthor struct { - user git.UserInfo - outcome git.AttributionOutcome -} - -func (a fixedAuthor) ResolveAuthor(context.Context, AuthorQuery) (git.UserInfo, git.AttributionOutcome) { - return a.user, a.outcome -} - -var saveProvider = k8stypes.NamespacedName{Namespace: "providers", Name: "workspace"} - -// saveSetup is what a save is served against: the GitTarget team-a/app, when it exists, its -// KubeProvider and the grants it holds, and what the source cluster's audit says. -type saveSetup struct { - // noTarget leaves team-a/app out. - noTarget bool - // provider is the GitTarget's KubeProvider; empty is providers/workspace, the one serving the request. - provider string - grants []configv1alpha3.GitTargetRemoteSaveRequestGrant - auditFacts metav1.ConditionStatus - author AuthorResolver -} - -// grantFrom admits the RemoteSaveRequests of namespaces. -func grantFrom(namespaces ...configv1alpha3.SourceNamespace) configv1alpha3.GitTargetRemoteSaveRequestGrant { - return configv1alpha3.GitTargetRemoteSaveRequestGrant{SourceNamespaces: namespaces} -} - -// shopSetup is a GitTarget that admits the requests of the namespace "shop". -func shopSetup(auditFacts metav1.ConditionStatus, author AuthorResolver) saveSetup { - return saveSetup{ - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, - auditFacts: auditFacts, - author: author, - } -} - -// saveServer is a server for setup, with request on the source cluster. -func saveServer( - t *testing.T, request *sourcev1alpha1.RemoteSaveRequest, setup saveSetup, finalizer *saveFinalizer, -) (*RemoteSaveRequestServer, dynamic.ResourceInterface) { - t.Helper() - scheme := runtime.NewScheme() - require.NoError(t, configv1alpha3.AddToScheme(scheme)) - provider := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}} - if setup.auditFacts != "" { - provider.Status.Conditions = []metav1.Condition{ - {Type: "AuditFactsReceived", Status: setup.auditFacts, Reason: "Test"}, - } - } - objects := []client.Object{provider} - if !setup.noTarget { - providerName := setup.provider - if providerName == "" { - providerName = "workspace" - } - objects = append(objects, &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a", UID: "uid-app"}, - Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{Name: providerName, Namespace: "providers"}, - AllowedRemoteSaveRequests: setup.grants, - }, - }) - } - c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(objects...). - WithStatusSubresource(&configv1alpha3.KubeProvider{}).Build() - - manager := &Manager{AuthorResolver: setup.author} - fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(request) - require.NoError(t, err) - obj := &unstructured.Unstructured{Object: fields} - obj.SetAPIVersion(sourcev1alpha1.GroupVersion.String()) - obj.SetKind("RemoteSaveRequest") - dc := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), - map[schema.GroupVersionResource]string{remoteSaveRequestGVR(): "RemoteSaveRequestList"}, obj) - server := &RemoteSaveRequestServer{ - Manager: manager, Client: c, Finalizer: finalizer, Log: logr.Discard(), - dynamicClient: func(context.Context, k8stypes.NamespacedName) (dynamic.Interface, error) { return dc, nil }, - } - return server, dc.Resource(remoteSaveRequestGVR()).Namespace("shop") -} - -func newRemoteSaveRequest(catchUp sourcev1alpha1.CatchUpPolicy) *sourcev1alpha1.RemoteSaveRequest { - return &sourcev1alpha1.RemoteSaveRequest{ - ObjectMeta: metav1.ObjectMeta{Name: "save", Namespace: "shop", UID: "uid-save", Generation: 1}, - Spec: sourcev1alpha1.RemoteSaveRequestSpec{ - GitTargetRef: sourcev1alpha1.GitTargetReference{Namespace: "team-a", Name: "app"}, - Message: "save: from the editor", - CatchUp: catchUp, - Window: &sourcev1alpha1.RemoteSaveRequestWindow{ - AttachTimeout: &metav1.Duration{Duration: 3 * time.Second}, - MaxDuration: &metav1.Duration{Duration: time.Second}, - }, - }, - } -} - -// serveOnce serves the request and returns its status as written to the source cluster. -func serveOnce( - t *testing.T, - server *RemoteSaveRequestServer, - resource dynamic.ResourceInterface, - request *sourcev1alpha1.RemoteSaveRequest, -) sourcev1alpha1.RemoteSaveRequestStatus { - t.Helper() - watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} - server.serve(context.Background(), logr.Discard(), watcher, resource, request) - got, err := resource.Get(context.Background(), request.Name, metav1.GetOptions{}) - require.NoError(t, err) - var written sourcev1alpha1.RemoteSaveRequest - require.NoError(t, runtime.DefaultUnstructuredConverter.FromUnstructured(got.Object, &written)) - return written.Status -} - -func condition(status sourcev1alpha1.RemoteSaveRequestStatus, conditionType string) *metav1.Condition { - return apimeta.FindStatusCondition(status.Conditions, conditionType) -} - -// A save attaches to its creator's window on the GitTarget it names, as its author found from the -// source cluster's audit, and reports the commit and the apply it asked for. -func TestRemoteSaveRequest_SavesTheNamedGitTargetAndReportsTheApply(t *testing.T) { - finalizer := &saveFinalizer{ - result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee", Branch: "main"}, - apply: git.SaveApply{ - Done: true, - Applied: true, - Reason: git.SaveApplyApplied, - AppliedRevision: "main@sha1:c0ffee", - }, - } - author := fixedAuthor{user: git.UserInfo{Username: "alice@example.com"}, outcome: git.AttributionResolved} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - server, resource := saveServer(t, request, shopSetup(metav1.ConditionTrue, author), finalizer) - - status := serveOnce(t, server, resource, request) - - require.Len(t, finalizer.attaches, 1) - attach := finalizer.attaches[0] - assert.Equal(t, "alice@example.com", attach.Author) - assert.Equal(t, git.AttributionResolved, attach.Attribution) - assert.Equal(t, "app", attach.GitTargetName) - assert.Equal(t, "team-a", attach.GitTargetNamespace) - assert.Equal(t, "save: from the editor", attach.Message) - assert.Equal(t, 3*time.Second, attach.AttachTimeout) - assert.True(t, attach.CatchUpNow) - - assert.Equal(t, "alice@example.com", status.Author) - assert.Equal(t, "main", status.Branch) - assert.Equal(t, "c0ffee", status.Commit) - assert.Equal(t, "main@sha1:c0ffee", status.AppliedRevision) - require.NotNil(t, condition(status, "Ready")) - assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) - assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) - require.NotNil(t, condition(status, "Applied")) - assert.Equal(t, metav1.ConditionTrue, condition(status, "Applied").Status) - assert.Equal(t, git.SaveApplyApplied, condition(status, "Applied").Reason) -} - -// On a cluster with no audit stream a save names nobody, and attaches as such: to a window whose -// edits name nobody. -func TestRemoteSaveRequest_OnAClusterWithNoAuditItNamesNobody(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow, Branch: "main"}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, - shopSetup(metav1.ConditionUnknown, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) - - status := serveOnce(t, server, resource, request) - - require.Len(t, finalizer.attaches, 1) - assert.Empty(t, finalizer.attaches[0].Author) - assert.Equal(t, git.AttributionUnresolved, finalizer.attaches[0].Attribution) - assert.Equal(t, saveReasonNoWindow, condition(status, "Ready").Reason) - assert.Nil(t, condition(status, "Applied"), "no catch-up was asked for") -} - -// On a cluster that sends audit events, a save no event names lost its fact: it is refused rather -// than attached to the window that should stay unattributed. -func TestRemoteSaveRequest_AnUnresolvedSaveOnAnAuditedClusterIsRefused(t *testing.T) { - finalizer := &saveFinalizer{} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, - shopSetup(metav1.ConditionTrue, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Empty(t, finalizer.attaches, "nothing is attached") - assert.Equal(t, metav1.ConditionTrue, condition(status, "Ready").Status) - assert.Equal(t, saveReasonAuthorUnresolved, condition(status, "Ready").Reason) -} - -// The GitTarget the request names takes it only when it exists, mirrors the cluster the request came -// from, and has a grant admitting both the request's namespace and its labels; nothing is admitted by -// default. -func TestRemoteSaveRequest_TheGitTargetMustAdmitIt(t *testing.T) { - selector := func(labels map[string]string) *metav1.LabelSelector { - return &metav1.LabelSelector{MatchLabels: labels} - } - shopLabelled := func(labels map[string]string) configv1alpha3.GitTargetRemoteSaveRequestGrant { - grant := grantFrom("shop") - grant.RequestSelector = selector(labels) - return grant - } - grants := func(g ...configv1alpha3.GitTargetRemoteSaveRequestGrant) []configv1alpha3.GitTargetRemoteSaveRequestGrant { - return g - } - tests := map[string]struct { - setup saveSetup - reason string - }{ - "no grant": {setup: saveSetup{}, reason: saveReasonNotAllowed}, - "another namespace": { - setup: saveSetup{grants: grants(grantFrom("billing"))}, - reason: saveReasonNotAllowed, - }, - "its namespace": {setup: saveSetup{grants: grants(grantFrom("shop"))}}, - "every namespace": {setup: saveSetup{grants: grants(grantFrom("*"))}}, - "its namespace and labels": { - setup: saveSetup{grants: grants(shopLabelled(map[string]string{"app": "cart"}))}, - }, - "its namespace, other labels": { - setup: saveSetup{grants: grants(shopLabelled(map[string]string{"app": "admin"}))}, - reason: saveReasonNotAllowed, - }, - "one complete grant of two": {setup: saveSetup{grants: grants( - grantFrom("billing"), shopLabelled(map[string]string{"app": "cart"}), - )}}, - "a selector that does not convert admits nothing": {setup: saveSetup{grants: grants( - configv1alpha3.GitTargetRemoteSaveRequestGrant{ - SourceNamespaces: []configv1alpha3.SourceNamespace{"shop"}, - RequestSelector: &metav1.LabelSelector{MatchExpressions: []metav1.LabelSelectorRequirement{ - {Key: "app", Operator: "Near", Values: []string{"cart"}}, - }}, - }, - )}, reason: saveReasonNotAllowed}, - "a GitTarget that mirrors another cluster": { - setup: saveSetup{provider: "elsewhere", grants: grants(grantFrom("*"))}, reason: saveReasonNotAllowed, - }, - "no such GitTarget": {setup: saveSetup{noTarget: true}, reason: saveReasonGitTargetNotFound}, - } - for name, tt := range tests { - t.Run(name, func(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - request.Labels = map[string]string{"app": "cart"} - request.Spec.Window.AttachTimeout = &metav1.Duration{} - server, resource := saveServer(t, request, tt.setup, finalizer) - - status := serveOnce(t, server, resource, request) - - if tt.reason == "" { - assert.Len(t, finalizer.attaches, 1) - assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) - return - } - assert.Empty(t, finalizer.attaches, "nothing is attached") - assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) - assert.Equal(t, tt.reason, condition(status, "Ready").Reason) - assert.Contains(t, condition(status, "Ready").Message, "team-a/app") - assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) - }) - } -} - -// replacingFinalizer recreates the GitTarget team-a/app as replacement the first time a save is sent, -// before any worker holds it, and then answers as the router does: an attach bound to the GitTarget -// that was replaced finds it gone. -type replacingFinalizer struct { - saveFinalizer - - client client.Client - replacement *configv1alpha3.GitTarget -} - -func (f *replacingFinalizer) ServiceCommitRequest( - ctx context.Context, attach git.AttachCommitRequest, -) (git.FinalizeResult, bool, error) { - var current configv1alpha3.GitTarget - if err := f.client.Get(ctx, client.ObjectKey{Namespace: "team-a", Name: "app"}, ¤t); err != nil { - return git.FinalizeResult{}, false, err - } - if attach.GitTargetUID != string(current.UID) { - return git.FinalizeResult{}, false, errGitTargetGone - } - if f.replacement != nil { - if err := f.client.Delete(ctx, ¤t); err != nil { - return git.FinalizeResult{}, false, err - } - if err := f.client.Create(ctx, f.replacement); err != nil { - return git.FinalizeResult{}, false, err - } - f.replacement = nil - return git.FinalizeResult{Phase: git.PhaseWaitingForWorker}, false, nil - } - return f.saveFinalizer.ServiceCommitRequest(ctx, attach) -} - -// A GitTarget recreated while a save waits for its worker admitted nothing: the save is attached to it -// only once its own grants admit the request, and is refused when they do not. -func TestRemoteSaveRequest_AReplacedGitTargetMustAdmitItAgain(t *testing.T) { - tests := map[string]struct { - grants []configv1alpha3.GitTargetRemoteSaveRequestGrant - reason string - }{ - "the replacement admits it": { - grants: []configv1alpha3.GitTargetRemoteSaveRequestGrant{grantFrom("shop")}, - reason: saveReasonCommitted, - }, - "the replacement admits nobody": {reason: saveReasonNotAllowed}, - } - for name, tt := range tests { - t.Run(name, func(t *testing.T) { - finalizer := &replacingFinalizer{ - saveFinalizer: saveFinalizer{ - result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}, - }, - replacement: &configv1alpha3.GitTarget{ - ObjectMeta: metav1.ObjectMeta{Name: "app", Namespace: "team-a", UID: "uid-replacement"}, - Spec: configv1alpha3.GitTargetSpec{ - KubeProviderRef: meta.NamespacedObjectReference{ - Name: "workspace", Namespace: "providers", - }, - AllowedRemoteSaveRequests: tt.grants, - }, - }, - } - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, shopSetup("", nil), &finalizer.saveFinalizer) - finalizer.client = server.Client - server.Finalizer = finalizer - - status := serveOnce(t, server, resource, request) - - assert.Equal(t, tt.reason, condition(status, "Ready").Reason) - if tt.reason == saveReasonNotAllowed { - assert.Empty(t, finalizer.attaches, "nothing is attached through the replacement") - return - } - require.Len(t, finalizer.attaches, 1) - assert.Equal(t, "uid-replacement", finalizer.attaches[0].GitTargetUID) - }) - } -} - -// A save that failed fails the request, and says why. -func TestRemoteSaveRequest_AFailureFailsTheRequest(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Err: errors.New("the GitTarget is suspended")}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - server, resource := saveServer(t, request, shopSetup("", nil), finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Equal(t, metav1.ConditionFalse, condition(status, "Ready").Status) - assert.Equal(t, saveReasonFailed, condition(status, "Ready").Reason) - assert.Contains(t, condition(status, "Ready").Message, "the GitTarget is suspended") - assert.Equal(t, metav1.ConditionTrue, condition(status, "Stalled").Status) - assert.Nil(t, condition(status, "Applied"), "nothing was committed to apply") -} - -// A request served again, after a restart or a dropped watch, keeps what the first serving decided. -// One being saved keeps its author: its status writes moved its resourceVersion, so its audit fact can -// no longer be found. One already committed keeps its commit, and only its apply is followed. -func TestRemoteSaveRequest_ServedAgainKeepsWhatItDecided(t *testing.T) { - t.Run("one being saved keeps its author", func(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - request.Status.Author = "alice@example.com" - request.Status.Conditions = []metav1.Condition{ - {Type: "Ready", Status: metav1.ConditionUnknown, Reason: saveReasonSaving}, - } - server, resource := saveServer(t, request, - shopSetup(metav1.ConditionTrue, fixedAuthor{outcome: git.AttributionUnresolved}), finalizer) - - status := serveOnce(t, server, resource, request) - - require.Len(t, finalizer.attaches, 1, "it is not refused for a fact it can no longer match") - assert.Equal(t, "alice@example.com", finalizer.attaches[0].Author) - assert.Equal(t, git.AttributionResolved, finalizer.attaches[0].Attribution) - assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) - }) - - t.Run("one committed keeps its commit and follows its apply", func(t *testing.T) { - finalizer := &saveFinalizer{ - result: git.FinalizeResult{Outcome: git.FinalizeNoOpenWindow}, - apply: git.SaveApply{Done: true, Reason: git.SaveApplyNothingToApply}, - } - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - request.Status.Commit = "c0ffee" - request.Status.Conditions = []metav1.Condition{ - {Type: "Ready", Status: metav1.ConditionTrue, Reason: saveReasonCommitted}, - {Type: "Applied", Status: metav1.ConditionUnknown, Reason: saveReasonApplying}, - } - server, resource := saveServer(t, request, shopSetup("", nil), finalizer) - - status := serveOnce(t, server, resource, request) - - assert.Empty(t, finalizer.attaches, "a decided save is never attached again") - assert.Equal(t, "c0ffee", status.Commit) - assert.Equal(t, saveReasonCommitted, condition(status, "Ready").Reason) - assert.Equal(t, metav1.ConditionTrue, condition(status, "Applied").Status) - assert.Equal(t, git.SaveApplyNothingToApply, condition(status, "Applied").Reason) - }) -} - -// A request is finished once Ready is decided, and, when it committed and asked to catch up, once -// Applied is decided too. -func TestRemoteSaveRequest_Finished(t *testing.T) { - at := metav1.NewTime(time.Now().Add(-time.Hour)) - ready := func(status metav1.ConditionStatus, reason string) metav1.Condition { - return metav1.Condition{Type: "Ready", Status: status, Reason: reason, LastTransitionTime: at} - } - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - _, finished := remoteSaveRequestFinished(request) - assert.False(t, finished, "no Ready yet") - - request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionUnknown, saveReasonSaving)} - _, finished = remoteSaveRequestFinished(request) - assert.False(t, finished, "still saving") - - request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonCommitted)} - _, finished = remoteSaveRequestFinished(request) - assert.False(t, finished, "committed, its apply not decided") - - request.Status.Conditions = append( - request.Status.Conditions, - metav1.Condition{ - Type: "Applied", - Status: metav1.ConditionTrue, - Reason: git.SaveApplyApplied, - LastTransitionTime: at, - }, - ) - finishedAt, finished := remoteSaveRequestFinished(request) - assert.True(t, finished) - assert.Equal(t, at.Time, finishedAt) - - applied := func(reason string) []metav1.Condition { - return []metav1.Condition{ready(metav1.ConditionTrue, saveReasonCommitted), { - Type: "Applied", Status: metav1.ConditionUnknown, Reason: reason, LastTransitionTime: at, - }} - } - request.Status.Conditions = applied(saveReasonApplying) - _, finished = remoteSaveRequestFinished(request) - assert.False(t, finished, "its apply is still followed") - request.Status.Conditions = applied(saveReasonLost) - finishedAt, finished = remoteSaveRequestFinished(request) - assert.True(t, finished, "an apply no worker knows of any more is decided") - assert.Equal(t, at.Time, finishedAt) - - request.Spec.CatchUp = sourcev1alpha1.CatchUpAsDefinedOnClaim - request.Status.Conditions = []metav1.Condition{ready(metav1.ConditionTrue, saveReasonNoWindow)} - _, finished = remoteSaveRequestFinished(request) - assert.True(t, finished) -} - -// The server serves the KubeProviders that enable RemoteSaveRequests, and stops serving one that no longer -// does. -func TestRemoteSaveRequest_ServesOnlyTheKubeProvidersThatEnableThem(t *testing.T) { - scheme := runtime.NewScheme() - require.NoError(t, configv1alpha3.AddToScheme(scheme)) - enabled := &configv1alpha3.KubeProvider{ - ObjectMeta: metav1.ObjectMeta{Name: "workspace", Namespace: "providers"}, - Spec: configv1alpha3.KubeProviderSpec{RemoteSaveRequests: configv1alpha3.RemoteSaveRequestsEnabled}, - } - disabled := &configv1alpha3.KubeProvider{ObjectMeta: metav1.ObjectMeta{Name: "other", Namespace: "providers"}} - c := fake.NewClientBuilder().WithScheme(scheme).WithObjects(enabled, disabled).Build() - server := &RemoteSaveRequestServer{ - Client: c, Log: logr.Discard(), - dynamicClient: func(context.Context, k8stypes.NamespacedName) (dynamic.Interface, error) { - return nil, errors.New("no cluster in this test") - }, - } - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() - - server.syncProviders(ctx) - assert.Len(t, server.watchers, 1) - assert.Contains(t, server.watchers, saveProvider) - - enabled.Spec.RemoteSaveRequests = configv1alpha3.RemoteSaveRequestsDisabled - require.NoError(t, c.Update(ctx, enabled)) - server.syncProviders(ctx) - assert.Empty(t, server.watchers) - server.stopAll() -} - -// A session serves every request it lists, and every one its watch delivers, writing each outcome -// back to the source cluster. -func TestRemoteSaveRequest_ASessionServesWhatItListsAndWatches(t *testing.T) { - finalizer := &saveFinalizer{result: git.FinalizeResult{Outcome: git.FinalizeCommitted, Commit: "c0ffee"}} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - server, resource := saveServer(t, request, shopSetup("", nil), finalizer) - watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} - readyReasonOf := func(name string) string { - got, err := resource.Get(context.Background(), name, metav1.GetOptions{}) - if err != nil { - return "" - } - var written sourcev1alpha1.RemoteSaveRequest - if runtime.DefaultUnstructuredConverter.FromUnstructured(got.Object, &written) != nil { - return "" - } - return readyReason(&written.Status) - } - - ctx, cancel := context.WithCancel(context.Background()) - done := make(chan error, 1) - go func() { done <- server.session(ctx, logr.Discard(), watcher) }() - - assert.Eventually(t, func() bool { return readyReasonOf("save") == saveReasonCommitted }, - 5*time.Second, 20*time.Millisecond, "the listed request is served") - - later := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - later.Name, later.UID = "later", "uid-later" - fields, err := runtime.DefaultUnstructuredConverter.ToUnstructured(later) - require.NoError(t, err) - obj := &unstructured.Unstructured{Object: fields} - obj.SetAPIVersion(sourcev1alpha1.GroupVersion.String()) - obj.SetKind("RemoteSaveRequest") - _, err = resource.Create(context.Background(), obj, metav1.CreateOptions{}) - require.NoError(t, err) - assert.Eventually(t, func() bool { return readyReasonOf("later") == saveReasonCommitted }, - 5*time.Second, 20*time.Millisecond, "the watched request is served") - - cancel() - require.NoError(t, <-done) -} - -// A finished request is deleted once it is older than the TTL, and kept before that or without one; -// it is never served again. -func TestRemoteSaveRequest_AFinishedRequestIsDeletedAfterTheTTL(t *testing.T) { - for name, tt := range map[string]struct { - ttl time.Duration - deleted bool - }{ - "past the TTL": {ttl: time.Minute, deleted: true}, - "within the TTL": {ttl: 2 * time.Hour}, - "no TTL": {}, - } { - t.Run(name, func(t *testing.T) { - finalizer := &saveFinalizer{} - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - request.Status.Conditions = []metav1.Condition{{ - Type: "Ready", Status: metav1.ConditionTrue, Reason: saveReasonNoWindow, - LastTransitionTime: metav1.NewTime(time.Now().Add(-time.Hour)), - }} - server, resource := saveServer(t, request, shopSetup("", nil), finalizer) - server.TTL = tt.ttl - watcher := &remoteSaveRequestWatcher{provider: saveProvider, route: "providers.workspace"} - dc, err := server.clusterClient(context.Background(), saveProvider) - require.NoError(t, err) - current, err := resource.Get(context.Background(), request.Name, metav1.GetOptions{}) - require.NoError(t, err) - - server.consider(context.Background(), logr.Discard(), watcher, dc.Resource(remoteSaveRequestGVR()), current) - - _, err = resource.Get(context.Background(), request.Name, metav1.GetOptions{}) - assert.Equal(t, tt.deleted, apierrors.IsNotFound(err), "deleted") - assert.Empty(t, finalizer.attaches, "a finished request is never served again") - }) - } -} - -// Each worker outcome is recorded with the reason the API documents, and the refusals and failures -// are Ready=False, Stalled=True. -func TestRemoteSaveRequest_RecordsEachOutcome(t *testing.T) { - for result, want := range map[*git.FinalizeResult]struct { - reason string - ready metav1.ConditionStatus - }{ - {Outcome: git.FinalizeCommitted, Commit: "c0ffee"}: {saveReasonCommitted, metav1.ConditionTrue}, - {Outcome: git.FinalizeNoOpenWindow}: {saveReasonNoWindow, metav1.ConditionTrue}, - {Outcome: git.FinalizeWindowMismatch}: {saveReasonWindowMismatch, metav1.ConditionTrue}, - {Outcome: git.FinalizeAlreadyPresent}: {saveReasonAlreadyPresent, metav1.ConditionTrue}, - {Outcome: git.FinalizeSaveInProgress}: {saveReasonSaveInProgress, metav1.ConditionFalse}, - {Outcome: "Unknown"}: {saveReasonFailed, metav1.ConditionFalse}, - {Err: errors.New("boom")}: {saveReasonFailed, metav1.ConditionFalse}, - {Err: git.ErrCommitRequestWithdrawn}: {saveReasonFailed, metav1.ConditionFalse}, - } { - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAsDefinedOnClaim) - status := &sourcev1alpha1.RemoteSaveRequestStatus{} - recordSaveOutcome(request, status, *result) - ready := condition(*status, "Ready") - require.NotNil(t, ready) - assert.Equal(t, want.reason, ready.Reason, "%+v", *result) - assert.Equal(t, want.ready, ready.Status, "%+v", *result) - assert.NotEmpty(t, ready.Message) - assert.Equal(t, want.ready == metav1.ConditionFalse, condition(*status, "Stalled") != nil, "%+v", *result) - } -} - -// Applied is False only for a commit the branch lost, and True for every other settled apply. -func TestRemoteSaveRequest_AppliedIsFalseOnlyForADroppedCommit(t *testing.T) { - for reason, want := range map[string]metav1.ConditionStatus{ - git.SaveApplyApplied: metav1.ConditionTrue, - git.SaveApplyNothingToApply: metav1.ConditionTrue, - git.SaveApplyNoApplier: metav1.ConditionTrue, - git.SaveApplyCommitDropped: metav1.ConditionFalse, - } { - request := newRemoteSaveRequest(sourcev1alpha1.CatchUpAfterPush) - status := &sourcev1alpha1.RemoteSaveRequestStatus{} - setSaveApplied(request, status, reason) - assert.Equal(t, want, condition(*status, "Applied").Status, reason) - assert.Equal(t, reason, condition(*status, "Applied").Reason) - } -} diff --git a/internal/watch/resource_policy.go b/internal/watch/resource_policy.go index 06cc1619..7df7f285 100644 --- a/internal/watch/resource_policy.go +++ b/internal/watch/resource_policy.go @@ -24,8 +24,9 @@ func isDefaultResourceExcluded(group, resource string) bool { groupResourceKey("flowcontrol.apiserver.k8s.io", "prioritylevelconfigurations"), groupResourceKey("batch", "jobs"), groupResourceKey("batch", "cronjobs"), - // A save is a request to GitOps Reverser, never state to mirror (commit_requests.go). - groupResourceKey("source.configbutler.ai", "remotesaverequests"): + // A save is a request to GitOps Reverser, never state to mirror. A CommitRequest may sit in a + // namespace a WatchRule mirrors, on a source cluster as on the control cluster. + groupResourceKey("configbutler.ai", "commitrequests"): return true default: return false diff --git a/test/e2e/source_cluster_e2e_test.go b/test/e2e/source_cluster_e2e_test.go index 32b53790..9888f641 100644 --- a/test/e2e/source_cluster_e2e_test.go +++ b/test/e2e/source_cluster_e2e_test.go @@ -37,7 +37,7 @@ import ( // install's own provider. It shows the startup converge landing in the workspace, an edit made // there withheld from the apply's revert by the REMOTE object's managedFields, two // CommitRequests that take over their edits' windows (the workspace sends no audit facts) and -// report the apply, and a RemoteSaveRequest created in the workspace itself. +// report the apply, and a CommitRequest created in the workspace itself. // // The whole suite is gated by skipUnlessSourceClusterEnabled() (env E2E_ENABLE_SOURCE_CLUSTER), // and the kcp specs additionally Skip when kcp is not installed — so a default `task test-e2e` @@ -803,114 +803,64 @@ spec: "the apply over the workspace edit is counted as a withheld collision", time.Minute) }) - // A save on the workspace's GitTarget that asks to catch up is applied once it is pushed, and - // each request reports the apply: Applied=True, with the revision Flux applied. The second save - // is made as soon as the first is pushed, so the two may share one apply. - // - // No write in the workspace names its author, since kcp sends no audit facts, and neither does - // the request, so each request attaches to the window of its edit. Every event from the workspace - // waits out the attribution grace first, so the requests wait longer than that for their window. - It("saves an edit in the workspace, and reports the apply on each of two saves", func() { - names := []string{"sc-flux-save-1", "sc-flux-save-2"} - commits := map[string]string{} - for i, name := range names { - value := fmt.Sprintf("saved-%d", i+1) - applyCatchUpSave(testNs, name, p.targetName) - r.editThroughAPI(timingConfigMap, value) - Eventually(func(g Gomega) { - g.Expect(commitRequestCondition(g, testNs, name, "Ready")).To(Equal("True")) - g.Expect(commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(Equal(commitRequestReasonCommitted)) - commits[name] = commitRequestField(g, testNs, name, "{.status.commit}") - g.Expect(commits[name]).NotTo(BeEmpty()) - }, 2*time.Minute, 2*time.Second).Should(Succeed()) - committed, inGit := r.committedValue(Default, timingConfigMap) - Expect(inGit).To(BeTrue()) - Expect(committed).To(Equal(value), "the save's commit holds its edit") - } - - last := commits[names[len(names)-1]] + // A save made in the workspace itself, next to the edit: a CommitRequest created there, served + // once the workspace's KubeProvider reconciles them and the CRD is installed. Before that, a + // request on this cluster for the GitTarget is refused, since the GitTarget mirrors the workspace. + // The GitTarget must admit the request by the workspace namespace: one is refused while it admits + // none. kcp sends no audit facts, so the request names nobody, like the edit, and attaches to the + // edit's window under the ordinary rule. Its own author is looked for first, for the whole + // attribution grace, so the edit is made once the request is waiting for a window. + It("saves an edit made in the workspace with a CommitRequest created there", func() { + By("refusing a CommitRequest on this cluster for a GitTarget that mirrors the workspace") + const here = "sc-flux-save-here" + applyCommitRequest(testNs, here, p.targetName, "e2e: made on the wrong cluster") Eventually(func(g Gomega) { - for _, name := range names { - applied := commitRequestCondition(g, testNs, name, "Applied") - reason := commitRequestField(g, testNs, name, `{.status.conditions[?(@.type=="Applied")].reason}`) - if applied == "False" { - StopTrying(fmt.Sprintf("CommitRequest %s is Applied=False (%s): only a commit the branch "+ - "lost may be", name, reason)).Now() - } - g.Expect(applied).To(Equal("True"), "CommitRequest %s Applied (%s)", name, reason) - g.Expect(reason).To(Equal("Applied")) - // A later save's commit holds the earlier one's, so one apply may settle both. - g.Expect(commitRequestField(g, testNs, name, "{.status.appliedRevision}")). - To(Or(ContainSubstring(commits[name]), ContainSubstring(last))) - } - g.Expect(commitRequestField(g, testNs, names[len(names)-1], "{.status.appliedRevision}")). - To(ContainSubstring(last)) - g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) - }, echoApplyBudget, 2*time.Second).Should(Succeed()) - r.eventuallyEverywhere(timingConfigMap, "saved-2", echoApplyBudget) - }) + g.Expect(commitRequestCondition(g, testNs, here, "Ready")).To(Equal("False")) + g.Expect(commitRequestField(g, testNs, here, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal("NotAllowed")) + g.Expect(commitRequestField(g, testNs, here, `{.status.conditions[?(@.type=="Ready")].message}`)). + To(ContainSubstring("create the CommitRequest there")) + }, 2*time.Minute, time.Second).Should(Succeed()) - // A save made in the workspace itself: a RemoteSaveRequest, served because the workspace's - // KubeProvider enables them. It names the GitTarget, which must admit it: one is refused while - // the GitTarget admits none. kcp sends no audit facts, so the request names nobody, like the - // edit, and attaches to the edit's window under the ordinary rule. Its own author is - // looked for first, for the whole attribution grace, so the edit is made once the request is - // waiting for a window. - It("saves an edit made in the workspace with a RemoteSaveRequest created there", func() { - By("installing the RemoteSaveRequest CRD in the workspace, and serving the workspace's RemoteSaveRequests") - _, err := kcp.wsKubectl( - hash, - "apply", - "-f", - "config/crd/bases/source.configbutler.ai_remotesaverequests.yaml", - ) + By("installing the CommitRequest CRD in the workspace, and reconciling the workspace's CommitRequests") + _, err := kcp.wsKubectl(hash, "apply", "--server-side", "-f", + "config/crd/bases/configbutler.ai_commitrequests.yaml") Expect(err).NotTo(HaveOccurred()) _, err = kcp.wsKubectl(hash, "wait", "--for=condition=Established", - "crd/remotesaverequests.source.configbutler.ai", "--timeout=60s") + "crd/commitrequests.configbutler.ai", "--timeout=60s") Expect(err).NotTo(HaveOccurred()) _, err = kubectlRunInNamespace(testNs, "patch", "kubeprovider", ws, "--type=merge", "-p", - `{"spec":{"remoteSaveRequests":"Enabled"}}`) + `{"spec":{"commitRequests":"ReconcileAndDeleteFinished"}}`) Expect(err).NotTo(HaveOccurred()) - requestField := func(g Gomega, name, jsonPath string) string { - out, getErr := kcp.wsKubectl( - hash, - "-n", - testNs, - "get", - "remotesaverequest", - name, - "-o", - "jsonpath="+jsonPath, - ) - g.Expect(getErr).NotTo(HaveOccurred()) - return strings.TrimSpace(out) - } + verifyResourceCondition("kubeprovider", ws, testNs, "CommitRequestsServed", "True", "Watching", "", "120s") - By("refusing a RemoteSaveRequest while the GitTarget admits none") + By("refusing a CommitRequest while the GitTarget admits none from the workspace") const refused = "sc-flux-workspace-save-refused" - createWorkspaceRemoteSaveRequest(kcp, hash, testNs, refused, "e2e: not admitted", testNs, p.targetName) + createWorkspaceCommitRequest(kcp, hash, testNs, refused, "e2e: not admitted", testNs, p.targetName, "30s") Eventually(func(g Gomega) { - g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("False")) - g.Expect(requestField(g, refused, `{.status.conditions[?(@.type=="Ready")].reason}`)). - To(Equal("NotAllowed")) + g.Expect(workspaceCommitRequestField(g, kcp, hash, testNs, refused, + `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("False")) + g.Expect(workspaceCommitRequestField(g, kcp, hash, testNs, refused, + `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("NotAllowed")) }, 2*time.Minute, time.Second).Should(Succeed()) - By("admitting the workspace namespace's RemoteSaveRequests on the GitTarget") + By("admitting the workspace namespace's CommitRequests on the GitTarget") _, err = kubectlRunInNamespace(testNs, "patch", "gittarget", p.targetName, "--type=merge", "-p", - fmt.Sprintf(`{"spec":{"allowedRemoteSaveRequests":[{"sourceNamespaces":[%q]}]}}`, testNs)) + fmt.Sprintf(`{"spec":{"allowedCommitRequests":[{"sourceNamespaces":[%q]}]}}`, testNs)) Expect(err).NotTo(HaveOccurred()) - By("creating a RemoteSaveRequest in the workspace, and editing once it waits for a window") + By("creating a CommitRequest in the workspace, and editing once it waits for a window") const name, message = "sc-flux-workspace-save", "e2e: save from the workspace" - createWorkspaceRemoteSaveRequest(kcp, hash, testNs, name, message, testNs, p.targetName) - saveField := func(g Gomega, jsonPath string) string { return requestField(g, name, jsonPath) } + createWorkspaceCommitRequest(kcp, hash, testNs, name, message, testNs, p.targetName, "30s") + saveField := func(g Gomega, jsonPath string) string { + return workspaceCommitRequestField(g, kcp, hash, testNs, name, jsonPath) + } Eventually(func(g Gomega) { - g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].reason}`)).To(Equal("Saving")) - }, 2*time.Minute, time.Second).Should(Succeed(), "the server picks the request up and waits for a window") - r.editThroughAPI(timingConfigMap, "saved-3") + g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("Unknown")) + }, 2*time.Minute, time.Second).Should(Succeed(), "the request is claimed and waits for a window") + r.editThroughAPI(timingConfigMap, "saved-1") - By("the save commits the edit, and reports the commit and the apply on the request") + By("the save commits the edit, and reports the commit, the apply and its deletion time on the request") var commit string Eventually(func(g Gomega) { g.Expect(saveField(g, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("True")) @@ -918,10 +868,12 @@ spec: commit = saveField(g, "{.status.commit}") g.Expect(commit).NotTo(BeEmpty()) g.Expect(saveField(g, "{.status.author}")).To(BeEmpty(), "kcp sends no audit facts") + g.Expect(saveField(g, `{.metadata.annotations.configbutler\.ai/delete-after}`)).NotTo(BeEmpty(), + "ReconcileAndDeleteFinished stamps a finished request for deletion") }, 2*time.Minute, 2*time.Second).Should(Succeed()) committed, inGit := r.committedValue(Default, timingConfigMap) Expect(inGit).To(BeTrue()) - Expect(committed).To(Equal("saved-3")) + Expect(committed).To(Equal("saved-1")) body, err := gitRun(p.repo.CheckoutDir, "show", "-s", "--format=%B", commit) Expect(err).NotTo(HaveOccurred()) Expect(strings.TrimSpace(body)).To(Equal(message)) @@ -932,38 +884,86 @@ spec: g.Expect(saveField(g, "{.status.appliedRevision}")).To(ContainSubstring(commit)) g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(commit)) }, echoApplyBudget, 2*time.Second).Should(Succeed()) - r.consistentlyEverywhere(timingConfigMap, "saved-3") + r.consistentlyEverywhere(timingConfigMap, "saved-1") + }) + + // Saves on the workspace's GitTarget that ask to catch up are applied once they are pushed, and + // each request reports the apply: Applied=True, with the revision Flux applied. The second save + // is made as soon as the first is pushed, so the two may share one apply. Every event from the + // workspace waits out the attribution grace first, so the requests wait longer than that for their + // window. + It("saves edits in the workspace, and reports the apply on each of two saves", func() { + names := []string{"sc-flux-save-1", "sc-flux-save-2"} + field := func(g Gomega, name, jsonPath string) string { + return workspaceCommitRequestField(g, kcp, hash, testNs, name, jsonPath) + } + commits := map[string]string{} + for i, name := range names { + value := fmt.Sprintf("saved-%d", i+2) + createWorkspaceCommitRequest(kcp, hash, testNs, name, "e2e: save "+name+" and apply it", testNs, + p.targetName, "45s") + r.editThroughAPI(timingConfigMap, value) + Eventually(func(g Gomega) { + g.Expect(field(g, name, `{.status.conditions[?(@.type=="Ready")].status}`)).To(Equal("True")) + g.Expect(field(g, name, `{.status.conditions[?(@.type=="Ready")].reason}`)). + To(Equal(commitRequestReasonCommitted)) + commits[name] = field(g, name, "{.status.commit}") + g.Expect(commits[name]).NotTo(BeEmpty()) + }, 2*time.Minute, 2*time.Second).Should(Succeed()) + committed, inGit := r.committedValue(Default, timingConfigMap) + Expect(inGit).To(BeTrue()) + Expect(committed).To(Equal(value), "the save's commit holds its edit") + } + + last := commits[names[len(names)-1]] + Eventually(func(g Gomega) { + for _, name := range names { + applied := field(g, name, `{.status.conditions[?(@.type=="Applied")].status}`) + reason := field(g, name, `{.status.conditions[?(@.type=="Applied")].reason}`) + if applied == "False" { + StopTrying(fmt.Sprintf("CommitRequest %s is Applied=False (%s): only a commit the branch "+ + "lost may be", name, reason)).Now() + } + g.Expect(applied).To(Equal("True"), "CommitRequest %s Applied (%s)", name, reason) + g.Expect(reason).To(Equal("Applied")) + // A later save's commit holds the earlier one's, so one apply may settle both. + g.Expect(field(g, name, "{.status.appliedRevision}")). + To(Or(ContainSubstring(commits[name]), ContainSubstring(last))) + } + g.Expect(field(g, names[len(names)-1], "{.status.appliedRevision}")).To(ContainSubstring(last)) + g.Expect(fluxLastAppliedRevision(g, ksName)).To(ContainSubstring(last)) + }, echoApplyBudget, 2*time.Second).Should(Succeed()) + r.eventuallyEverywhere(timingConfigMap, "saved-3", echoApplyBudget) }) }) }) -// createWorkspaceRemoteSaveRequest creates the RemoteSaveRequest name in namespace of a workspace, saving the -// GitTarget targetNS/target and asking to catch up. It waits up to 30s for a window: an edit from a cluster with no audit stream reaches its window -// only after the attribution grace. -func createWorkspaceRemoteSaveRequest(kcp *kcpTunnel, hash, namespace, name, message, targetNS, target string) { +// createWorkspaceCommitRequest creates the CommitRequest name in namespace of a workspace, saving the +// GitTarget targetNS/target and asking to catch up, as JSON. It waits up to attachTimeout for a window: +// an edit from a cluster with no audit stream reaches its window only after the attribution grace. +func createWorkspaceCommitRequest( + kcp *kcpTunnel, hash, namespace, name, message, targetNS, target, attachTimeout string, +) { GinkgoHelper() - f, err := os.CreateTemp("", "e2e-remotesaverequest-*.yaml") + f, err := os.CreateTemp("", "e2e-commitrequest-*.json") Expect(err).NotTo(HaveOccurred()) defer func() { _ = os.Remove(f.Name()) }() - _, err = fmt.Fprintf(f, `apiVersion: source.configbutler.ai/v1alpha1 -kind: RemoteSaveRequest -metadata: - name: %s - namespace: %s -spec: - gitTargetRef: - namespace: %s - name: %s - message: %q - catchUp: AfterPush - window: - attachTimeout: "30s" - maxDuration: "4s" -`, name, namespace, targetNS, target, message) + _, err = fmt.Fprintf(f, `{"apiVersion": "configbutler.ai/v1alpha3", "kind": "CommitRequest", +"metadata": {"name": %q, "namespace": %q}, +"spec": {"gitTargetRef": {"namespace": %q, "name": %q}, "message": %q, "catchUp": "AfterPush", +"window": {"attachTimeout": %q, "maxDuration": "4s"}}} +`, name, namespace, targetNS, target, message, attachTimeout) Expect(err).NotTo(HaveOccurred()) Expect(f.Close()).To(Succeed()) _, err = kcp.wsKubectl(hash, "create", "-f", f.Name()) - Expect(err).NotTo(HaveOccurred(), "failed to create RemoteSaveRequest %s/%s in the workspace", namespace, name) + Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s/%s in the workspace", namespace, name) +} + +// workspaceCommitRequestField reads a jsonpath field off a CommitRequest in a workspace. +func workspaceCommitRequestField(g Gomega, kcp *kcpTunnel, hash, namespace, name, jsonPath string) string { + out, err := kcp.wsKubectl(hash, "-n", namespace, "get", "commitrequest", name, "-o", "jsonpath="+jsonPath) + g.Expect(err).NotTo(HaveOccurred()) + return strings.TrimSpace(out) } // applyFluxIntoWorkspace creates, in flux-system, a GitRepository following main and a Kustomization @@ -1013,25 +1013,3 @@ spec: g.Expect(status).To(Equal("True"), "Kustomization %s is not Ready: %s", name, message) }, 3*time.Minute, 2*time.Second).Should(Succeed(), "the Kustomization must apply its folder into the workspace") } - -// applyCatchUpSave creates the CommitRequest name on gitTarget, asking its applier to catch up once -// it is pushed. It waits 45s for its window: a write from a cluster with no audit stream reaches the -// window only after the attribution grace. -func applyCatchUpSave(namespace, name, gitTarget string) { - GinkgoHelper() - _, err := kubectlRunWithStdin(namespace, fmt.Sprintf(`apiVersion: configbutler.ai/v1alpha3 -kind: CommitRequest -metadata: - name: %s - namespace: %s -spec: - gitTargetRef: - name: %s - message: "e2e: save %s and apply it" - catchUp: AfterPush - window: - attachTimeout: "45s" - maxDuration: "4s" -`, name, namespace, gitTarget, name), "apply", "-f", "-") - Expect(err).NotTo(HaveOccurred(), "failed to create CommitRequest %s", name) -} From c02b50665a12979cef4e0e050a6650c3923b5bfe Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:17:33 +0000 Subject: [PATCH 27/33] docs: one CommitRequest, served here or on a source cluster; cert-manager only for attribution The chart renders its self-signed Issuer only when it mints a certificate: the audit ingress certificates with attribution, or the metrics certificate with metrics TLS. The default install renders no cert-manager object, so it needs no cert-manager, and the quickstart moves the cert-manager step under named commit authors. The CommitRequest design spec, the status conditions guide, the architecture, security model, components, definitions and chart README describe one kind, picked up on the control cluster or on a source cluster and served by one core, with its author from audit and no admission webhook. The attribution guide asks the audit policy to keep covering commitrequests creates. UPGRADING records what changed since 0.54.0, and commitrequest-crd.yaml is listed among the release assets. Co-Authored-By: Claude Opus 5.5 --- .coverage-baseline | 2 +- .github/RELEASES.md | 3 +- AGENTS.md | 6 +- README.md | 2 +- SECURITY.md | 2 +- charts/gitops-reverser/README.md | 21 +-- charts/gitops-reverser/templates/_helpers.tpl | 18 +++ .../templates/audit-certificates.yaml | 2 +- .../templates/certificates.yaml | 8 +- charts/gitops-reverser/values.yaml | 3 + cmd/main_chart_certificates_test.go | 39 +++++ config/samples/commitrequest.yaml | 12 +- docs/UPGRADING.md | 33 ++++ docs/architecture.md | 144 +++++++++--------- docs/attribution-setup-guide.md | 9 +- docs/components.md | 6 +- docs/definitions.md | 5 +- docs/future/source-side-save-plan.md | 18 +-- docs/interpreting-metrics.md | 2 +- docs/quickstart.md | 39 +++-- docs/security-model.md | 12 +- docs/spec/commitrequest-design.md | 143 ++++++++++++++--- docs/spec/status-conditions-guide.md | 42 +++-- test/e2e/cluster/audit/policy.yaml | 8 +- test/e2e/quickstart_framework_e2e_test.go | 2 +- 25 files changed, 391 insertions(+), 190 deletions(-) create mode 100644 cmd/main_chart_certificates_test.go diff --git a/.coverage-baseline b/.coverage-baseline index b0184ecd..a03b4942 100644 --- a/.coverage-baseline +++ b/.coverage-baseline @@ -1 +1 @@ -80.6 +82.5 diff --git a/.github/RELEASES.md b/.github/RELEASES.md index d4453592..b11b51c5 100644 --- a/.github/RELEASES.md +++ b/.github/RELEASES.md @@ -88,7 +88,8 @@ Push to main 4. **When Release PR is Merged**: - GitHub Release created with tag (e.g., `v0.2.0`) — first as a **draft**, so every signed - asset (`crds.yaml`, `install.yaml`, SBOM, `.sigstore.json` signatures, `.intoto.jsonl` attestations) + asset (`crds.yaml`, `commitrequest-crd.yaml`, `install.yaml`, SBOM, `.sigstore.json` signatures, + `.intoto.jsonl` attestations) can be attached before it goes public; immutable releases reject post-publish uploads, so `publish-release` flips the draft to published only after every asset is in place. - The linux/amd64 + linux/arm64 image digests already built and scanned by that diff --git a/AGENTS.md b/AGENTS.md index d3f6af75..7a66c73f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -96,9 +96,9 @@ See [Documentation checks](./CONTRIBUTING.md#documentation-checks). ### Webhook Code (`internal/webhook/`) -- Implement admission webhook interface correctly -- Add proper validation/mutation logic -- Update webhook configuration in `config/webhook/` +- It is the audit ingress and the push receiver: HTTP handlers for what kube-apiserver and Git + hosts post. GitOps Reverser ships no admission webhook; validation lives in the schema, CEL and + the reconciler (`docs/spec/where-validation-lives.md`) ### API Changes (`api/v1alpha3/`) diff --git a/README.md b/README.md index 71582845..b7e4149b 100644 --- a/README.md +++ b/README.md @@ -116,7 +116,7 @@ commits. If both touch the same resource, the API wins Follow the [installation walkthrough](docs/quickstart.md) to capture ConfigMaps from a demo namespace into a disposable repo. You need a Kubernetes cluster, `kubectl`, Helm 3, and Git write access. -The walkthrough includes cert-manager setup; Redis and audit delivery are optional. +Redis, audit delivery and cert-manager are optional: they come in with named commit authors. The demo captures a live change: diff --git a/SECURITY.md b/SECURITY.md index 9b419074..125b5eda 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -58,7 +58,7 @@ Security reports are especially relevant for areas such as: - queue and Valkey/Redis handling - Git credential handling - SOPS and Secret processing -- admission webhook validation +- `CommitRequest` admission and serving, including on source clusters ## Supported versions diff --git a/charts/gitops-reverser/README.md b/charts/gitops-reverser/README.md index c9c610ce..24da08ae 100644 --- a/charts/gitops-reverser/README.md +++ b/charts/gitops-reverser/README.md @@ -27,14 +27,14 @@ For the chart's optional starter `quickstart` block, see [`docs/configuration.md ## Prerequisites - Kubernetes 1.36 (probably works for older versions, but not tested) -- cert-manager (for TLS certificates) - **Valkey or Redis: optional but advised.** The default install runs without it in `configured-author` mode (watches cold-replay on restart). Add it to unlock warm-restart resume - cursors, CommitRequest author capture (the admission webhook), and attributed-author mode; enabling - any of those requires a non-empty `queue.redis.addr`. -- **Optional, attributed-author mode only:** kube-apiserver audit delivery, which adds - mirrored-resource commit-author attribution. Without it the operator still mirrors state, authored by - the configured committer. + cursors and attributed-author mode; enabling either requires a non-empty `queue.redis.addr`. +- **Optional, attributed-author mode only:** kube-apiserver audit delivery, which adds commit-author + attribution for mirrored resources and `CommitRequest`s, and cert-manager, which mints the audit + ingress certificates (or bring the Secrets yourself with `servers.audit.tls.certManager: false`). + Without them the operator still mirrors state, authored by the configured committer. The default + install renders no certificate, so it needs no cert-manager. ## Installation @@ -198,7 +198,7 @@ nodeSelector: | `quickstart.gitBranch.branch` | Branch the starter `GitBranch` writes. Must match `quickstart.gitProvider.branchPatterns` | `main` | | `quickstart.gitTarget.path` | Repository path used by the starter `GitTarget`; set `.` only to deliberately target the repo root | `live-cluster` | | `quickstart.watchRule.rules` | Rules used by the starter `WatchRule` | `configmaps create/update/delete` | -| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors and for the admission webhook to actually record CommitRequest authors (admission runs as a no-op without it); **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | +| `queue.redis.addr` | Redis/Valkey endpoint (`host:port`). Optional but advised: empty means cold-replay on restart. Set it for warm-restart cursors; **required** when `attribution.enabled=true` unless `attribution.transport=memory` | `""` | | `queue.redis.auth.existingSecret` | Name of a pre-created Secret holding the Redis password (only used when `queue.redis.addr` is set) | `""` | | `queue.redis.auth.existingSecretKey` | Key within the Secret that holds the password | `password` | | `queue.redis.auth.username` | Optional Redis ACL username | `""` | @@ -220,7 +220,6 @@ nodeSelector: | `kubeProvider.secretRef.key` | Key within that Secret. Empty reads `value` then `value.yaml` | `""` | | `kubeProvider.allowedGitTargets` | Rendered as the provider's `spec.allowedGitTargets`: which namespaces' `GitTarget`s may mirror through it, and what. Each grant has a `consumerNamespaces` block (`from`: `Same`, `Names`, `Selector`, `All`), and `sourceNamespaces`, `clusterResources` or both beside it. **Unset, the field is omitted and the CRD defaults it to the release namespace mirroring its own**; a list is the whole policy, and `[]` admits nobody, the release namespace included: a refused `GitTarget` names `spec.allowedGitTargets`. `quickstart.enabled` adds a grant for the quickstart namespace, and [`values-playground.yaml`](values-playground.yaml) grants every namespace its own. With an `inCluster` provider a grant bypasses live namespace RBAC through the operator's identity | `null` | | `kubeProvider.allowedApplierClaims` | Rendered as the provider's `spec.allowedApplierClaims`: which namespaces' `FluxClaim`s and `ArgoCDClaim`s may hold an applier through it (`consumerNamespaces`), and in which `applierNamespaces` it may live, such as `flux-system`. Admission to mirror is no admission to apply. Unset, the CRD defaults it to the release namespace holding appliers in its own; `[]` admits nobody | `null` | -| `servers.admission.enabled` | Install the validate-operator-types admission webhook that captures CommitRequest authors (a form of author attribution). Enabled by default; a no-op until `queue.redis.addr` is set | `true` | | `rbac.create` | Create the manager ClusterRole and its binding | `true` | | `rbac.watchTypes.mode` | Which types a `WatchRule` may read. `any` grants cluster-wide read on everything, including every Secret in the cluster. `selected` grants read on `rbac.watchTypes.selected` only, so the reverser cannot list or watch Secrets (it keeps `get` on named Secrets it is pointed at). See [`docs/rbac.md`](../../docs/rbac.md) | `any` | | `rbac.watchTypes.selected` | Types to grant when `mode: selected`, as `{apiGroups, resources}` entries (verbs are always `get,list,watch`). Required and non-empty in that mode; `namespaces`, `customresourcedefinitions` and `apiservices` come from the manager role and must not be restated | `[]` | @@ -233,13 +232,9 @@ nodeSelector: | `service.ports.receiver` | Port of the `-receiver` Service when `servers.receiver.enabled=true` | `80` | | `service.ports.metrics` | Service port for metrics | `8080` | | `servers.healthProbe.bindAddress` | Liveness/readiness probe bind address (`--health-probe-bind-address`) | `:8081` | -| `servers.admission.port` | Admission webhook container/Service port | `9443` | -| `servers.admission.timeoutSeconds` | Admission webhook timeout (failurePolicy is Ignore) | `2` | -| `servers.admission.tls.certManager` | Mint the admission serving cert via cert-manager (false = BYO via `secretNameOverride`) | `true` | -| `servers.admission.tls.secretNameOverride` | Override Secret name for the admission serving cert | `-admission-server-cert` | | `certManager.enabled` | Use cert-manager to mint the chart's serving/client certs | `true` | | `certManager.issuer.name` | Name of the (shared) cert-manager issuer | `selfsigned-issuer` | -| `certManager.issuer.create` | Create the self-signed issuer (set false to reuse an existing one) | `true` | +| `certManager.issuer.create` | Create the self-signed issuer when a certificate needs it (set false to reuse an existing one) | `true` | | `servers.metrics.tls.certManager` | Mint the metrics serving cert via cert-manager (used when `servers.metrics.tls.enabled`) | `true` | | `servers.audit.tls.certManager` | Let the chart mint audit TLS Secrets (server + kube-apiserver client) via cert-manager when `attribution.enabled=true` | `true` | | `servers.audit.tls.rootCA.secretNameOverride` | Override Secret name for the audit root CA | `-audit-root-ca` | diff --git a/charts/gitops-reverser/templates/_helpers.tpl b/charts/gitops-reverser/templates/_helpers.tpl index 741a3fe2..d4a7e117 100644 --- a/charts/gitops-reverser/templates/_helpers.tpl +++ b/charts/gitops-reverser/templates/_helpers.tpl @@ -123,3 +123,21 @@ The audit route of the KubeProvider the chart renders: ..tls.certManager. (Deliberately NOT duplicated per # server — that one shared resource is the exception to the "servers as peers" rule.) +# The issuer renders only when some certificate does: the audit ingress certificates +# with attribution.enabled, or the metrics certificate with servers.metrics.tls.enabled. +# The default install renders neither, so it needs no cert-manager. certManager: enabled: true issuer: diff --git a/cmd/main_chart_certificates_test.go b/cmd/main_chart_certificates_test.go new file mode 100644 index 00000000..037e778d --- /dev/null +++ b/cmd/main_chart_certificates_test.go @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// The default install mints no certificate, so it renders no cert-manager object at all, and installs +// on a cluster without cert-manager. The self-signed Issuer renders exactly when a certificate needs +// it: attribution with audit TLS, or metrics TLS. +func TestChart_TheIssuerRendersOnlyForACertificate(t *testing.T) { + _, err := os.ReadFile("../charts/gitops-reverser/templates/certificates.yaml") + require.NoError(t, err) + kinds := func(setValues ...string) []string { + var out []string + for _, template := range []string{"templates/certificates.yaml", "templates/audit-certificates.yaml"} { + for _, doc := range helmTemplateDocs(t, template, setValues...) { + for _, line := range strings.Split(doc, "\n") { + if kind, ok := strings.CutPrefix(line, "kind: "); ok { + out = append(out, kind) + } + } + } + } + return out + } + + assert.Empty(t, kinds(), "the default install needs no cert-manager") + assert.Contains(t, kinds("servers.metrics.tls.enabled=true"), "Issuer") + assert.Contains(t, kinds("attribution.enabled=true", "queue.redis.addr=valkey:6379"), "Issuer") + assert.NotContains(t, kinds("attribution.enabled=true", "queue.redis.addr=valkey:6379", + "servers.audit.tls.certManager=false"), "Issuer", "Secrets you bring yourself need no Issuer") +} diff --git a/config/samples/commitrequest.yaml b/config/samples/commitrequest.yaml index 7b9ce546..bc12076a 100644 --- a/config/samples/commitrequest.yaml +++ b/config/samples/commitrequest.yaml @@ -5,7 +5,13 @@ # reported in .status.commit once it has been pushed. # # Frontends typically use metadata.generateName so each "save" is a fresh -# object. The referenced GitTarget must live in the same namespace. +# object. The referenced GitTarget is in the same namespace unless +# gitTargetRef.namespace names another, whose spec.allowedCommitRequests must +# admit this one. The request's author comes from its own audit event. +# +# For a GitTarget that mirrors another cluster, create the request on that +# cluster instead, with gitTargetRef.namespace set, once its KubeProvider sets +# spec.commitRequests and the cluster has the CommitRequest CRD. apiVersion: configbutler.ai/v1alpha3 kind: CommitRequest metadata: @@ -24,8 +30,8 @@ spec: # opens. Next first closes the open window, under its own message. attach: CurrentOrNext # How long to wait for a window, from when the branch worker registers the - # request. "2s" covers a write's wait for its audit fact. - attachTimeout: "2s" + # request. "5s" covers a write's wait for its audit fact. + attachTimeout: "5s" # Close the attached window this long after the attach. Add idleTimeout (at # most maxDuration) to also close it after that much silence. maxDuration: "2s" diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 75003d1e..558979ad 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -7,6 +7,39 @@ guidance that the changelog's breaking-change entries link to. We are pre-1.0, so breaking changes bump the **minor** version (release-please is configured with `bump-minor-pre-major`) rather than the major. Read the relevant entry before upgrading across it. +## The admission webhook is gone; a `CommitRequest`'s author comes from audit + +GitOps Reverser installs no admission webhook. The chart no longer renders the +`ValidatingWebhookConfiguration`, its `Certificate` and Secret, or port `9443` on the controller +`Service`, and the command-author records in Redis (`author:v1:command:*`) are no longer written; +those that remain expire on their own. + +- **Values and flags.** Drop `servers.admission` from your values, and the `--admission-webhook`, + `--admission-webhook-bind-address` and `--admission-webhook-cert-*` flags from a hand-written + Deployment: the controller refuses to start with one of them. +- **cert-manager is needed only for a certificate.** The chart renders its self-signed `Issuer` only + when it mints one: the audit ingress certificates with `attribution.enabled`, or the metrics + certificate with `servers.metrics.tls.enabled`. The default install needs no cert-manager. +- **A request's author.** A `CommitRequest`'s author comes from its own audit event, as a write's + does, and is reported in `status.author`. The `AuthorAttributed` condition is gone. Keep `create` + of `commitrequests` in your audit policy; the write catch-all of the + [tuned policy](../test/e2e/cluster/audit/policy.yaml) covers it. A request no audit event names, on + a cluster that sends audit events, ends `Ready=True` with reason `AuthorUnresolved` and saves + nothing. With attribution off, a request names nobody, as it did with the webhook off. +- **Status while it runs.** A `CommitRequest` in progress is `Ready=Unknown`, reason `Saving` until the + branch worker reports a phase, rather than `Ready=False`. Automation that waits for `Ready=True` or + `Stalled=True` is unaffected. +- **`spec.window.attachTimeout` defaults to `5s`**, since the request now waits for its own audit event + as its edits do. A request that sets it keeps its value. +- **A `GitTarget` that mirrors another cluster.** A `CommitRequest` on the cluster GitOps Reverser runs + in, for such a `GitTarget`, is refused with reason `NotAllowed`. Create it on the cluster the + `GitTarget` mirrors instead, as described in + [serving the `CommitRequest`s of a source cluster](configuration.md#serving-the-commitrequests-of-a-source-cluster). + +A `CommitRequest` may also name a `GitTarget` in another namespace now, in +`spec.gitTargetRef.namespace`, which that `GitTarget` admits in `spec.allowedCommitRequests`. Omitted, +the list admits the `GitTarget`'s own namespace, as before. + ## A `FluxClaim` applies the tip a minute after GitOps Reverser's own pushes A push of GitOps Reverser's own asks no apply at once, and in 0.54.0 asked none at all, so a diff --git a/docs/architecture.md b/docs/architecture.md index 8609dc80..4b4bc0f9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -239,13 +239,12 @@ optional and only add *who* did something: |---|---|---|---| | **Discovery** (CRD/APIService) | *what types exist* | yes | the API surface: served GVRs, scope, preferred version, subresources; rules resolve against it | | **Watch** (per claimed `(GVR, scope)`) | *what changed* | yes | the object body, ordered per type; the only object-state source, with deletes reconciled via `sendInitialEvents` replay + mark-and-sweep | -| **Audit webhook** (`/audit-webhook/`) | *who changed mirrored state* | no | a post-persist attribution fact, partitioned by source provider and joined to the watch event by resourceVersion ([Optional Attribution](#optional-attribution)) | -| **Validating admission webhook** (`/validate-operator-types`) | *who issued a command* | no | the submitter of a `CommitRequest`, captured at admission and keyed 1:1 by UID ([CommitRequest Finalize](#commitrequest-finalize)) | +| **Audit webhook** (`/audit-webhook/`) | *who changed mirrored state, and who created a `CommitRequest`* | no | a post-persist attribution fact, partitioned by source provider and joined to the watch event, or to the `CommitRequest`, by resourceVersion ([Optional Attribution](#optional-attribution)) | -With both optional sources off, the product still mirrors state correctly, and every commit is authored +With the optional source off, the product still mirrors state correctly, and every commit is authored by the committer. -### Why audit, not admission, attributes a mirrored change +### Why audit, not admission, attributes a change For a mirrored change, taking the author from a validating admission webhook is tempting, because the request already carries `userInfo`. We tried it; the edge cases are not fixable, because **admission runs *before* @@ -263,13 +262,11 @@ the write reaches etcd**: Audit avoids both: kube-apiserver posts a `ResponseComplete` event *after* the write persisted, carrying the resourceVersion that joins it to the watch event. -The one place admission *is* the right source is the `CommitRequest`, and only for its **command** meaning -(*who issued the save*) and not for mirroring the object. That holds because it is **our own type with a -memory model we control**: the controller reconciles off its own watch, which only delivers persisted -objects, so the author is captured at admission and read back 1:1 by UID: no resourceVersion join, no -persistence proof from audit needed. (We still can't guarantee a create is never blocked, but it holds -across our edge cases.) If a `WatchRule` also selects CommitRequests, that *mirroring* runs through the -normal pipeline and is audit-attributed like any other resource. See +A `CommitRequest`'s author comes from audit too, for the same reasons: its own `ResponseComplete` event, +joined by the UID and resourceVersion it was created with, from the same index and with the same grace +as the edits it saves. So the request and its edits are attributed by one pipeline, on the cluster both +were made on, and a request on a source cluster needs nothing from the control cluster. A +`CommitRequest` is never mirrored, whatever a `WatchRule` selects. See [CommitRequest Finalize](#commitrequest-finalize). ### Proving it on new Kubernetes versions @@ -407,24 +404,27 @@ One `KubeProvider` grant answers two questions in different planes: ### CommitRequest - **Source**: [api/v1alpha3/commitrequest_types.go](../api/v1alpha3/commitrequest_types.go) -- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) +- **Core**: [internal/controller/commitrequest_core.go](../internal/controller/commitrequest_core.go) -A one shot "save now" signal that finalizes the open commit window for a same namespace `GitTarget` -instead of waiting for the silence timer. The **entire spec is immutable**. Key fields: +A one shot "save now" signal that finalizes the open commit window for a `GitTarget` instead of waiting +for the silence timer. It is created on the cluster the `GitTarget` mirrors: the control cluster, or a +source cluster whose `KubeProvider` sets `spec.commitRequests`. The **entire spec is immutable**. Key +fields: -- `spec.gitTargetRef.name`: target whose window the request attaches to. +- `spec.gitTargetRef`: target whose window the request attaches to, admitted by its + `spec.allowedCommitRequests`. On a source cluster the namespace is required. - `spec.message`: optional verbatim commit message (1–1024 chars, no control characters). -- `spec.window`: `attach` (`CurrentOrNext` or `Next`), `attachTimeout` (default `2s`, the wait for a +- `spec.window`: `attach` (`CurrentOrNext` or `Next`), `attachTimeout` (default `5s`, the wait for a window), and the timers that replace the target's for the attached window: `idleTimeout` (none by default) and `maxDuration` (default `2s`). A request never opens a window; only writes do. - `spec.whenNothingToCommit`: `Resolve` (default) or `CommitEmpty`, which records the message in an empty commit when the request ends with nothing to commit. -- `status.conditions`: kstatus-compatible. **Ready** is the summary (True once the request reached a - terminal outcome that is not an error: a pushed commit, or a benign no-commit); - **Reconciling**/**Stalled** are the kstatus progress/blocked pair; **AuthorAttributed** reports - whether the `/validate-operator-types` validating admission webhook named the submitter; an absent record - means the request claims no actor and can attach only to an unnamed window. **Pushed** reports whether the - commit reached the remote. The `Ready` +- `status.author`: the creator, as the request's audit event names them; empty when it names nobody, + and then the request can attach only to an unnamed window. +- `status.conditions`: kstatus-compatible. **Ready** is the summary (Unknown while the save runs, True + once the request reached a terminal outcome that is not an error: a pushed commit, or a benign + no-commit); **Reconciling**/**Stalled** are the kstatus progress/blocked pair. **Pushed** reports + whether the commit reached the remote. The `Ready` condition's `reason` carries `Committed`, `NoWindow`, `WindowMismatch`, `AlreadyPresent`, or `FinalizeFailed`. A benign no-commit (e.g. `NoWindow`) is `Ready=True`, `Stalled=False`, a correct non-error outcome, whereas a `FinalizeFailed` is `Ready=False`, `Stalled=True`. @@ -743,8 +743,8 @@ flowchart TD TIMER -->|silence timer| BASE TIMER -->|buffer limit or resync boundary| BASE - CR[CommitRequest persisted] --> ADMISSION[Admission submitter lookup
named or unnamed] - ADMISSION --> ATTACH[Attach to matching open window] + CR[CommitRequest persisted] --> AUTHOR[Author from its audit fact
named or unnamed] + AUTHOR --> ATTACH[Attach to matching open window] ATTACH -->|same author + GitTarget| BASE ATTACH -->|no matching window| BENIGN[Ready=True, Pushed=False] @@ -757,9 +757,7 @@ flowchart TD The important boundary is the open window: it accepts only the same author and `GitTarget`. A `CommitRequest` from another author never closes someone else's window; it resolves as a benign no-commit -instead. The one exception is opt-in, on a cluster that has never sent an audit fact: there no write -can name its author, and a request may claim a window whose writes name none (step 2 of the -`CommitRequest` flow). +instead. ### Remote moved while we were writing @@ -1116,9 +1114,8 @@ queued behind it on the same `(GitTarget, GVR, scope)` goroutine. That cost, and is in [Watch Event Ordering](#watch-event-ordering) and [attribution-removal-wait-options.md](design/attribution-removal-wait-options.md). -The CommitRequest controller does **not** read this audit index. A CommitRequest's submitter is named by -the `/validate-operator-types` validating admission webhook instead, captured synchronously at admission, -never joined from an audit fact (see [CommitRequest Finalize](#commitrequest-finalize)). +A `CommitRequest`'s author is read from this index too, on the route of its `GitTarget`'s `KubeProvider`, +with the same grace (see [CommitRequest Finalize](#commitrequest-finalize)). ### Author and committer identity in Git @@ -1760,50 +1757,53 @@ unsafely: logged per-resource, counted in the resync summary as `placementSkippe ## CommitRequest finalize -- **Controller**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) - -A `CommitRequest` finalizes the open commit window for its GitTarget. The request author is resolved from -the `/validate-operator-types` validating admission webhook, which captures the authenticated submitter at -admission (keyed by the object's UID) before the object is visible. The lookup is **present-or-never**: if -the record is missing, the request is marked `AuthorAttributed=False`, claims no actor, and still attaches -immediately. That is not a failure: - -1. The controller stamps the in-progress conditions (`Reconciling=True`) and settles - `AuthorAttributed` synchronously from the admission author cache. There is no audit wait on this path. -2. The controller eagerly **attaches** the request to the worker (`AttachCommitRequest`), which - waits for a window until `registration + attachTimeout`. The worker binds it to an open window only - when the author state and GitTarget match, first come first served, and replaces the window's timers - with the request's own, so time spent waiting never shortens the collection. A request that names its - submitter may also claim a window on its `GitTarget` whose writes name no author, when it asks - (`spec.window.author`), the grant admitting it allows it (`unattributedWindows` in the `GitTarget`'s - `spec.allowedCommitRequests`), and the - `GitTarget`'s `KubeProvider` has never received an audit fact (`AuditFactsReceived` is not `True`). - With `SavedBy` the window takes the request's message, the commit names no author, and the message - ends with a `Saved-By` trailer; with `TakeOver` the submitter authors the commit, and the message ends - with an `Author-Source: CommitRequest /` trailer. Once the provider has received a - fact, a write that names no author means its fact went missing, and its window stays another's. A - request that asks for more than its `GitTarget` allows fails at first sight with - `TakeOverNotAllowed`. It **never finalizes - another author's window**; a window carries at most one request. The controller reports the phase the - worker reports: `WaitingForWindow`, `CollectingWindow`, `WaitingForPush`. -3. The window finalizes when the request's timers close it (or when it closes for any other reason). +- **Core**: [internal/controller/commitrequest_core.go](../internal/controller/commitrequest_core.go) +- **Control-cluster path**: [internal/controller/commitrequest_controller.go](../internal/controller/commitrequest_controller.go) +- **Source-cluster path**: [internal/controller/commitrequest_server.go](../internal/controller/commitrequest_server.go) + +A `CommitRequest` finalizes the open commit window for its GitTarget. It is picked up in one of two +ways: by the `CommitRequestReconciler` on the control cluster, or by the `CommitRequestServer`'s watch +of a source cluster, through the `KubeProvider` whose `spec.commitRequests` reconciles it. Both hand it +to one core, which takes one step at a time, each decided again from the object as it is stored +([contract](spec/commitrequest-design.md)): + +1. **The owner.** The `GitTarget`'s `KubeProvider` alone picks the path that serves the request: the + control-cluster path for an `inCluster` provider, that provider's watch for a kubeconfig. Any other + path writes nothing but `GitTargetNotFound`, once `attachTimeout` has passed with no `GitTarget`, + and only into an empty status. Every status write is conditional on the resourceVersion it was + decided from. +2. **Admission and the author.** A grant in the `GitTarget`'s `spec.allowedCommitRequests` must admit + the request: `consumerNamespaces` on the control cluster, `sourceNamespaces` on a source cluster. + The request's author comes from its own audit fact, on the route of the `GitTarget`'s + `KubeProvider`, awaited with the attribution grace; a request no fact names, on a provider that has + received audit facts, is `AuthorUnresolved` and saves nothing. +3. **The claim.** The owner writes the first status, `status.author` and `Ready=Unknown` (`Saving`), + and only then **attaches** the request to the worker (`AttachCommitRequest`), which waits for a + window until `registration + attachTimeout`. The worker binds it to an open window only when the + author state and GitTarget match, first come first served, and replaces the window's timers with the + request's own, so time spent waiting never shortens the collection. It **never finalizes another + author's window**; a window carries at most one request. The owner reports the phase the worker + reports: `WaitingForWindow`, `CollectingWindow`, `WaitingForPush`. +4. The window finalizes when the request's timers close it (or when it closes for any other reason). If a finalize closes an open window, the worker always schedules a push, so a window closed by an otherwise no-op resync is not stranded. -4. Outcomes resolve on push and are reported as conditions: a pushed commit sets `Ready=True` / +5. Outcomes resolve on push and are reported as conditions: a pushed commit sets `Ready=True` / `Pushed=True` with `branch`/`commit`; a benign no-commit sets `Ready=True` with the reason on `Ready` and `Pushed=False`; a failure sets `Ready=False` / `Stalled=True` with a message. **Between the window's - finalize and that push the request is still in flight**, which matters because the controller keeps + finalize and that push the request is still in flight**, which matters because the owner keeps re-sending its attach until it reads an outcome: the worker marks the request committed rather than forgetting it, so a re-send is recognized as the same request and cannot resolve early or claim a later window. A worker that stops without pushing fails the request instead of leaving it to time out. `AlreadyPresent` therefore means the remote confirmed there was nothing to add, not that a local plan - found no diff. + found no diff. A conflict on the outcome's write re-applies the outcome onto the fresh object; the + save is never restarted. +6. **Retention.** A finished request is stamped `configbutler.ai/delete-after` and deleted once it has + passed: always on the control cluster, and on a source cluster only with + `commitRequests: ReconcileAndDeleteFinished`. -The CommitRequest submitter is not recoverable from object state alone, so without an admission record the -request cannot claim an actor. The final Git author remains the attached watch window's author: configured -committer when attribution was disabled, or the explicit unresolved author when live attribution ran but -could not resolve. There is no audit-fact join for the request itself: its submitter is captured at admission -by the validating webhook, not derived from the audit attribution index. +A request whose audit event names nobody claims no actor. The final Git author remains the attached +watch window's author: configured committer when attribution was disabled, or the explicit unresolved +author when live attribution ran but could not resolve. *** @@ -1836,10 +1836,12 @@ Controllers watch their dependencies so dependents reconcile quickly after spec at all (the installation's `inCluster` consent, and which namespace holds its audit route), and projects its readiness to dependent targets. A provider created, deleted or moved to another route wakes the others carrying that route, so an `AuditRouteConflict` clears without waiting for a requeue. -- `CommitRequestReconciler` runs with `MaxConcurrentReconciles=1` and attributes/attaches as above; its - optional `AuthorLookup` is the command-author cache populated by the `/validate-operator-types` webhook - (wired whenever the admission webhook is enabled, independent of `--author-attribution`, and nil - otherwise, so the request claims no actor). +- `CommitRequestReconciler` serves the control cluster's `CommitRequest`s through the shared core, four + at a time, so one request's author wait does not hold up the others. It reads through the uncached + API reader, and shares the watch plane's author resolver and fact streams, which are nil with + attribution off. The `CommitRequestServer` serves the source clusters' requests through the same core, + one watch per `KubeProvider` that reconciles them, and reports each to the `KubeProviderReconciler`, + which writes it as `CommitRequestsServed`. Dependency watches use narrow predicates to avoid status-only heartbeat churn: a `KubeProvider` also admits a `Ready` transition, and a `Namespace` admits only label changes. `GitProvider`, `GitBranch`, @@ -1874,7 +1876,7 @@ flowchart TD Hq -->|no| J[Configured-author: no fact streams or index; audit webhook skipped] I --> K[Setup + register Watch Manager] J --> K - K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + CommitRequest controllers; admission webhooks] + K --> L[Register GitProvider + GitReceiver + KubeProvider + GitBranch + GitTarget
+ FluxClaim + ArgoCDClaim + CommitRequest controllers; CommitRequest server] L --> M[Add cert watchers + health checks] M --> N[mgr.Start] ``` @@ -2022,7 +2024,7 @@ Current limitations: | [internal/types/](../internal/types/) | shared resource identity/reference and sensitivity policy | | [internal/typeset/](../internal/typeset/) | type followability registry, lookup model, and the relevance filter (controller-owned → don't mirror) | | [internal/watch/](../internal/watch/) | discovery catalog, watch manager, per-`(GVR, scope)` raw watches with `sendInitialEvents` replay + mark-and-sweep, the author resolver, watched type tables, event router | -| [internal/webhook/](../internal/webhook/) | `/audit-webhook` ingress and attribution fact extraction (no body joiner), the `/validate-operator-types` admission handler, and the `/git-push` receiver | +| [internal/webhook/](../internal/webhook/) | `/audit-webhook` ingress and attribution fact extraction (no body joiner), and the `/git-push` receiver | *** diff --git a/docs/attribution-setup-guide.md b/docs/attribution-setup-guide.md index 5e3ff512..62341e03 100644 --- a/docs/attribution-setup-guide.md +++ b/docs/attribution-setup-guide.md @@ -26,8 +26,8 @@ queue: Redis/Valkey holds attribution facts and watch cursors. If its data is lost, recent facts are lost and watches cold-replay. For a disposable single-pod setup, `attribution.transport: memory` needs -no Redis but loses facts whenever the reverser restarts. It also leaves Redis-backed features such -as `CommitRequest` author capture inactive unless you configure Redis separately. +no Redis but loses facts whenever the reverser restarts, and leaves the watch resume cursors +inactive unless you configure Redis separately. The chart's `KubeProvider`, `local` in the release namespace, admits only the release namespace by default. Grant the namespaces whose `GitTarget`s may mirror through it in @@ -119,6 +119,11 @@ other subresources. Keep manual scale writes if you want their authors recorded. The example includes Secret writes at `RequestResponse` level. To keep Secret bodies out of the webhook, exclude core `secrets` too. Those writes then have no audit evidence for attribution. + +Keep `create` of `commitrequests` in the `configbutler.ai` group covered, at `Metadata` level at +least: a `CommitRequest`'s author comes from its own audit event, as a write's does. Without it a +request names nobody, and cannot save the window of an author the audit names. The write catch-all +covers it. The audit policy controls evidence collection; `WatchRule` and `ClusterWatchRule` control what gets written to Git. diff --git a/docs/components.md b/docs/components.md index 5d3f20e6..ec157ac1 100644 --- a/docs/components.md +++ b/docs/components.md @@ -78,7 +78,7 @@ The operator's own API surface and the reconcilers behind it. |---|---|---| | CRD types | [`api/v1alpha3/`](../api/v1alpha3/) | the ten kinds users apply | | Reconcilers | [`internal/controller/`](../internal/controller/) | one per kind, plus shared condition and status helpers | -| Admission handlers | [`internal/webhook/`](../internal/webhook/) | the always-allow observer and `validate-operator-types`, which captures a `CommitRequest` submitter | +| CommitRequest core | [`internal/controller/commitrequest_core.go`](../internal/controller/commitrequest_core.go) | serves a `CommitRequest` for both paths: the control-cluster reconciler, and the server that watches source clusters' requests through their `KubeProvider`s | | Access grants | [`internal/authz/`](../internal/authz/) | one admission decision per act, read from the grant lists on the referenced object (`allowedGitBranches`, `allowedGitTargets`, `allowedApplierClaims`) with the scope that applies, in their own package because several call sites need the same answer | | Rule store | [`internal/rulestore/`](../internal/rulestore/) | compiled `WatchRule` and `ClusterWatchRule` cache the data plane reads | @@ -188,7 +188,9 @@ flowchart LR | Target watches | **objects** of claimed types | source cluster | raw `dynamic ... Watch()`, no informer and no cache ([`internal/watch/target_watch.go:1420`](../internal/watch/target_watch.go#L1420)) | Two more paths reach the operator without being watches at all. The audit webhook is a **push** from -kube-apiserver, and the admission webhooks are synchronous requests. Neither observes state. The +kube-apiserver; it observes no state. The `CommitRequest` server's watch of a source cluster's +requests is a raw `dynamic ... Watch()` too, one per `KubeProvider` that reconciles them, scoped to +the namespaces it lends. The applier claim reconciler reads each applier claim's `Kustomization` or `Application` with a GET on its own tick rather than a watch, so a cluster without Flux or Argo CD needs no informer for them. diff --git a/docs/definitions.md b/docs/definitions.md index 83b68675..aac5f0f2 100644 --- a/docs/definitions.md +++ b/docs/definitions.md @@ -42,7 +42,8 @@ without a side, or "namespace" without a direction. Fix the side first and the r vocabulary follows. **Control cluster.** The cluster the operator runs in, and where its own ten kinds live. When a -document says "a namespace" with no qualifier, it means a control-cluster namespace. +document says "a namespace" with no qualifier, it means a control-cluster namespace. `CommitRequest` +is the one operator kind that can also live on a source cluster, next to the edits it saves. **Source cluster.** The cluster whose objects are mirrored into Git. A `GitTarget` names it through its `KubeProvider`. It is the same cluster as the control cluster on a single-cluster install, @@ -179,7 +180,7 @@ its `maxDuration` after opening, whichever comes first, or earlier for a flush, `CommitRequest` holds it. Windows do not merge across authors, which is why attribution can be per-commit. -**Attach.** What a `CommitRequest`, or a `RemoteSaveRequest` made on a source cluster, does to a commit +**Attach.** What a `CommitRequest`, on the control cluster or on a source cluster, does to a commit window: it gives the window its message and replaces the window's timers with its own. A request attaches to at most one window and never opens one. One word for the act, in the API (`window.attach`, `attachTimeout`), the code and the docs: not diff --git a/docs/future/source-side-save-plan.md b/docs/future/source-side-save-plan.md index 6ca3f687..fb8ebbe3 100644 --- a/docs/future/source-side-save-plan.md +++ b/docs/future/source-side-save-plan.md @@ -9,17 +9,13 @@ It also plans the second step: letting a save take over a window whose writes na opt-in, for a source cluster that cannot send audit events at all. There it is the only way a save can put its author's name on the commit. -Status: implemented on `feat/gitops-api-asks-converge-root-ca`. The two request kinds are a pair: -`CommitRequest` (`configbutler.ai/v1alpha3`, formerly `CommitRequest`) on the control cluster, and -`RemoteSaveRequest` (`source.configbutler.ai/v1alpha1`, `internal/watch/remote_save_requests.go`) on -the source cluster. Each names its GitTarget, which admits it in a grant list: -`spec.allowedCommitRequests` (by control-cluster namespace; omitted, its own namespace) and -`spec.allowedRemoteSaveRequests` (by source namespace and labels; omitted, none). A KubeProvider -also serves its cluster's remote requests only with `spec.remoteSaveRequests: Enabled`, and the -GitTarget must mirror that cluster. The takeover is opt-in, as described below, bounded per grant. -The open questions are answered: its own group, a default `attachTimeout` of `5s`, the reason -`AuthorUnresolved`, and a second save of one author on one GitTarget is refused while the first is in -progress (`SaveInProgress`). +Status: superseded. It was built on `feat/gitops-api-asks-converge-root-ca` as a second kind, +`RemoteSaveRequest` in `source.configbutler.ai/v1alpha1`, with a takeover for clusters that send no +audit events. The [review that replaced it](../review/20261027-save-request-simplification.md) folded +both kinds into one `CommitRequest`, served on the control cluster and on a source cluster through one +core, and removed the takeover with the admission webhook. The +[configuration reference](../configuration.md#serving-the-commitrequests-of-a-source-cluster) describes +what shipped; this page keeps the reasoning that led there. This revision takes in gitops-api's review of the first draft, and a later API review of the implementation: see [After the API review](#after-the-api-review). diff --git a/docs/interpreting-metrics.md b/docs/interpreting-metrics.md index ef3c0842..bcbbe616 100644 --- a/docs/interpreting-metrics.md +++ b/docs/interpreting-metrics.md @@ -216,7 +216,7 @@ boundary, the commit, the push. Background: | `git_next_retry_timestamp_seconds` | gauge | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name` | When the next attempt at the pending writes is due, as a Unix timestamp. No sample while no retry is scheduled. | | `git_materialization_failures_total` | counter | `gitbranch_namespace`, `gitbranch_name`, `provider_namespace`, `provider_name`, `reason` | Attempts to commit decided writes that stopped before any push: `reason` is `unreachable` or `parent_unavailable`. Nothing is lost; the writes wait for the retry. A failure `git_pushes_total` cannot see, because no push cycle started. | | `git_branch_targets` | gauge | `gitbranch_namespace`, `gitbranch_name`, `gittarget_namespace`, `gittarget_name`, `source_cluster` | Always 1. The **join** between the GitTarget-labeled half of the pipeline and the GitBranch-labeled half: the `GitTarget`'s `spec.gitBranchRef`. Published per configured GitTarget, whether or not its GitBranch exists or its worker runs. | -| `commit_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `CommitRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `save_in_progress` (refused: an earlier save of the same author on the target was in progress) / `failed`. A target that never resolved publishes the empty pair. See the counting note below. | +| `commit_requests_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | One per `CommitRequest` terminal **decision**. `outcome` is `committed` / `no_window` / `window_mismatch` / `already_present` / `save_in_progress` (refused: an earlier save of the same author on the target was in progress) / `author_unresolved` (no audit event named the request's creator, on a cluster that sends them) / `failed`. Requests served on a source cluster count here too. A target that never resolved publishes the empty pair. See the counting note below. | | `placements_total` | counter | `source`, `disposition`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new document at a resolved path. | | `placement_refusals_total` | counter | `reason`, `gittarget_namespace`, `gittarget_name`, `group`, `version`, `resource` | One per new resource the writer declined. Every increment is a resource **absent** from the mirror. | | `placement_kustomization_entries_total` | counter | `outcome`, `gittarget_namespace`, `gittarget_name` | `added` / `no_change` / `failed`. | diff --git a/docs/quickstart.md b/docs/quickstart.md index 5e204126..f24b6588 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -10,26 +10,14 @@ cluster, which the starter target names as its source. ![Config basics: a WatchRule or ClusterWatchRule selects what a GitTarget mirrors; the GitTarget writes its folder through a GitBranch and a GitProvider with a credentials Secret, and reads the cluster through a KubeProvider. The direction is always from the cluster into Git](images/config-basics.excalidraw.svg) -**Prerequisites:** a Kubernetes cluster with `kubectl`, Helm 3, and cert-manager for TLS. +**Prerequisites:** a Kubernetes cluster with `kubectl` and Helm 3. The default install renders no +certificate, so it needs no cert-manager; [named users](#want-named-users-on-your-commits) do. To write the objects yourself instead of letting the chart generate them, start from [`config/samples/`](../config/samples/): one manifest per object, consistent enough to apply together. [`configuration.md`](configuration.md) explains every field. -## 1. Install cert-manager - -Skip this step if cert-manager is already installed and healthy. - -The controller mounts an admission certificate at startup, so cert-manager must be healthy *before* -you install the chart. (`--set servers.admission.enabled=false` drops the dependency; the -[chart README](../charts/gitops-reverser/README.md) covers bring-your-own certificates.) - -```bash -kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.3/cert-manager.yaml -kubectl wait --for=condition=ready pod -l app.kubernetes.io/instance=cert-manager -n cert-manager --timeout=300s -``` - -## 2. Create a Git repo and a deploy key +## 1. Create a Git repo and a deploy key Create an (empty) repository the operator will write to, then generate a key. Add the public half as a **deploy key with write access**; the private half becomes the Secret in the next step: @@ -39,7 +27,7 @@ ssh-keygen -t ed25519 -C "gitops-reverser@cluster" -f /tmp/gitops-reverser-key - # Add /tmp/gitops-reverser-key.pub to your Git provider as a deploy key (write access) ``` -## 3. Create the demo namespace and Git credentials +## 2. Create the demo namespace and Git credentials Do this **before** installing. The starter `GitProvider` is only re-checked about every 5 minutes, so if the Secret is missing at install time your first commit can be minutes late; having it ready up @@ -72,7 +60,7 @@ credentials Secrets are accepted as-is (they must have **write** access). See [`github-setup-guide.md`](github-setup-guide.md) for the full GitHub guide and HTTPS/PAT fallback. -## 4. Install GitOps Reverser with the demo enabled +## 3. Install GitOps Reverser with the demo enabled Point the starter `GitProvider` at your repo and install: @@ -89,14 +77,14 @@ Replace `OWNER/REPO` with your repository (angle brackets would be parsed by the This install has these defaults: -- **No Redis**, so warm-restart cursors and author capture stay inactive. Add one with +- **No Redis**, so warm-restart cursors and author attribution stay inactive. Add one with `--set queue.redis.addr=HOST:PORT` (plus `queue.redis.auth.existingSecret` if it needs auth). - **Cluster-wide read on every watchable type, including Secrets** (`rbac.watchTypes.mode=any`). [`rbac.md`](rbac.md) explains how to narrow it. - **SOPS is on for the starter target**, so a `sops-age-key` Secret is generated in the demo namespace with a backup reminder. Only matters once you mirror Secrets; back it up if you keep the demo. -Wait for the controller. On a fresh install this waits on cert-manager issuing the certificate: +Wait for the controller: ```bash kubectl rollout status deployment/gitops-reverser -n gitops-reverser --timeout=300s @@ -134,7 +122,7 @@ A `Reason` that names a missing or unauthorized dependency (`KubeProviderNotFoun streams still settling is initialization, and resolves on its own. `describe` prints every condition with its message, which is where the specific cause is written. -## 5. Test it +## 4. Test it ```bash kubectl create configmap test-config --from-literal=key=value -n gitops-reverser-quickstart-demo @@ -166,3 +154,14 @@ To tear the demo down: `helm uninstall gitops-reverser -n gitops-reverser` and Enable audit attribution to use the Kubernetes actor as the Git author while keeping the committer unchanged. The [attribution setup guide](attribution-setup-guide.md) covers receiver configuration, API server delivery, and verification. It requires control over kube-apiserver settings. + +The audit ingress serves TLS with certificates cert-manager mints, so install cert-manager first, +and wait until it is healthy *before* you enable attribution: the controller mounts the audit +certificates at startup. Skip this if cert-manager is already installed and healthy, or bring the +Secrets yourself with `servers.audit.tls.certManager: false`; the +[chart README](../charts/gitops-reverser/README.md) covers bring-your-own certificates. + +```bash +kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.21.3/cert-manager.yaml +kubectl wait --for=condition=ready pod -l app.kubernetes.io/instance=cert-manager -n cert-manager --timeout=300s +``` diff --git a/docs/security-model.md b/docs/security-model.md index 20ad641c..6d3e33e7 100644 --- a/docs/security-model.md +++ b/docs/security-model.md @@ -47,8 +47,13 @@ To mirror live state into Git the controller must: or `applierAccess.argocd`. Apart from the applier an applier claim holds, the controller never writes to a watched type. Its -write targets are Git, that applier, and the Secrets it generates itself (the signing key, and the -age key under `generateWhenMissing`). +write targets are Git, that applier, the Secrets it generates itself (the signing key, and the +age key under `generateWhenMissing`), and the status of the `CommitRequest`s it serves. On a source +cluster that last write is opt-in: a `KubeProvider` with `spec.commitRequests` set writes +`CommitRequest` status there, and, with `ReconcileAndDeleteFinished`, deletes finished requests, +through its own credential, in the namespaces it lends. It never writes the CRD. + +GitOps Reverser installs no admission webhook, so it never sits in the write path of the API server. ### Which namespaces a rule may read from @@ -113,13 +118,14 @@ namespace it already knows. Narrowing that further is |---|---| | Git credentials Secret | Grants push access to your repository. | | SOPS/age key material | Decrypts (and the public key encrypts) Secret data written to Git. | -| Redis/Valkey queue | Buffers decoded audit events in transit; not an audit archive. | +| Redis/Valkey queue | Holds each `GitTarget`'s watch resume cursors, and, with the `redis` attribution transport, the attribution facts decoded from audit events, keyed by audit route, type and object, until their TTL; not an audit archive. | | Audit ingress (`/audit-webhook/`, such as `/audit-webhook/gitops-reverser.local` for the chart's provider) | Accepts audit traffic; protected by mutual TLS via cert-manager. A route is held by one namespace: a `KubeProvider` elsewhere that carries it is `Stalled` (`AuditRouteConflict`) and reads none of its facts. | | Push receiver (`--receiver-bind-address`, off by default) | Plain HTTP for Git hosts' push webhooks. Each `GitReceiver` route is a path derived from its token, whose Secret lives in the receiver's own namespace, and a delivery must carry a valid signature or token for it before its body is read. A delivery is only a hint: it wakes the branch worker of every `GitProvider`, in any namespace, whose `spec.url` names the repository it reports, and that worker reads the remote with its own provider's credential. The payload's repository URL is only compared with `spec.url`, never fetched, so a delivery cannot choose a URL, a credential or a commit. A worker holds at most one pending look however many deliveries arrive, so deliveries never queue up behind its writes; but they are not rate-limited, so a token holder, in whatever namespace, who keeps delivering can keep a matching worker reading its remote continuously. That availability cost is accepted: there is no per-worker cooldown. The `202` carries no body, so it does not say whether another namespace tracks a repository. | | Generated Secret material | Signing keys and generated age keys live in cluster Secrets. | | Grants (`allowedGitBranches`, `allowedGitTargets`, `allowedApplierClaims`) | Nothing is admitted automatically, an object's own namespace included. A `from: Selector` grant admits every control-cluster namespace whose labels match, so it is only as strong as the right to set those labels: whoever can label a namespace can admit it, including a tenant allowed to edit its own namespace. Select on labels only the platform can set, or use `from: Names`. A `Names` grant also admits a namespace deleted and recreated with the same name. Grants decide who may use a provider's identity; every operation still runs as that identity, never as the referrer. | | Another provider on the same repository | The branch claim is keyed by repository and branch, not by provider, so grants cannot stop a `GitBranch` that claims a branch through a second `GitProvider` for the same URL with a copied credential. On Gitea and Forgejo a read-only token opens a push session. Keep tenants from creating `GitProvider`s where they share a repository. | | `KubeProvider.spec.inCluster` and `--in-cluster-kube-providers` | `inCluster` wields the operator's own identity in the cluster it runs in. It is honored only for a `KubeProvider` the flag lists by exact `namespace/name` (no wildcards, empty by default); any other is `Stalled` (`InClusterNotAuthorized`) and never used. Anyone who can create a `KubeProvider` under a listed name wields that identity through it, so list only names in namespaces you control. | +| `GitTarget.spec.allowedCommitRequests` | Lets whoever may create a `CommitRequest` in an admitted namespace save their own commit window on the `GitTarget`: its message, its timers and an apply. `consumerNamespaces` admit namespaces of the cluster GitOps Reverser runs in; `sourceNamespaces` admit namespaces of the cluster the `GitTarget` mirrors, by name only, and never one its `KubeProvider` does not lend it. A request's author comes from its own audit event, never from a name it carries, so a grant decides who may save, not who authored the commit. | | `KubeProvider.spec.allowedApplierClaims` | Lets the namespaces a grant admits suspend and apply a `Kustomization` or `Application` in its `applierNamespaces`, such as `flux-system`, through the provider's identity. Admission to mirror does not admit to apply. A delegation, not owner consent: it does not prove the namespace owns the applier or its repository. An applier claim only acts while an eligible `GitTarget` names it, and the binding check compares the `GitProvider` URL, not write access. A denial hands the applier back. | ## Secret data the controller writes to Git diff --git a/docs/spec/commitrequest-design.md b/docs/spec/commitrequest-design.md index 7407e5cb..4dc5baef 100644 --- a/docs/spec/commitrequest-design.md +++ b/docs/spec/commitrequest-design.md @@ -1,6 +1,6 @@ # CommitRequest window finalization -> **spec**: current behavior, reviewed at `c12f7970` on 2026-10-20. +> **spec**: current behavior, reviewed at `670032d2` on 2026-10-27. > The code depends on this document; change one, change the other. > Index: [`../INDEX.md`](../INDEX.md) @@ -8,11 +8,83 @@ A `CommitRequest` is a one-shot “save now” command for one `GitTarget`. It d `CommitRequest` object. Instead, it can attach a message to a matching open commit window and asks the worker to close that window after the requested collect delay. +## Two ways in, one core + +A `CommitRequest` is the one operator kind that can also live on a source cluster. It is picked up +in one of two ways, and both hand it to one core +([`commitrequest_core.go`](../../internal/controller/commitrequest_core.go)), which decides everything +else once: + +- **On the control cluster**, the `CommitRequestReconciler` serves it with the operator's own client, + as any operator kind. +- **On a source cluster**, the `CommitRequestServer` watches the requests of every `KubeProvider` + whose `spec.commitRequests` is `Reconcile` or `ReconcileAndDeleteFinished`, through that provider's + credential, in the namespaces it lends its `GitTarget`s in `spec.allowedGitTargets`. It writes + status, and with `ReconcileAndDeleteFinished` deletes finished requests, through the same + credential. It never writes the CRD, and reports its state in the `KubeProvider`'s + `CommitRequestsServed` condition. + +The core takes one step at a time, each decided again from the object as it is stored. Its +responsibilities, in order: admission by `spec.allowedCommitRequests`; the request's author; the +attach and its outcome; the apply `catchUp: AfterPush` asked for; and retention. + +### Which path owns a request + +The named `GitTarget`'s `KubeProvider` alone picks the path that serves a request, its owner: + +| The request is on | The `GitTarget`'s `KubeProvider` | What happens | +| --- | --- | --- | +| the control cluster | `inCluster` | the control-cluster path serves it | +| the control cluster | a kubeconfig, with a `Reconcile` value | left alone: the kubeconfig may point back at the control cluster, and then its watch serves it | +| the control cluster | a kubeconfig, with `commitRequests: Ignore` | `NotAllowed`, saying to create the request on that cluster | +| a source cluster that provider P watches | P | P's watch serves it | +| a source cluster that provider P watches | any other provider | left alone | +| a source cluster | `gitTargetRef` names no namespace | not picked up, and no status | +| either | the `GitTarget` does not exist | `GitTargetNotFound` once `attachTimeout` has passed since creation | + +Three rules keep several observers of one request safe: + +1. **A path writes only what it knows to be true.** Only the owner serves a request and writes its + progress. A path that is not the owner writes only `GitTargetNotFound`, and only into an empty + status, because nothing else is true for every observer. +2. **Every status write is conditional on the resourceVersion its decision was made from.** On a + conflict the writer reads the request again and decides again, so a `GitTargetNotFound` decided + from an older version never overwrites an owner that has started. +3. **The owner claims a request by writing its first status, and attaches only after that.** The + first write records the author and `Ready=Unknown`, reason `Saving`. If it conflicts and the fresh + object already holds an outcome, the owner stops without saving. After the claim only the owner + writes, so a later conflict comes from its own writes, and an outcome the worker produced is + written onto the fresh object as it is; the save is never restarted. + +Ownership is structural, not "the process that wrote the status": after a restart, the owner is +still the path the `KubeProvider` picks, finds its own claim, and resumes with the author it +recorded. + +A control-cluster request for a `GitTarget` that mirrors another cluster is refused rather than +served, because its author would come from the control cluster's audit and its edits' from the +other cluster's. A request on a source cluster must name the `GitTarget`'s namespace, because that +namespace belongs to the control cluster; the path cannot refuse a request without one, since a +kubeconfig pointing back at the control cluster sees the control cluster's requests, where an +omitted namespace is valid. Known gap: a control-cluster request whose `GitTarget`'s provider has a +`Reconcile` value and points somewhere else gets no status from anyone. + +### Admission + +A grant in the `GitTarget`'s `spec.allowedCommitRequests` sets exactly one of two fields, enforced in +CEL. `consumerNamespaces` admits the request's namespace on the control cluster, by its labels there, +and only on the control-cluster path. `sourceNamespaces` admits it on a source cluster, by name or +`"*"`, only on the source-cluster path, and only when the `KubeProvider` lends that namespace to the +`GitTarget` in `spec.allowedGitTargets`. Omitted, the list is `[{consumerNamespaces: {from: Same}}]`. +A request no grant admits is `NotAllowed`, with a message naming the field that applies. The +`GitTarget` that admitted a request is recorded in `status.gitTargetUID`: a `GitTarget` of that name +with another UID replaced it and must admit the request again, and its refusal is written only once +no worker holds the request. + ## Request and window contract -The request identifies the target in `spec.gitTargetRef.name`, may provide `spec.message`, and sets -`spec.window` (which window to attach to, how long to wait for one, and the timers that close it) and -`spec.whenNothingToCommit`. It is handled by the target’s single branch worker, +The request identifies the target in `spec.gitTargetRef`, may provide `spec.message`, and sets +`spec.window` (which window to attach to, how long to wait for one, and the timers that close it), +`spec.whenNothingToCommit` and `spec.catchUp`. It is handled by the target’s single branch worker, so resource events and the attach request share one FIFO. The worker attaches a request only when all of these match an open window: @@ -21,9 +93,8 @@ The worker attaches a request only when all of these match an open window: 2. the named actor, if either side has one; and 3. whether each side names an actor (`AttributionOutcome.NamesActor`). -The third rule keeps independently configured command admission and live-resource attribution from being -coupled. A request with no named submitter can attach to either a configured-author or an unresolved live -window, but never to a named actor’s window. A request with a named submitter can attach only to that +A request with no named author can attach to either a configured-author or an unresolved live +window, but never to a named actor’s window. A request with a named author can attach only to that actor’s named window. Therefore one user’s request never finalizes another user’s work. A window is only ever opened by a write; a request attaches to one and never opens one. On first @@ -39,9 +110,10 @@ runs from the attach, and so does the first idle interval. A waiting request get the target's own timers are applied, so a target with `idleTimeout: 0s` does not close the window before the request can attach. -`attachTimeout` defaults to `"2s"` rather than `"0s"` because the write a request exists to publish -reaches the worker strictly after the request does: a watch event is held until its audit fact -arrives, so a zero wait is shorter than the smallest window-open latency the pipeline can produce. +`attachTimeout` defaults to `"5s"` rather than `"0s"`, and counts from the worker's registration, +which is after the request's own author was found. The write a request exists to publish reaches +the worker through the same audit join, and on a busy cluster after it, so a zero wait is shorter +than the window-open latency the pipeline can produce. The two durations are separate settings with separate clocks: `attachTimeout` runs from registration, `maxDuration` from the attach. Each is a pointer, so an omitted value and an explicit `"0s"` stay distinguishable. A zero `attachTimeout` attaches to a window already open, or gives up at @@ -75,17 +147,25 @@ pushed commit is required; `Ready=True` also includes successful no-commit outco ## Authorship -The validating admission webhook captures a command submitter before the `CommitRequest` persists. The -controller performs one best-effort, present-or-never lookup: +A request's author comes from its own audit event, on the cluster it was created on, the same way a +watched write's does: the core looks it up in the same fact index, on the audit route of the +`GitTarget`'s `KubeProvider`, keyed by the request's UID and the resourceVersion it was created with, +and waits up to `--author-attribution-grace`. It acquires the fact stream of `commitrequests` on that +route, so the fact follower reads those audit facts at all. + +| Outcome | The request | +|---|---| +| attribution is off | names no author, and nothing waits for one | +| a fact names the creator | names them, in `status.author` | +| no fact, on a provider whose `AuditFactsReceived` is not `True` | names no author: the cluster sends no audit | +| no fact, on a provider that has received facts | `Ready=True`, reason `AuthorUnresolved`: its fact went missing, so it saves nothing rather than take the window that should stay unattributed | -| Admission result | `AuthorAttributed` | Request claim | -|---|---|---| -| submitter record found | `True` / `AttributedFromAdmission` | that named actor | -| capture ran but no record | `False` / `CommitterFallback` | no actor | -| webhook disabled or Redis unavailable | `False` / `AuthorCaptureDisabled` | no actor | +The wait runs whenever attribution is on; whether the provider has received facts is checked only +after it, so a request is not refused while its provider's first audit event is still in transit. +The author is written in the claim: every status write moves the resourceVersion the fact is keyed +by, so a later lookup could never match. -`AuthorAttributed=False` is not a statement about the eventual Git author. The matched live window decides -that: +The author decides which window the request attaches to. The matched window decides the Git author: - configured-author, replay, and resync windows use the configured committer; - a resolved live attribution fact uses the authenticated actor; and @@ -96,9 +176,7 @@ author states. ## Lifecycle and outcomes -The controller stamps its conditions on first reconcile, immediately attaches the request, and polls the -worker. There is no audit wait for a `CommitRequest`; a delayed admission record cannot arrive after the -object is visible. +The owner claims the request, attaches it at once, and polls the worker. | Outcome | Conditions | |---|---| @@ -107,18 +185,23 @@ object is visible. | Nothing to commit, with `whenNothingToCommit: CommitEmpty` | `Ready=True`, `Pushed=True`, reason `NoWindow` or `AlreadyPresent`; `status.commit` is the empty commit | | Window produced no diff, and the remote agreed | `Ready=True`, `Pushed=False`, reason `AlreadyPresent` | | Terminal write failure, withdrawal before the worker holds it, shutdown with an unresolved held write, or a target that may not be written | `Ready=False`, `Pushed=False`, `Stalled=True`, reason `FinalizeFailed` | +| Refused by the grants, or made on the wrong cluster | `Ready=False`, `Stalled=True`, reason `NotAllowed` | +| The `GitTarget` did not exist within `attachTimeout` | `Ready=False`, `Stalled=True`, reason `GitTargetNotFound` | +| An earlier save of the same author on the `GitTarget` still in progress | `Ready=False`, `Stalled=True`, reason `SaveInProgress` | +| No audit fact names the creator, on a cluster that sends them | `Ready=True`, `Pushed=False`, reason `AuthorUnresolved` | A retryable materialization or push failure keeps a held request in `WaitingForPush`; it is not a terminal outcome. Publication has to settle the request or the worker must establish that it will no longer execute it before the controller can report failure. -`Reconciling=True` is the normal in-progress state, with the phase the worker reports as its reason: -`Progressing`, `WaitingForWorker`, `WaitingForWindow`, `CollectingWindow`, `WaitingForPush`. Once the +`Ready=Unknown` with `Reconciling=True` is the normal in-progress state, with the phase the worker +reports as its reason: `Saving` (claimed, not yet registered), `WaitingForWorker`, `WaitingForWindow`, +`CollectingWindow`, `WaitingForPush`. Once the request is `CollectingWindow` or `WaitingForPush` the worker holds it, and only the worker ends it. Before that, a request still unresolved `attachTimeout + maxDuration + 120s` after creation is withdrawn from the worker, which cancels it for good, and fails with `FinalizeFailed`; a GitTarget that never started a worker is named in the message. The withdraw rides the worker's queue behind every attach, and is a no-op for a -request the worker holds, so the controller's failure and the worker's answer cannot disagree. +request the worker holds, so the reported failure and the worker's answer cannot disagree. The withdraw goes to the worker that accepted the request's attach, not to whichever worker the GitTarget names now: a worker shared by several GitTargets outlives a deleted one, and a retired worker finishes its @@ -188,6 +271,16 @@ in the words its `GitTarget` uses. While an outage has paused the branch's intak refused at admission and the controller sends it again; past its safety window, it fails with `FinalizeFailed` only after withdrawal confirms that the worker does not hold it. +### Retention + +A finished request (`Ready=True` or `Stalled=True`) is stamped `configbutler.ai/delete-after`, now +plus `--commit-request-ttl`, and deleted once that time has passed; the delete is conditional on the +version that was judged. The annotation is the contract: removing it keeps the request, and a value +already there is never replaced. A request that asked to catch up is deleted only once `Applied` is +settled. On the control cluster this always applies. On a source cluster only +`commitRequests: ReconcileAndDeleteFinished` stamps and deletes; with `Reconcile` nothing is stamped, +and an annotation stamped before a switch back to `Reconcile` stays without effect. + ### Capacity, replay, and restart limits The budget charges every kept item its payload bytes, its message, and a fixed overhead for its diff --git a/docs/spec/status-conditions-guide.md b/docs/spec/status-conditions-guide.md index 938df9c5..c6e35da5 100644 --- a/docs/spec/status-conditions-guide.md +++ b/docs/spec/status-conditions-guide.md @@ -334,34 +334,32 @@ trio; the bounded "did the work actually happen" signal lives on the `Pushed` do ```go const ( - TypeReady = "Ready" // True at a non-error terminal outcome - TypeReconciling = "Reconciling" // True until the request reaches a terminal outcome - TypeStalled = "Stalled" // True when the finalize failed (kstatus Failed) - TypeAuthorAttributed = "AuthorAttributed" // Whether admission captured the command submitter - TypePushed = "Pushed" // True once the commit is in the remote repository + TypeReady = "Ready" // Unknown while the save runs; True at a non-error terminal outcome + TypeReconciling = "Reconciling" // True until the request reaches a terminal outcome + TypeStalled = "Stalled" // True when the request failed (kstatus Failed) + TypePushed = "Pushed" // True once the commit is in the remote repository ) ``` A request's progress runs through the phases the branch worker reports: waiting for a window, -collecting in it, and waiting for the push. `AuthorAttributed` settles at first sight because command -authorship is captured synchronously at admission; it never has an `Unknown` or audit-wait state. +collecting in it, and waiting for the push. Its author is found from its audit event before its first +status write, which records it in `status.author`, so no condition carries it. Canonical reads: -- in progress: `Reconciling=True` reason `Progressing` or `WaitingForWorker`, then `WaitingForWindow`, - `CollectingWindow` or `WaitingForPush` as the worker reports it, `AuthorAttributed` settled, +- in progress: `Ready=Unknown`, `Reconciling=True` reason `Saving` or `WaitingForWorker`, then + `WaitingForWindow`, `CollectingWindow` or `WaitingForPush` as the worker reports it, `Pushed=Unknown` → kstatus InProgress - committed: `Ready=True`, `Pushed=True`, `Stalled=False`, reason `Committed` → kstatus Current -- benign no-commit (nothing to save / already present / foreign open window): `Ready=True`, `Pushed=False`, - `Stalled=False`, with the specific reason on `Ready` (`NoWindow`, `AlreadyPresent`, `WindowMismatch`) - → kstatus Current (a correct, non-error outcome). With `whenNothingToCommit: CommitEmpty`, nothing to - save or already present records an empty commit instead: `Pushed=True`, the reason still naming the - cause -- failed finalize: `Ready=False`, `Stalled=True`, reason `FinalizeFailed` → kstatus Failed - -`AuthorAttributed=True` (`AttributedFromAdmission`) means the command submitter was captured. `False` -(`CommitterFallback`) means capture ran but no admission author record exists; `False` -(`AuthorCaptureDisabled`) means capture is disabled. Both claim no actor and can attach only to an unnamed -window. That condition does not itself determine the Git author: the attached watch window is -configured-author (the committer) when watch attribution is disabled, or explicitly unresolved when watch -attribution ran but could not name an actor. +- benign no-commit (nothing to save / already present / foreign open window / a missing audit fact): + `Ready=True`, `Pushed=False`, `Stalled=False`, with the specific reason on `Ready` (`NoWindow`, + `AlreadyPresent`, `WindowMismatch`, `AuthorUnresolved`) → kstatus Current (a correct, non-error + outcome). With `whenNothingToCommit: CommitEmpty`, nothing to save or already present records an + empty commit instead: `Pushed=True`, the reason still naming the cause +- failed: `Ready=False`, `Stalled=True`, reason `FinalizeFailed`, `NotAllowed`, `GitTargetNotFound` or + `SaveInProgress` → kstatus Failed + +`status.author` empty means no audit fact named the creator, or attribution is off: the request +claims no actor and can attach only to an unnamed window. That does not itself determine the Git +author: the attached watch window is configured-author (the committer) when watch attribution is +disabled, or explicitly unresolved when watch attribution ran but could not name an actor. diff --git a/test/e2e/cluster/audit/policy.yaml b/test/e2e/cluster/audit/policy.yaml index bfc033c8..2aa32aca 100644 --- a/test/e2e/cluster/audit/policy.yaml +++ b/test/e2e/cluster/audit/policy.yaml @@ -53,7 +53,7 @@ rules: - deletecollection # No resources listed = Matches everything else (including CRDs). # This also captures `commitrequests.configbutler.ai` create events, which - # drive the "finalize the open commit window now" path. If this catch-all - # is ever narrowed, keep at least Metadata-level capture for - # `commitrequests` create events — the consumer reads the object body from - # the apiserver, so Metadata is sufficient. + # name a CommitRequest's author, as a write's event names its author. If this + # catch-all is ever narrowed, keep at least Metadata-level capture for + # `commitrequests` create events — the author is joined by uid and + # resourceVersion, which Metadata carries. diff --git a/test/e2e/quickstart_framework_e2e_test.go b/test/e2e/quickstart_framework_e2e_test.go index 2745336e..36a35d93 100644 --- a/test/e2e/quickstart_framework_e2e_test.go +++ b/test/e2e/quickstart_framework_e2e_test.go @@ -138,7 +138,7 @@ func (r *quickstartFrameworkRun) helmInstallReadmeQuickstart() { chart = "charts/gitops-reverser" } - // ONE apply, exactly as the README's step 4 does it. This works because the chart installs no + // ONE apply, exactly as the quickstart's install step does it. This works because the chart installs no // admission webhook — the starter GitTarget is gated by nothing, so it is admitted while the // Deployment it rolls is still coming up. Namespace authorization is enforced at // reconcile instead (checkSourceAuthorization, inside the Validated gate), which does not need From 3f71841c13b826d9750f35e7e7dc2bfe04d68ff8 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:20:05 +0000 Subject: [PATCH 28/33] fix(commitrequest): report a granted Role at once, and retry a source cluster's client A namespace whose list was denied stayed reported Forbidden until its next watch ended, up to five minutes after the Role was granted: a list that succeeds now clears it. A watcher whose cluster client could not be built gave up for good while reporting Watching: it now retries, and reports the cluster Unreachable until it can. Co-Authored-By: Claude Opus 5.5 --- internal/controller/commitrequest_server.go | 43 ++++++++++++++++--- .../controller/commitrequest_server_test.go | 5 +++ 2 files changed, 41 insertions(+), 7 deletions(-) diff --git a/internal/controller/commitrequest_server.go b/internal/controller/commitrequest_server.go index 20797552..1927c12a 100644 --- a/internal/controller/commitrequest_server.go +++ b/internal/controller/commitrequest_server.go @@ -122,9 +122,10 @@ type commitRequestWatcher struct { mu sync.Mutex // forbidden are the namespaces whose list or watch was denied, and undeletable those where a - // finished request could not be deleted. + // finished request could not be deleted. unreachable is why the cluster's client could not be built. forbidden map[string]bool undeletable map[string]bool + unreachable error } // CommitRequestsEvents names a KubeProvider whose report changed, so its reconcile writes it at once. @@ -387,6 +388,9 @@ func (w *commitRequestWatcher) report(deletes bool) CommitRequestsReport { if deletes { verbs += ", and patch and delete on commitrequests" } + if w.unreachable != nil { + return unreachableCommitRequests(w.unreachable) + } if len(w.forbidden) > 0 { return CommitRequestsReport{Status: metav1.ConditionFalse, Reason: ReasonCommitRequestsForbidden, Message: fmt.Sprintf("the credential may not list or watch CommitRequests in %s: grant it %s, "+ @@ -425,9 +429,8 @@ func (s *CommitRequestServer) run( release := s.FactStreams.Acquire(queue.FactStreamKeyFor(route, commitRequestGVR().GroupResource())) defer release() } - dc, err := s.Clusters.ApplierDynamicClient(ctx, watch.SourceClusterID(watcher.provider)) + dc, err := s.clusterClient(ctx, log, watcher) if err != nil { - log.Info("Cannot reach the KubeProvider's cluster for CommitRequests; will retry", "err", err.Error()) return } resource := dc.Resource(commitRequestGVR()) @@ -455,6 +458,28 @@ func (w *commitRequestWatcher) markUndeletable(namespace string) { w.mark(w.undeletable, namespace, true) } +// clusterClient builds the client of the watcher's cluster, retrying until it can or ctx ends; until +// then the watcher reports the cluster unreachable. +func (s *CommitRequestServer) clusterClient( + ctx context.Context, log logr.Logger, watcher *commitRequestWatcher, +) (dynamic.Interface, error) { + for { + dc, err := s.Clusters.ApplierDynamicClient(ctx, watch.SourceClusterID(watcher.provider)) + watcher.mu.Lock() + watcher.unreachable = err + watcher.mu.Unlock() + if err == nil { + return dc, nil + } + log.Info("Cannot reach the KubeProvider's cluster for CommitRequests; will retry", "err", err.Error()) + select { + case <-ctx.Done(): + return nil, ctx.Err() + case <-time.After(commitRequestReconnect): + } + } +} + // watchNamespace lists and watches one namespace's CommitRequests, or every namespace's, reconnecting // after a failure, until ctx ends. func (s *CommitRequestServer) watchNamespace( @@ -462,8 +487,10 @@ func (s *CommitRequestServer) watchNamespace( resource dynamic.NamespaceableResourceInterface, namespace string, ) { for ctx.Err() == nil { - err := s.session(ctx, watcher, resource.Namespace(namespace)) - watcher.mark(watcher.forbidden, namespaceLabel(namespace), apierrors.IsForbidden(err)) + err := s.session(ctx, watcher, resource.Namespace(namespace), namespaceLabel(namespace)) + if apierrors.IsForbidden(err) { + watcher.mark(watcher.forbidden, namespaceLabel(namespace), true) + } if err != nil && ctx.Err() == nil { log.Info("The watch of CommitRequests ended; reconnecting", "namespace", namespaceLabel(namespace), "err", err.Error()) @@ -475,14 +502,16 @@ func (s *CommitRequestServer) watchNamespace( } } -// session lists the requests and watches them, until the watch ends. +// session lists the requests and watches them, until the watch ends. A list that succeeds clears a +// denial reported for the namespace, so a Role granted since is reported at once. func (s *CommitRequestServer) session( - ctx context.Context, watcher *commitRequestWatcher, resource dynamic.ResourceInterface, + ctx context.Context, watcher *commitRequestWatcher, resource dynamic.ResourceInterface, namespace string, ) error { list, err := resource.List(ctx, metav1.ListOptions{}) if err != nil { return err } + watcher.mark(watcher.forbidden, namespace, false) for i := range list.Items { s.consider(ctx, watcher, &list.Items[i]) } diff --git a/internal/controller/commitrequest_server_test.go b/internal/controller/commitrequest_server_test.go index 12fea160..54b5cad9 100644 --- a/internal/controller/commitrequest_server_test.go +++ b/internal/controller/commitrequest_server_test.go @@ -135,6 +135,11 @@ func TestCommitRequestWatcher_Report(t *testing.T) { every := &commitRequestWatcher{namespaces: []string{"*"}, forbidden: map[string]bool{}, undeletable: map[string]bool{}} assert.Contains(t, every.report(false).Message, "every namespace") + + every.unreachable = errors.New("no route to host") + report = every.report(false) + assert.Equal(t, ReasonCommitRequestsUnreachable, report.Reason) + assert.Contains(t, report.Message, "no route to host") } // On a source cluster a request is admitted by a grant's sourceNamespaces, and only in a namespace the From 36bb4b4311f5cf8b7fcad0d8b95ddf8558e04a28 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:21:01 +0000 Subject: [PATCH 29/33] docs(review): phase 1 of the one-CommitRequest design is built Co-Authored-By: Claude Opus 5.5 --- docs/review/20261027-save-request-simplification.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/review/20261027-save-request-simplification.md b/docs/review/20261027-save-request-simplification.md index c0f5a8b7..2be9ad39 100644 --- a/docs/review/20261027-save-request-simplification.md +++ b/docs/review/20261027-save-request-simplification.md @@ -4,9 +4,9 @@ Review of `feat/gitops-api-asks-converge-root-ca` (PR #456, at `70d07d91`), on o much of the `SaveRequest` / `RemoteSaveRequest` machinery can go, and in particular whether the admission webhook can go entirely. -Status: design decided on 2026-10-27, nothing built yet. The first version of this review proposed -one kind on the source cluster only, named `SaveRequest`. The decisions below replace that -proposal. +Status: design decided on 2026-10-27; phase 1 is built in PR #456, phase 2 of the CRD signal is not. +The first version of this review proposed one kind on the source cluster only, named `SaveRequest`. +The decisions below replace that proposal. Line references are to `70d07d91`, before the change. ## Decisions From cecd5388243d254dc58a1726b760be8a3d38b00f Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:35:18 +0000 Subject: [PATCH 30/33] test(e2e): make an edit a save was created for once the save is claimed A CommitRequest is registered with the branch worker once its own audit event named its author, no longer at creation, and its edit is attributed the same way. On the e2e GitTargets, whose windows close on the first silence, an edit made right after the request could reach the worker first and commit alone: the Flux applier's catch-up save ended NoWindow. The two specs that save before they edit wait for the claim, Ready=Unknown, and configuration.md and UPGRADING say so. Co-Authored-By: Claude Opus 5.5 --- docs/UPGRADING.md | 3 +++ docs/configuration.md | 7 +++++++ test/e2e/bi_directional_echoes_e2e_test.go | 1 + test/e2e/commit_request_e2e_test.go | 12 ++++++++++++ test/e2e/flux_applier_e2e_test.go | 1 + 5 files changed, 24 insertions(+) diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 558979ad..6c1ae59d 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -31,6 +31,9 @@ those that remain expire on their own. `Stalled=True` is unaffected. - **`spec.window.attachTimeout` defaults to `5s`**, since the request now waits for its own audit event as its edits do. A request that sets it keeps its value. +- **A request is registered once its author is found**, no longer at creation. A client that creates a + request and then makes the edit it saves, on a `GitTarget` whose windows close on the first silence + (`idleTimeout: "0s"`), waits for `Ready=Unknown` before the edit, or the edit can commit alone. - **A `GitTarget` that mirrors another cluster.** A `CommitRequest` on the cluster GitOps Reverser runs in, for such a `GitTarget`, is refused with reason `NotAllowed`. Create it on the cluster the `GitTarget` mirrors instead, as described in diff --git a/docs/configuration.md b/docs/configuration.md index 4b2b2e5d..2749da5b 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -3078,6 +3078,13 @@ up to `--author-attribution-grace` each. That is rare when the audit policy cove [suggested alert](interpreting-metrics.md#suggested-alerts) on `tier="absent"` says when it is not. Widening `attachTimeout` hides that delay rather than removing it. +A request is registered with the branch worker only once its own audit event has named its author, +and its edit is attributed the same way, so a request created just before its edit does not always +reach the worker first. On a `GitTarget` whose windows close on the first silence (`idleTimeout: +"0s"`), such an edit can commit alone before the request is there to attach to it. Make the edit +once the request is claimed (`Ready=Unknown`), or give the `GitTarget` an `idleTimeout` that covers +the gap. + `maxDuration`, and `idleTimeout` when set, cover the span of the save's own writes after the attach. The worst case from registration to the finalize is `attachTimeout` plus `maxDuration`; the push follows, with its cooldown and any retries. diff --git a/test/e2e/bi_directional_echoes_e2e_test.go b/test/e2e/bi_directional_echoes_e2e_test.go index 3e1b6e4a..927839be 100644 --- a/test/e2e/bi_directional_echoes_e2e_test.go +++ b/test/e2e/bi_directional_echoes_e2e_test.go @@ -397,6 +397,7 @@ func echoScenarios(run func() *echoRun) { By("saving an API edit with a CommitRequest while a Git-side push changes the same object") name := applyCommitRequestWithGenerateName(r.testNs, "echo-save-", r.targetName, "e2e: save the edit") + waitForCommitRequestClaimed(r.testNs, name) r.editThroughAPI("saved", "s-save") r.pushFromOutside("saved", "s-git") diff --git a/test/e2e/commit_request_e2e_test.go b/test/e2e/commit_request_e2e_test.go index f6bb6259..b8945f06 100644 --- a/test/e2e/commit_request_e2e_test.go +++ b/test/e2e/commit_request_e2e_test.go @@ -488,6 +488,18 @@ spec: Expect(err).NotTo(HaveOccurred(), fmt.Sprintf("failed to apply CommitRequest %s/%s", namespace, name)) } +// waitForCommitRequestClaimed waits until the request is claimed: Ready=Unknown, written once its own +// audit event named its author and before it is attached. A save made before its edit waits for this +// on a GitTarget whose windows close on the first silence (e2eCommitWindow): the request is +// registered only after its author is found, and its edit, attributed the same way, could otherwise +// reach the worker first and commit alone. +func waitForCommitRequestClaimed(namespace, name string) { + GinkgoHelper() + Eventually(func(g Gomega) { + g.Expect(commitRequestCondition(g, namespace, name, "Ready")).To(Equal("Unknown")) + }, time.Minute, 250*time.Millisecond).Should(Succeed(), "CommitRequest %s/%s is claimed", namespace, name) +} + // commitRequestField reads a jsonpath field off a CommitRequest object. func commitRequestField(g Gomega, namespace, name, jsonPath string) string { out, err := kubectlRunInNamespace(namespace, "get", "commitrequest", name, diff --git a/test/e2e/flux_applier_e2e_test.go b/test/e2e/flux_applier_e2e_test.go index 8a85d720..8b3a2d06 100644 --- a/test/e2e/flux_applier_e2e_test.go +++ b/test/e2e/flux_applier_e2e_test.go @@ -380,6 +380,7 @@ spec: maxDuration: "4s" `, request, testNs, destName), "apply", "-f", "-") Expect(err).NotTo(HaveOccurred()) + waitForCommitRequestClaimed(testNs, request) _, err = kubectlRunInNamespace(testNs, "patch", "configmap", configMap, "--type=merge", "-p", `{"data":{"test-key":"saved"}}`) Expect(err).NotTo(HaveOccurred()) From dada53a42643e7618ff8dbb702602111bb28b02c Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 10:40:03 +0000 Subject: [PATCH 31/33] fix(commitrequest): follow a rotated credential, re-read audit state, and keep retention to the owner - A source cluster's watcher restarts when its KubeProvider's credential version changes, and builds its client fresh from that credential rather than taking the shared cached one, so its watches and status writes move to the rotated kubeconfig together. - Whether a provider has received audit facts is read after the author wait, not from the provider fetched before it: facts that arrive during the wait refuse a request whose own fact is still missing. - Only the path that owns a finished request, or refused it because no path serves its GitTarget, follows its apply and deletes it. Another path that deletes finished requests leaves it alone whatever delete-after it carries; once the GitTarget is gone, any deleting path may. Co-Authored-By: Claude Opus 5.5 --- docs/spec/commitrequest-design.md | 11 ++- internal/controller/commitrequest_core.go | 26 +++-- .../controller/commitrequest_core_test.go | 82 ++++++++++++++++ internal/controller/commitrequest_decide.go | 62 ++++++++---- internal/controller/commitrequest_server.go | 36 ++++--- .../controller/commitrequest_server_test.go | 94 ++++++++++++++++++- internal/watch/cluster_context.go | 42 ++++++--- 7 files changed, 294 insertions(+), 59 deletions(-) diff --git a/docs/spec/commitrequest-design.md b/docs/spec/commitrequest-design.md index 4dc5baef..130f54d9 100644 --- a/docs/spec/commitrequest-design.md +++ b/docs/spec/commitrequest-design.md @@ -21,8 +21,9 @@ else once: whose `spec.commitRequests` is `Reconcile` or `ReconcileAndDeleteFinished`, through that provider's credential, in the namespaces it lends its `GitTarget`s in `spec.allowedGitTargets`. It writes status, and with `ReconcileAndDeleteFinished` deletes finished requests, through the same - credential. It never writes the CRD, and reports its state in the `KubeProvider`'s - `CommitRequestsServed` condition. + credential. A rotated kubeconfig restarts the watch on a client built from the new credential, so + watches and status writes never keep the old one. It never writes the CRD, and reports its state in + the `KubeProvider`'s `CommitRequestsServed` condition. The core takes one step at a time, each decided again from the object as it is stored. Its responsibilities, in order: admission by `spec.allowedCommitRequests`; the request's author; the @@ -281,6 +282,12 @@ settled. On the control cluster this always applies. On a source cluster only `commitRequests: ReconcileAndDeleteFinished` stamps and deletes; with `Reconcile` nothing is stamped, and an annotation stamped before a switch back to `Reconcile` stays without effect. +Retention belongs to the path that owns the request, or that refused it because no path serves its +`GitTarget`. Another path that deletes finished requests, such as a second provider watching the same +namespace, or the control cluster under a kubeconfig that points back at it, leaves the request +alone, whatever `delete-after` it carries. Once the `GitTarget` is gone no path owns the request, +and any path that deletes finished requests may. + ### Capacity, replay, and restart limits The budget charges every kept item its payload bytes, its message, and a fixed overhead for its diff --git a/internal/controller/commitrequest_core.go b/internal/controller/commitrequest_core.go index c9ace55a..b9505383 100644 --- a/internal/controller/commitrequest_core.go +++ b/internal/controller/commitrequest_core.go @@ -294,7 +294,10 @@ func (c *commitRequestCore) claim( if !admitted { return c.decide(ctx, p, cr, commitRequestOutcome{err: refusal}, false) } - author, refused := c.creator(ctx, cr, provider) + author, refused, err := c.creator(ctx, cr, provider) + if err != nil { + return 0, err + } if refused { return c.decide(ctx, p, cr, commitRequestOutcome{authorUnresolved: true}, false) } @@ -315,11 +318,15 @@ func (c *commitRequestCore) claim( // // The lookup matches the request's resourceVersion, which every status write moves, so it is made // only before the claim, and the claim records the answer. +// +// Whether the provider has received audit facts is read after the wait, not from provider: the first +// facts may arrive during it, and a request whose own fact is still missing then must be refused +// rather than take the unnamed window. A provider that cannot be read then is an error, retried. func (c *commitRequestCore) creator( ctx context.Context, cr *configbutleraiv1alpha3.CommitRequest, provider *configbutleraiv1alpha3.KubeProvider, -) (string, bool) { +) (string, bool, error) { if c.authors == nil { - return "", false + return "", false, nil } c.streams.ensure(provider.AuditRoute()) user, outcome := c.authors.ResolveAuthor(ctx, watch.AuthorQuery{ @@ -333,11 +340,16 @@ func (c *commitRequestCore) creator( ExactCapable: true, }) if outcome.NamesActor() { - return user.Username, false + return user.Username, false, nil + } + if outcome == git.AttributionNotAttempted { + return "", false, nil + } + var current configbutleraiv1alpha3.KubeProvider + if err := c.reader.Get(ctx, provider.Key(), ¤t); err != nil { + return "", false, fmt.Errorf("read KubeProvider %s after the author wait: %w", provider.Key(), err) } - refused := outcome != git.AttributionNotAttempted && - apimeta.IsStatusConditionTrue(provider.Status.Conditions, KubeProviderConditionAuditFactsReceived) - return "", refused + return "", apimeta.IsStatusConditionTrue(current.Status.Conditions, KubeProviderConditionAuditFactsReceived), nil } // writeClaim writes the first status of a request: its author, the GitTarget that admitted it, and diff --git a/internal/controller/commitrequest_core_test.go b/internal/controller/commitrequest_core_test.go index c93ae03a..bda11e6b 100644 --- a/internal/controller/commitrequest_core_test.go +++ b/internal/controller/commitrequest_core_test.go @@ -445,6 +445,88 @@ func TestCommitRequestCore_ARequestNoFactNames(t *testing.T) { }) } +// factsArriveDuringTheWait names nobody, and while it waits the provider's first audit facts arrive: +// its AuditFactsReceived turns True. +type factsArriveDuringTheWait struct { + c client.Client + provider types.NamespacedName +} + +func (f factsArriveDuringTheWait) ResolveAuthor(ctx context.Context, _ watch.AuthorQuery) ( + git.UserInfo, git.AttributionOutcome, +) { + var provider configv1alpha3.KubeProvider + if err := f.c.Get(ctx, f.provider, &provider); err == nil { + apimeta.SetStatusCondition(&provider.Status.Conditions, metav1.Condition{ + Type: KubeProviderConditionAuditFactsReceived, Status: metav1.ConditionTrue, Reason: "Received", + }) + _ = f.c.Update(ctx, &provider) + } + return git.UserInfo{}, git.AttributionUnresolved +} + +// A provider whose first audit facts arrive while a request waits for its own is one that sends them: +// the request whose fact is still missing is refused, never attached to the unnamed window. +func TestCommitRequestCore_FactsArrivingDuringTheWaitRefuseAnUnnamedRequest(t *testing.T) { + c := newCommitRequestClient(t, nil, newCommitRequest("save")) + f := committed() + authors := factsArriveDuringTheWait{c: c, provider: types.NamespacedName{Namespace: "default", Name: localProvider}} + serveStep(t, &commitRequestCore{reader: c, finalizer: f, authors: authors}, controlPath(c), "save") + + assert.Empty(t, f.calls, "nothing is attached") + ready := commitRequestReady(t, c, "save") + require.NotNil(t, ready) + assert.Equal(t, crReasonAuthorUnresolved, ready.Reason) +} + +// A finished request is kept, its apply followed and its delete-after honored, only by the path that +// keeps it: the GitTarget's own, or the one that refused it because no path serves it. Another path that +// deletes finished requests leaves it alone, whatever delete-after it carries. Once its GitTarget is +// gone, no path owns it any more, and any path that deletes may. +func TestCommitRequestCore_RetentionIsTheKeepersAlone(t *testing.T) { + past := time.Now().Add(-time.Minute).UTC().Format(time.RFC3339) + for _, tc := range []struct { + name string + target *configv1alpha3.GitTarget + providers []client.Object + path func(client.Client) commitRequestPath + goneTarget bool + wantDeleted bool + }{ + {name: "another provider's watcher that deletes", target: mirroringTarget("team-a"), + providers: []client.Object{kubeconfigProvider("team-a", true), kubeconfigProvider("team-b", true)}, + path: func(c client.Client) commitRequestPath { return watcherPath(c, "team-b", true) }}, + {name: "the control cluster, under a watcher pointing back at it", target: mirroringTarget("team-a"), + providers: []client.Object{kubeconfigProvider("team-a", true)}, path: controlPath}, + {name: "the owner that deletes", target: mirroringTarget("team-a"), + providers: []client.Object{kubeconfigProvider("team-a", true)}, + path: func(c client.Client) commitRequestPath { return watcherPath(c, "team-a", true) }, + wantDeleted: true}, + {name: "the control cluster that refused it", target: mirroringTarget("team-a"), + providers: []client.Object{kubeconfigProvider("team-a", false)}, path: controlPath, wantDeleted: true}, + {name: "any path that deletes, once the GitTarget is gone", target: mirroringTarget("team-a"), + providers: []client.Object{kubeconfigProvider("team-a", true), kubeconfigProvider("team-b", true)}, + path: func(c client.Client) commitRequestPath { return watcherPath(c, "team-b", true) }, + goneTarget: true, wantDeleted: true}, + } { + t.Run(tc.name, func(t *testing.T) { + cr := finishedWithDeleteAfter("save", past) + cr.Spec.GitTargetRef.Namespace = "default" + c := newCommitRequestClient(t, nil, append([]client.Object{cr, tc.target}, tc.providers...)...) + if tc.goneTarget { + require.NoError(t, c.Delete(context.Background(), tc.target)) + } + core := &commitRequestCore{reader: c, finalizer: committed(), ttl: time.Hour} + after := serveStep(t, core, tc.path(c), "save") + + assert.Equal(t, !tc.wantDeleted, commitRequestExists(t, c, "save")) + if !tc.wantDeleted { + assert.Equal(t, commitRequestUnownedRecheck, after, "looked at again, in case its owner changes") + } + }) + } +} + // A finished request is stamped with delete-after and deleted once it has passed on a path that // deletes finished requests, and left as it is on one that does not. Removing the annotation keeps it. func TestCommitRequestCore_Retention(t *testing.T) { diff --git a/internal/controller/commitrequest_decide.go b/internal/controller/commitrequest_decide.go index 4bc3fd10..cf07c63a 100644 --- a/internal/controller/commitrequest_decide.go +++ b/internal/controller/commitrequest_decide.go @@ -143,29 +143,50 @@ func (c *commitRequestCore) writeOutcome( } } -// serveFinished is the step of a request whose save ended: following the apply it asked for, as its -// owner, and then its retention. +// serveFinished is the step of a request whose save ended: following the apply it asked for, and then +// its retention, both only on a path that keeps the request. func (c *commitRequestCore) serveFinished( ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, ) (time.Duration, error) { - if !followsApply(cr) { - return c.expireFinished(ctx, p, cr) + if !followsApply(cr) && !expires(p, cr) { + return 0, nil } - // A GitTarget that is gone is asked for nothing, which is true for every observer. - target, found, err := c.gitTarget(ctx, cr.GitTargetKey()) + keeps, err := c.keeps(ctx, p, cr) if err != nil { return 0, err } - if found { - _, owner, _, err := c.owns(ctx, p, target) - if err != nil { - return 0, err - } - if !owner { - return commitRequestUnownedRecheck, nil - } + if !keeps { + return commitRequestUnownedRecheck, nil + } + if followsApply(cr) { + return c.followApply(ctx, p, cr) } - return c.followApply(ctx, p, cr) + return c.expireFinished(ctx, p, cr) +} + +// keeps decides whether path p keeps a finished request: follows its apply and deletes it once its +// delete-after time has passed. That is the owner, or the path that refused the request because no path +// serves its GitTarget. A request whose GitTarget is gone, or that names none, has no owner left, and +// what is asked of it is true for every observer, so any path keeps it. +// +// Without this, a path that deletes finished requests would delete one that another path owns and +// keeps, whenever it carries a delete-after annotation: set by a person, or by an earlier mode. +func (c *commitRequestCore) keeps( + ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, +) (bool, error) { + if cr.Spec.GitTargetRef.Name == "" { + return true, nil + } + key, named := p.gitTargetKey(cr) + if !named { + return false, nil + } + target, found, err := c.gitTarget(ctx, key) + if err != nil || !found { + return !found, err + } + _, owner, refusal, err := c.owns(ctx, p, target) + return owner || refusal != "", err } // stampDeleteAfter writes the delete-after annotation onto a request that has just finished, on a path @@ -218,10 +239,10 @@ func (c *commitRequestCore) stampDeleteAfter( func (c *commitRequestCore) expireFinished( ctx context.Context, p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest, ) (time.Duration, error) { - value, ok := cr.Annotations[CommitRequestDeleteAfterAnnotation] - if !p.deletes || !ok || !cr.DeletionTimestamp.IsZero() { + if !expires(p, cr) { return 0, nil } + value := cr.Annotations[CommitRequestDeleteAfterAnnotation] deleteAfter, parsed := parseDeleteAfter(value) if !parsed { // Kept rather than guessed at: an annotation a person mistyped is not consent to delete. @@ -243,6 +264,13 @@ func (c *commitRequestCore) expireFinished( return 0, nil } +// expires reports whether path p may delete cr at all: a path that deletes finished requests, and a +// request that carries a delete-after annotation and is not already being deleted. +func expires(p commitRequestPath, cr *configbutleraiv1alpha3.CommitRequest) bool { + _, ok := cr.Annotations[CommitRequestDeleteAfterAnnotation] + return p.deletes && ok && cr.DeletionTimestamp.IsZero() +} + // parseDeleteAfter reads the annotation's RFC 3339 time, reporting false for anything else. func parseDeleteAfter(value string) (time.Time, bool) { t, err := time.Parse(time.RFC3339, value) diff --git a/internal/controller/commitrequest_server.go b/internal/controller/commitrequest_server.go index 1927c12a..dbc0ba8b 100644 --- a/internal/controller/commitrequest_server.go +++ b/internal/controller/commitrequest_server.go @@ -71,11 +71,12 @@ const ( const commitRequestCRDFix = "apply it with kubectl apply --server-side -f commitrequest-crd.yaml, from the " + "release of GitOps Reverser this cluster is served by" -// CommitRequestClusters reaches a source cluster through a KubeProvider's identity. *watch.Manager -// satisfies it. +// CommitRequestClusters reaches a source cluster through a KubeProvider's identity, with clients built +// for the call, and says which version of the credential each was built from. *watch.Manager satisfies +// it. type CommitRequestClusters interface { - ApplierDynamicClient(ctx context.Context, clusterID string) (dynamic.Interface, error) - SourceDiscovery(ctx context.Context, clusterID string) (discovery.DiscoveryInterface, error) + SourceDynamicClient(ctx context.Context, clusterID string) (dynamic.Interface, string, error) + SourceDiscovery(ctx context.Context, clusterID string) (discovery.DiscoveryInterface, string, error) } // CommitRequestsReport is what the server last found of one KubeProvider's CommitRequests: the @@ -216,9 +217,9 @@ func (s *CommitRequestServer) syncProvider( "err", err.Error()) return } - report := s.probeCRD(ctx, key) + report, credential := s.probeCRD(ctx, key) if report.Reason == "" { - report = s.runWatcher(ctx, provider, namespaces) + report = s.runWatcher(ctx, provider, namespaces, credential) } else { s.stopWatcher(key) } @@ -264,13 +265,16 @@ func (s *CommitRequestServer) lentNamespaces( } // probeCRD reports, by discovery, a cluster that serves no CommitRequest CRD in v1alpha3, or that -// could not be read; an empty reason means the CRD is there. -func (s *CommitRequestServer) probeCRD(ctx context.Context, provider k8stypes.NamespacedName) CommitRequestsReport { - dc, err := s.Clusters.SourceDiscovery(ctx, watch.SourceClusterID(provider)) +// could not be read; an empty reason means the CRD is there. It also returns the version of the +// credential it was read with. +func (s *CommitRequestServer) probeCRD( + ctx context.Context, provider k8stypes.NamespacedName, +) (CommitRequestsReport, string) { + dc, credential, err := s.Clusters.SourceDiscovery(ctx, watch.SourceClusterID(provider)) if err != nil { - return unreachableCommitRequests(err) + return unreachableCommitRequests(err), "" } - return commitRequestCRDReport(dc) + return commitRequestCRDReport(dc), credential } // commitRequestCRDReport reads, through dc, whether its cluster serves commitrequests in @@ -319,9 +323,11 @@ func unreachableCommitRequests(err error) CommitRequestsReport { Message: "the cluster's API could not be read: " + err.Error()} } -// runWatcher starts, restarts or keeps provider's watcher for namespaces, and reports how it goes. +// runWatcher starts, restarts or keeps provider's watcher for namespaces, and reports how it goes. A +// changed credential restarts it, so its watches and the status it writes move to the new credential +// together, rather than keep the client it started with. func (s *CommitRequestServer) runWatcher( - ctx context.Context, provider *configbutleraiv1alpha3.KubeProvider, namespaces []string, + ctx context.Context, provider *configbutleraiv1alpha3.KubeProvider, namespaces []string, credential string, ) CommitRequestsReport { key := provider.Key() if len(namespaces) == 0 { @@ -330,7 +336,7 @@ func (s *CommitRequestServer) runWatcher( Message: "no GitTarget mirroring through this KubeProvider is lent a namespace in spec.allowedGitTargets"} } deletes := provider.DeletesFinishedCommitRequests() - config := fmt.Sprintf("%s|%t|%s", provider.AuditRoute(), deletes, strings.Join(namespaces, ",")) + config := fmt.Sprintf("%s|%t|%s|%s", provider.AuditRoute(), deletes, strings.Join(namespaces, ","), credential) s.mu.Lock() if s.watchers == nil { s.watchers = map[k8stypes.NamespacedName]*commitRequestWatcher{} @@ -464,7 +470,7 @@ func (s *CommitRequestServer) clusterClient( ctx context.Context, log logr.Logger, watcher *commitRequestWatcher, ) (dynamic.Interface, error) { for { - dc, err := s.Clusters.ApplierDynamicClient(ctx, watch.SourceClusterID(watcher.provider)) + dc, _, err := s.Clusters.SourceDynamicClient(ctx, watch.SourceClusterID(watcher.provider)) watcher.mu.Lock() watcher.unreachable = err watcher.mu.Unlock() diff --git a/internal/controller/commitrequest_server_test.go b/internal/controller/commitrequest_server_test.go index 54b5cad9..756f02e7 100644 --- a/internal/controller/commitrequest_server_test.go +++ b/internal/controller/commitrequest_server_test.go @@ -5,6 +5,8 @@ package controller import ( "context" "errors" + "slices" + "sync" "testing" "time" @@ -194,18 +196,56 @@ func TestCommitRequestCore_AdmissionBySide(t *testing.T) { }) } -// fakeClusters serves one source cluster: its dynamic client, and its discovery. +// fakeClusters serves one source cluster: its dynamic client, and its discovery, under one credential. type fakeClusters struct { dynamic dynamic.Interface discovery discovery.DiscoveryInterface } -func (f fakeClusters) ApplierDynamicClient(context.Context, string) (dynamic.Interface, error) { - return f.dynamic, nil +func (f fakeClusters) SourceDynamicClient(context.Context, string) (dynamic.Interface, string, error) { + return f.dynamic, "v1", nil } -func (f fakeClusters) SourceDiscovery(context.Context, string) (discovery.DiscoveryInterface, error) { - return f.discovery, nil +func (f fakeClusters) SourceDiscovery(context.Context, string) (discovery.DiscoveryInterface, string, error) { + return f.discovery, "v1", nil +} + +// rotatingClusters serves one source cluster whose credential the test rotates. Each credential reaches +// the cluster through a client of its own, so a client built from an old one sees none of what the new +// one does. +type rotatingClusters struct { + discovery discovery.DiscoveryInterface + + mu sync.Mutex + credential string + clients map[string]dynamic.Interface + // built are the credentials a client was built from, in order. + built []string +} + +func (r *rotatingClusters) rotate(credential string) { + r.mu.Lock() + defer r.mu.Unlock() + r.credential = credential +} + +func (r *rotatingClusters) SourceDynamicClient(context.Context, string) (dynamic.Interface, string, error) { + r.mu.Lock() + defer r.mu.Unlock() + r.built = append(r.built, r.credential) + return r.clients[r.credential], r.credential, nil +} + +func (r *rotatingClusters) builtFrom() []string { + r.mu.Lock() + defer r.mu.Unlock() + return slices.Clone(r.built) +} + +func (r *rotatingClusters) SourceDiscovery(context.Context, string) (discovery.DiscoveryInterface, string, error) { + r.mu.Lock() + defer r.mu.Unlock() + return r.discovery, r.credential, nil } // sourceCommitRequest is a CommitRequest as it is stored on a source cluster. @@ -280,6 +320,50 @@ func TestCommitRequestServer_ServesARequestOnASourceCluster(t *testing.T) { assert.False(t, ok, "a provider that ignores CommitRequests reports nothing") } +// A rotated kubeconfig moves a provider's watcher to the new credential: its watches, and the status it +// writes, run on a client built from it, rather than on the one the watcher started with. +func TestCommitRequestServer_ARotatedCredentialMovesTheWatcher(t *testing.T) { + provider := kubeconfigProvider("edge", true) + c := newCommitRequestClient(t, nil, mirroringTarget("edge"), provider) + gvr := commitRequestGVR() + listKinds := map[schema.GroupVersionResource]string{gvr: "CommitRequestList"} + before := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), listKinds) + after := dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(), listKinds, + sourceCommitRequest("default", "save")) + clusters := &rotatingClusters{ + discovery: fakeDiscovery(nil, commitRequestResources("configbutler.ai/v1alpha3", "commitrequests")), + credential: "v1", + clients: map[string]dynamic.Interface{"v1": before, "v2": after}, + } + f := committed() + s := &CommitRequestServer{Client: c, Finalizer: f, Clusters: clusters, Log: logr.Discard()} + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + s.syncProviders(ctx) + report, ok := s.CommitRequestsReport(provider.Key()) + require.True(t, ok) + require.Equal(t, ReasonCommitRequestsWatching, report.Reason) + require.Eventually(t, func() bool { return len(clusters.builtFrom()) > 0 }, 10*time.Second, + 10*time.Millisecond, "the watcher runs on the first credential") + + clusters.rotate("v2") + s.syncProviders(ctx) + + require.Eventually(t, func() bool { + u, err := after.Resource(gvr).Namespace("default").Get(ctx, "save", metav1.GetOptions{}) + if err != nil { + return false + } + cr, convErr := commitRequestFromUnstructured(u) + if convErr != nil { + return false + } + ready := apimeta.FindStatusCondition(cr.Status.Conditions, ConditionTypeReady) + return ready != nil && ready.Reason == crReasonCommitted + }, 10*time.Second, 50*time.Millisecond, "the request is seen, and served, through the new credential") + assert.Equal(t, []string{"v1", "v2"}, clusters.builtFrom()) +} + // A KubeProvider carries CommitRequestsServed and status.commitRequests only while it serves its // cluster's CommitRequests. func TestKubeProviderReconciler_CommitRequestsServed(t *testing.T) { diff --git a/internal/watch/cluster_context.go b/internal/watch/cluster_context.go index 4cb5b405..e51edb13 100644 --- a/internal/watch/cluster_context.go +++ b/internal/watch/cluster_context.go @@ -741,37 +741,53 @@ func (m *Manager) ApplierDynamicClient(ctx context.Context, clusterID string) (d if m.sourceClusterDeclared(clusterID) { return m.clusterDynamicClient(ctx, clusterID) } - cfg, _, inCluster, err := m.resolveSourceConfig(ctx, clusterID) + dc, _, err := m.SourceDynamicClient(ctx, clusterID) + return dc, err +} + +// SourceDynamicClient returns a dynamic client for the cluster clusterID names, with that KubeProvider's +// identity, built for this call from the provider as it resolves now, and the version of the credential +// it was built from. Never the cached client of a declared cluster: that one is rebuilt only on the +// credential refresh's own cadence, so a caller that restarts on a new version could still be handed the +// old credential. It is what the CommitRequest server watches a cluster's requests through. +func (m *Manager) SourceDynamicClient(ctx context.Context, clusterID string) (dynamic.Interface, string, error) { + if clusterID == configPlaneClusterID { + return nil, "", errors.New("no cluster named: the config plane is not a KubeProvider") + } + cfg, version, inCluster, err := m.resolveSourceConfig(ctx, clusterID) if err != nil { - return nil, err + return nil, "", err } // Tests inject a fake client for the local cluster without a REST config at all. if inCluster && m.dynamicClient != nil { - return m.dynamicClient, nil + return m.dynamicClient, version, nil } dc, err := dynamic.NewForConfig(cfg) if err != nil { - return nil, fmt.Errorf("build dynamic client for cluster %q: %w", describeCluster(clusterID), err) + return nil, "", fmt.Errorf("build dynamic client for cluster %q: %w", describeCluster(clusterID), err) } - return dc, nil + return dc, version, nil } // SourceDiscovery returns a discovery client for the cluster clusterID names, with that KubeProvider's -// identity, built for this call: what the CommitRequest server asks whether the cluster serves the -// CommitRequest CRD. It carries the dial timeout every discovery client here does. -func (m *Manager) SourceDiscovery(ctx context.Context, clusterID string) (discovery.DiscoveryInterface, error) { +// identity, built for this call, and the version of the credential it was built from: what the +// CommitRequest server asks whether the cluster serves the CommitRequest CRD, and whether the credential +// its watch runs on changed. It carries the dial timeout every discovery client here does. +func (m *Manager) SourceDiscovery( + ctx context.Context, clusterID string, +) (discovery.DiscoveryInterface, string, error) { if clusterID == configPlaneClusterID { - return nil, errors.New("no cluster named: the config plane is not a KubeProvider") + return nil, "", errors.New("no cluster named: the config plane is not a KubeProvider") } - cfg, _, _, err := m.resolveSourceConfig(ctx, clusterID) + cfg, version, _, err := m.resolveSourceConfig(ctx, clusterID) if err != nil { - return nil, err + return nil, "", err } dc, err := discovery.NewDiscoveryClientForConfig(discoveryRESTConfig(cfg)) if err != nil { - return nil, fmt.Errorf("build discovery client for cluster %q: %w", describeCluster(clusterID), err) + return nil, "", fmt.Errorf("build discovery client for cluster %q: %w", describeCluster(clusterID), err) } - return dc, nil + return dc, version, nil } // sourceClusterDeclared reports whether any GitTarget's Declare captured this cluster id. From eaa9f8379a036a64c14411bf5b5fdc32c570892d Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 11:15:30 +0000 Subject: [PATCH 32/33] docs: design the mutation-capture lab on its own cluster, and revise the gitops-api open asks - docs/design/mutation-capture-lab-own-cluster.md: the lab gets its own k3d cluster with real audit, admission, conversion and aggregated-API deliveries, certificates issued before the cluster exists so it never restarts, its own k3s pin, lab-* tasks, and the requirements that keep it working beside the e2e cluster. - docs/future/gitops-api-open-asks.md: revision 17; links to gitops-api's design docs point at its repository, not the untracked checkout. - .gitignore: docs/review/* is local; tracked reviews stay tracked. - .devcontainer: common-utils 2.7.1. Co-Authored-By: Claude Opus 5.5 --- .devcontainer/devcontainer-lock.json | 8 +- .gitignore | 1 + .../mutation-capture-lab-own-cluster.md | 239 +++++++++++++ docs/future/gitops-api-open-asks.md | 330 +++++++++++++----- 4 files changed, 482 insertions(+), 96 deletions(-) create mode 100644 docs/design/mutation-capture-lab-own-cluster.md diff --git a/.devcontainer/devcontainer-lock.json b/.devcontainer/devcontainer-lock.json index f4dfa882..c14fe30d 100644 --- a/.devcontainer/devcontainer-lock.json +++ b/.devcontainer/devcontainer-lock.json @@ -1,9 +1,9 @@ { "features": { - "ghcr.io/devcontainers/features/common-utils:2.7.0": { - "version": "2.7.0", - "resolved": "ghcr.io/devcontainers/features/common-utils@sha256:3f63bbee7418dc940d9e5e7ab93ef3049acfe4dba3085b17ee45fe307fdba4e5", - "integrity": "sha256:3f63bbee7418dc940d9e5e7ab93ef3049acfe4dba3085b17ee45fe307fdba4e5" + "ghcr.io/devcontainers/features/common-utils:2.7.1": { + "version": "2.7.1", + "resolved": "ghcr.io/devcontainers/features/common-utils@sha256:e76a6a8a35d1ad824daad1540725e0ec9f8ed2dc00d2850e990a9830013c7d08", + "integrity": "sha256:e76a6a8a35d1ad824daad1540725e0ec9f8ed2dc00d2850e990a9830013c7d08" }, "ghcr.io/devcontainers/features/docker-outside-of-docker:1.10.1": { "version": "1.10.1", diff --git a/.gitignore b/.gitignore index d2f1bdfd..5eb73908 100644 --- a/.gitignore +++ b/.gitignore @@ -67,3 +67,4 @@ loadtest /external-sources/ /docs/local-only/* hack/release-notes/node_modules +docs/review/* diff --git a/docs/design/mutation-capture-lab-own-cluster.md b/docs/design/mutation-capture-lab-own-cluster.md new file mode 100644 index 00000000..bcbeb4f0 --- /dev/null +++ b/docs/design/mutation-capture-lab-own-cluster.md @@ -0,0 +1,239 @@ +# Mutation-capture lab on its own cluster + +Status: proposed, 2026-10-27. Not built. + +The [mutation-capture lab](../spec/mutation-capture-lab-design.md) records what Kubernetes emits +through watches, audit, admission, CRD conversion, and an aggregated API, and commits it as a +corpus under [`test/mutationlab/corpus/`](../../test/mutationlab/corpus/). That corpus is evidence +the product relies on: `internal/queue/fact_index_corpus_test.go` and +`fact_index_selector_corpus_test.go` replay it as fixtures. It is also how a new Kubernetes +version is judged: capture again, and the corpus diff is the behavioral changelog. + +This page designs the lab as a cluster of its own, which can run beside the e2e cluster, and lists +the requirements that keep the capture faithful and the lab working. + +## What broke, and why it could + +The lab never owned its wiring. `task lab-e2e` swapped the lab image into the product's controller +Deployment on the e2e cluster ([`swap-image.sh`](../../test/mutationlab/swap-image.sh)) and borrowed +everything around it: + +| The lab needs | It came from | Since `fd6f6933` | +|---|---|---| +| an admission webhook registration for `/validate-all` | the product's `ValidatingWebhookConfiguration` | deleted | +| a serving certificate, port 9443, and its mount | the product's `Certificate`, `Service`, and Deployment | deleted | +| the CRD conversion webhook (row 14) | the CA bundle and Service read off that registration | broken with it | +| audit deliveries | the API server's webhook, aimed at the product's NodePort and route, with the product's certificates | works | +| a pod, an identity, and RBAC | the product's Deployment and ServiceAccount | works, by accident | + +The driver's readiness check waits for an admission record, so the run now fails before its first +scenario, although audit still flows. + +The deletion only exposed the root cause: nothing runs the lab, so a product change could remove +what the lab depended on, and every check stayed green. + +## Decision + +The lab gets its own k3d cluster, `gitops-reverser-capture-lab`, which runs the lab and nothing of +the product. Every source the corpus records arrives the way it does on any cluster: + +- the API server's **audit webhook backend** posts batches to the lab over mutual TLS; +- a **`ValidatingWebhookConfiguration`** sends admission reviews to the lab's Service; +- the **CRD conversion webhook** of the scenario Widget CRD calls the lab's Service; +- the **sample aggregated API server** (`wardle`) serves flunders behind an `APIService`; +- the lab **watches** through its own ServiceAccount. + +Nothing is replayed, proxied, or read from a file. That is the point: a new Kubernetes version is +tested by creating the lab cluster on it, and only real deliveries say what that version does. + +A namespace on the e2e cluster cannot get there. The API server takes one audit webhook, set when +the cluster is created, and on the e2e cluster it belongs to the product. Sharing it brings back +the swap. + +## Design + +```text + host: task lab-capture ── go test driver ── port-forward :18081 ─────────────┐ + │ + k3d cluster gitops-reverser-capture-lab (one node, k3s pinned by the lab) │ + ┌────────────────────────────────────────────────────────────────────────────┼─┐ + │ kube-apiserver │ │ + │ audit webhook backend ── mTLS ── 127.0.0.1:30444 (NodePort) ─┐ │ │ + │ ValidatingWebhookConfiguration mutation-capture-lab ─────────┤ │ │ + │ CRD conversion (Widget) ─────────────────────────────────────┤ │ │ + │ APIService v1alpha1.wardle.example.com ── wardle-server │ │ │ + │ ▼ ▼ │ + │ namespace mutation-capture-lab: Deployment mutation-capture-lab │ + │ :9444 audit (mTLS) · :9443 /validate-all + /convert · :8080 records API │ + │ watches: configmaps, deployments, pods, widgets, flunders │ + └─────────────────────────────────────────────────────────────────────────────┘ +``` + +### The cluster + +[`start-cluster.sh`](../../test/e2e/cluster/start-cluster.sh) creates it, with: + +- `CLUSTER_NAME=gitops-reverser-capture-lab`, so the context is `k3d-gitops-reverser-capture-lab` + and its stamps live under `.stamps/cluster/k3d-gitops-reverser-capture-lab/`; +- `K3S_IMAGE` from `LAB_K3S_IMAGE`, which defaults to the image recorded in + [`CLUSTER.md`](../../test/mutationlab/corpus/CLUSTER.md), independent of the e2e cluster's; +- `AUDIT_DIR_REL` pointing at the lab's stamp directory, which holds the e2e audit policy, copied + from the one tracked file, and the lab's final audit webhook kubeconfig; +- `K3D_AGENT_COUNT=0`. The e2e cluster runs an agent because it restarts its server node to load + the audit kubeconfig; the lab never restarts, so one node is enough. + +The API server's audit flags, batch settings included, come from the script unchanged, so both +clusters deliver audit the same way. + +### Certificates, before the cluster exists + +The e2e cluster restarts its server because cert-manager issues the audit certificates only after +the cluster is up, and the API server reads its audit kubeconfig only at startup. The lab issues +its certificates first, with a small Go command, into the lab's stamp directory: + +- one CA; +- a serving certificate for `mutation-capture-lab.mutation-capture-lab.svc`, used by the audit, + admission, and conversion servers; +- a client certificate for the API server. + +The audit kubeconfig is then complete when the cluster is created: server +`https://127.0.0.1:30444/audit-webhook`, the CA, the client certificate, and the Service's DNS +name as `tls-server-name`. The lab requires and verifies that client certificate. The swapped lab +never did, because the swap passed no client CA. + +After creation, the serving certificate and CA go into the cluster as Secrets, and the CA is +written into the registration's `caBundle` when it is applied. The driver writes it into the Widget +CRD's conversion `caBundle` from the stamp directory. cert-manager is not installed: the lab's own +certificates have no rotation to manage, and cert-manager arrives on the e2e cluster only through +the whole Flux stack. + +### The deployment + +`test/mutationlab/deploy/`, a kustomization, holds: + +- namespace `mutation-capture-lab`; +- ServiceAccount `mutation-capture-lab`, and a ClusterRole that grants `get`, `list`, and `watch` + on what `--watch-resources` names, nothing else; +- the Deployment, one replica, so every audit batch reaches the one store; +- Service `mutation-capture-lab`: ClusterIP 443 to 9443 for admission and conversion, NodePort + 30444 to 9444 for audit, and 8080 for the records API; +- `ValidatingWebhookConfiguration` `mutation-capture-lab`, with the rules the corpus was captured + under: `resources: ["*"]` top-level only, `CREATE`, `UPDATE`, `DELETE`, `scope: "*"`, + `matchPolicy: Equivalent`, `failurePolicy: Ignore`, `timeoutSeconds: 1`. Rows 5 and 6 record no + admission for `/status` and `/scale` because those rules match no subresources; Kubernetes would + send them if asked. + +The sample aggregated API comes from +[`test/e2e/setup/manifests/sample-apiserver/`](../../test/e2e/setup/manifests/sample-apiserver/), +which gains a `kustomization.yaml` so the e2e setup and the lab apply the same directory. + +### Tasks + +The lab's tasks move into their own Taskfile, `test/mutationlab/Taskfile-lab.yml`, included flat +like the e2e one. The verbs follow the playground's: + +| Task | Does | +|---|---| +| `lab-bootstrap` | issues the certificates, creates the cluster, builds and imports the lab image, applies the deployment and the sample API server, and waits until every source is live | +| `lab-capture` | runs the scenarios against the lab cluster and compares the capture to the corpus | +| `lab-corpus-update` | runs the scenarios and writes the capture as the new corpus, `CLUSTER.md` included | +| `lab-status` | shows the cluster, its k3s version, the lab pod, and how many records it holds | +| `lab-cleanup` | deletes the cluster and its stamp directory | + +`lab-e2e`, `swap-image.sh`, and the lab's dependency on `_controller-deployed` are removed. + +Testing a new Kubernetes version: + +```bash +task lab-cleanup +task lab-bootstrap LAB_K3S_IMAGE=rancher/k3s:v1.38.0-k3s1 +task lab-corpus-update +git diff test/mutationlab/corpus/ +``` + +### Beside the e2e cluster + +Both clusters can run at once. What each one owns: + +| Concern | e2e | lab | +|---|---|---| +| cluster and context | `gitops-reverser-test-e2e` | `gitops-reverser-capture-lab` | +| stamps | `.stamps/cluster/k3d-gitops-reverser-test-e2e/` | `.stamps/cluster/k3d-gitops-reverser-capture-lab/` | +| API port on the host | picked by k3d | picked by k3d | +| NodePort 30444 | inside its own node container | inside its own node container | +| run lock | `e2e.lock` in its stamps | `lab.lock` in its stamps | +| host port-forward | Gitea, Prometheus, Valkey, Flux, Argo CD | records API on 18081 | +| k3s image | the e2e pin | `LAB_K3S_IMAGE` | + +`start-cluster.sh` switches the current kube context to the cluster it creates. The lab passes a +new `K3D_SWITCH_CONTEXT=false`, and the script's devcontainer kubeconfig rewrite reads the named +context rather than the current one. Otherwise bootstrapping the lab would move a person's +`kubectl`, or an e2e run started without `CTX`, onto the lab cluster. + +## What is reused + +| Piece | How | +|---|---| +| `start-cluster.sh` | as is, plus `K3D_SWITCH_CONTEXT` and a context-scoped kubeconfig rewrite | +| the audit policy | the same tracked file, so the corpus shows what the product's clusters audit | +| the audit flags and batch settings | from the script, unchanged | +| the sample aggregated API server | the same manifests, through a new `kustomization.yaml` | +| the lab binary, recorders, normalizer, corpus, and driver | unchanged, apart from the client-CA flag, the context, and the readiness check | +| the content-hashed lab image and its build | unchanged | +| `lab-port-forward.sh` | with the lab's namespace and selector | +| the audit kubeconfig template | from `generate-audit-webhook-kubeconfig.sh`, fed from files instead of Secrets | + +Not reused: `inject-webhook-tls.sh`, whose node restart the lab avoids, and the Flux stack. + +## Requirements + +Each one names the failure it prevents. + +1. **Real deliveries only.** Every record arrives from the API server or the aggregated API server + through its own mechanism: the audit webhook backend, a webhook registration, a CRD conversion + webhook, a watch. No log scraping, replay, or proxy. Otherwise a new Kubernetes version is + judged through a translation that may hide what changed. +2. **Mutual TLS on audit, verified.** The API server presents its client certificate and the lab + checks it, as a production receiver does. Otherwise the lab records deliveries such a receiver + would refuse. +3. **The registration's match rules stay those the corpus was captured under.** Otherwise rows 5 + and 6 change meaning while looking like a Kubernetes change. +4. **One audit policy file, shared with e2e.** Otherwise the corpus drifts from what the product's + clusters audit. +5. **Self-contained.** No lab manifest, script, or task names a product resource or depends on a + product task. A static check enforces it. Otherwise the next product change breaks the lab + unnoticed, as this one did. +6. **Coexistence.** The lab has its own cluster name, stamps, lock, and port, and never switches + the current kube context. Otherwise running it beside e2e corrupts one of the two. +7. **Every source live before the first scenario.** `lab-bootstrap` ends by proving audit, + admission, conversion, the aggregated API, and the watch each with a probe write it then finds + among the records. Today's check covers admission alone. +8. **The version is pinned and recorded by the tool.** `lab-corpus-update` writes the k3s image + and the server version into `CLUSTER.md`. Otherwise the corpus and its provenance disagree, as + they did when e2e moved to 1.37 and the corpus stayed on 1.36.4. +9. **Something runs it.** A scheduled CI job runs `lab-bootstrap` and `lab-capture`, and fails on + any corpus diff. Without it the other requirements hold only until the next unnoticed change. + +## Build order + +1. `K3D_SWITCH_CONTEXT` and the context-scoped rewrite in `start-cluster.sh`, and the + `kustomization.yaml` for the sample API server. +2. The certificate command, and the lab's client-CA verification. +3. `test/mutationlab/deploy/` and `Taskfile-lab.yml` with `lab-bootstrap`, `lab-status`, and + `lab-cleanup`. +4. The driver: the lab context, the CA from the stamps, and the five-source readiness check. Then + `lab-capture` and `lab-corpus-update`; delete `lab-e2e` and `swap-image.sh`. +5. A capture on the pinned k3s v1.36.4, committed on its own: its diff must be empty, or explained + line by line, because only the delivery path changed. +6. A second commit that moves the pin to the e2e cluster's version, so its diff is the Kubernetes + change alone. +7. The scheduled CI job, the static self-containment check, and the lab's README and + [spec](../spec/mutation-capture-lab-design.md#isolated-test-setup) rewritten to match. + +## Open questions + +- **CI cadence.** Nightly plus manual dispatch is the proposal. A per-PR job would catch a break + in the PR that causes it, for a few runner minutes per PR. +- **Memory beside e2e.** One k3s node with the lab and the sample API server should stay near + 1 GiB, but that is an estimate until measured. If it is more, `lab-capture` can take the e2e lock + as well, so the two run in turn. diff --git a/docs/future/gitops-api-open-asks.md b/docs/future/gitops-api-open-asks.md index 0d9927de..2a03b48a 100644 --- a/docs/future/gitops-api-open-asks.md +++ b/docs/future/gitops-api-open-asks.md @@ -1,94 +1,240 @@ -# The gitops-api team's open asks - -Status: **open asks from a production user**, condensed on 2026-10-24 from their standing list -(revision 15, 2026-10-20, written against `v0.53.0`). The asks keep their numbers. Closed asks are -left out: #30 shipped as `GitReceiver` (#432), #33 went with the prefetch in #425, #22 is fixed -(`pkg/manifestanalyzer/repo.go`), #26 and #27 shipped (`git_branch_targets`, -`commit_requests_total`), #25 shipped in 0.44.0 (#343), and #6 is answered by immutability that -fails at admission with "delete and recreate". Since revision 15: #32 is a row in -[the Flux guide](../bi-directional-flux.md#troubleshooting); #31 is -`apply_claimed_resource_reconciled_total`, labeled by applier kind and claim; ask #23 is pinned at the -fact index, for a finalizer cleared by a PATCH or an UPDATE, in either order -(`TestFactIndex_AFinalizerClearedByAControllerNamesTheDeleter`); and #11 shipped in 0.41.0 (#290): -create and update write through one encoder, `internal/yamlstyle`, and a test fails on a second -one. A file GitOps Reverser did not write is re-indented once, on its first edit. Kind and field -names are updated to `main` (`FluxClaim` for `FluxTrigger`, `KubeProvider` for `ClusterProvider`, -`ClaimedResourceReconciled` for `ResumedElsewhere`). - -Their suggested order, as of revision 15: #34 when the e2e budget allows; then #10; #28 when two -owners share a cluster. #28 is deferred: see below for why. - -## 34. An e2e for an in-cluster applier with a remote source cluster - -**Ask.** One e2e where the `GitTarget`'s `KubeProvider` names a remote cluster and the `FluxClaim`'s -names the operator's own, with an API edit landing during an apply. - -**Why.** It is their production topology: Flux runs in the management cluster and applies into a -remote workspace through `spec.kubeConfig`; GitOps Reverser reads that workspace remotely and holds -the `Kustomization` locally. They proved the hold, the apply cycle, the receiver and `ForeignCommit` -in it, but never saw the apply window withhold anything (`apply_window_collisions_total` stayed at -zero), so the part that reads `managedFields` on the remote object is untested in this topology. -An e2e that provokes one collision would settle it. - -Two smaller things they noted, both answered. `OperationUnresolved` on the `GitTarget` and the -`WatchRule` is not recorded again: each object's readiness takes the reason of the one it depends -on (`ApplierClaimReady` on the `GitTarget`, `GitTargetReady` on the rule), so one operation shows on -three objects by design. And `apply_requests_total{outcome="sent"}` counts only an applier that was -asked: an apply that is not needed is never sent, so it is not counted. - -## 28. Two operator instances cannot share one config-plane cluster - -**Ask.** A way for two instances sharing one config-plane cluster to partition the CR set, so that -exactly one instance reconciles a given `GitTarget`. Smallest useful first, and they stack: - -1. **`--watch-namespaces`**, a `Cache.DefaultNamespaces` pass-through. It partitions the namespaced - kinds, which since #430 includes `KubeProvider`; only `ClusterWatchRule` stays cluster-scoped. -2. **An explicit owner selector**, a label selector or a `spec.instance`-style field paired with an - `--instance-name` flag, which covers the cluster-scoped kinds too. - -**Why.** They hit it moving onto a cluster that already runs an instance for its owner's own -workloads: two populations, different source clusters, providers, credentials and namespaces, and -still both instances would reconcile both sets. RBAC is no workaround: controller-runtime issues a -cluster-wide List, so a narrowed ClusterRole fails the cache sync and the manager exits. - -**What sharing costs them**, for calibration: one Valkey holding both populations' attribution -facts; one upgrade cadence; one RBAC posture that is the union of both; and, customer-visible, a -restart for one owner's reasons costs the other's users their commit message, because -`RenderMatchesLive` starts `Unknown` after a restart and an open window is dropped. They are not -blocked: one shared instance works. - -**Deferred, not built.** Step 1 was built and dropped before merge (#445). It removed two of the -four costs above, the shared Valkey and the restart reaching the other owner's users. It could not -remove the other two: - -- **One upgrade cadence stays.** The CRDs are cluster-scoped, so both instances run against one - schema, and an API break (we make them without migrations) breaks whichever instance lags. -- **One RBAC posture stays.** The ClusterRole is still cluster-wide. - -It is also not the HA mechanism the chart's replica guard -(`charts/gitops-reverser/templates/validate-replica-count.yaml`) asks for. Two instances serving -disjoint namespaces never contend for a `GitTarget`; two replicas of one instance would, and need -leader or ownership coordination instead. Against that, it adds a lasting cost: a second tenancy -model beside one instance serving many namespaces through grants (`consumerNamespaces`, -cross-namespace provider references). Every cross-namespace reference gains a "namespace this -instance does not serve" failure, and every cluster-scoped kind needs its own ownership check. - -The cost their users see is the restart dropping an open window. That is better fixed inside one -instance, for every installation: the durable pending writes in [the TODO](../TODO.md). A second -owner who needs its own upgrade cadence or RBAC needs its own cluster; #28 would give it neither. - -## 10. Sibling inference can place an object in another namespace's directory - -**Ask.** Consider the namespace when inferring where a *new* document goes. Cosmetic. +# gitops-reverser in production: how we use it, and what we'd still like + +_From the [gitops-api](https://github.com/ConfigButler/gitops-api) team. **Revision 17, 2026-10-26** +(first sent 2026-07-09). Written to be readable from the gitops-reverser repo — no gitops-api +knowledge assumed. Every ask is traced to something we measured on a running system. **We run +`v0.54.0`.**_ + +## Where this stands + +**`0.54.0` closed every ask we had open but three, and we adopted it the day it was tagged.** It +reshaped every connection object with no in-place migration, so we rebuilt our cluster rather than +migrate it; loop conformance 21/21 and our editor click-through pass on it. #30 shipped as +`GitReceiver` and deleted more of our code than any release since #1: one receiver for the whole +platform, so our relay now forwards GitHub's delivery verbatim, with no per-repository token, Role +or receiver. Adoption write-up: [reverser-0.54-adoption.md](https://github.com/ConfigButler/gitops-api/blob/main/docs/design/reverser-0.54-adoption.md). + +**One new ask, #35, and unlike the last few it costs us something today.** `0.54.0` stopped +applying the forward leg after the reverser's own pushes, which is right for the cluster and leaves +the applier's own state — its applied revision, its source artifact — behind the branch until +somebody else pushes. We read that state to publish. Details and a proposal below. + +**We accept your reasoning for deferring #28**: a namespace split leaves the CRDs and the +ClusterRole shared, and the cost that reaches users (a restart dropping an open window) is better +fixed inside one instance — which is the second vote below. + +| Open | What it is | +|---|---| +| **35** 🟡 new | Converge the applier on the reverser's own pushes, at a bounded rate | +| **34** 🟡 | An e2e for an in-cluster applier with a remote source cluster — our production topology | +| **10** ⚪ | Sibling inference can place an object in another namespace's directory — cosmetic | +| **28** ⏸ | Two instances on one config-plane cluster — deferred, by agreement | + +Plus two **votes for items already on your TODO**, with our evidence: filtering `kube-root-ca.crt`, +and the durable worker queue. + +## Who we are + +**gitops-api lets an external user edit their config-as-data by talking to a Kubernetes API.** They +log in with GitHub, they `kubectl` (or click) against an API surface that is theirs, and their edit +becomes a **Git commit authored by them** in their own repository. gitops-api itself **never writes +Git** — that is entirely gitops-reverser's job, and it is why the product exists. The attribution +_is_ the product. + +Each tenant is a [kcp](https://kcp.io) **workspace**: a nodeless, near-namespace-cheap logical +cluster with its own CRDs and its own OIDC. + +```text + ┌───────────────────────────────────────────┐ + a human's edit ────▶│ kcp workspace (one per tenant) │ + (Dex/OIDC token) └───────────────┬───────────────────────────┘ + │ shard-wide audit firehose, tagged `kcp.io/cluster` + ▼ + ┌──────────────────────────────────────┐ + │ ONE gitops-reverser (STOCK) + ONE │ a KubeProvider per tenant; + │ Valkey, for the whole platform │ attribution partitioned by + └───────────────┬──────────────────────┘ spec.attribution.auditRoute + ┌─────────────┴─────────────┐ + ▼ ▼ + tenant A's repo tenant B's repo ← attributed commits + ▲ + │ Flux `Kustomization` with spec.kubeConfig → the workspace, + └─ held by a FluxClaim through the ONE in-cluster KubeProvider + (forward leg: git → cluster) +``` + +**We run the stock image, with no patches.** One reverser mirrors our management cluster in-cluster +and every tenant's workspace remotely. A GitHub App's single webhook reaches it through our server, +which forwards each push to one `GitReceiver`. + +We also **link** `pkg/manifestanalyzer`: our onboarding wizard's folder step runs your acceptance +gate so the folders it offers are the folders your writer will accept. The reverser is therefore +pinned twice for us — the image we run and the Go module we link — with a test that fails the build +when they part. + +--- + +## Open asks + +### 35. Converge the applier on the reverser's own pushes, at a bounded rate + +**Ask.** A periodic converge: when an applier's last applied commit is behind the branch tip, and +nothing else is pending for it, apply the tip — **at most once per interval**. For example +`FluxClaim.spec.convergeInterval` / `ArgoCDClaim.spec.convergeInterval`, defaulting to something like +`1m`, with `0` keeping `0.54.0`'s behaviour. The decision stays in the branch worker, like every +other one; the claim's own periodic reconcile may be the natural clock. Batched by construction: a +burst of saves inside one interval costs one apply, not one per push. + +It is a timer on a reason you already have. A `Converge` already applies the tip "because nothing +says the applier runs it", and already asks nothing when the applier's last succeeded operation +applied that commit. Today it fires once at worker start and once per new `GitTarget`; this asks for +it on a clock as well. + +**Why.** `applier-echoes.md` S2 is right about the cluster: after the flush, an apply following the +worker's own push changes no object, so it protects nothing there. But the applier's **own state** +is read by other things, and on `0.54.0` that state stops moving: + +- the `Kustomization`'s `status.lastAppliedRevision`; +- the `GitRepository`'s artifact. It fetches only when the claim asks it to apply, so it stays at + the last foreign commit as well; +- everything downstream of those two: Flux's commit-status notifications, `flux get kustomizations`, + any promotion or packaging step that consumes "what was applied". + +Measured on `0.54.0`, after one conformance run and one editor click-through: branch tip +`be3b08d2`, artifact **and** applied revision `c81500d0`, the last commit somebody else pushed — +**15 commits behind, and it moves only when somebody pushes in GitHub.** On `0.53.0` the quiet-branch +`CatchUp` kept them level. + +It costs us one feature. We publish each repository's folder as an OCI image from what the forward +leg **applied** (applied revision == artifact revision), downloaded from source-controller. On +`0.54.0` an edit made in our editor is mirrored, committed and attributed perfectly, and then never +published until an unrelated push happens. We cannot work around it from our side: the only lever +we have is to fetch or reconcile the Flux objects ourselves, and reconcile ownership tells us — rightly +— that a fetch of ours between your check and your resume applies a commit nobody checked. + +**Why not after every push.** That was `0.53.0`'s `CatchUp` minus the quiet period, and it would make +every save cost an apply. A bounded interval keeps the applier eventually level for whoever reads its +state, and the cluster pays at most one no-op apply per interval per applier. + +### 34. An e2e for an in-cluster applier with a remote source cluster + +**Ask.** One e2e where the `GitTarget`'s `KubeProvider` names a remote cluster (`secretRef`) and the +`FluxClaim`'s names the operator's own (`inCluster`), with an API edit landing during an apply. + +**Why.** It is our production topology: Flux runs in the management cluster and applies into a remote +workspace through `spec.kubeConfig`; the reverser reads that workspace remotely and holds the +`Kustomization` locally. We have proven the hold, the apply cycle, the receiver and `ForeignCommit` +in it (21/21 in our conformance probe, on `0.53.0` and on `0.54.0`). We have **not** seen the apply +window withhold anything in it — `apply_window_collisions_total` stayed at zero across every run — +so the part of #29 that reads `managedFields` on the remote object is untested in this topology by +anyone. An e2e that provokes one collision would settle it. + +### 10. Sibling inference can place an object in another namespace's directory — ⚪ cosmetic + +**Ask.** Consider the namespace when inferring where a _new_ document goes. **Why.** The cold-start layout is namespace-first, but once a layout exists, placement follows -siblings. Two tenants syncing a repository named `infra` produce two objects named `infra` in -different namespaces, and one lands in the other's directory. Nothing is corrupted, but -`git log -- tenants/tenant-a/` shows another tenant's history. - -## A possible #24: say when a stored field was pruned - -From their 0.43 upgrade: a deleted field is pruned by the API server, so the object stays -`Ready=True` and quietly uses a default or a wider scope. They had to write a test asserting the -*absence* of the old spellings. When an applied object still carries a field the CRD no longer -serves, a condition or `Event` on that object would make a migration self-diagnosing. Not filed as an ask. +siblings. Two tenants syncing a repo called `infra` produce two objects named `infra` in different +namespaces, and one lands in the other's directory. **Nothing is corrupted** — both documents and +both namespaces are correct, and deleting one removes exactly that document. Only the path suffers: +`git log -- tenants/tenant-a/` shows another tenant's history. It may well be intentional. + +### 28. Two reverser instances cannot share one config-plane cluster — ⏸ deferred, by agreement + +We asked for a way to partition the CR set between two instances on one cluster (`--watch-namespaces`, +then an owner selector), because we are moving onto a cluster that already runs a reverser for its +owner. You built the first step and dropped it before merge (#445): it removes the shared Valkey and +the cross-owner restart, but not the shared CRDs, the shared upgrade cadence or the shared +ClusterRole, and it adds a second tenancy model beside grants. **Agreed.** We run one shared +instance; the cost that would make us ask again is the restart below, and you want that fixed for +everyone. Our analysis: [reverser-instance-topology.md](https://github.com/ConfigButler/gitops-api/blob/main/docs/design/reverser-instance-topology.md). + +## Votes for your own TODO + +Both are already on your list. These are the measurements behind our vote, not new asks. + +**Filter `kube-root-ca.crt`** (`docs/TODO.md`, "Filter more cluster-generated noise"). Our tenants +mirror ConfigMaps, so every tenant repository carries the workspace's cluster CA certificate — a +file nobody wrote, that means nothing outside that one cluster. Every rebuild of the cluster gives +it a new CA, and the first snapshot commits the change as a `message_source="reconcile"` commit by +the committer. That is the only commit-without-a-human our alarm +(`git_commits_total{author_kind="committer"}`) sees, and it fires once per rebuild on every +tenant. A default filter, with an opt-back-in for whoever really wants it, would remove both. + +**The durable worker queue** (`docs/TODO.md`, "Make the worker queue durable"). This is the one cost +of sharing an instance that our users can see. `RenderMatchesLive` starts `Unknown` after a restart, +an open commit window is dropped, and a save made in that window loses the message its author typed +and its attribution. `0.54.0`'s #443 now holds live edits while fidelity is pending, which fixes the +edit; as far as we can tell the window's message is still dropped. It is the last item standing +between us and "a deploy of the reverser is invisible to the people editing". + +## Process notes + +- **Retire a field by rejecting it first.** `0.44.0` kept `eventTemplate`/`groupTemplate` in the + schema for one release with a CEL rule that refuses them, and `0.53.0` did the same for + `allowAnySourceNamespace`. Both failed loudly in our operator instead of silently changing + behaviour. When a field is simply removed, the API server prunes it and the object stays + `Ready=True`, so we write tests asserting the _absence_ of old spellings. Making + rejection-before-removal the default would make those tests unnecessary. +- **`0.54.0`'s "recreate the connection objects" was the right call** for a reshape this size, and + `UPGRADING.md`'s eight steps were complete. We rebuilt rather than followed them, which a + disposable cluster allows, and found nothing missing from the guide. + +--- + +## Shipped — the record + +One line each: what happened, and whether we are happy. + +| # | Ask | Shipped | Verdict | +|---|---|---|---| +| 1 | Separate the config plane from the watched cluster | `v0.38.0` (#249/#251), #259 | **Happy** — one reverser now serves our whole platform; it deleted a tier of our architecture | +| 3 | A Redis key prefix | #221 | **Happy** — defaulted to the historic root, no migration | +| 4 | Export `pkg/manifestanalyzer` | #221 | **Happy** — the foundation of our wizard's folder step | +| 5 | A `CommitRequest` that can assert an author | the admission webhook records the submitter (#220 itself closed unmerged) | **Happy** — a save's author comes from who filed it | +| 6 | Movable `GitTarget` destination, or fail loudly | immutability that fails at admission with "delete and recreate" | **Happy** — we delete and recreate, now for a `GitBranch` too | +| 7 | Degrade without `apiregistration.k8s.io` | #223, `v0.34.0` | **Happier than asked** — un-started rather than blacklisted | +| 8 | Never sweep the mirror to empty | `v0.39.0` (#260), `prune.mode: OnEvent` | **Very happy** — an API default, not a guard we maintain | +| 9 | RBAC that matches the runtime need | #223, `v0.34.0` | **Happy** | +| 11 | Serialize `[CREATE]` and `[UPDATE]` bodies identically | `0.41.0` (#290), one encoder | **Happy** — the mirror's `git diff` is readable | +| 12 | Strip `kcp.io/cluster` from Git writes | #227, `v0.35.0` | **Happy, and grateful** — fixed before we reported it | +| 13 | Tag releases so Go can resolve them | `v0.37.0` | **Happy** — what made #4 pinnable | +| 14 | Watch a differently-named namespace than the rule's own | `v0.39.0` (#259) | **Happier than asked** | +| 15 | Make a lost audit route loud | `0.44.0` (#341), `AuditFactsReceived` | **Very happy** — a latched condition tells _not yet_ from _never_ | +| 16–21 | The analyzer asks: permanence, the report contract, `Key()`, rendered types, `FleetRoot`, `ReadScope` | `0.40.0`/`0.40.1` (#273, #275) | **Very happy** — three better than asked, two corrections of our ask | +| 22 | `ReasonRefusedStructural` no longer claims "permanent" | `0.41.0` (#290) | **Happy** | +| 23 | Deletion-as-intent names the deleter, not the finalizer-clearer | `0.41.0` (#289), pinned for UPDATE and PATCH in `0.54.0` | **Happy** | +| 26, 27 | Join the pipeline's halves; count `CommitRequest` outcomes | `git_branch_targets`, `commit_requests_total` | **Happy** — better than asked | +| 29 | Wake the reconciler only for commits the reverser did not make | `v0.53.0`, reconcile ownership | **Happy** — the forward-leg race is gone for every writer ([write-up](https://github.com/ConfigButler/gitops-api/blob/main/docs/design/reverser-feedback-forward-leg-race.md)) | +| 30 | One receiver endpoint for a GitHub App | `0.54.0` (#432), `GitReceiver` | **Very happy** — deleted our per-repository tokens, Roles and HMAC relay | +| 31 | Count "resumed elsewhere" in a metric | `0.54.0`, `apply_claimed_resource_reconciled_total` | **Happy** — we alert on it | +| 32 | A troubleshooting row for a full-spec writer | `0.54.0`, the Flux guide | **Happy** | +| 33 | Say why `prefetch` is false | `0.54.0` (#425), prefetch removed | **Happy** | +| — | `WatchRule` readiness lag (never filed) | `0.44.0` (#343) | **Happy** — rule readiness is usable as an alert | + +**#2 — ignore writes by an identity or field manager. ⚫ WITHDRAWN 2026-07-14, our mistake.** The +forward leg's echo is self-terminating: the applier applies the commit the reverser just pushed, the +content matches, the echo commits nothing. We built the filter ourselves and **it caused a +two-and-a-half-minute flip-flop**, every commit unattributed. **Please never add +`excludeUsers`/`excludeFieldManagers` on our account** — +[why](https://github.com/ConfigButler/gitops-api/blob/main/docs/design/forward-leg-loop.md). The right +shape was the one you already had: sanitize the content, do not filter the writer. + +## What is _not_ upstream's problem + +So a wishlist does not become a blame list: + +- **The audit muxer** was our integration with kcp, not a reverser concern — and `v0.39.0` let us + delete it. +- **kcp's front-proxy** panicking on an anonymous request, and its per-workspace authenticator + answering `Unauthorized` for a minute after a fresh start. kcp's, not yours. +- **Broken label references in our own alerts**, found while mapping our dashboards onto a new + metric surface. Ours, every time — which is why we now test our rules against `0.54.0`'s labels. + +## Suggested order + +1. **#35** — the only open ask with a cost to us today, and a timer on a reason you already have. +2. **#34** whenever the e2e budget allows — our production topology, and the one part of #29 nobody + has seen work in it. +3. The two TODO items above, in whichever order your plan already has them. +4. **#10** — cosmetic, whenever. From c7fc27ac76c21c6e44b92fe38d13e02ec9ec74a3 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Tue, 27 Oct 2026 11:18:44 +0000 Subject: [PATCH 33/33] fix(build): drop the removed source CRDs from helm-sync's generates The source.configbutler.ai CRDs are gone, so the charts/gitops-reverser/crds/source.configbutler.*.yaml glob matched no file and Task never treated helm-sync as up to date. CI's cache check after `task lint` failed on it. Co-Authored-By: Claude Opus 5.5 --- Taskfile-build.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/Taskfile-build.yml b/Taskfile-build.yml index 5e122e2d..f9a77384 100644 --- a/Taskfile-build.yml +++ b/Taskfile-build.yml @@ -93,7 +93,6 @@ tasks: - exclude: cmd/**/zz_generated.deepcopy.go generates: - charts/gitops-reverser/crds/configbutler.*.yaml - - charts/gitops-reverser/crds/source.configbutler.*.yaml - charts/gitops-reverser/config/role.yaml cmds: - |