Repository navigation
feat(persistence): bounded directory discovery and advisory index repair - #74
Open
PoisonGodv wants to merge 1 commit into
Open
PoisonGodv wants to merge 1 commit into
PoisonGodv wants to merge 1 commit into
Conversation
Add an opt-in, single-primary Redis directory protocol slice for the leased progress store introduced for agentic-ai-java#7: - RedisProgressDirectory: explicit fixed-capacity directory where claim and membership registration complete atomically in one Lua call, with bounded lexicographic continuation paging (limit 1..1000) and a controlled registerExisting migration path for legacy authority. - RedisProgressAdvisoryIndex and RedisProgressIndexRepairer: advisory discovery/repair scan that records bounded issues for corrupt records and advances past them without treating the index as state truth. - RedisLeasedProgressStore gains the directory mode integration points; existing API, default behavior, and stored formats are unchanged. - RedisProgressReferenceConsumer: reference consumer that revalidates revision, state, reservation, owner, and fence atomically after claim and again before dispatch, with exact int64 delivery tokens for ack/retry/backoff. - Tests: directory, repairer, and protocol suites including six adversarial review probes, all executed against real Valkey 8.1.2. Refs agentic-ai-java#8. This is a protocol slice submitted for design review, not a complete Worker: capacity, migration, and retention policy decisions are listed as open maintainer decisions in the PR description. Signed-off-by: lnw <951448078@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe what this PR does / why we need it
Issue #8 asks for Redis polling claim, expiry recovery, and an advisory repair index without introducing MQ or Redis Streams. Its full scope also includes a complete Worker and independent-JVM takeover; this PR does not deliver those.
This PR submits an opt-in, single-primary protocol slice for design review, building on the
LeasedProgressStorefrom #69 (#7): a bounded membership directory, an advisory index with a bounded repair scan, and a reference consumer that demonstrates the claim-to-dispatch safety checks. Existing APIs, default behavior, stored formats, and configuration prefixes are unchanged; nothing new is active unless directory mode is explicitly enabled. The module README documents the new opt-in modes and their boundaries.Does this pull request fix one issue?
NONE. Refs #8. This is a partial protocol slice submitted for design review, not a complete Worker, and it does not claim to meet #8's acceptance criteria for crash recovery, dual-Worker single-owner guarantees, or bounded polling budgets end to end.
Describe how you did it
RedisProgressDirectory: an explicit, immutable-capacity directory (1..10000 retained session/run identities). Claim and membership registration complete atomically in the same Lua call: the wrapper limits the originalCLAIMscript's KEYS to the authority hashes and performs the membershipZADDonly after an OK reply, preserving the existing type/data/fence validation. Capacity and wrong-type rejections leave no orphan authority, and concurrent contenders cannot over-admit.page(after, limit)uses exclusive lexicographic continuation with limit 1..1000, and there is no pending SCAN page. A quiescent full sweep covers registered membership; an insertion behind the finished cursor appears on the next sweep.registerExistingcall; duplicate registration is idempotent.RedisProgressAdvisoryIndex/RedisProgressIndexRepairer: advisory publication plus a repair scan that records bounded issues for corrupt records and advances past them, without treating the index as state truth. One repair call performs one page query, at mostlimitprojections, and at mostlimitpublications.HINCRBYthen string read; no Lua double conversion) with ack/retry/backoff. Poll moves delivered scores against RedisTIME; exact token comparisons prevent an old delivery from acking or retrying a newer delivery or revision. Retry invalidates its token, so repeated retries cannot extend backoff. Preflight reserves room for the maximum 1000-token batch and fails closed near int64 exhaustion.RedisProgressReferenceConsumer: after claim, metadata is read atomically together with reservation, live owner/fence, and server-time validation; candidate revision and an explicit runnable predicate are checked; the guard renews and revalidates immediately before dispatch. Revision/state changes,expires=0, reservation mismatch, and stale fences are rejected; commit is fenced and ack follows commit. This is a reference implementation for review, not a production Worker; unknown application states require an explicit adapter and need not be runnable.Measured/mechanical boundaries, stated so reviewers can judge them directly:
Describe how you verified it
All Redis-touching tests run against real Valkey 8.1.2 via Testcontainers (AOF,
appendfsync always); they are disabled when Docker is absent rather than silently passing.Two executions back this submission:
expires=0, contrary to the existing store's canonical nonnegative expiry contract) was corrected before the final run without changing the failing assertion.Formatting and lint: scoped Spring JavaFormat 0.0.39 apply/validate passed, and root/module Checkstyle reported zero violations, in the review run. License-eye repository check on the implementation delivery: 773 checked, 585 valid, 188 ignored, 0 invalid. The review-probe test file added afterward carries the repository Apache header but was not covered by a new license-eye run. Whole-repository build and CI were not run locally.
To verify locally with Docker available:
./mvnw -B -pl :argi-graph-persistence-redis -am testSpecial notes for reviews
Requesting design review before any further Worker work, because three policy decisions belong to maintainers rather than this slice:
registerExistingis the right migration contract, given that mixed writers must be quiesced and all future producers enforced through directory mode; other legacy keyspaces are not auto-migrated.Further limits: a single authoritative primary is assumed (no Cluster/Sentinel/failover certification; Lua gives isolation, not rollback); external effect fencing and idempotency remain application responsibilities; independent-JVM takeover and OS-kill recovery were not run and belong to the #8 follow-up.
This submission was prepared with AI assistance (Claude) under a two-stage workflow: implementation, then an independent review whose adversarial probes are included as
RedisProgressProtocolReviewTestand whose findings are summarized above.