AnalyticsStore.md

AnalyticsStore

AnalyticsStore is an independently named user resource. Canonical events live in ZoneTree; bounded generations compute aggregates from currently authorized captures. Orleans owns request execution and the silo-local storage lifetime.

Requirement	Acceptance	Completion gate
REQ-ANALYTICS-001	AC-ANALYTICS-001	Configure three independent AnalyticsStore resources with immutable declared dimensions and numeric metrics. SDK and official MCP callers ingest and independently query each store. Resource kind, schema, generated Orleans aliases and IDs remain stable.
REQ-ANALYTICS-002	AC-ANALYTICS-002	Existing authorized Batch commits canonical event replacement/deletion, expected revisions, row metadata and links to node-local ZoneTree through acknowledgement from every configured copy across the KeyLoad:ReplicaCount copies. Failed CAS changes neither event nor generation. Same operation replay returns the original receipt.
REQ-ANALYTICS-003	AC-ANALYTICS-003	Bounded event reads enforce current persisted scope, row and field policies. Aggregate inputs contain only currently visible rows, with raw-use authorization for selected dimensions and metrics. Fresh persisted subject/policy, resource lifetime, read generation and store generation are revalidated before publishing output.
REQ-ANALYTICS-004	AC-ANALYTICS-004	The centrally pinned ZoneTree.Analytics 1.0.2 executes each aggregate from a complete authorized canonical capture in a fresh derived generation. Track, Flush, Query and Dispose calls are synchronous. accepted/processed/dropped/rejected/error counters and the complete raw-event query plan are checked. Limits refuse incomplete results rather than returning a partial aggregate.
REQ-ANALYTICS-005	AC-ANALYTICS-005	The silo-local ZoneTree owner (the node's Orleans Grain Service) owns derived directories, engines and shutdown. Cancellation joins the actual operation and disposal; a failed join blocks subsequent generation acquisition and physical owner release.
REQ-ANALYTICS-006	AC-ANALYTICS-006	Event, generation and timestamp-index key families participate in bounded partition inventory, snapshots/backup and resource cleanup. Effect capture includes the store and every linked resource through the existing observed resource binder; a retired resource prevents new effects.
REQ-ANALYTICS-007	AC-ANALYTICS-007	HTTP/SDK, official MCP and versioned T-SQL EXEC catalog calls share the same typed operations and connection-owned Orleans execution. A public functional scenario proves three independent stores through create, ingest, replacement, deletion, raw read and complete aggregate.
REQ-ANALYTICS-008	AC-ANALYTICS-008	Full solution build, meaningful TUnit operations, real process recovery and Aspire-owned cluster SDK/MCP gates pass. No performance, endurance, production-readiness or full-SQL claim follows from source presence.
REQ-ANALYTICS-009	AC-ANALYTICS-009	Half-open timestamp reads seek their native index and read the corresponding canonical events. Timestamp moves, deletion, reinsertion, current row policy, ordinal ID output and cold reopen preserve exact results in both storage modes. One query prepares native dimension/metric keys once before projecting its rows.

The schema declares ordinally sorted unique dimension and metric names. Events have an ID, UTC timestamp, string dimensions, finite numeric metrics, optional persisted row access, canonical entity links and monotonic event revisions. Schema changes that would reinterpret retained events are rejected. Replacement requires the exact prior revision, with zero reserved for an absent event. Deletion records a canonical tombstone. A store generation advances with every successful mutation. Mutation receipt revision is that store generation; event CAS revisions remain separate and are returned with event reads.

Reads use a half-open UTC time range and ordinal ID continuation. Continuation is explicitly a new read rather than a persisted snapshot token. Aggregation supports Count, Sum, Average, Min and Max, grouped by declared dimensions. Count selects no source metric. KeyLoad sets no group, record, byte or lifetime ceiling (ADR-149): it sizes the engine's ingestion queue by the captured row count, never truncates dimension values and sets the engine query timeout only from the caller's explicit expiry. The output is not measured or capped. Canonical timestamps may be ahead of system UTC. Each disposable generation configures the engine's ingestion skew to include its latest already captured timestamp. Original timestamps, capture limits and complete-ingestion checks stay authoritative; the requested range alone cannot expand ingestion skew.

The timestamp index contains the event ID under (store, timestamp, id) in the canonical key codec. Upsert moves the index entry with its event; deletion removes it. Transactional resources commit event, index and generation together. Direct resources retain their explicit nontransactional contract: a read can encounter an older index hint during a concurrent replacement and validates the current canonical event before accepting it. The index never supplies event contents or permissions. Reads deduplicate IDs and retain ordinal public ordering.

AnalyticsTimestampSeekTests exercises a two-event window among 515 events, requires fewer than eight examined range entries, then verifies timestamp moves, deletion/reinsertion and two cold opens for both storage modes. The native projection prepares its selected field names and numeric keys once per query, including repeated aggregate selections over the same source metric. The same canonical capture, native engine, complete-ingestion checks and publication revalidation remain in use. Local investigation records the measured work and its qualification limits.

An aggregate can include one dimensionMatch. It compares the exact stored value of a declared dimension, including case, whitespace and empty strings. The caller needs field-use permission even when the dimension is absent from groupBy. Matching applies to authorized canonical rows before native ingestion.

Initial entity links target existing, readable, nondeleted collection documents in the same atomic partition. They use the existing canonical EntityRef identity and fresh document/row authorization at the mutation cut. This stage does not claim a cross-partition transaction or a general resolver for every model. Aggregate queries do not group by links or dereference linked content. Raw link disclosure remains subject to current access and field policy.

The selected published package is ZoneTree.Analytics 1.0.2 under its original MIT license. Its package metadata identifies source commit 3fae17a7c94b5b444c457fdd00ed119d13179487. Each query builds its own bounded raw-event generation from the captured current canonical events and forces the RawEvents query source. Updates and deletes are reflected by that new capture, without a second authoritative database or a stale rollup fallback. Public schema names map to ordinal keys; public names are never truncated to fit identifier bounds.

Each admitted generation has its own derived-user-analytics/<generation> directory under the silo-local ZoneTree owner. The physical owner keeps the root stable while concurrent generations open and close. Core captures immutable authorized data under its read cut and releases the gated view before tracking events and awaiting the native flush and query. The silo-local owner retains the original operation and engine through async disposal, then removes the generation's directory after successful joined close. The connection operation holds the shared analytical admission through the execution join and fresh source/authorization validation before output publication. The final validation uses the remaining original read budget; it does not renew the explicit expiry. The internal same-process result and snapshot never become an extra public or inter-grain payload. The raw execution files are never read as authority, do not acknowledge a database mutation and are excluded from canonical backup authority. Their raw-event, rollup and session stores disable WAL through the published engine options: these generations are rebuilt from the authorized canonical capture. Canonical data retains its own caller-side WAL publication and acknowledgement contract. Interrupted disposable generations are removed at the next opening under the acquired physical lock; they are never reused as trusted projections.

Existing OpenTelemetry logs, traces and metrics remain the foundation. User analytics events and dimensions are not telemetry labels or diagnostic payloads. Measured performance is a separate Linux GitHub gate.

Core owns the canonical AnalyticsStore feature and its adapter; Server owns generation lifetime on the silo-local ZoneTree owner; Clustering reuses the existing connection read capability path; Abstractions, Client and ClientApi own typed public contracts and existing transport composition.

Applicable decisions are ADR-071 for canonical storage, ADR-125 for connection execution, ADR-130 for catalog-driven T-SQL EXEC, and ADR-136 for the silo-local ZoneTree owner.

flowchart LR
    Caller[SDK, official MCP, T-SQL EXEC] --> Connection[Existing connection grain]
    Connection --> Canonical[Fresh authorization and bounded canonical cut]
    Canonical --> Rows[ZoneTree events and replicated mutations]
    Canonical --> Capture[Owned authorized aggregate inputs]
    Capture --> Execution[Silo-local owner bounded generation]
    Execution --> Join[Actual query and joined disposal]
    Join --> Recheck[Fresh policy, lifetime and store generation]
    Recheck --> Result[Complete caller result]

Loading

AC-ANALYTICS-001/002/007 map to the added real FunctionalGateUserAnalytics scenario. AC-ANALYTICS-002/003/004 map to AnalyticsStoreTests and the additional policy-publication tests. AC-ANALYTICS-006 maps to AnalyticsSharedIntegrationTests, including actual family pages, family inventory and D2 enrollment. AC-ANALYTICS-005 maps to physical-host lifecycle tests, and AC-ANALYTICS-008 requires the build, TUnit, process-recovery and Aspire cluster runs. Source presence and narrow build or recovery results do not close these complete current-source analytics qualification gates.