feat(metamodel): @of is optional on @agg:collect — the whole-object rollup (#335) - #345
Merged
Conversation
…ract it lands in (#335) Design for making @Of optional on origin.aggregate @agg:collect. Ships in two halves: the rollup itself, and two holes in the queryable-projection contract that the rollup would otherwise land a column into — the filter allowlist never consults isArray, and @Sortable has no subtype validation at all. Both are live today. Records the decision NOT to mint a subtype or @kind for "document" vs "relational" projections: ADR-0037's subtype test fails, @kind is the physical-storage axis with read-only-ness derived from it, and the query contract is already per-field, which is the right level. Four claims in the issue did not survive verification against the code, including that the @objectRef-must-be-a-value rule already exists — it is payload-scoped and never reaches a projection-hosted field, so the branch's main #270 guard would have been absent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ng it (#335) A parallel change (#342) already moved metamodelVersion 0.10 -> 0.11, and that bump is unreleased — v0.24.0 shipped 0.10. The gate baselines against the last release tag so a cycle needs one bump, not one per PR, so this change needs no edit at all. Records the two conditions that would change it: 0.11 shipping before this lands, or Half B being judged breaking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… it (#335) A rule with no negative fixture is a blind spot one layer up — "no fixture covers it" and "every port enforces it" look identical on a green suite. Three findings in the parallel #342 work were exactly this shape, including one where the ungated arm was the precise place a port had diverged. Makes Half B's negative cases shared conformance fixtures rather than port-local unit tests, and records the measurement that motivates it: a structural scan of 1321 JSON + 124 YAML files finds zero fields carrying both isArray:true and filterable/sortable:true, so nothing in the corpus exercises an array field through the filter tier at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…#335) "Potentially breaking" was too flat. Three bounds now stated: in-repo cost measured at zero, what breaks was already emitting SQL that cannot execute, and — the distinction that actually matters — nothing ever taught adopters to write it. That last point separates this from a sibling change in the same window that is the same category and a different magnitude: a retired spelling that appeared as a worked example in a shipped skill has a plausible acquisition route; a silently wrong emission nobody was told to write does not. Also records that this version question gets asked when #335 is cut, on its own evidence, rather than folded into a ruling already made about a different change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…urface it lands in (#335) Fifteen tasks, 86 steps, TDD throughout. Half B first (two loader rules making the filter/sort contract honest about array columns) so Half A lands into a correct tier, then the loader change across four ports, the seven byte-gated prose copies, the TypeScript view column and its lowering, and a real-engine round-trip on both dialects. Carries the traps that have bitten this repo before as inline warnings rather than lore: `dotnet test` printing Passed! over a failed compile, a Maven or ci-local run piped through tail reporting tail's exit status, the integration round-trip living in a package no per-package run touches, and the fresh-worktree precondition that looks like a regression in codegen-ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…335) Ports the two loader validation rules from TypeScript (task 3) to C#, Java and Python: - @filterable: true on an ARRAY field now errors ERR_FILTERABLE_UNSUPPORTED_ SUBTYPE (extends the existing no-operator-band check in each port). - @Sortable: true on an ARRAY field, or a field subtype with no operator band, now errors ERR_SORTABLE_UNSUPPORTED_SUBTYPE — a wholly new pass in all three ports, wired into each port's validation pipeline. Array-ness is read through each port's resolving accessor (C#/Java field.ResolvedIsArray()/isArrayType(), Python field.resolved_is_array()), matching the accessor already used by the @agg:collect check, per ADR-0039. Java additionally gains the ERR_SORTABLE_UNSUPPORTED_SUBTYPE message-prefix constant in ErrorMessageConstants (the ErrorCode enum member already existed; the message ledger did not). Turns fixtures/conformance/error-filterable-array-field and error-sortable-array-field green on all three ports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
Both corrections were established by execution, not review. Task 5's model was illegal in two ways unrelated to this feature: a projection cannot `extends` an entity, and a projection's identity must `extends` an entity identity rather than declare fresh @fields. Replaced with the shape the corpus already uses, verified to load clean and to produce exactly the one intended error for the whole-object form. Task 9's grep returns 9 files, not 7 — two are this plan and its spec quoting the sentence. Scoped the grep to product files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#335) An @of-absent origin.aggregate @agg:collect is now a WHOLE-OBJECT rollup: the carrying field must be a field.object (isArray, @objectref) and the collect assembles the related rows as an array of that value object rather than an array of one scalar column. @via is required (no @Of entity to infer the relationship from), @distinct is refused (a guaranteed no-op on the object form), and @orderby resolves against the @via terminal entity via a new _viaTerminalEntityNode helper. The @objectref must resolve to an object.value — #210's payload-scoped value-only rule never reaches a projection-hosted field, so this branch enforces it itself (ERR_SUBTYPE_RULE_VIOLATION), closing the same hole 0.20.16 (#270) closed for payload-typed fields. The scalar collect-with-@Of path is untouched.
…ollect (#335) Fix round 1 on the whole-object collect review: two of the six must-enforce rules had no test arm, so deleting the _validateViaPath/_checkAggregateCardinality call and the entire _viaTerminalEntityNode call site still passed all six original tests. - Add "@via provably to-one" arm (ERR_ORIGIN_CARDINALITY via a single reference hop, inherently to-one). - Add a 3-entity A.b.c chain arm proving @orderby resolves against the @via TERMINAL entity (C) rather than the head (A) or a middle hop (B). - Gate the @orderby call on `hops !== undefined` — _viaTerminalEntityNode does not reproduce _validateViaPath's malformed-@via guard, so an invalid @via was emitting a second, misleadingly-scoped @orderby error alongside the real one. - The three arms that previously discriminated only by the shared ERR_INVALID_ORIGIN code now also assert each rule's distinctive message fragment. Both new arms verified by commenting out their corresponding source lines (cardinality call; the whole hasOrderBy block) and confirming the matching arm — and only that arm — fails, then restoring.
…via terminal (#335) The lowering for an @of-absent origin.aggregate @agg:collect (the whole- object rollup added in the prior commit) now projects EXACTLY the declared value object's members, matched by name against the @via terminal entity: - An unmatched member fails with a new code, ERR_COLLECT_MEMBER_UNRESOLVED. Failing open here is how #270 turned a curated value object into the full entity, invisible in a diff because the metadata still read as curated. - A member whose field.<subType> differs from the matched entity field's fails with ERR_INVALID_ORIGIN — the object-form analogue of the scalar @Of path's element-type check, same #185 type-preserving doctrine. Both sides are read with the resolving accessor (children()), never own*(), per ADR-0039 — a value object or entity may bring members via extends, and an own-only read would silently skip them, which is exactly the bug class this guard exists to prevent. ERR_COLLECT_MEMBER_UNRESOLVED is registered across all six error ledgers (ERROR-CODES.json, TS/Python/C#/Java ErrorCode enums, and Java's separate ErrorMessageConstants message ledger — discovered on an earlier #335 task to be a sixth site, not five). ERR_INVALID_ORIGIN is reused, not re-added. Two new arms added to the existing validation-collect-whole-object.test.ts (the model helper gained an optional voFields parameter so SupplierBrief's member list can vary per test without touching any existing call site). Each arm's pin was verified by commenting out its corresponding check, confirming that arm alone failed, then restoring to a clean diff. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…rms (#335) Half A conformance fixtures for the @of-absent origin.aggregate @agg:collect whole-object rollup: one positive fixture plus eight negative arms covering every rule the loader enforces on the @of-absent branch (the scope grew from the brief's original six to eight — @distinct and @orderBy-terminal landed after the brief was written). Models are lifted from the verified validation-collect-whole-object.test.ts harness; jsonPaths and the positive expected.json are copied verbatim from the runner's actual output, never hand-derived. error-collect-no-object-ref intentionally asserts two errors (ERR_INVALID_ORIGIN from the new rule plus the pre-existing ERR_OBJECT_FIELD_WITHOUT_OBJECT_REF, which fires on any field.object missing @objectref regardless of collect) — both are inherent to the shape. error-collect-via-to-one's value object is narrowed to a single id member (matching the mis-cardinality path's actual terminal) so member-resolution can't raise a second, unrelated ERR_COLLECT_MEMBER_UNRESOLVED alongside the intended ERR_ORIGIN_CARDINALITY. Conformance suite: 588 -> 606 pass (+18 = 9 fixtures x lint + conformance). Full metadata package: 2507 -> 2525 pass, 0 fail.
…335) The per-member type-agreement check added in Task 6 cites the #185 type-preserving doctrine but implemented only half of it: it compared `member.subType !== match.subType` and ignored array-ness entirely. A value-object member declared `field.string` matching a terminal-entity `field.string isArray: true` (or the reverse) passed validation, and the lowering would project an array into a scalar member. The sibling this rule names, `_checkPassthroughType`, already compares BOTH axes via the `field.<subType>[]` label and carries a comment explaining why subtype names never contain "[]", so equal labels mean same subType AND same array-ness. This check now does the same thing, so the two halves of one doctrine are enforced the same way. Found by the whole-branch review. Caught before Task 8 deliberately: porting a known-incomplete rule into three more ports and then fixing four is strictly worse than fixing the reference implementation first. Also corrects the ERR_FILTERABLE_UNSUPPORTED_SUBTYPE ledger description, which still described only the subtype cause after this branch gave the code a second trigger (array field) in all four ports. Its sibling ERR_SORTABLE_UNSUPPORTED_SUBTYPE already documented both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…even (#335) Task 7 shipped eight rules and nine fixtures after a scope correction the ledger recorded but this plan never absorbed: member TYPE agreement and @orderBy-resolves- against-the-@via-terminal had no place in the original six. Following the stale text literally would have under-ported two rules into three ports — precisely the silent cross-port omission the shared corpus exists to prevent. Also records the trap that makes Task 8's own signal untrustworthy: five of the eight negative fixtures assert ERR_INVALID_ORIGIN, which is ALSO what an unported loader emits for the old "missing @Of" error. Those five pass vacuously before any porting work begins, so "conformance green" does not mean the rules were implemented. The step now names the guard-neutralisation technique the Task 7 reviewer used, and repeats the no-git-stash prohibition (worktrees share one stash list). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…ct rules (#335) All four cleanup reviewers independently flagged the same thing, and the efficiency pass measured it: `_validateViaPath` already holds the terminal entity in `currentObj` when it returns, and throws it away — so `_viaTerminalEntityNode` existed only to recover it by walking the identical path a second time, and the whole-object branch called that helper TWICE. Three walks where one would do. On a 1103-object estate the two redundant walks measured ~56ms of a 778ms load — 7.2%. Worse than the cost: the duplicate re-derived the ADR-0042 package-resolution rule by hand, so the loader carried two independent @via path-walkers that had to be kept in agreement. The branch's own comments admitted the divergence ("_viaTerminalEntityNode does not reproduce _validateViaPath's malformed-shape guard") and both call sites needed a redundant `hops !== undefined` gate to paper over it. Those comments were the smell, not the invariant. `_validateViaPath` now returns `{ hops, terminal }` — the shape `_validateFromPath` already uses for the same reason. `terminal` is defined exactly when `hops` is (all five early exits return undefined; the sole success path has walked to the end), so the gate is structural rather than commented. `_viaTerminalEntityNode` is deleted: 44 lines, and three ports that no longer have to reimplement it in Task 8. Also extracted, because the helper set IS the cross-port porting contract — every port mirrors this file's `_check*`/`_validate*` decomposition 1:1, and Java's validateOriginNode is already ~330 lines: - `_checkCollectMembers` — the member-resolution + type-agreement rules. Measured nesting in the @of-absent arm drops from 10 levels to 4. - `_typeLabel` — the `field.<subType>[]` idiom, previously inline in four places with its invariant re-explained each time. `_checkPassthroughType` now shares it. Fixture fix (found by the altitude pass): error-collect-orderby-not-terminal declared `name` only on B, the MIDDLE hop. Task 7's review proved it discriminates middle-resolution — but a port resolving @orderby against the HEAD (the likelier mistake, since the walk starts there) also errored and so PASSED. Entity A now declares `name` too. Verified by mutation: resolving against the head fails the fixture now and did not before. Python's `@sortable` check inlined the literal "sortable" where the other three ports used their constant; now imports FIELD_ATTR_SORTABLE. Verified: typecheck clean, 2526 pass / 0 fail, conformance 606/0. The extracted helper was mutation-checked (disabling it fails 3 unit tests + 2 fixtures, so it is reachable and pinned, not dead code). Python conformance is unchanged at 4 failed / 451 passed — the known interim red for Half A, which Task 8 ports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
#335 changes the registered vocabulary: `@of` becomes OPTIONAL on `origin.aggregate @agg: collect`, which is a rule relaxation an adopter can observe, plus eight new load errors. Under ADR-0035 Amendment 2 that moves the metamodel contract's number, and pre-1.0 a breaking metamodel change moves the MINOR. It is 0.12, NOT the 0.11 the design spec's §A7 claimed. 0.11 is CLAIMED — by #342's XOR relaxation, shipping in 0.24.1; origin/main's 2af77e7 names it the sole claimant precisely so a parallel branch would not reason "v0.24.0 shipped 0.10, so 0.11 is free" and collide. This branch already carries #342 in its ancestry, which is what its 0.11 was recording; #335's own change needs the next number. No gate would have caught this. #335's only registry footprint is `@of`'s byte-gated DESCRIPTION PROSE, and check-metamodel-version.mjs WARNs on prose rather than classifying it — a rule can change with no machine-readable footprint (#210's only manifest edit was a `rules` string). Confirmed empirically: the gate reported "declared 0.11 — ok" before this commit and "declared 0.12 — ok" after. Human decision, made by Doug. Written by `check-metamodel-version.mjs --set 0.12` so all five sites move together — the manifest plus all four port constants. A partial edit only fails in the forgotten port's registry-conformance lane, which is release-tag/dispatch-only for three of them. Verified: all five sites read 0.12 with no residual 0.11, gate green, typecheck clean, metadata 2526/0, Python registry-conformance 10/10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
Five of the eight negative conformance fixtures for the @of-absent @agg:collect asserted ERR_INVALID_ORIGIN — which is ALSO what a loader that still REQUIRES @Of emits for that exact metadata. The corpus compares only `code` + `source` and never message text (spec/conformance-tests.md), so those five passed vacuously against three ports that implement none of the rules: Python conformance failed 4 of the 9, not 9 of the 9. ERR_COLLECT_WHOLE_OBJECT now carries the five refusals raised by the whole-object arm itself — carrier is not a field.object declaring @objectref; @via absent; @distinct declared; an @orderby key that does not resolve against the @via TERMINAL entity; a value-object member whose declared field.<subType>[] disagrees with the matched terminal field's. The three arms that already had distinct codes keep them (ERR_SUBTYPE_RULE_VIOLATION for a non-value @objectref, ERR_ORIGIN_CARDINALITY for a to-one @via, ERR_COLLECT_MEMBER_UNRESOLVED for an unmatched member). The @orderby arm runs through _validateOrderByKeys, shared with the scalar @Of arm and with origin.first, so the helper gained a trailing `code: ErrorCode = "ERR_INVALID_ORIGIN"` parameter. Only the whole-object call site passes the new code; the other two keep their envelope byte-for-byte, so no existing fixture, port or adopter-visible behaviour changes. Registered in all SIX ledgers (ERROR-CODES.json, the TS/Python/C#/Java enums, and Java's separate ErrorMessageConstants). C#'s FixtureLint rejects an unregistered code in a fixture, which is independent proof the registration took in the shared ledger. Signal verified by execution on every unported port, not by reading: Python 9 failed / 568 passed (was 4 failed / 568 passed), C# 9 failed / 979 passed, Java 606 run / 9 failures — in each case exactly the nine #335 fixtures, so Task 8 now has a real red-before/green-after gate per rule. TS is unchanged green: metadata 2526/0, conformance 606/0, typecheck clean, dotnet build with no `error CS`, `mvn -pl metadata compile` clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
Ports Half A's eight loader rules for an @of-absent origin.aggregate @agg:collect to C#, Java and Python. The nine shared conformance fixtures are the contract; all four ports now agree on every one. The eight rules, in the order they fire: the carrying field must be a field.object declaring @objectref (ERR_COLLECT_WHOLE_OBJECT); that @objectref must resolve to an object.value (ERR_SUBTYPE_RULE_VIOLATION — #210's rule is payload-scoped and never reaches a projection-hosted field, so this branch enforces it itself); @via is required (ERR_COLLECT_WHOLE_OBJECT); @distinct is refused (ERR_COLLECT_WHOLE_OBJECT); the @via path must be to-many (ERR_ORIGIN_CARDINALITY); every @orderby key must resolve against the @via TERMINAL entity (ERR_COLLECT_WHOLE_OBJECT); every value-object member must match a terminal field by name (ERR_COLLECT_MEMBER_UNRESOLVED); and a matched member must agree on BOTH type axes (ERR_COLLECT_WHOLE_OBJECT). Each port mirrors the TS decomposition 1:1 rather than inlining, because the helper set IS the porting contract and Java's validateOriginNode was already ~330 lines: WalkedViaPath (validateViaPath now returns hops AND the terminal, so the terminal is not recovered by a second walk carrying a second copy of the ADR-0042 package-resolution rule), _checkCollectMembers, and _typeLabel. validateOrderByKeys gains an optional error code — Java as a 7-arg overload, C# and Python as a defaulted parameter — so only the whole-object call site reports ERR_COLLECT_WHOLE_OBJECT while the scalar @Of and origin.first sites keep their envelope byte-for-byte. ADR-0039 throughout: resolving reads for @objectref (C# Attr, Java hasMetaAttr/getMetaAttr, Python get_meta_attr), resolving children() on both the value object and the terminal entity, and the resolving array accessor inside the type label (C# ResolvedIsArray, Java isArrayType, Python resolved_is_array). @via stays an OWN read — origin.* never inherits (ADR-0029). Green is not the evidence here, and this cut does not rely on it. Every one of the 24 (8 rules x 3 ports) guards was neutralised individually against a backed-up copy and the corpus re-run: in all 24 cases EXACTLY the one fixture that gates that rule failed, and no other. Each file was restored and `diff -q` confirmed byte-identical to its backup afterwards; no `git stash` was used, since worktrees share one stash list. Two C# mutations initially tripped CS0162 under warnings-as-errors, so those were re-run with a runtime-opaque condition rather than a statically dead one — a detail worth keeping: a neutralisation that will not compile proves nothing. Suites: Python 1849 passed / 0 failed; C# 1696 passed / 0 failed across all four test projects with no `error CS`; Java 1494 run / 0 failures / BUILD SUCCESS; TypeScript unchanged at 2526 / 0 with typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…copies (#335) Until now the loader was LOOSER than the contract it ships: five ports accept an @of-absent @agg:collect while every registry copy still said @Of was "Required for count/sum/avg/min/max/collect". Five prose strings change, byte-identically across the copies that carry them: - @Of — required for count/sum/avg/min/max; OPTIONAL for collect, where absent means a whole-object rollup of the field's declared @objectref value object. - @agg and the origin.aggregate type description — collect is an array rollup of the @Of column OR of the carrying field.object's value object. - @distinct — not supported on a whole-object collect, with the reason (a guaranteed no-op whenever the value object carries the primary key). - the origin.aggregate `rules` string — the whole rule, including the one thing a port could not infer from the others. That last point is the reason this is not a one-line edit. The VO-member↔column binding is NAME MATCHING (member name == terminal field name), deliberately not `extends` — which keeps one value object collectable from two different entities, something `extends` would forbid. It is a NEW implicit convention in a metamodel whose stated doctrine is "extends is THE mechanism", so leaving it undocumented would have left four ports to each infer it from the TypeScript. It is now written into the byte-gated contract, along with the @via-terminal @orderby scope and the both-axes type agreement. Six copies carry the long form and are regenerated or edited in lockstep; `server/csharp/.../Origin/OriginSchema.cs` carries its own shorter wording (it is a C# runtime schema, not part of the byte-gated manifest) and is edited to match in substance. `expected-registry.json` and the `metamodel-docs` fixtures are GENERATED — emitted from the TS registry and by `scripts/regen-metamodel-docs.ts` rather than hand-edited; the regenerated manifest differs from the committed one in exactly the five intended strings and nothing else. §A7 of the design spec is corrected in the same commit, since it documents this same contract. It claimed the change rode #342's unreleased `metamodelVersion` 0.11 and needed no edit. The re-check that section itself demanded then fired: origin/main's 2af77e7 names 0.24.1 as the sole claimant of 0.11, so #335 needs 0.12 — already shipped in 7c99536a7. The superseded reasoning is recorded rather than deleted, because a reviewer independently re-derived it and called it correct: "v0.24.0 shipped 0.10 and main already carries the unreleased bump" is true and still wrong, because it never asks who else has claimed 0.11. Registry conformance green in all four ports (TS 8/0 — the only runner that compares description prose; Python 84/0; C# 25/0; Java RegistryManifest 3/0, which compares structure rather than prose). TS metadata 2526/0. `check-metamodel-version.mjs` reports the four prose changes as WARN and passes: "additive since v0.24.0; declared 0.12 — ok". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…ollup (#335) CLOSES THE MERGE GATE. Since the loader started accepting an @of-absent @agg:collect, `extract-view-spec.ts` still bailed on `if (!of_) continue;` ABOVE the collect branch — so the metadata LOADED clean and codegen silently DROPPED the column. Proven empirically before this commit: the positive conformance fixture extracted `columns = [productId]` with `supplierBriefs` simply gone, i.e. a CREATE VIEW missing a column the generated type still declares. This branch was not mergeable until this landed. A new `collectObjectAgg` SelectColumn kind carries a MEMBER LIST rather than one source column — deliberately a separate kind, not an arm of `collectAgg`: a union would force `isInflationSensitive` and `buildGroupBy` to re-narrow. Both are updated; a whole-object collect is unconditionally inflation-sensitive, because the loader refuses @distinct on this form, so it can never be the distinct case that makes `collectAgg` safe. Extraction resolves the related entity from @via's TERMINAL hop, exactly as the any/all arm does — there is no @Of entity. The value object's declared member list IS the exposure: a field the terminal entity has but the VO omits is not projected, which is the #270 guarantee this rollup has to keep. The JSON key is the MEMBER name while the value reads the terminal entity's PHYSICAL column; those differ whenever a field carries @column, and a test pins that they stay distinct. SQL: `jsonb`, not `json`. Verified against a real PG 15 — `json` has neither an equality nor an ordering operator, so the `json_agg(json_build_object(…) ORDER BY …)` form does not run at all. Postgres gets COALESCE(jsonb_agg(jsonb_build_object(…) ORDER BY <pk> ASC) FILTER (WHERE <pk> IS NOT NULL), '[]'::jsonb); SQLite gets json_group_array(json_object(…)) with json_array() as the empty value. In-aggregate ORDER BY needs SQLite >= 3.44, which is not a new constraint — the scalar collect already emits it and D1's baseline is pinned there. Default element order is the related entity's PK ascending, not "value ascending": ordering rows by a serialized object is meaningless. An explicit @orderby leads with the PK appended as a tie-break. The SCALAR arm keeps its no-tie-break behaviour deliberately — adding one would change emitted SQL for every existing project using @orderby — and three no-churn tests pin that asymmetry, byte for byte, in the same file. Tasks 10 and 11 of the plan are ONE commit rather than two. Splitting them would leave an intermediate commit that does not BUILD: adding the union member without its emit branch fails `tsc` with TS2345 where view-ddl-emit falls through to renderFirst. That exhaustiveness failure is a feature of the union — it is what guarantees no kind can be added and silently unlowered — but it makes the split commit a bisect trap. codegen-ts 1312 pass / 0 fail (7 new DDL + 5 new extract), package typecheck clean, whole-workspace `bun run --filter '*' build` and `typecheck` clean. The two issue-214-read-half-compile failures seen mid-task were the documented unbuilt-workspace precondition (missing `@metaobjectsdev/runtime-ts/drizzle-fastify`), not a regression — they pass after the build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…SQLite (#335) Golden SQL is not evidence for new DDL, and this cut is the proof of why. Both view-lifecycle probes gain a `WeekBrief` value object and a `weekBriefs` column carrying an @of-less collect, beside the existing scalar `weekLabels` collect so both arms run through one view on one engine. Each asserts the full emit -> apply -> introspect -> re-diff-is-EMPTY convergence gate, then the actual values: two related rows give an array of objects in related-PK order, zero related rows give [] and not null. The SQLite emitter is CHANGED here, by what the probe found. On SQLite 3.44.0 — D1's pinned baseline — the in-aggregate ORDER BY clause destroys the JSON subtype: json_group_array(json_object(...)) -> nests correctly json_group_array(json_object(...) ORDER BY ...) -> array of QUOTED STRINGS json_group_array(json(json_object(...)) ORDER BY ...)-> array of QUOTED STRINGS So the column came back as ["{\"id\":1,...}", ...] — an array of strings — while the generated type declared an array of objects, and a json() wrapper on the argument does not survive the ORDER BY either. Dropping the ORDER BY was not an option: element order would stop being deterministic and an author's @orderby would silently do nothing. The emitter now builds the ordered array first and re-wraps it element by element through json_each, which iterates in array order, so the ordering survives while json(value) restores each element's subtype. Still a grouped LEFT JOIN — unlike origin.first it needs no correlation info, so a multi-hop @via lowers exactly as a single-hop one does. Postgres was already correct and is verified, not assumed by symmetry. The unit test in codegen-ts is updated to pin the new SQLite shape, so the two tiers cannot drift. Second finding, pinned rather than normalised away: a `field.long` member arrives inside the rollup as a JSON NUMBER, while the same value as a top-level BIGINT column arrives from node-postgres as a STRING. That asymmetry is inherent to JSON — jsonb has no bigint — and is lossy above 2^53. Both id types are asserted explicitly with a comment, so a future change to either tier has to face it. Ran against a throwaway Testcontainers Postgres, not a local one: this box has sibling-project Postgres containers up, `METAOBJECTS_TEST_PG_URL` was confirmed unset, and startPostgres() provisions its own `metaobjects-test-<uuid>` container on a fresh port. view-lifecycle-pg 17/0, view-lifecycle-sqlite 1/0, codegen-ts projection suite 148/0, package typecheck clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
… replaced, not deleted (#335) The retirement guide's "delete the child and change nothing else" advice is correct for a payload or object.value host and SILENTLY WRONG for a view-kind projection: a projection field with no origin.* is treated as a plain base column, so the emitted DDL becomes `SELECT base."supplierBriefs"` against a column that does not exist. The metadata loads clean; the failure surfaces at apply. §2 now splits by host — 2a keeps the delete for payload/value hosts, 2b replaces the child with `origin.aggregate @agg: collect @via: …` for projections and names the three constraints the old subtype did not have (@via required, @distinct refused, members must match the @via terminal on both type axes), each of which is a load error, so a mistake there fails loudly. The section that stated the gap as a future promise is replaced by the migration itself, now that #335 has shipped. fixtures/conformance/README.md's "coverage genuinely lost" entry is closed with the reasoning, not just a status flip: the array-of-value-object-carrying-an-origin shape lives in the dedicated `collect-whole-object` fixture rather than back in `flattened-kitchen-sink`, because that fixture's concern is `@storage: flattened` and a whole-object rollup exercises none of it — folding them would make each harder to read in a five-language investigation. It also names where the shape is round-tripped against real engines. TS conformance 606/0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
…will look (#335) The CHANGELOG entry leads with the gap rather than the feature: a projection could roll related rows up into an array of one COLUMN and had no way to roll them into an array of OBJECTS, so the shape needed a second round-trip or a hand-written view — unmanaged, and therefore invisible to `meta verify --db`. It records the rulings that will otherwise be re-litigated: why the declared value object is the exposure and binds BY NAME rather than by `extends`; why @distinct is refused by choice rather than engine limit; why the new error code exists at all (the corpus compares code + source and never message text, so five negative fixtures passed against three ports containing none of the rules); why jsonb and not json; why SQLite needs the json_each re-wrap; and the bigint-inside-jsonb asymmetry an adopter will hit above 2^53. Half B's array filter/sort rules are recorded in the same entry. The authoring skill is updated in all SIX copies (the source plus its five byte-gated conformance mirrors). That matters for a specific reason: #342 shipped with the skill teaching a form the loader had just made ILLEGAL. This is the same failure in the other direction — a skill that cannot express the new form leaves adopters reaching for a hand-written view, which is exactly what the feature exists to remove. agent-context-conformance 5/0. `downstream-metadata-decisions.md` gains it as a Step 0 entry rather than a feature note, because "I need an array of objects" is the case most often mistaken for missing vocabulary: the @Of form reads as the only form, and the natural next thought is a custom view. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #335.
The gap
A projection could roll related rows up into an array of one column, and had no way to roll them up into an array of objects.
origin.aggregate @agg: collectrequired@of, so "every supplier's{id, name}for this product" was inexpressible — the shape had to be a second round-trip, or a hand-written view, which is unmanaged and invisible tometa verify --db. Whenorigin.collectionretired in 0.24.0 this became the one coverage gap the retirement guide had to state rather than close.What lands
@ofbecomes OPTIONAL oncollect, and omitting it means a whole-object rollup. The carryingfield.object @isArray @objectRefcollects each related row as its declared value object.{ "field.object": { "name": "supplierBriefs", "isArray": true, "@objectRef": "SupplierBrief", "children": [ { "origin.aggregate": { "@agg": "collect", "@via": "Product.suppliers" } } ] }}@viaterminal entity's fields by name; a field the entity has but the value object omits is simply not projected. That carries the Prelude to #210: retire redundant payload origin-typing (Kotlin, Python) and close Java's KNOWN_GAPS entry as moot #270 guarantee (a curated value object must not silently become the full entity) down to the DDL tier. Name matching rather thanextendsis deliberate — it keeps one value object collectable from two different entities. The convention is written into the byte-gatedorigin.aggregateregistry prose so no port has to infer it from TypeScript.field.objectdeclaring@objectRef; that@objectRefmust name anobject.value;@viais required; the path must be to-many;@distinctis refused;@orderBykeys must resolve against the@viaterminal entity; every value-object member must match a terminal field; and a matched member must agree on both type axes (subtype and array-ness).ERR_COLLECT_WHOLE_OBJECT, for the five refusals that would otherwise have sharedERR_INVALID_ORIGIN. Not taxonomy for its own sake: the shared corpus compares error code + source and never message text, andERR_INVALID_ORIGINis exactly what a loader that still requires@ofemits for this same metadata — so five of the eight negative fixtures passed against three ports containing none of the rules. Proven: the unported ports went 4 → 9 failures once the code was distinct.COALESCE(jsonb_agg(jsonb_build_object(...) ORDER BY <pk> ASC) FILTER (WHERE <pk> IS NOT NULL), '[]'::jsonb)—jsonbnotjson, because PG'sjsontype has neither an equality nor an ordering operator.Half B — array filter/sort validation
Two commits (
9a09724,1de6601) give@sortablethe subtype validation@filterablealready had, and stop an array field claiming a filter-operator band.This makes previously-loading metadata fail to load, so it is breaking on the metadata contract — but it is the previously-wrong-acceptance ruling this release already invokes twice (#342, and before that the 0.19.1
@minclamp and the 0.21.6likecase-sensitivity fix): such a declaration emitted alike/eqrule against atext[]column, i.e. SQL that cannot execute. It was never validly expressible. Bounds, stated rather than hoped:isArray: trueand@filterable/@sortable: true.Residual exposure is not zero and the changelog says so: an adopter who wrote the form independently auto-adopts a load error, because pre-1.0
^0.24.xresolves it.Two findings the real-engine round-trip paid for
Both were invisible to emitted-SQL text, and only the emit → apply → introspect → re-diff gate could see them:
ORDER BYdestroys the JSON subtype, so the column returned an array of quoted strings while the generated type declared an array of objects. The obviousjson(json_object(...))wrapper does not survive theORDER BY(seven variants probed). Fix: build the ordered array, then re-wrap element-by-element throughjson_each, which iterates in array order — ordering survives andjson(value)restores the subtype. DroppingORDER BYwas rejected (an author's@orderBywould silently do nothing); a correlated ordered subquery also works but loses multi-hop@via.field.longmember arrives inside the rollup as a JSON number while the same value as a top-level BIGINT arrives from node-postgres as a string. Inherent to JSON, lossy above 2^53. Pinned with a comment rather than normalised away.Verification
scripts/ci-local.shon this exact tree (merge-basebcef4d9c8), every lane read from its trailingOK —line, never piped:--quick--only ts-slow--only java--only csharp--only pythongit diff origin/main -- server/typescript/packages/codegen-ts/test/golden/is empty.metamodelVersionmoves 0.11 → 0.12. 0.11 is index @expr is unreachable: @fields is required on both index.lookup and identity.secondary, and @fields+@expr silently drops the fields #342's sole claimant per the rule[0.24.1]states; this branch complying with that rule is why it moves.Coordination
The 0.24.1 release cut owns the merge — landing this is part of that cut, not a separate action.
🤖 Generated with Claude Code
https://claude.ai/code/session_015WgNA6uDN9qxu2nA471rAq