Repository navigation
Add binding-local views to the Axiom adapter, registry, compose and closure (S2b PR 1) - #1482
Conversation
AxiomEngine takes an optional views sequence of AxiomView declarations: a logical engine entity no frame entity maps to (rulespec-be's Child), one bool person selector column, and an outside value per output. The adapter gathers the selected person rows in table order, runs the module's program on the logical entity, checks the engine's array against the selected count, and scatters it back through the same mask; every other row holds the outside value, cast through the output's graph dtype. An empty selection makes no engine call, and a null or non-bool selector refuses. View outputs resolve to the person entity, so callers that key on (binding entity, variable) read them unedited. Each view's program compiles under its own cache key, re-checks the pinned RuleSpec tree first, refuses dense relations, and contributes its derived metadata, so a Child-only module has metadata. The selector joins the person entity's relation_dependencies; view_root_inputs() reports each view's roots. axiom_engine_ref writes views only when present, so every existing reference is unchanged. The view code stays in axiom.py, because source_hash hashes defining files and does not follow imports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
closure_partition no longer refuses a binding that declares views, and BINDING_LOCAL_VIEW_MESSAGE is gone. Per binding, each view's logical entity (Child) joins the live entities and maps to the person table, so a view's roots partition against the closure's Child entries and the existing frame check reads them on the person slice. Roots on that entity stay unmapped on every other binding, and unreached_roots still reads only the binding's engine entity, so it cannot excuse a view root. engine_root_inputs adds each view's roots under its logical entity through AxiomEngine.view_root_inputs(), and refuses a view binding on an injected adapter. binding_view_entities reads a binding's logical entities for the closure and compose. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
The registry parses a binding's views block into AxiomView declarations
(logical_entity, selector {column}, outside {name: value}) and builds the
adapter with them. It refuses an empty list, views beside relations, a
binding off the person entity, a logical entity that names a mapped engine
entity, a repeated logical entity, a selector with any field but column, a
structural selector, a selector that names a binding output, and an outside
key that names no binding variable. Once the adapter is pinned, it checks
each view's program for relations in place of the binding entity's program,
requires each view's outside keys to equal the binding's variables on its
logical entity, and refuses a view input named like a root input of the
binding entity's program. _check_engine compares an injected adapter's views
with the binding's and refuses views on an injected Python adapter.
binding_view_dependencies reports each selector as a whole-table person
slice, the same columns AxiomEngine.relation_dependencies reports to the
rules kernels. Compose unions them with the relation dependencies for every
rules node, refuses a view node whose person slice omits an input the
closure encodes (other than through a graph node) or defaults on the view's
logical entity, and requires each selector on the closure producer's person
slice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
No NZ CI test builds an axiom_engine_ref for NZ's rows: NZ's engine-free graph injects Python toy engines. The new golden holds references generated at main 4a36edc for NZ's six binding rows (unreached_roots included) over a synthetic tree, and for the registry and adapter relation fixtures, all built engine-free through build_transport_registry and axiom_engine_ref. The test compares them byte for byte, so a reference that gains "views": [] or "views": null without a declared view fails. A second test, gated on POPULACE_RULESPEC_NZ, builds NZ's six real references from the tree alone and compares them with a frozen copy the NZ hub writes at the base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
SimulateRulesKernel._require_relation_inputs reads the adapter's relation_dependencies, which now carry each view's selector, so a node whose person slice omits or masks the selector refuses before the engine runs. The test pins that path for a view adapter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
The placement lint now walks every import node in axiom.py, relative and function-body imports included, and a second test requires AxiomView and every view helper to carry axiom.py's module name. A helper moved into a module axiom.py already imports (materialize, concept_mapping) would leave the simulate.* hashes unmoved by an edit to it; the module-name test refuses that move, which the import pin alone passes. P4 gains an output clause: with views=None, materialize returns the same names, order, dtypes and values, through the same engine calls, as a reference copy of main 4a36edc's no-view path kept in the test. NZ's CI graph injects toy engines and never runs AxiomEngine.materialize, so this is the executable guard on view-free outputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
_support_roster mapped each defaulted closure entry to a frame entity through entity_names alone, so a defaulted Child root, which PR 2 writes on the person table, mapped to no entity and missed the closure_default class. _closure_defaults now maps a binding-local view's logical entity to the person entity for entries of that binding's module. Without views the mapping is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6Jc8SMdMbiMGS2yvWB
|
Ready for hub review Head: Commits
Kernel hashesMethod:
8 + 3 = the spec's 11, with no difference. Tests
MutantsEach ran on a scratch worktree at
Merge-tree
Judgment calls
Not done
Generated by Claude Code |
test_nz_real_refs_match_their_frozen_copy compares NZ's six real references with a frozen copy, and wrote none without a rulespec-nz tree. This adds the copy, generated at PR 1's base (main 4a36edc) from a git archive of rulespec-nz 8dc2507d, the commit NZ's bindings pin. The base run froze the file, and a second base run passed against it. At this branch's head the test passes too, so the six references keep their bytes with views in place. The CI golden test passed at the base as well, which checks that axiom_engine_refs.json came from the base. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guide said binding-local views remained separate work. It now describes the declaration, the gather and scatter, the outside values, the registry's refusals, the selector slice, the closure's treatment of view roots, and the row-locality assumption that the skip-gated real-engine differential tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
S2b PR 1 engine-gated run (brief 6041): failures (1 test, differential (b); the cause sits in the test's input typing, not in the adapter) Eight of the nine files pass at PR 1's head and at the PR 1 + PR 2 merge, with no setup skips. Build
Each file ran on its own as Run A: PR 1 head
|
| file | passed | failed | skipped |
|---|---|---|---|
frame shared/test_axiom_relations.py |
64 | 0 | 0 |
frame shared/test_axiom_views.py |
54 | 1 | 0 |
frame shared/test_axiom_adapter.py |
44 | 0 | 1 |
frame shared/test_axiom_graph_adapter.py |
167 | 0 | 3 |
build shared/test_nz_axiom_input_closure.py |
21 | 0 | 0 |
frame us/test_rules_engine_contract.py |
8 | 0 | 0 |
build shared/test_axiom_engine_ref_goldens.py |
2 | 0 | 0 |
build shared/test_transport_views.py |
42 | 0 | 0 |
build shared/test_transport_registry.py |
59 | 0 | 0 |
Skips:
test_axiom_adapter.py:118, "axiom engine is installed here":test_engine_methods_describe_installation_when_missingskips by design once the engine is installed [ran, readtest_axiom_adapter.py:118-122].test_axiom_graph_adapter.py:2019(3 tests), "git here records the decomposed name as the file system lists it…":_decomposed_checkoutskips on a Linux file system, where git records an NFD file name as listed. The guard reads git and the file system, not the engine [readtest_axiom_graph_adapter.py:2000-2022].
No test skipped for a missing engine, dense extension or RuleSpec variable. test_real_dense_relation_fixture and test_nz_real_refs_match_their_frozen_copy both ran and passed (-k selection, 2 passed) [ran]. The three TestBelgianPilotSlice tests ran and passed [ran]. The golden test passed, so I didn't diff the references.
Run B: local merge of PR 2 into PR 1 [ran]
git merge --no-ff 8a4e7a4719dc7a20ae4e6afa3d02a7a85312e0f6 on top of ef39861676582382ff62de60aeed3f6f025de2b8 merged with no conflict. Merge commit f485801c5fa75651dbed9c84909c1f58ff6fa9a7 (local, unpushed), tree 85044a15c272696fc86bea1c03eaf566d03f1659, merge base 3995d92d8f26b5b4d6334d429e80b9f8d63f448a. Against PR 1's head, the merge changes five files (transport/column_kernels.py, frame/concept_mapping.py and three test files). It leaves uv.lock, test_axiom_views.py and F/adapters/axiom.py unchanged.
| file | run A passed / failed / skipped | run B passed / failed / skipped |
|---|---|---|
test_axiom_relations.py |
64 / 0 / 0 | 64 / 0 / 0 |
test_axiom_views.py |
54 / 1 / 0 | 54 / 1 / 0 |
test_axiom_adapter.py |
44 / 0 / 1 | 44 / 0 / 1 |
test_axiom_graph_adapter.py |
167 / 0 / 3 | 167 / 0 / 3 |
test_nz_axiom_input_closure.py |
21 / 0 / 0 | 21 / 0 / 0 |
test_rules_engine_contract.py |
8 / 0 / 0 | 8 / 0 / 0 |
test_axiom_engine_ref_goldens.py |
2 / 0 / 0 | 2 / 0 / 0 |
test_transport_views.py |
42 / 0 / 0 | 42 / 0 / 0 |
test_transport_registry.py |
59 / 0 / 0 | 59 / 0 / 0 |
The skips match run A's, with the same reasons. The one failure matches run A's: same test, same line and same engine error [ran].
Failure: test_differential_b_real_child_program_against_the_view
Where it fails. The full-batch execute at test_axiom_views.py:980 raises RuntimeError: type mismatch: right side of comparison is not numeric, from axiom_rules_engine/dense.py:414 [ran]. None of the shape or view checks runs.
At the base. test_axiom_views.py doesn't exist at 4a36edcf3600 [ran ls], so PR 1 adds this failure.
Cause: the test's input typing. _real_inputs (test_axiom_views.py:923-941) picks each input's type from its name suffix: _years/_count/_months/_rank become int64, _income/_amount/_eur become float64, _region becomes a region code in 1–3, and every other name becomes bool [read]. The engine refuses a comparison with a bool on one side and a number on the other (engine/src/dense.rs:7811-7826, with is_numeric at dense.rs:7369-7374) [read]. Bisecting the 54 outputs one at a time showed 31 of them failing with this error [ran]. Then I traced which inputs cause it [ran, read the rule files]:
- 6 bool conditions end in
_region, so they get a region code:flanders_family_benefits_child_domiciled_in_dutch_language_region,brussels_family_benefits_child_domiciled_in_brussels_capital_region,german_speaking_community_family_benefits_child_domiciled_in_german_language_region,wallonia_family_benefits_child_legal_domicile_in_french_language_region,wallonia_family_benefits_child_without_legal_domicile_effectively_resides_in_french_language_region,wallonia_family_benefits_parents_eu_or_reg883_and_economic_activity_in_french_region. For example,if flanders_family_benefits_child_domiciled_in_dutch_language_region:(rulespec-be/be-vlg/statutes/family_benefits/eligibility.yaml:60) fails on an integer column. - 1 bool condition ends in
_months:german_speaking_community_family_benefits_foreign_stay_or_settlement_authorized_more_than_three_months(be-dg/statutes/family_benefits/eligibility.yaml:72). - 7 numeric inputs fall through to bool:
flanders_family_benefits_disability_tier,flanders_family_benefits_orphan_status_code,wallonia_family_benefits_disability_tier,german_speaking_community_family_benefits_disability_category(each compared== 1in the regions'amounts.yaml),german_speaking_community_family_benefits_benefit_year(>=against a freeze start year),german_speaking_community_family_benefits_article_10_extension_months_usedandgerman_speaking_community_family_benefits_months_since_disappearance(each compared inbe-dg/statutes/family_benefits/eligibility.yaml). - 1 decimal falls through to bool:
german_speaking_community_family_benefits_current_index_factor, which the engine refuses with "expected decimal-compatible dense column".
With those 15 inputs retyped, the engine runs the full batch [ran].
With the typing corrected. I ran the test unchanged except for _real_inputs, which an out-of-tree pytest plugin replaced with a version that retypes those 15 names. I didn't edit the checkout [ran]:
- Child roots the program reads: 107; relations: none; outputs: 54, all
decimalunder the graph dtypes [ran]. - Batch sizes: full 40; shuffled 40; single-row 1; random subset 15 in one run [ran]. The view frame holds 80 person rows, 40 of them selected by
is_child[ran]. - Shuffled, single-row and random-subset batches gave every row the full batch's value for all 54 outputs [ran].
- Through the view (
AxiomView("Child", "is_child", outside)), selected rows equal the full batch's values (:1032), and unselected rows equaloutside(:1033), for all 54 outputs [ran]. - The P3 check (
test_axiom_views.py:1034-1038) compares two float64 sums with==, one over 80 rows with zeros interleaved and one over 40 rows. It failed in both runs I made. The first failed withassert np.float64(58559298.47) == (np.float64(58559298.470000006) + (0.0 * 40)). The second, which recorded the check instead of asserting it, found 51 of 54 sums equal and 3 that differ by 9.1e-13 to 1.8e-12 (flanders_family_benefits_disability_supplement_monthly_amount,flanders_family_benefits_selected_monthly_amount,wallonia_family_benefits_disability_supplement_monthly_amount). All 54 agree tortol=1e-12[ran]. Row-level equality holds, so these differences come from summation order, not from the view.
Reproducibility. CompiledDenseProgram.root_inputs returned the 107 names in a different order in each of three processes (order hashes b2bd9620e1a2, 3d52ddf5b419, 755552b2c107) [ran]. _real_inputs draws from default_rng(530849) in that order, so the seed doesn't fix which input gets which values, and the P3 pairs differ between runs. The engine fills root_inputs in first-reference order during compilation (engine/src/dense.rs:3837-3845) and keeps its compiler indexes in HashMaps (dense.rs:2298-2305) [read]. I didn't trace which iteration sets the visit order. Sorting program.root_inputs before drawing would fix the batch per seed.
Fixing the test would mean three changes: type those 15 inputs by name, compare the P3 sums with a tolerance, and sort the root names before drawing. Those are suggestions; this run changed nothing.
What I couldn't do
- Rerun differential (b) at the base: the test doesn't exist there.
- Report differential (b)'s shape results from the test as written: it fails before its first shape check. The shape and view results above come from the plugin run.
Generated by Claude Code
|
S2b PR 1 mechanism review (brief 6039): CHANGES WANTED Head I found no P1. The gather, scatter, outside-value, cache and closure mechanics held against everything I tried. I found two P2s, both small: an order-dependent closure check, and a mutant that no test catches. The P3s are test-quality notes.
FindingsP2-1. Two bindings on one module path, one with a view and one without: the closure check depends on binding order
Ran: the
So the same bindings pass or fail the closure check depending on file order. No binding does this today, and NZ binds no views. But Belgium binding one module twice (by period, say) is plausible, and the failure would point away from its cause. Fix, either:
Add a test for both orders. P2-2. Mutant survives:
|
| Declared | Cast to graph dtype | Refused |
|---|---|---|
| decimal | float32 → float64 | int32, bool |
| integer | int32, uint64 → int64 | float32, bool |
| judgment | int8, uint8 → int64 | float64 |
| bool | bool | int32, float32 |
The outside array and the scattered values always share one dtype, so out[mask] = … never casts silently.
_outside_values, at 0 rows and at 3:
- Decimal:
2**53+1is refused and2**53is accepted.np.float64(0.25)passes, because it subclassesfloat.infis refused atAxiomView.-0.0passes. - Integer:
True,3.0and2**63are refused. - Judgment:
2is refused and-1passes. - Bool:
1is refused. np.int64(3)andnp.True_are refused atAxiomView, which accepts only Python numbers. That's stricter than_outside_values, but consistent with JSON input.
Shapes. A fake engine returned length 1 with 2 rows selected, (selected, 1), and length 4 with 2 selected (4 = n). Each is refused with "selects 2 row(s)".
Counterfactual and solve-zero, which the spec doesn't cover. With the fake view adapter:
_evaluatewith an overrideperson.age = 10reaches the gather: the engine saw ages[10, 10], and the outputs were[100.0, -7.5, 106.0].- Solve-zero's trial on
person.incomereaches the view:[100.0, -7.5, 320.0]. validate_input_overridereturns early without relations, andvariables()includes the view roots, so view inputs are admitted.- Both kernels call
SimulateRulesKernel._require_relation_inputsper component in_prepare(F/rules_kernels.py:333), andrelation_dependencies("person")now carries the selector. A masked person slice or one without the selector refuses there;test_every_rules_kernel_requires_the_whole_selectorpins it.
Closure. Built on the test_transport_views.py fixture with a person frame slice of {is_child}:
- An encoded (
concept_binding)Child.agethe slice lacks lands inframe_missing. - An encoded (
graph_node) one does not. - An unclosed
Child.child_incomeis agap, and a defaulted one the slice lacks isframe_missing. - For a module bound without views, its
Childclosure entries land inunexpectedand its roots inunmapped_entity. unreached_rootscan't excuse a view root: it is checked againstlive_roots[binding][engine_entity], and the registry refuses aChildinput named like aPersonroot.- The cross-binding question (the union passed to
without_unreached): for a binding without views,Childisn't in its live roots, so the union can't reach them. The one cross-binding effect I found is P2-1.
Engine-free paths can't compile a view. In packages/*/src, the transport callers of variables() and view_root_inputs() are engine_root_inputs (inside the closure kernel at run time), _check_views (registry, after axiom_engine_ref) and the counterfactual bridge (at run time). Compose reaches the selector only through binding_view_dependencies, which builds no adapter. rules_engine_input_refs in F/concept_mapping.py isn't used for Axiom.
Mutants (re-run of the hand-off's, plus mine)
Method. Each mutant was applied and restored by script. Rows 1 to 17 ran against test_axiom_views.py, test_transport_views.py, test_transport_evidence_kernels.py::TestClosurePartition and test_axiom_engine_ref_goldens.py (111 tests at baseline, 1 to 1.3 minutes each). The three survivors were re-run on the full 7-file set (394 tests).
| Mutant | Result |
|---|---|
scatter without the mask (out[:selected] =) |
red, 9 (P1, P2, P5, P6, diff (a), …) |
| outside values → 0 | red, 8 (P1, P2, P3, empty selection, …) |
| outside values → NaN | red, 13 |
gather sorted by age |
red, 4 (P1, P5, P6, diff (a)) |
| a null selector read as false | red, 1 (null-selector test) |
selector dropped from relation_dependencies |
red, 4 (kernel refusal ×2, dependency, agreement) |
| view program also cached under the frame key | red, 3 |
Child roots put on Person in engine_root_inputs |
red, 1 |
| own: view entities made live on every binding | red, 1 (test_a_binding_local_view_partitions_its_logical_entity_roots) |
| own: selector added for every entity | red, 1 |
own: compose guard drops defaulted |
red, 1 |
own: scatter through np.where on a resized, misaligned array |
red, 8 |
| own: selector dtype check accepts ints | red, 1 |
own: view_root_inputs keeps only the first view |
survives, full set (P2-2) |
| own: outside keys only a superset | survives, full set (P3-1) |
own: binding_to_frame gets every binding's view entities |
survives, full set; equivalent: the map is consulted only for a binding's live roots, which already exclude other bindings' view entities |
own: _view_program metadata merge overwrites, not setdefault |
survives; equivalent: one name has one derived item per module |
I did not re-run the hand-off's views: []/null ref, closure-refusal-kept, and placement-lint rows. The golden and lint tests pass at head, and I read both.
PR claims confirmed
- Tests: at
ef398616,-n 4, the 7 files above: 394 passed, 5 skipped, each skip gated on its environment:- differential (b): no dense extension or
POPULACE_RULESPEC_BE; test_axiom_relations.py:798: no dense extension;test_nz_axiom_input_closure.py:535and:610: no rulespec-nz;- the real-tree golden: no
POPULACE_RULESPEC_NZ.
- differential (b): no dense extension or
- Differential (a) compares the view path with an
AxiomEnginewhose person entity ischild(entity_names={"child": "Child", …}), through_batch_from_tableand_graph_output, comparing values and dtypes, as ruling 8 asks. - Separate cache keys (
_view_programsapart from_programs), the pinned-tree re-check before each view compile, andaxiom_engine_ref's refusal after a view compile: read in code, each pinned by its test, and the cache mutant turns red. - Absent views add no ref key:
if engine._views is not None. The golden test passes at head. - The docs commit (
ef398616) matches the code: the registry's refusal list, the selector slice,unreached_rootsexcusing only engine-entity roots, and the support roster.
Not run
- Differential (b), on the real engine and a rulespec-be tree, as the brief says.
test_nz_real_refs_match_their_frozen_copy: no rulespec-nz tree.- Kernel-hash measurement, the merge-tree checks against Encode person_role bindings and their closure defaults on the person table (S2b PR 2) #1480 and Give a person with no household no household's row, and read malformed holdout rows alike in both guards #1481, and the wider S2D-AGREED item 6 file list: these belong to the other lens.
- I have not read the other reviewer's comment.
Generated by Claude Code
|
S2b PR 1 integration review (brief 6040): APPROVE Review head
FindingsP3-1. The registry pins outside values that the adapter refuses only when the view runs. I ran
P3-2.
P3-3. The compose guard on view inputs needs a closure resource. P3-4. The CI golden couples to NZ's live binding file.
P3-5. Docs wording. Every sentence in the new "Binding-local views" section matches the code; I checked each refusal, the selector-slice rule, the closure and roster behaviour, and absent-means-absent. One overstatement: "a real-engine differential … tests the assumption". That test is written but has never run. "is written to test" is accurate until the Linux run lands. Also, What I ran and what it showed
PR claims confirmed
Not run
Generated by Claude Code |
The first review of the fix round (P2) found that 0d6201a's selector refusal also refused an override of a selector that a Person program reads as a root input. The registry accepts that shape, variables() lists the selector, and before 0d6201a the counterfactual bridge applied such an override. validate_input_override now names a selector only when no program reads it, which is exactly when the bridge refuses the override as outside the engine's inputs, so the check changes no override the bridge applies. test_a_selector_that_a_program_reads_stays_overridable drives the counterfactual bridge with a Person program that reads is_child: the override reaches that program's input and the rows the view selects. It fails on 0d6201a's check. The refusal test now also runs through the bridge, which calls no program. The guide states the rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
S2b PR 1 engine-gated run, round 2 (brief 6042): pass Every file in run A (PR 1's head) and run B (PR 1 merged with PR 2) passed on the real engine, with no failures and no setup skips [ran]. Differential (b) now passes, and three fresh processes built byte-identical input batches from seed 530849 [ran]. Build
I ran each file with Run A: PR 1's head, and run B: PR 1 + PR 2For run B, I made a local merge commit of
Skips, identical in both runs and matching 6041's:
The engine-gated tests ran instead of skipping: The added tests explain the higher counts [read,
No test that passed at The only warnings came from 9 Differential (b)I reproduced the test's steps in a script that loads
Seed reproducibilityI ran three fresh processes. Each one compiled the program as the test does, built
What I could not do
Generated by Claude Code |
`_real_inputs` said the pinned engine refuses a wrong-typed input by name.
At axiom-rules-engine 753ce7d, `src/dense.rs` reports a bool, text or date
column as a type mismatch when a program needs a number from it ("expected
decimal-compatible dense column" for an arithmetic operand, "{side} side of
comparison is not numeric" for a comparison), and neither message names the
input. The docstring now says that.
`axiom_ref_goldens.py`'s module docstring wrote `"\n"` in a non-raw string,
so it rendered a line break where the regeneration recipe needs the two
characters. It now escapes the backslash.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the merge base 875506e, main gained #1473, #1506, H2 (#1485), #1501, #1480 and #1488. The merge is clean. Two files changed on both sides, in separate hunks: compose.py (#1488 on main's side) and test_transport_evidence_kernels.py (H2 on main's side). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Landed as squash At merge time:
|
S2b PR 1 of 3 from Belgium's port (d048; design thread #259). It adds binding-local views to the Axiom adapter, the registry, compose and the input closure, so the adapter can run a
Childmodule over the person rows a selector picks. #1480 writesChildinputs and defaults as person columns, and review brief 6040 ran the two PRs together. This PR branched from main4a36edcf.02265bb5merges main875506e8into it, and36800b14merges main3e7a7045, after #1480 and #1488 landed; it edits none of #1480's files. Reviews 6039, 6040 and 6041 ran atef398616, and commits 11 to 18 answer their findings. A first review of that fix round asked for one change, which commit 20 makes. The fix round's two reviews atf1d48f79approve, and commit 21 makes review B's two docstring fixes. The NZ hub cleared the delta sincef1d48f79at 22:05Z, and CI passed on the head.What it changes
A rules binding may declare
views:F/=packages/microcosm-frame/src/microcosm/frame/andT/=packages/microcosm-build/src/microcosm/build/transport/. Line numbers refer to this PR's head,36800b14.Adapter (
F/adapters/axiom.py).AxiomView(:342) holds the logical entity, the selector column and the outside values. Two views compare equal and hash alike when their canonical JSON matches (:381-387), so outside values0,0.0andFalsestay distinct.AxiomEngine(views=...)refuses an empty list, views beside relations, output dtypes other thangraph, a logical entity that a frame entity maps to, two views on one logical entity, and a selector that names a structural column or a view output (_validate_views,:632).materializegroups requested names by execution root (:1072)._materialize_view(:1172) gathers the selected person rows with one boolean mask, in table order, checks the engine's array against the selected count, and scatters it back through the same mask. Every other row holds the outside value, which_outside_values(:2307) casts through_graph_output. An empty selection makes no engine call._view_selection(:1147) refuses an absent, null or non-bool selector._view_program(:1433) compiles each view under its own cache key, re-checks a pinned tree before it compiles, refuses dense relations, and merges its derived metadata._derived_metadata(:1521) loads view programs, so aChild-only module has metadata.variable_metadataresolves a view output toperson(:733).validate_input_overriderefuses, by name, an override of a view's selector that no program reads, before its early return for adapters without relations (:833-841). A selector that a program reads as a root input stays an engine input: an override changes that input and the rows each view selects.relation_dependencies("person")(:803) carries each selector.view_root_inputs()(:686) reports each view's roots.axiom_engine_refwritesviewsonly for an adapter that has them (:2026), and refuses to pin an adapter whose view program compiled first (:1974).Registry (
T/registry.py)._binding_views(:285) parses each view and refuses malformed or conflicting declarations._check_enginecompares every Axiom adapter's views with the binding's, so an injected adapter must match (:448), and refuses views on an injected non-Axiom adapter (:467)._check_views(:471) runs after the reference pins the adapter. It checks relations on each view's program, outside keys against the binding's variables on the logical entity, and any view input that shares its name with a root input of the binding entity's program. It casts each outside value through the adapter's_outside_values(:502), so the registry refuses at build a value the adapter would refuse at run time, such astrueon a decimal output or0.5on a judgment.binding_view_dependencies(:355) returns each selector as a person column.Compose (
T/compose.py)._require_relation_inputs(:4257) unions view and relation dependencies. It refuses a view node whose person slice omits an input that the closure encodes (other than through a graph node) or defaults on the logical entity (:4320)._check_closure_relation_slicesrequires each selector (:2566)._closure_defaults(:2278) maps a view binding's defaultedChildroots toperson, so the support roster classes the columns #1480 writes asclosure_default.Closure (
T/evidence_contract.py,T/evidence_kernels.py).closure_partitiondrops the reserved refusal andBINDING_LOCAL_VIEW_MESSAGE. Per binding, it treats each view's logical entity as live and maps it to the person table (evidence_contract.py:1067,:1046). When bindings share a module path, it checks that module against the union of their checked roots per engine entity, in any binding order, and refuses bindings that give one path different module SHA-256 values (evidence_contract.py:1164-1177).engine_root_inputsadds view roots under the logical entity (evidence_kernels.py:345) and refuses views on an injected non-Axiom adapter (:348).Goldens.
tests/golden/axiom_engine_refs.jsonholds 10 references that the cloud builder (brief 6037) generated at the base, for NZ's six binding rows over a synthetic tree and for the relation fixtures. Its test reads those six rows from a frozen copy,test_support/microcosm_build/axiom_ref_bindings.py, so an edit to NZ's live bindings can't move it.tests/golden/nz_real_axiom_engine_refs.jsonholds NZ's six real references as the base computes them from a git archive of rulespec-nz8dc2507d, the commit NZ's bindings pin. A reference that gains aviewskey without a declared view fails both.Docs.
docs/transport-declared-units.mddescribes the declaration, the gather and scatter, the outside values, the refusals, the selector slice and how the closure partitions view roots. It also states the row-locality assumption thattest_differential_b_real_child_program_against_the_viewis written to test, that a reference keeps the views list in declaration order and eachoutsidemapping in sorted key order, and when an input override may change a selector.Commits
2d8b6271addsAxiomViewand the view gather, scatter, cache, metadata, dependencies and reference to the adapter, withtest_axiom_views.py.a2108456partitions view roots on the logical entity in the closure.f5a06700parsesviewsin the registry and adds the compose guards, withtest_transport_views.py.c04320ddpins NZ-shaped and relation engine references to goldens from the base.446e0316adds the changelog fragment.5933b6fftests that every rules kernel refuses a view node without the whole selector slice.4d4a99b4pins view code toaxiom.pyand the no-view outputs to the base path.881d78aaclasses a view's defaulted roots as person columns in the support roster.6872f50cfreezes NZ's real-tree engine references at the base.ef398616documents binding-local views in the declared-units guide.The fix round, one commit per item of its brief:
b31d4b60checks each shared path against the union of its bindings' roots, so the closure check doesn't depend on binding order, and refuses differing module SHA-256 values (6039 P2-1).679f21fctests two views on one binding (6039 P2-2).94709352checks outside declarations fully at registry build (6039 P3-1, 6040 P3-1).959ea22akeeps outside value types in view equality and hashing (6040 P3-2).ad854d25makes the view properties reach their edges and all four dtypes (6039 P3-2).0d136ed5types differential (b)'s inputs by an explicit table, and checks that an output varies across the batch and that the selector isn't a root input (6041; 6039 P3-3).1cf06b54freezes the CI golden's NZ rows (6040 P3-4).0d6201adcorrects the guide's wording and names a view selector in the override refusal (6040 P3-5; 6039 P3-4).02265bb5merges main875506e8. git mergesT/compose.pyandtest_transport_evidence_kernels.pywithout conflicts.f1d48f79accepts an override of a selector that a program reads, and names only a selector that no program reads (the fix round's first review, P2).760b0b4acorrects the two docstrings that review B flagged:_real_inputssaid the pinned engine names a wrong-typed input, andaxiom_ref_goldens.pyrendered its"\n"as a line break.36800b14merges main3e7a7045, which adds Give a stage launched on Modal a telemetry-only collector token #1473, Re-pin the US Chronicle feed to add Federal Student Aid's loan portfolio tables #1506, H2 (Bind NZS co-residence from household composition (H2) #1485), Tighten five G tests that checked less than their names claim #1501, Encode person_role bindings and their closure defaults on the person table (S2b PR 2) #1480 and Prove extension calibration nodes' measurement producers in compose #1488. git mergesT/compose.pyandtest_transport_evidence_kernels.pywithout conflicts.No commit takes 6040 P3-3, the compose view guard without a closure resource; PR 3 does.
Reach
Kernel hashes [executed, the NZ hub's method:
make_composed_fixture(root)in a temporary directory, thenimplementation_hash(ref)for each of its registry's 45 kernels. The fixture bindsToyRulesEngine, so the threesimulate.*kernels hash again with_adapter_classes = (AxiomEngine,)]:AxiomEngine(3)2d8b6271adapterexport.prepare@1,export.readback@1,transport.input_closure@1simulate.rules_by_ref@1,simulate.counterfactual@1,simulate.solve_zero@1a2108456closuretransport.input_closure@1,transport.donor_receipt@1,transport.donor_ranges@1,transport.support_bounds@1,transport.weights_audit@1,gates.battery@1f5a06700toef3986164a36edcftoef398616b31d4b60closure uniontransport.input_closure@1,transport.donor_receipt@1,transport.donor_ranges@1,transport.support_bounds@1,transport.weights_audit@1,gates.battery@1679f21fc,94709352959ea22aview equalityexport.prepare@1,export.readback@1,transport.input_closure@1simulate.rules_by_ref@1,simulate.counterfactual@1,simulate.solve_zero@1ad854d25to1cf06b540d6201adselector refusalexport.prepare@1,export.readback@1,transport.input_closure@1simulate.*ef398616to0d6201adf1d48f79selector overridesexport.prepare@1,export.readback@1,transport.input_closure@1simulate.*760b0b4adocstringsBase to
ef398616moves these 11 and no others. Bothtransport.createhashes stay atcd21a414f12ab1b9. After a one-line edit to each file at the base, these kernels move [executed]:F/adapters/axiom.py:export.prepare@1,export.readback@1,transport.input_closure@1and the threesimulate.*underAxiomEngine. The export kernels hash it as a module (T/terminal_kernels.py:57,:558,:644at the head),transport.input_closure@1lists it (T/evidence_contract.py:169), and thesimulate.*kernels hash each bound adapter class (F/rules_kernels.py:111-116,:126-135) [read].T/evidence_kernels.py:transport.donor_ranges@1,transport.donor_receipt@1,transport.input_closure@1,transport.support_bounds@1andtransport.weights_audit@1.T/evidence_contract.py: those five andgates.battery@1, which hashes it (T/gate_kernels.py:759) [read].T/registry.pyandT/compose.py: none.No US or UK runtime kernel hashes an edited file: no module under
uk_runtime/,us_runtime/or the US and UK adapters imports or names one [executed,git grepatef398616, and again at02265bb5over its 340 US and UK runtime modules and the five PolicyEngine adapter modules].microcosm/build/frame_serializer_registry.py:95namesaxiom.pyas a write site, not a hash input [read].US primary-QRF worker identity: the binding at
ef398616equals the one at4a36edcf, and the one at0d6201adequals it, byte for byte [executed, fresh processes].Main to head [executed, the same method]: from main
875506e8tof1d48f79, the same 11 kernels move. The other 37 of the registry's 45 keep their hashes, among themtransport.create@1and@2(f003dbed400ad679at both). The US primary-QRF worker identity atf1d48f79equals main875506e8's, byte for byte [executed, fresh processes]; main's own PRs moved it between4a36edcfand875506e8. After the merge, from main3e7a7045to the head36800b14, the same 11 kernels move and the other 37 keep their hashes (transport.create@1and@2at289aae38fc7b1db0on both sides), and the US identity at the head equals3e7a7045's, byte for byte [executed, fresh processes].NZ's node keys. The 8 toy-registry kernels re-key the NZ nodes that use them, and the three
simulate.*re-key wherever a binding runsAxiomEngine. S2D-AGREED item 6 asks for a stated re-run of such a change once NZ's M4, its first calibrated population, has passed, and M4 hasn't passed [NZ, ask 7, and again at 22:05Z on 11/04]. Until then, PR 1 costs NZ cache misses on those nodes and no stated re-run. NZ's rules bindings hold no engine pin yet, andbuild_transport_registryraises without one (T/registry.py:129-132,:559-566) [read], so NZ can't build its registry from them until G8-x, its first real-data run, sets both pins [NZ].Tests
The cloud builder's hand-off comment (#1482 (comment)) gives its counts, skips and mutants at
881d78aa.test_nz_real_refs_match_their_frozen_copyfroze its copy on a first run at the base, passed on a second, and passed at881d78aaagainst that copy, which6872f50ccommits byte for byte [executed]. At the base,test_engine_refs_match_the_base_golden_byte_for_bytepassed too, soaxiom_engine_refs.jsonmatches the base's references.test_differential_b_real_child_program_against_the_viewruns the realChildprogram from rulespec-be'sregional_routing.yamlon a full batch, then on shuffled, single-row and random-subset batches and through a view over a person table, and checks that each row gets the value the full batch gives it. It skips without the Axiom dense extension and a rulespec-be export (POPULACE_RULESPEC_BE). Brief 6042 ran it on Linux, on the pinned engine build, where it passed (#1482 (comment)).The fix round [builder, executed at
0d6201ad, one file per process]: ten files, 593 passed, 42 skipped and none failed.test_axiom_views.py57 passed and 1 skipped,test_axiom_relations.py63 and 1,test_axiom_adapter.py20 and 25,test_axiom_graph_adapter.py165 and 5, the UStest_rules_engine_contract.py0 and 8,test_transport_views.py51,test_transport_registry.py59,test_transport_evidence_kernels.py156,test_nz_axiom_input_closure.py19 and 2, andtest_axiom_engine_ref_goldens.py3. The skipped tests need the Axiom dense extension,POPULACE_RULESPEC_BE, an NZ checkout, or a filesystem that keeps NFC and NFD names apart. Both golden JSON files and NZ's spec and bindings keep their bytes.Each new test failed on the code before its fix, or under the mutant the review named [builder, executed]:
Vehicle.vehicle_valuedropped out and the test failed.Personkey, andtrueon a decimal output,2and0.5on a judgment all passed registry build.0against0.0passed.test_real_inputs_preserve_types_regions_and_seed_across_root_ordersfailed on the inputs' dtype kinds. The typing table agrees with the test cases rulespec-be530849eholds forregional_routing.yamland the modules it imports: 101 inputs with examples, 0 mismatches. Six roots have no case but generated zeros, so the check leaves them out, and the checker's run over the whole export stops on an unrelated#relationkey. Differential (b) itself skips here.test_ci_golden_source_ignores_live_nz_bindingsraisedFileNotFoundError.test_input_overrides_name_a_view_selector_in_the_refusaldidn't raise.Commit 20 [executed at
f1d48f79, one file per process]:test_axiom_views.py58 passed and 1 skipped,test_axiom_relations.py63 and 1,test_counterfactual_kernels.py37,test_quantized_solve_zero.py134,test_axiom_adapter.py20 and 25,test_axiom_graph_adapter.py165 and 5,test_transport_views.py51,test_transport_registry.py59, andtest_axiom_engine_ref_goldens.py3, with a rulespec-nz export at8dc2507d. The skips have the reasons above. On02265bb5's code with this head's test file,test_a_selector_that_a_program_reads_stays_overridablefails with "Input overrides cannot change view selector 'is_child'."On main
3e7a7045[executed on the head's tree98800a01, one file per process, with a rulespec-nz export at8dc2507d]:test_transport_evidence_kernels.py156 passed,test_transport_views.py51,test_transport_registry.py59,test_axiom_engine_ref_goldens.py3,test_nz_axiom_input_closure.py19 and 2 skipped,test_axiom_views.py58 and 1,test_nz_transport_toy_graph.py40,test_transport_measurement_proof.py49,test_transport_column_kernels.py36,test_concept_mapping.py266,test_group_encode.py82,test_axiom_relations.py63 and 1,test_counterfactual_kernels.py37,test_quantized_solve_zero.py134 andtest_transport_composed_graph.py469.test_nz_spec_package.pylists the checkout's files withgit ls-files, so it ran in a scratch git repository whose index reproduces that tree: 1,203 passed. The skips have the reasons above. Fromgit diff 875506e8 f1d48f79togit diff 3e7a7045 36800b14, 14 of this PR's 16 files keep their patch-ids; the other two carry commit 21's docstrings.After CI. Main moved to
fc5bb901after CI run 41915381015 started: #1513 (US), #1491 (UK) and #1507 (NZ's ageing factors from the September 2026 QES), 51 files in all, none of them this PR's.git merge-treemerges the head withfc5bb901cleanly, into treeabff51a7, which adds to the head's tree exactly main's changes since3e7a7045: the same 51 files, with the same patch-id. Onabff51a7[executed, one file per process, with a rulespec-nz export at8dc2507d], each engine-free test file that main changed since3e7a7045passes:test_nz_transport_declarations.py60 and 2 skipped,test_nz_target_references.py20 and 41 skipped,test_nz_reference_aging.py6 and 16 skipped,test_nz_reference_intervals.py15 and 1 skipped,test_nz_money_normalization.py56,test_us_esi_premiums.py175,test_uk_company_cars.py34,test_uk_company_cars_stage_health.py6,test_uk_measure_simulation.py65,test_uk_measurement_period.py21,test_uk_population_targets.py32,test_uk_target_references.py34 and 5 skipped,test_uk_terminal_gates.py62 andtest_transport_composed_graph.py469. Their skips need NZ's or the UK's Chronicle feed, which the local run doesn't stage. The 15 files of the run on98800a01above pass here with the same counts, andtest_nz_spec_package.pypasses 1,203 in a scratch git repository. #1491 also changed the UK spine parity fixtures, and none of the 52 files outsidetools/that nameuk_spineloads a module this PR changes [executed: each imported in its own process onabff51a7].Still to do
ef398616. 6039, the mechanisms (Add binding-local views to the Axiom adapter, registry, compose and closure (S2b PR 1) #1482 (comment)), wanted changes: two P2s and four P3s. 6040, the integration (Add binding-local views to the Axiom adapter, registry, compose and closure (S2b PR 1) #1482 (comment)), approved with five P3s. 6041, the engine-gated run (Add binding-local views to the Axiom adapter, registry, compose and closure (S2b PR 1) #1482 (comment)), found differential (b) failing on the test's input typing. Commits 11 to 18 answer each finding but 6040 P3-3, which PR 3 takes.02265bb5: one P2, which commit 20 answers. Its sandbox allowed no temporary files, so it ran its mutants in memory and couldn't run the tests that need a temp directory.f1d48f79, both GPT-6.1 Sol through Subfleet. A2, of commit 20 and the mutants on scratch copies, approves with no P1 or P2: the five named mutants fail their tests, and on02265bb5's code the new selector test fails. Its one P3: in a whole-file run on a loaded host,TestClosurePartition::test_property_every_live_root_lands_in_exactly_one_classfailed Hypothesis's slow-input-generation health check once, and it passed alone at the head and at main875506e8. That test predates this PR (92a3d6bf), so this PR leaves it to a follow-up. B, of differential (b), the goldens, the main merge, reach and this body, approves with no P1 or P2 and two P3 docstring fixes, which commit 21 makes.f1d48f79, at 18:55Z: patch-ids, the diff, the merge with maine2918f4f, the kernel table, and NZ's real-tree references.f1d48f79, at 22:05Z: commit 21 and the merge of main3e7a7045. Its own runs agree: the same 11 kernels move, andtest_nz_transport_declarations.pypasses 62, none skipped, on main plus NZ: re-declare the ageing factors from the September 2026 QES #1507 plus this PR.f1d48f79: run 41882901707.36800b14: run 41915381015 passed all 11 checks. Both engine-free jobs rantest_transport_evidence_kernels.py, the one file both H2 and this PR edit: each job's pytest command lists it, and each job's--durations=25report lists two of its tests. Each job passed 25,982 tests, skipped 291 and xfailed 3, and none failed.test_differential_b_real_child_program_against_the_viewon Linux, on the pinned engine build: brief 6042 passed it, with every other engine-gated file, atf1d48f79and merged with Encode person_role bindings and their closure defaults on the person table (S2b PR 2) #1480's head then (17e26433).nz/axiom_rules_bindings.jsonstill leaves null on main. So the stated re-run hasn't come due, and this PR costs NZ cache misses on the 11 kernels' nodes. The NZ hub reports the day M4 passes.🤖 Generated with Claude Code