[DTrim99]

Fixes #776. (#1077 was closed as a duplicate of #776.)
Refs #1020, #1052, PolicyEngine/policyengine-us#9732

The blocker is gone: PolicyEngine/policyengine-us#9737, which models the ICHIA/CHIPRA §214 lawfully-residing option, merged on October 5, 2026. This branch is now merged up to main at 79a533c7 (November 3, 2026), which includes #1052's ACS local immigration stage and #1496's spec-engine repin.

Problem

The default US build stores no years_since_us_entry, so every person gets the policyengine-us default of 5: past the federal five-year bar (8 USC 1613). PolicyEngine/policyengine-us#9732 removes that default, which would set the clock to 0, and is held until the data supplies the column. Without a stored clock, the bar, the Missouri SSP waiting period and the Washington TANF and RCA windows all run on a default instead of the survey. The CPS ASEC already carries the raw signal in PEINUSYR (year of entry to stay).

Design

The immigration stage (us_runtime/immigration.py) now owns a third person column, years_since_us_entry (float). The two status labels are unchanged.

• Foreign-born (PRCITSHP 4/5): the time period minus an arrival year, clipped at 0.
• US-born (PRCITSHP 1-3, which includes people born in Puerto Rico or the territories and those born abroad to a US parent): age (A_AGE). This is Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052's convention; citizens short-circuit every engine consumer.
• Pre-US: store years_since_us_entry from PEINUSYR (blocked on engine ICHIA option) #776 bases. A base labelled before this change carries both labels but no clock, and passes through with_us_immigration_inputs unchanged; the engine default still applies to it. A clock stored without the labels is refused as a partial surface.

Band mapping (verified against the Census data dictionaries)

Each code was checked in the PEINUSYR value lists of cpsmar23.pdf to cpsmar26.pdf (https://www2.census.gov/programs-surveys/cps/techdocs/):

code	interval	vintages
1	before 1950 (lower edge = birth year)	all
2 / 3 / 4 / 5 / 6	1950-59 / 60-64 / 65-69 / 70-74 / 75-79	all
7	1980-1981	all
8-26	two-year bands, 1982-1983 to 2018-2019	all
27	2020-2023	ASEC 2023 (income 2022)
27 / 28	2020-2021 / 2022-2024	ASEC 2024 (income 2023)
27 / 28	2020-2021 / 2022-2025	ASEC 2025 (income 2024)
27 / 28 / 29	2020-2021 / 2022-2023 / 2024-2026	ASEC 2026 (income 2025)

Census re-bins the top codes every year, so a code is read in the codebook of the person's own vintage. The survey year is source_year + 1. A frame without source_year is read as one file of the period's income year.

The stage refuses rather than guesses in three cases:

• a foreign-born person with code 0 (NIU);
• a code outside the vintage's codebook (for example 29 in ASEC 2025);
• any vintage without a reviewed codebook (ASEC 2027 needs a new entry).

Straddle rule

Every consumer compares the clock with a whole number of years: >= 5 for the bar and the MO SSP wait, and < window for the ORR window. A band midpoint would put a straddling band wholly on one side, so the arrival year is drawn uniformly over the band's calendar years, using the stage's seeded blake2b keyed draw:

• Salt. It has its own salt, immigration:arrival_year, so the status labels are untouched.
• Key. It is keyed on source_year:source_person_id, so support clones share one entry year.
• Seed protocol. It is registered as seed site immigration_arrival_year_draw.

Two bounds apply before the draw:

• The band is first intersected with the years the person can have arrived in, no earlier than survey year - A_AGE - 1. That also bounds the open "Before 1950" band.
• If the age contradicts the band, the band wins at its last year.

With the 2024 period, no two-year band straddles five: 2018-2019 gives 5-6 years and 2020-2021 gives 3-4. In an odd period every two-year band does, and the wide top codes straddle the one-year window, so the draw keeps the rule exact in expectation for any period.

National pool and ACS rows

The column joins the immigration operator output family (operator_boundary), so the multispine pool's source_operator_immigration transfer family becomes (immigration_status_str, ssn_card_type, years_since_us_entry). On ACS rows the clock is a chained-QRF target fitted after the joint status pair, so it is conditioned on the transferred status. It is registered as a discrete numeric target, which keeps transferred values on the donors' whole-year support. #1052 / #1020 remain the route to exact YOEP years on ACS rows.

Catalog unit

Since #1484 every numeric pool input needs a catalog unit. policyengine-us 2.2.1 declares unit = "year" on years_since_us_entry, and "year" also marks calendar years (birth_year), so the generator refuses to map it on its own. The clock is reviewed into _REVIEWED_UNIT_BY_INPUT as a duration, ("year", "years"), and years joins hours and weeks in the unit enum of spec_engine/schema/catalogs.schema.json.

Relationship with #1052 (merged November 3)

#1052 added the ACS local lane's immigration stage (us_runtime/acs_local_immigration.py). It fills only missing cells: exact YOEP clocks on ACS rows, and, on donor rows without a clock, the PEINUSYR bin year of immigration._asec_arrival_year. The two now fit together this way:

• One definition. The ACS local stage no longer redefines the clock's constants; it reuses the immigration stage's. YEARS_SINCE_US_ENTRY_COLUMN is US_IMMIGRATION_YEARS_SINCE_ENTRY_COLUMN, ACS_LOCAL_IMMIGRATION_COLUMNS is US_IMMIGRATION_OWNED_PERSON_COLUMNS, and ENGINE_DEFAULT_YEARS_SINCE_US_ENTRY, FIVE_YEAR_BAR_YEARS and the foreign-born PRCITSHP codes come from the immigration stage. The public names Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052's tests and tools import are unchanged.
• Donor rows. A donor built after this PR carries this PR's band-drawn clock, which the local stage keeps. Its bin-year fallback now applies only to donors labelled before US: store years_since_us_entry from PEINUSYR (blocked on engine ICHIA option) #776. Its module docstring says so.
• ACS rows. The local lane's ACS transfer excludes runtime-owned immigration columns, so the clock does not reach local-lane ACS rows by transfer; Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052's exact YOEP clock still fills them. Only the national pool's ACS rows get the clock by transfer.
• Whole-file gate. Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052's acs_local_immigration_signal_gate runs us_immigration_composition_gate on the whole file, and that gate now also grades a stored clock. A missing clock cell is therefore named for its spine and once more for the file; the gate fails in the same cases as before. One Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052 test expectation and the gate docstring say so.
• Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052's NEVER_DEFAULT_FILLED entry covers the ACS local lane; the national lane has no default-fill register (its ACS rows are filled by transfer).

The arrival-clock caveat María raised on #1052 applies here too: neither the ASEC nor the ACS records when qualified status was granted, so the clock runs from physical entry. That is modeling decision 2 below. #1052's adjustment_lag_sensitivity receipt sizes the affected group for the ACS local lane; the national lane has no equivalent receipt yet.

Registries and pins

Values are main (31c89390) → this branch.

Registry or pin	Change
source_stages.json	outputs and notes; frozen generation-0 digest re-pinned (generator, test_us_bundle_core_contracts, test_support/us_spec_bundle): 1683f9cd… → d11d99fd…
Spec bundle	regenerated (tools/generate_us_bundle_from_constants.py): sources.yaml, imputation.yaml, catalogs.yaml, battery.yaml, spine.yaml, engine_abi.lock.json (policyengine-us still 2.2.1); spec_sha256 fe3901b5… → c46f0ea7…
Catalog	177 → 178 columns; closed-catalog check 177 → 178; producer/catalog closure 135 → 136 physical outputs; new unit years
Stacked spine	battery metric (monetary_sign_separated 81 → 82); surface counts 136 → 137, 72 → 73 late, 29 → 30 ASEC-source, 120 → 121 transfer; completion receipt 72 → 73
Late-producer registry	72 → 73 targets, 53 → 54 numeric
Pool engine contracts	tools/repin_us_pool_engine_contracts.py: provisioning 762/123 → 761/124 (declared-absent / materialized), remaining-stage manifest d4fb9de9… → 7ae93314… (still 1,059 rows; also in the engine ABI lock test and the H3 parity fixture, regenerated with tools/graph_us_post_transfer_fixture.py)
Bundle generator invariants	late DAG (19, 72) → (19, 73), late float dtypes 52 → 53, graph counts (2747, 92, 230, 35, 0, 214) → (2748, 93, 232, 35, 0, 214)
Field-usage ledger	42,210 → 42,267 fields (32,429 → 32,467 authored, 9,781 → 9,800 resolved); nine pointer claims; coverage evidence docs/evidence/spec-engine/us-f0-coverage.json regenerated
Inventory coverage	nine digests (seed map 99fafd88… → 4db18a3a…, seed protocol 61b4206a… → 89789314…, …) and the count vector (also test_support/spec_engine_inventory_coverage.py)
Seed protocol	new site immigration_arrival_year_draw (owner immigration_status); 53 → 54 sites, 112 → 113 bindings
Release input coverage	release_input_coverage.POST_REFERENCE_ECPS_REQUIRED_INPUTS and the manifest tool; manifest regenerated: 171 → 172 required, 6 reviewed exclusions, 177 → 178 columns; required, NO reviewed exclusion (the #368/#462/#978 pattern; today's artifacts are red until a national build carries it)
acs_transfer	_DISCRETE_NUMERIC_TARGETS
us_runtime exports	new names exported
Test pins	count and digest pins in 31 test and test-support files (schedule f1fb754e → 3b9801f8, payload 76a79c9d → 3f37bc91, authority bf681aed → b9ce079f and four of its components, the pool transfer-target digest 8b234f9a → dd0b1be5, the three ACS transfer execution-contract digests, the pool-tool manifest golden bfaa7ffc → ba39a694, the plan lock's 54 seed sites, the loader golden vector 5892f4b1 → f5950b7e)
test_spec_only_country_packages	its scanner refuses any key holding the token entry; years_since_us_entry is the first release-coverage column name with it, so a plain snake_case key directly under release_input_coverage_manifest.json's columns is admitted, and its row is still scanned
docs/us-multispine-operator-ordering.md	updated rows, counts and digests

Where main and this PR each moved a count from the same base value (the pool input surface 142 → 143, the producer/catalog closure 134 → 135, pool provisioning 763/122 → 762/123), git merged the edit once; those were corrected to the observed values (144, 136, 761/124).

Summary statistic and gate

us_immigration_composition_summary now reports the weighted entry-clock distribution:

• per non-citizen status and for all non-citizens;
• the population and share under the five-year bar;
• 0-4, 5-9, 10-19 and 20+ year bands.

The weighted LPR share under five years is by_status.LEGAL_PERMANENT_RESIDENT.under_5_years_share. It flows into the base builder's summary JSON and the fiscal-refresh gate details. us_immigration_composition_gate fails a stored clock that is incomplete, negative, or the engine default for every non-citizen. A frame without the clock still passes, as before.

Needs confirmation from code owners

Two changes outside the immigration stage need confirmation from whoever owns that code:

1. Spec-only test guard (Declare the NZ transport and entitlement graphs #1440). That test refuses any key containing entry, so years_since_us_entry failed it. I added a narrow exemption for column names listed in the release-input-coverage manifest. Please check that this is the intended scope.
2. Catalog unit. Since Read US catalog units from engine source and retire the unit waiver #1484, every numeric input needs a unit. years now joins hours and weeks in spec_engine/schema/catalogs.schema.json, and the clock is reviewed in as a duration (see "Catalog unit" above).

Tests

After merging main, every file below was run under WSL one at a time against policyengine-us 2.2.1. Engine-contract files import the engine; the rest are engine-free.

File	Result
engine_free/us/test_us_immigration	118 passed
engine_free/us/test_us_acs_local_immigration (#1052)	79 passed
engine_free/us/test_us_acs_local_release_tool (#1052)	214 passed, 1 skipped
engine_free/us/test_us_multispine_pool / test_us_late_producer_dag	64 / 30 passed
engine_free/us/test_us_acs_transfer / test_us_spine_blindness	103 / 504 passed
engine_free/us/test_us_spec_bundle / test_us_bundle_core_contracts / test_us_catalog_units	4 / 17 / 2 passed
engine_free/us/test_us_spine_agreement / test_us_l0_refit_export / test_us_stored_input_register	14 / 50 / 28 passed
engine_free/us/test_us_puf_support_base_builder / test_us_fiscal_refresh_builder	72 / 350 passed (2 skipped)
engine_free/shared/test_spec_engine_* (loader, seeds, battery, compiler_ir, field_usage, imputation, legacy_adapter, identity_contracts, calibration, inventory_coverage, plan_lock, engine_abi)	10 / 11 / 4 / 10 / 14 / 4 / 6 / 8 / 12 / 13 / 6 / 10 passed
engine_free/shared/test_spec_only_country_packages / test_release_input_coverage	37 / 86 passed
engine_contract/us: coverage_tool, inventory_coverage, release_input_coverage, stacked_authority_semantics, imputation_lineage_spec, us_bundle_core_contracts, typed_closure, us_spec_bundle, engine_abi, us_graph, take_up_semantics	7 / 2 / 34 / 4 / 8 / 4 / 13 / 22 / 1 / 6 / 16 passed
microcosm-graph engine_workflow/us/test_acceptance_h_parity	1 passed

ruff check . is clean, ruff format is clean on every touched file, and tools/ci_test_plan.py verify is ok. --check is clean for repin_us_spec_engine_contracts, generate_us_bundle_from_constants, spec_engine_coverage (42,267/42,267 fields, 41/41 inventory checks) and repin_us_pool_engine_contracts.

The full engine-free and engine-us jobs, and the engine-workflow pool-tool and H3 parity files, ran on Modal against main at 31c89390 with the same runner:

Job	main 31c89390	this branch 530083ad
engine-free (713 files)	25,269 passed, 285 skipped, 3 xfailed, 3 failed	25,401 passed, 285 skipped, 3 xfailed, the same 3 failed
engine-us (contract, scenario, workflow)	367 passed, 6 skipped	367 passed, 6 skipped
engine-workflow pool tool + H3 parity	32 passed	32 passed

The three engine-free failures are the same on main: a root-user permission test, a worker-recycling test that needs Python 3.13.15 (the runner has 3.13.12), and a transport-evidence module count. The Modal runner drops the container's inherited PYTHONPATH; with it, about 600 tests error on a duplicated idna distribution on main as well.

New tests in test_us_immigration.py:

• TestEntryClockBands: every fixed band in every vintage; each vintage's top codes; clip at zero; birth-year bounds; age contradictions; the frame without source_year; refusals.
• TestEntryClockStraddleRule: a straddling band lands on both sides at about 50/50; no straddle across five in 2024; the one-year window; seeded reproducibility; clone consistency.
• TestEntryClockUsBorn: PRCITSHP 1-3 get their age.
• TestEntryClockStageOutput: the ASEC labels and every other column are byte-identical to the unchanged label kernels, and the clock is appended last as float64.
• TestEntryClockFrameIntegration and TestEntryClockSummaryAndGate.

Not yet run on real data. No build was run.

Modeling decisions (for review)

1. Period, not interview year. The clock is time_period - arrival (as US: store years_since_us_entry from PEINUSYR (blocked on engine ICHIA option) #776 and Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052 specify), and the pooled vintages are not restated. With a 2024 period and the 2024-2026 ASEC pool, the 2025 and 2026 vintages count arrivals after 2024 as 0 years, and their 2020-2021 arrivals as 3-4 years rather than the 4-6 years they had at interview. That raises the pooled share under the bar relative to measuring at interview. For scale, the ASEC 2026 file has 1,047 persons in code 27 against 2,862 in codes 28-29 (all persons, unweighted; experiments/us-asec-2025-source/receipts/measurements_2025.json). The alternative, survey year - arrival, keeps each vintage's own duration distribution; it is a one-line change if reviewers prefer it.
2. Entry, not status grant. The bar runs from the grant of qualified status, but the CPS measures physical entry. For people who adjusted status after arriving, the clock runs long. For the five-year bar (Medicaid, CHIP through Medicaid, Washington TANF) that under-applies the bar, the conservative direction. For a window that closes, such as Washington RCA (years_since_us_entry < window_years), it runs the other way and can wrongly exclude a recently qualified person. This is the caveat María raised on Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052, and it applies here unchanged. Modelling an adjustment lag (from the DHS Yearbook) is deferred.
3. Draw within band, not midpoint. See the straddle rule above. The kernel's existing midpoint table (_ARRIVAL_YEAR_MIDPOINTS) still drives the DACA and Cuban/Haitian tests, so the labels stay byte-identical. The clock never feeds a label.
4. Pre-US: store years_since_us_entry from PEINUSYR (blocked on engine ICHIA option) #776 bases pass through. Rederiving only the clock on an old base would silently change existing --base-h5 lanes, so it is not done. The release-input-coverage gate makes the missing column visible.
5. ACS rows via transfer. This is not exact YOEP. Assign immigration status and years since entry on ACS rows in the ACS local lane (#1020) #1052 owns that route for the ACS local lane; the national pool's ACS loader does not keep YOEP yet (ACS local lane: immigration_status_str and ssn_card_type default-filled CITIZEN for all ACS persons; years_since_us_entry never populated #1020).

Before merge / on the next rebuild

• On a from-scratch rebuild, check:
  • the weighted LPR share under five years (under_5_years_share), against DHS LPR-flow evidence; Derive years_since_us_entry from CPS PEINUSYR (and ACS YOEP) so policyengine-us can drop its default #1077 estimated about 26% of LPRs arrived in 2020 or later;
  • the clock on ACS rows against ASEC rows (spine agreement and battery);
  • no PEINUSYR refusals (foreign-born code 0, or an unreviewed vintage);
  • Medicaid, SNAP and ACA calibration shifts;
  • the pe-us#9732 microsim.

🤖 Generated with Claude Code