[frahlg]

Problem

Item C5 of the random-rig triage (27 Oct). A site meter that answers every poll with the same total and phase values counts as fresh, because the dispatch gate checks only UpdatedAt, and each repeated answer is a new sample.

Seed 207537: the rig's meter repeated one sample (2667 W, L1/L2/L3 2.4/5.5/-0.2 A) for 39.9 s. Dispatch acted on it the whole time. The battery PI wound from +5 kW to -5 kW, and an EV rise from 0 to 6 A went out 38 s into the freeze ("no raise on a stale meter" failed). #1879 (merged) covers only a meter whose UpdatedAt stops moving.

Change

Owner rule: total and phases unchanged over N polls while FTW's own orders moved by more than a set amount means stale. Stop dispatch and raise no EV.

• control.State.ObserveSiteMeterFreeze(store, now) (new go/internal/control/meter_freeze.go) runs every tick under the control lock. It compares each new meter sample (total W, plus l1..l3 A, W and V when present) with the one before. When the values change, it records where every battery and charger stands: FTW's last order (PrevTargets, evCommanded; none counts as 0 W) and the device's own reading.
• It reads the meter and the devices through health-gated accessors (telemetry.Store.OnlineMeterSample, OnlineRawWByType). A watchdog-offline driver drops out, so its cached reading never counts.
• A device's move counts as the smaller of its order's move and its reading's move, in the reading's direction. So an order counts only when the device followed it: a full battery, a car that does not draw, a charger that has not applied an offer yet, or a battery that moves against its order counts 0. The moves are summed with their sign, since the meter sees only the net change.
• Held from the first repeat. Once the net move is more than 500 W and the meter still repeats the values from before it, battery dispatch treats the sample as held (siteMeterHeld, as in fix(control): act only on new site-meter samples in battery dispatch #1879): the PI holds its targets instead of acting on the same error again, and other paths may only move a battery toward 0 W.
• Stale after 30 s. If the meter repeats the same values for at least 3 more polls and 30 s after the move, it counts as stale. If the devices move back under 500 W first, the wait starts over. Any changed meter value clears both at once.
• cmd/ftw: siteDispatchGateLocked is the one gate the tick and siteDispatchNow (operator setpoints) share. Setup's check before it enables control now uses the same gate (siteMeterGateLocked) with the candidate's fuse. A frozen meter gives site_meter_stale: drivers return to their default mode, TickWithDispatch(false) stands the chargers down, and the battery gets no new orders.

Why these numbers

• 500 W. Site meters resolve the total to 1–10 W and phase currents to 0.01–1 A (1 A ≈ 230 W). Batteries track an order to within tens of watts, and inverters ignore steps below about 100–200 W. 500 W is 50 times the coarsest total step and above every inverter dead band. Each EV step is at least 1380 W (6 A on one phase).
• 30 s and 3 polls. Swedish HAN meters and DSMR 4 send a reading every 10 s, so a driver that polls every second sees up to 10 identical samples by design. A telegram lost to a CRC error leaves the old values for 20 s, and the PI makes its large moves right after a telegram that shows a load step. 30 s is three telegram periods, so one lost telegram cannot trip it. The 3 polls mean a slow poller is judged on several answers, not one.
• Why an EV order waits for the car. The rig's OCPP charger accepts a new limit at once, and the car draws it about 30 s later (seed 207537: offer at 157 s, draw at 188 s; command_response_s: 40, startup_delay_s: 30). Real cars take as long to wake. Until then a live meter with no noise repeats itself. Counting the accepted offer would mark that live meter stale and send the chargers to 0 W.

For seed 207537 the battery reading shows a 924 W move about 1.6 s into the freeze. From then the PI holds the battery, and the meter goes stale about 31.6 s into the freeze, before the 38 s EV rise. The control test that replays the seed with a one-tick battery lag holds the battery between 3208 and 4075 W and goes stale at 34 s. The runner should confirm this on the rig.

Second review changes (77bcd3d)

• CI "no new direct telemetry reads" (meter_freeze.go 0 → 5): the check now reads through the two new health-gated accessors. The script reports "none added".
• Seed 207537's battery swing: the PI now holds from the first repeat after a 500 W move. Before, the swing ran the full 10 kW inside the 30 s window.
• The 3-poll rule has a test (20 s poller).
• Setup's control check uses the shared gate.
• Risk names the second-controller case and the EV-only gap.
• Not changed: an accepted EV offer still does not count until the car draws it. See Risk and the question for the owner.

Tests

go/internal/control/meter_freeze_test.go:

• TestFrozenMeterWithMovingOrdersIsStale (triage section 3, item 5): the 207537 sample repeats while ComputeDispatch winds the battery, which follows one tick later. Battery orders stay within 3208..4075 W; stale at 34 s, before the 38 s EV rise. One changed value reopens the gate.
• TestFrozenMeterHoldLetsLiveMeterLoopSettle: a closed loop with ComputeDispatch. With a 10 s meter polled every 2 s, the battery reaches the target without overshooting more than 1 kW into export; with a meter that sends a new reading every tick, it settles too. Never stale.
• TestFrozenMeterSlowPollerNeedsThreeRepeats: one answer every 20 s after a 3 kW move. Not stale after 1 or 2 repeats, stale after 3.
• TestFrozenMeterAcceptedOfferWaitsForTheCar: the rig's timing, offer accepted at 1 s, car draws at 32 s, noise-free live meter. Never stale.
• TestFrozenMeterQuietSiteStaysFresh, TestFrozenMeterTenSecondTelegramsDoNotFlap, TestFrozenMeterLostTelegramDoesNotTrip, TestFrozenMeterOrdersNotFollowedDoNotCount, TestFrozenMeterOppositeMovesCancel, TestFrozenMeterMoveAndBackDoesNotCount, TestFrozenMeterFirstOrderMatchingAutonomousBatteryDoesNotCount: never stale.
• TestFrozenMeterNewlyPluggedEVCounts, TestFrozenMeterFromStartIsCaught, TestFrozenMeterWaitsForTheMoveToLand (not stale at 29 s, stale at 30 s): stale.

go/internal/telemetry/telemetry_test.go: TestOnlineMeterSampleSkipsOfflineMeter, TestOnlineRawWByTypeSkipsOfflineDrivers.

go/cmd/ftw/site_meter_stale_test.go: TestSiteDispatchGateClosesOnFrozenMeter drives siteDispatchNow to site_meter_stale before 38 s, checks that setup's gate with a candidate fuse gives the same verdict, and that a changed value reopens it. TestFrozenSiteMeterClosesDispatchGate: a frozen verdict does not replace an existing reason.

Fail before, pass after, by undoing each change on its own:

• PI hold off: TestFrozenMeterWithMovingOrdersIsStale fails (orders swing -5000..4075 W) and TestFrozenMeterHoldLetsLiveMeterLoopSettle fails (10 s meter overshoots to 2000 W export).
• meterFreezeSamples = 0: TestFrozenMeterSlowPollerNeedsThreeRepeats fails.
• An accepted EV offer counted without the car: TestFrozenMeterAcceptedOfferWaitsForTheCar fails at 31 s.
• Earlier rounds: window back to 15 s, absolute sum, devices without an order skipped, no restart after a move back, counting moves against the order, and the gate ignoring the verdict each fail their test.

go test -race -count=1 passes on ./internal/control, ./internal/telemetry and ./cmd/ftw. make verify passes. .github/scripts/check-telemetry-reads.sh origin/master: "none added". TestSite/siterig was not run here; the runner does that.

Risk

• False stale. It needs a meter that repeats its total, every phase current and power, and voltage exactly for 30 s and 3 polls after FTW's devices visibly moved more than 500 W net. A live meter that sends voltage does not do that. A meter that sends no voltage and whole-amp currents (some DSMR 4) has only its total W to move. Two lost telegrams in a row on a 10 s meter (30 s gap) is the edge left. A false stale stops dispatch until the meter's values change, returns drivers to default and clears a manual battery hold (clearBatteryManualHoldForDispatchBlock).
• Second controller. The check assumes the meter shows any move of FTW's devices. A second controller on the site can cancel such a move exactly: an inverter export limiter that curtails PV as the battery discharges, or a third-party battery with its own CT. With a meter that sends no voltage, this can hold the battery from the first repeat and then give a false stale after 30 s. FTW has no such case on record.
• Held from the first repeat. On a 10 s meter polled more often, the PI now acts once per new reading after a move over 500 W, instead of again on each repeat. The closed-loop test shows this settles and overshoots less (under 1 kW instead of 2 kW export). Plan-following paths may only move a battery toward 0 W on such a repeat, as on any held sample; the next reading lifts that.
• EV-only site. An EV order counts only once the charger reports the car drawing it. At a site with no battery, a frozen meter is found 30 s after that. With the rig's charger (car draws about 30–40 s after the offer), that is 60–70 s after the raise, so a 40 s freeze there goes unseen and the first EV raise on a frozen meter goes out. Counting the accepted offer instead would catch it within 30 s, but it marks a live noise-free meter stale while the car wakes (TestFrozenMeterAcceptedOfferWaitsForTheCar).
• A meter that refreshes more slowly than every 30 s (for example a cloud-polled meter) now goes stale whenever FTW moves a device. Dispatching on such a meter is the hazard this fixes.
• Not covered: a hybrid inverter whose meter and battery readings freeze together. The battery reading does not move, so the check does not fire. That is no worse than today.

Stack and merge order

Based on master (merged up to 7840c33). Stands alone. It touches go/cmd/ftw/main.go (the freshness block in the tick and setup's control check), go/internal/control/dispatch.go (one State field, one line where dispatch reads the meter), go/internal/telemetry/store.go (two accessors) and go/cmd/ftw/site_dispatch_safety.go.

Open PRs that touch the same files:

• No other open PR touches meter_freeze.go or telemetry/store.go.
• Draft fix(control): fuse protection as one function #1789, which is in conflict, rewrites site_dispatch_safety.go, including the return line of siteDispatchNow that this PR changes. Whichever lands second needs a small hand merge.
• Other open PRs change other lines in main.go or dispatch.go.

Question for the owner

At a site with no battery, should an EV offer the charger accepted count as a move before the car draws it? Yes catches a frozen meter within 30 s of the first raise, but marks a live meter with no noise stale while the car wakes, which on the rig sends the chargers to 0 W. This PR answers no.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T18:57:47.408049Z	c3f148c	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3f148c7cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Establish baselines for devices first commanded during a freeze

If the meter starts repeating before the first battery dispatch, its baseline contains no batteries: NewState initializes PrevTargets empty, and ObserveSiteMeterFreeze runs before ComputeDispatch populates it. This branch then skips that battery forever because f.devices is refreshed only when the meter values change. Even subsequent confirmed multi-kilowatt battery movements never mark the meter frozen, leaving dispatch enabled indefinitely. Establish a baseline when a newly commanded device first appears rather than requiring a changed meter reading.

AGENTS.md reference: AGENTS.md:L66-L66

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Account for cancellation between managed device movements

For a supported total-only meter, a battery increasing discharge by 2 kW while an EV increases consumption by 2 kW legitimately leaves the site reading unchanged. Summing absolute device movements reports 4 kW of movement instead of zero, so after 15 seconds this healthy meter is declared stale and all dispatch, including EV charging, is stopped. Account for the signed net effect at the site boundary before concluding that the meter must have changed.

AGENTS.md reference: AGENTS.md:L63-L63

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Reset the detection window when device movement disappears

If a battery moves above the threshold and returns to its baseline before the next legitimate 10-second meter telegram, the meter can correctly keep reporting the same values. However, once movedAt is set, this branch never checks movement again: it declares the meter frozen 15 seconds later even when devicesMovedW is now zero. This shuts down healthy dispatch for a transient the meter never sampled. Clear or restart the pending verdict when the qualifying movement no longer exists.

Useful? React with 👍 / 👎.