Repository navigation
[Bug] A nested shorter window of an exact sum does not reuse the newest panes of the longer window's chain #924
Description
Example
sum(sum_over_time(m_gauge[1m])) is the merge of the newest panes of the pane chain already kept for [2m]. Today the planner builds a second chain.
Workload base.json (exact sum (sum_over_time(m_gauge[2m])) every 60 s):
{"query_workload":{"language":"promql","query_batch":null,"repeating_queries":[{"query":"sum (sum_over_time(m_gauge[2m]))","demand":{"fixed_interval":60000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}}]},"data_workload":{"arrival":{"ingesting":{"rows_per_second":1333.3333333333333}},"sources":{"m_counter":{"series_count":10000,"metric_type":"counter","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}},"m_gauge":{"series_count":10000,"metric_type":"gauge","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}}}}}Workload related.json (adds exact sum (sum_over_time(m_gauge[1m])) every 60 s):
{"query_workload":{"language":"promql","query_batch":null,"repeating_queries":[{"query":"sum (sum_over_time(m_gauge[2m]))","demand":{"fixed_interval":60000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}},{"query":"sum (sum_over_time(m_gauge[1m]))","demand":{"fixed_interval":60000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}}]},"data_workload":{"arrival":{"ingesting":{"rows_per_second":1333.3333333333333}},"sources":{"m_counter":{"series_count":10000,"metric_type":"counter","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}},"m_gauge":{"series_count":10000,"metric_type":"gauge","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}}}}}cargo run -p asap-devtools --bin stage_pipeline -- --workload base.json --enforce-accuracy-guarantees --out base.out.json
cargo run -p asap-devtools --bin stage_pipeline -- --workload related.json --enforce-accuracy-guarantees --out related.out.json
# same two commands with --exhaustive: identical selected plans and costs| Workload | Selected plan | Cost per second |
|---|---|---|
base.json |
P4-m1 |
0.004156 |
related.json |
P15-m2 |
0.008290 |
Witness: base.json plan with the shorter query derived by a SummaryMerge of the newest panes of the longer query's pane chain covering 60000 ms |
witness-m1 |
0.004156 |
Expected: related.json costs about 0.004156 (the witness), not 0.008290.
Re-verified on current main (stage_pipeline built at 65655b6); the numbers above are from that run.
Current behavior
The shorter window doubles the cost (an extra 0.004134 per second) instead of adding nothing. Same class in SQL: SUM(value) over 5 minutes plus over 1 minute (seed 11 case 7): 0.015642 selected vs 0.007842 witness.
Expected behavior and invariant
For the workload with the added query, the selected cost should not exceed the base cost plus the cost of deriving the added query from the base plan, within a small tolerance. The cheaper plan (the witness) is priced by Stage 2 and Stage 3 with the same models as the search and passes Stage 3's checks, but the search never selects it.
Minimal reproduction
The two stage_pipeline commands above. planner_fuzz found it (seeds and relation names are under Status and related work). Running with --exhaustive selects the same plan and the same cost, so no candidate rule generates the cheaper plan; it is a missing candidate, not a search miss.
Why it happens
Not diagnosed. The nested-window rule that the docs describe does not produce a candidate where the shorter window merges the newest panes of the longer window's chain for exact sums.
Regression test
Plan base.json and related.json and assert cost(related) - cost(base) <= derive_cost + tolerance. Today it fails with the numbers above.
Status and related work
Related: #873 (nested rate/increase windows, which covers rate and increase only) and #731 (shared window grids). Found by planner_fuzz seeds 11 and 12, relation d-nested-shorter-window.
Activity
zzylol commentedon Oct 27, 2026
Reproduced the original exact-sum example on main
7680b6bcwith candidate-connection MILP and enforced accuracy: base 0.0041564500, with the shorter window 0.0082898001 at a 30 s budget.Confirmed candidate-generation cause:
crates/logical-asap-optimization/src/asap_aware_cse.rs::pane_candidatesrequirespane_count >= 2. The shorter[1m]query evaluated every 60 s has exactly one pane and is excluded. Consequently the shared-state configuration pass has no one-pane reader to extend onto the longer query's two-pane chain. This is a candidate-domain gap, not merely a solver failing to choose an existing shared reader.Allowing the legal one-pane candidate makes the regression pass: related workload 0.0041564833 versus base 0.0041564500. The additional cost is the short reader's merge/finalize work.
The change and regression are local and uncommitted. Broader pane-boundary and execution validation is still needed before treating this as a merge-ready fix; the issue should remain open.
zzylol commentedon Oct 27, 2026
Implemented in PR #943. The pane rule now admits one-pane readers, allowing the shorter sum to consume the newest pane of the longer chain. Original paired-window cost is 0.0041564833/s versus single-query 0.0041564500/s. The SQL 5-minute/1-minute sum variant also passes.
Final local validation: 690 focused tests passed, plus workspace format and all-target/all-feature Clippy checks.