Repository navigation
[Bug] The same query at a longer repeat interval recomputes instead of reusing the shorter-interval result #925
Description
Example
The same query asked every 120 s is answered by every second evaluation of the same query asked every 60 s. Today the planner builds a separate plan for the 120 s copy.
Workload base.json (exact sum (sum_over_time(m_gauge[2m])) every 60 s):
{"query_workload":{"language":"promql","query_batch":null,"repeating_queries":[{"query":"sum (sum_over_time(m_gauge[2m]))","demand":{"fixed_interval":60000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}}]},"data_workload":{"arrival":{"ingesting":{"rows_per_second":1333.3333333333333}},"sources":{"m_counter":{"series_count":10000,"metric_type":"counter","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}},"m_gauge":{"series_count":10000,"metric_type":"gauge","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}}}}}Workload related.json (adds the same query every 120 s):
{"query_workload":{"language":"promql","query_batch":null,"repeating_queries":[{"query":"sum (sum_over_time(m_gauge[2m]))","demand":{"fixed_interval":60000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}},{"query":"sum (sum_over_time(m_gauge[2m]))","demand":{"fixed_interval":120000},"requirements":{"accuracy":{"max_error":0.0,"failure_probability":0.0,"accuracy_scope":"whole_result"},"max_latency_ms":null},"predictability":{"predictable":{"known_at":null}}}]},"data_workload":{"arrival":{"ingesting":{"rows_per_second":1333.3333333333333}},"sources":{"m_counter":{"series_count":10000,"metric_type":"counter","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}},"m_gauge":{"series_count":10000,"metric_type":"gauge","values_non_negative":true,"sample_interval":15000,"stream_norms":null,"max_distinct_per_evaluation":null,"distinct_values_per_label":{"instance":50,"job":5,"region":3}}}}}cargo run -p asap-devtools --bin stage_pipeline -- --workload base.json --enforce-accuracy-guarantees --out base.out.json
cargo run -p asap-devtools --bin stage_pipeline -- --workload related.json --enforce-accuracy-guarantees --out related.out.json
# same two commands with --exhaustive: identical selected plans and costs| Workload | Selected plan | Cost per second |
|---|---|---|
base.json |
P4-m1 |
0.004156 |
related.json |
P25-m2 |
0.008290 |
Witness: base.json plan with the second query reading the first query's result node |
witness-m2 |
0.004156 |
Expected: related.json costs about 0.004156 (the witness), not 0.008290.
Re-verified on current main (stage_pipeline built at 65655b6); the numbers above are from that run.
Current behavior
The longer interval doubles the cost instead of adding nothing. Same class in SQL: SUM(value) over 2 minutes every 60 s and every 120 s (seed 11 case 9): 0.001064 selected vs 0.000544 witness. Also on a seed workload (seed 13 case 108, PromQL quantile_over_time(0.75, data[6h]) at a longer interval): 1650.9 selected vs 1581.4 witness (--exhaustive refuses that workload for exceeding 10000 candidates, so only the default search was run there).
Expected behavior and invariant
For the workload with the added query, the selected cost should not exceed the base cost plus the cost of deriving the added query from the base plan, within a small tolerance. The cheaper plan (the witness) is priced by Stage 2 and Stage 3 with the same models as the search and passes Stage 3's checks, but the search never selects it.
Minimal reproduction
The two stage_pipeline commands above. planner_fuzz found it (seeds and relation names are under Status and related work). Running with --exhaustive selects the same plan and the same cost, so no candidate rule generates the cheaper plan; it is a missing candidate, not a search miss.
Why it happens
Not diagnosed. Queries with the same expression but different repeat intervals are planned independently; the shorter-interval query's result (or state) is not offered as an input to the longer-interval one, although its evaluation times are a subset.
Regression test
Plan base.json and related.json and assert cost(related) - cost(base) <= derive_cost + tolerance. Today it fails with the numbers above.
Status and related work
Related: #731 (shared window grids across intervals). Found by planner_fuzz seeds 11, 12 and 13, relation i-other-repeat-interval.
Activity
zzylol commentedon Oct 27, 2026
Reproduced on main
7680b6bcwith current candidate-connection MILP, enforced accuracy, and a 30 s budget. There are two distinct layers:(1/120)/(1/60+1/120)=1/3. The normalized product is about 1.94e-7, belowMINIMUM_NORMALIZED_LATENCY_TERM=1e-6insearch/milp/direct_cost.rs::DirectCostProfile::validate. The report records this UnsupportedDirectCost reason and zero direct-cost validated plans. Subsequent refinement returns a recomputing plan costing 0.0124000252 within the budget.The cheaper plan is legal: taking the single-query selected pane plan and attaching both query roots passes the same Stage 2/3 checks at 0.0041564750. Thus this is not an executor inability to serve the slower reader.
Controlled diagnostic: temporarily lowering only the numerical guard on the local diagnostic branch allowed the cost regression to pass (0.0041494500). The original guard has been restored. Lowering it is NOT proposed as a production fix: small lifetime coefficients can be lost within solver feasibility tolerances and invalidate cost/optimality claims.
A proper fix must preserve numerical soundness (better-scaled/restricted pricing or a reliable validated shared-plan seed/fallback), in addition to scalar CSE. All work remains uncommitted; this issue is not resolved.
zzylol commentedon Oct 27, 2026
Implemented in PR #943 without lowering either numerical guard. A bounded maintained-state seed avoids raw-scan latency ranges; if monetary coefficients still cannot be certified, a structural seed proposes a plan for full evaluator validation. Restricted objectives never become global lower bounds. Original mixed-repeat cost is 0.0041494500/s versus single-query 0.0041564500/s; the SQL mixed-repeat regression also passes. These are validated incumbents, not whole-catalog optimality certificates.
Final local validation: 690 focused tests passed, plus workspace format and all-target/all-feature Clippy checks.