Repository navigation
A partial measured cost table can still mis-rank plans: calibrate per operator kind #791
Description
Problem, by example
Take a per-series p99 over 5 min, quantile_over_time(0.99, m[5m]), queried every minute over 10,000 rows/s and priced on CPU only. There are two candidates:
- EXACT: an exact rebuild from raw samples at query time (Scan, TimeRange, Aggregate(quantile)).
- B1: KLL panes maintained at ingestion time.
The deployment's measured cost table covers the KLL (PaneChain 150 ns/row, SummaryMerge 2,000 ns/state, SummaryEstimate 5,000 ns/group), TimeRange (2 ns/row) and Project (1 ns/row). It does not cover Scan or the quantile Aggregate.
| Real cost of the unmeasured operators | Stage 3 selects (partial table) | Full table selects |
|---|---|---|
| Scan 40 ns/sample, quantile 20 ns/row | B1 | EXACT; B1 costs 1.42x more |
| Scan 100 ns/sample, quantile 50 ns/row | B1 | B1 |
These numbers come from the test a_partial_table_can_still_mis_rank_plans in crates/plan-selection/src/lib.rs (PR #789). The gap widens as the unmeasured operators get cheaper; review measured up to 2.6x.
Expected: a partial table should rank plans the way the full table does whenever the unmeasured operators' real costs follow the measured operators of the same kind.
Why it happens
#771 / PR #789 prices a node with no entry as its formula CPU times one calibration factor (CalibrationFit::factor). The factor is Σ measured ÷ Σ formula time over every measured node of every candidate. Here it is dominated by the KLL's 150 ns against the formula's 2 operations per row, so the unmeasured Scan and quantile Aggregate are priced at the sketch's ratio (about 100x) rather than at anything an exact operator was measured at. The factor is a heuristic, documented as such in the crate's "Measured operation costs" docs. Full tables are what make the choice right today.
Proposal
Fit one factor per operator class instead of one global factor, falling back up the hierarchy:
- the same operator kind at the same phase;
- the same operator kind at either phase;
- the same class: sketch summary operators, exact accumulators, relational operators (Scan, Filter, Aggregate, Sort, ...);
- the global factor.
An alternative is to refuse, or warn on, a table that misses an operator kind the candidates use, and leave coverage to the runtime's measurement tool. The two can be combined.
Scope and acceptance
- In scope: the calibration in
price_at_scale/select_plan, and reporting which factor priced each node inNodeCost::detail. - Out of scope: the runtime's
measure_costscoverage. - Acceptance:
a_partial_table_can_still_mis_rank_plansflips to EXACT once the table measures one exact relational operator near the real scan and aggregate cost (e.g. TimeRange or Filter at 30 ns), andcalibration_keeps_unmeasured_nodes_on_the_measured_scalestill selects B1.
Status and related
This is a follow-up of #771 and PR #789, which documents the limitation. Related: #761 (Stage 3 CPU under-prediction).
Activity