Repository navigation
SQL top-k ordered by an alias (ORDER BY c DESC LIMIT k) does not bind #619
Description
Problem
SELECT src_ip, COUNT(*) AS c FROM flows GROUP BY src_ip ORDER BY c DESC LIMIT 2 (ordering by the alias) lowers to a Top-K aggregate over the per-src_ip count, unlike ORDER BY COUNT(*) DESC LIMIT 2, which stays a sort and limit. Two things go wrong for it on the SQL path:
- The Pass 1 DAG does not bind. The frontend's
TopKaggregate has nooutput_names. Compiling the lowered (or the all-pass-through composed) DAG withcompile_physical_asap_dagand binding it fails:invalid DAG: node 2: invalid DAG: aggregate output names differ from measures. The same happens for every Pass 1 choice that composes ([0, 0],[0, 1],[0, 2]). - Top-K sketch alternatives never compose. Pass 1 offers
CmsWithHeapandCountSketchWithHeapfor the Top-K target, but composing them fails:unsupported local realization: Top-K item identity for closed schemas. That is 6 of the 9 combinations, so Stage 1 rejects them.
plan_stages prices P1–P3, the three pass-through Top-K plans (inner count pass-through, exact Count or HydraCms). Stage 2 turns the pass-through Top-K into Sort and Limit, so those physical plans do bind and return a 3, b 2. Only the Pass 1 DAG that bypasses Stage 2 fails, and the sketch alternatives are dead weight.
Repro
Run against flows(ts Timestamp, src_ip Utf8), ε = 0.1, δ = 0.01, on stack/509-viewer-stages + the SQL COUNT(*) sketch fix. Use the catalog()/binds helpers in crates/integration-tests/tests/pass1_sql_coverage.rs:
let sql = "SELECT src_ip, COUNT(*) AS c FROM flows GROUP BY src_ip ORDER BY c DESC LIMIT 2";
let root = lower_sql(sql, &catalog(), target).await.unwrap();
binds(&physical_common::compile_physical_asap_dag(&root).unwrap());
// Err("invalid DAG: node 2: invalid DAG: aggregate output names differ from measures")
let inventory = enumerate_local_logical_candidates(vec![(0, QueryRoot::Operator(root))], &BTreeMap::new()).unwrap();
// targets: [PassThrough, CmsWithHeap, CountSketchWithHeap] (Top-K), [PassThrough, Count, HydraCms] (count)
for choice in enumerate_choices(&inventory, usize::MAX) { compose_logical_candidate(&inventory, &choice) /* ... */ }
// [0, *]: composes; bind: aggregate output names differ from measures
// [1, *], [2, *]: unsupported local realization: Top-K item identity for closed schemasOutput from plan_stages (with executor_models()): costs P1, P2, P3, selected P2. P4–P9 are rejected with Stage 1: unsupported local realization: Top-K item identity for closed schemas.
Expected
- The Top-K aggregate the SQL frontend lowers carries output names, so its DAG binds the way the
ORDER BY COUNT(*)form's does. - Pass 1 either offers no heap-sketch Top-K alternatives for SQL rows (there is no item identity to rank), or ranks the grouping column so they compose.
Found while fixing SQL COUNT(*) per-group sketches (PR to follow).
Activity