Skip to content

SQL top-k ordered by an alias (ORDER BY c DESC LIMIT k) does not bind #619

Open

Description

@zzylol

Problem

SELECT src_ip, COUNT(*) AS c FROM flows GROUP BY src_ip ORDER BY c DESC LIMIT 2 (ordering by the alias) lowers to a Top-K aggregate over the per-src_ip count, unlike ORDER BY COUNT(*) DESC LIMIT 2, which stays a sort and limit. Two things go wrong for it on the SQL path:

  1. The Pass 1 DAG does not bind. The frontend's TopK aggregate has no output_names. Compiling the lowered (or the all-pass-through composed) DAG with compile_physical_asap_dag and binding it fails: invalid DAG: node 2: invalid DAG: aggregate output names differ from measures. The same happens for every Pass 1 choice that composes ([0, 0], [0, 1], [0, 2]).
  2. Top-K sketch alternatives never compose. Pass 1 offers CmsWithHeap and CountSketchWithHeap for the Top-K target, but composing them fails: unsupported local realization: Top-K item identity for closed schemas. That is 6 of the 9 combinations, so Stage 1 rejects them.

plan_stages prices P1–P3, the three pass-through Top-K plans (inner count pass-through, exact Count or HydraCms). Stage 2 turns the pass-through Top-K into Sort and Limit, so those physical plans do bind and return a 3, b 2. Only the Pass 1 DAG that bypasses Stage 2 fails, and the sketch alternatives are dead weight.

Repro

Run against flows(ts Timestamp, src_ip Utf8), ε = 0.1, δ = 0.01, on stack/509-viewer-stages + the SQL COUNT(*) sketch fix. Use the catalog()/binds helpers in crates/integration-tests/tests/pass1_sql_coverage.rs:

let sql = "SELECT src_ip, COUNT(*) AS c FROM flows GROUP BY src_ip ORDER BY c DESC LIMIT 2";
let root = lower_sql(sql, &catalog(), target).await.unwrap();
binds(&physical_common::compile_physical_asap_dag(&root).unwrap());
// Err("invalid DAG: node 2: invalid DAG: aggregate output names differ from measures")
let inventory = enumerate_local_logical_candidates(vec![(0, QueryRoot::Operator(root))], &BTreeMap::new()).unwrap();
// targets: [PassThrough, CmsWithHeap, CountSketchWithHeap] (Top-K), [PassThrough, Count, HydraCms] (count)
for choice in enumerate_choices(&inventory, usize::MAX) { compose_logical_candidate(&inventory, &choice) /* ... */ }
// [0, *]: composes; bind: aggregate output names differ from measures
// [1, *], [2, *]: unsupported local realization: Top-K item identity for closed schemas

Output from plan_stages (with executor_models()): costs P1, P2, P3, selected P2. P4–P9 are rejected with Stage 1: unsupported local realization: Top-K item identity for closed schemas.

Expected

  • The Top-K aggregate the SQL frontend lowers carries output names, so its DAG binds the way the ORDER BY COUNT(*) form's does.
  • Pass 1 either offers no heap-sketch Top-K alternatives for SQL rows (there is no item identity to rank), or ranks the grouping column so they compose.

Found while fixing SQL COUNT(*) per-group sketches (PR to follow).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions