Scaling families: planning time, rounds and gap as one sharing structure grows #918

Problem, by example

We measure planning time only on fixed workloads, so we cannot tell how it grows. The one curve we have is accidental: W-RangeHeavy quantile queries over one window under the legacy search (2026-10-21):

There is no such curve for --search memo-milp, so we cannot show that validity constraints, stronger cuts, clusters and decoding change the growth rather than one data point (#879).

Why it isn't done today

The eval runs the paper workloads (W-RangeHeavy, W-RollupHeavy, W-Mixed, 8 users) whole. They mix every structure at once, so a slow run does not say which structure is responsible.

Proposal

Parameterized workload families, one structure each, scaled by one parameter n, after the pathology families in "EXPLAIN Yourself! Finding Query Planner Stalls Across DBMSes" (arXiv:2608.23402):

1. Same-window queries: n queries over one metric and one window (quantile_over_time(q_i, m[1h]), sum_over_time, …).
2. Nested windows: one query per window w_1 < … < w_n over one metric.
3. Readers per unit: n queries that can share one unit with different accuracy targets and repeat intervals (strictest-reader size, union rate, max retention).
4. Groups per query: one query reading n groups of one metric, e.g. max_over_time(m[5m]) - min_over_time(m[5m]) + …, plus queries sharing each group.
5. Repeat intervals: n distinct repeat intervals over one window.
6. Groupings: n groupings of one metric that roll up (the W-RollupHeavy CUBE shape).

For each family and n = 1, 2, …, until the time limit: record wall time, peak RSS, MILP rounds, variables, constraints, gap, and whether it was certified. Plot time and rounds against n for each search mode.

Scope and acceptance

• In scope: the generator for the six families (a devtools binary or planner_fuzz subcommand), a run script, and a results table checked into the eval directory.
• Acceptance: the table for --search legacy reproduces the RangeHeavy curve above for family 1; the same run under --search memo-milp gives a curve per family; both are recorded before and after the memo-MILP speed-ups merge.

Status and related work

• Decided 2026-10-23: open as an issue, implement after the current memo-MILP work.
• Related: Tracking: plan every whole evaluation workload within 3 minutes (exact pruning, memo-group search) #879, Memo search solver: MILP over the memo for all cost models #888, planner_fuzz: search for workloads that plan slowly or leave the MILP gap open #917 (planning-time fuzz mode).