Skip to content

Plan selection: warn when the measured cost table is missing or has no Scan, TimeRange or Aggregate entry #843

Open

Description

@zzylol

Example

Run #838's workload (one quantile_over_time(0.5, data[1m]) every 10 s over 10,000 series) without a measured cost table:

cargo run -p asap-devtools --bin stage_pipeline -- --workload wl.json --out out.json

Today the run succeeds silently. Every node is priced from the formulas at ResourceSpeed::illustrative() (1 ns per CPU operation, 0.1 ns per scanned byte), so a raw-sample scan costs about 4.2 ns per sample. A runtime measured about 890 ns per sample for a remote read (#761). The only hint is "source": "analytical" on each candidate, which is easy to miss.

The same thing happens with a table that has sketch entries but no Scan, TimeRange or Aggregate entry: those operators stay on the formula prices, scaled by one calibration factor fitted on the sketch kernels.

What should happen: stage_pipeline and select_plan warn, for example:

warning: no measured operation cost table; CPU is priced from illustrative formulas (1 ns per operation)
warning: measured operation cost table has no entry for Scan, TimeRange, Aggregate; these are priced from the formulas

Why it happens

stage_pipeline falls back to MeasuredOperationCosts::default() when --measured-operation-costs is not given (crates/devtools/src/bin/stage_pipeline.rs). select_plan (crates/plan-selection/src/lib.rs) accepts an empty or partial table without comment. The per-node price_source records which nodes were priced from the formulas, but nothing summarizes it.

Proposal

  • select_plan returns the list of operator kinds the candidates use that the table has no entry for (empty table: all of them), and whether the table is empty.
  • stage_pipeline prints one warning line to stderr for an empty table, and one for missing Scan, TimeRange or Aggregate entries. It also writes the same list into stage3_plan_selection in the output JSON.
  • No change to prices or to the selected plan.

Acceptance

  • A test runs select_plan with an empty table and with a sketch-only table and checks the reported missing kinds.
  • A stage_pipeline run without --measured-operation-costs prints the warning.

Related

#761 (Stage 3 under-predicts CPU; closed as covered by #789), #791 (per-operator-kind calibration for partial tables), #835 (executor-side measured table).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    cost-modelCost formulas, statistics, ranking, and selectiondev-toolsDeveloper tools, diagnostics, and demosenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions