Repository navigation
asap-planner facade review: orchestration and deployment input assembly #1017
Description
Part of #1010. Line numbers are on main at 958778400.
Example: one call through every stage
Same query as the tracking issue (avg by (job) (quantile_over_time(0.5, data[5m]) / quantile_over_time(0.5, data[5m]) + quantile_over_time(0.9, data[5m])), every minute, max_error 0.01, failure_probability 0.001). A runtime plans it with one call:
let (plan, report) = asap_planner::plan_installable(UserInput { workload, frontend_specific, models, pass }).await?;What happens inside, on main:
| Step | Crate | Code |
|---|---|---|
| Default supported summaries to the executor's list; add the executor compile check | asap-planner |
plan_installable (crates/planner/src/installable.rs:240), executor_summaries_when_unrestricted |
| Stage 0: lower each query | asap-planner → frontends |
e2e_plan (crates/planner/src/lib.rs:277) → lower → lower_promql_query_workload / lower_sql_dialect / lower_metricsql_query_workload |
| Run the optimization pass | asap-plan-selection |
optimize(pass, OptimizationInput) (crates/plan-selection/src/search/mod.rs:410), default pass Stage1ToStage3Pass |
| Stage 1 Pass 1 | asap-plan-selection → asap-logical-asap-optimization |
run_stage1_to_stage3_with (search/pipeline.rs:163) → generate_per_computation_candidates_with_memo, LogicalMemo::from_roots |
| Pass 1 memo, Pass 2, state groups, Stage 2, Stage 3 | asap-plan-selection |
candidate_connection_search → search_domain (search/route.rs) |
| Installable plan | asap-planner |
installable_plan_of (installable.rs:269) |
So the orchestration and the public API (OptimizationPass, optimize, PlanningModels, Stage1ToStage3Pass) live in the Stage 3 crate, and asap-planner re-exports them (crates/planner/src/lib.rs:41).
Functionality
- The only entry point: frontends, then Stages 1 → 2 → 3, then the installable plan.
- Assemble the deployment's inputs (cost model, accuracy model and evidence, capabilities) and give each stage the part it reads.
Concepts and interfaces
| Concept | Code on main |
|---|---|
| user input | UserInput { workload, frontend_specific, models, pass } (crates/planner/src/lib.rs:125), FrontendInput (:108) |
| deployment inputs (doc: "TODO: define this data structure", #525) | PlanningModels { accuracy, evidence, plan_selection, deployment_capabilities, executor_compile_check } (crates/plan-selection/src/search/mod.rs:68) |
| deployment capabilities | DeploymentCapabilities { supported_summaries, raw_data_retained, query_time_results_retained } (crates/logical-asap-optimization/src/deployment_capabilities.rs:48); fields read by Stage 1, Stage 3 and Stage 2 respectively |
| pluggable optimization pass | trait OptimizationPass (search/mod.rs:397), PassRegistry; one implementation, Stage1ToStage3Pass |
| outputs | PlanOutput (e2e_plan), (InstallablePlan, Option<SelectedPlanReport>) (plan_installable) |
Interface decided in #1047 (2026-10-29)
Decided in #1047, the stage-boundary types. Its sub-issue is #1048. Names follow #1050 within #1047's scope. Principles: a stage or pass reads only the previous step's output type and calls no other stage; the asap-planner facade runs the stages in order; each stage prunes only what its own knowledge rules out and records it as PrunedCandidate<C> { group, candidate, reason: PruneReason }; queries are entry indices. Planning stays as fast as on main: P6 and P7 report before/after stage_pipeline runs.
| Decided | |
|---|---|
| Functionality | Validate; Stage 0 → Pass 1; per pane-width round, Pass 2 → Stage 2 → Stage 3; keep the best validated selection with that round's Pass 2 and Stage 2 outputs; build_selected_physical_plan; installable_plan_of. Owns the total time budget and the caller's query Ids |
| Input | UserInput { workload, frontend_specific, models: PlanningModels, pass }, unchanged |
| Deployment inputs by stage | Pass 1: supported summaries, accuracy model and evidence. Stage 2: query_time_results_retained. Stage 3: PlanSelectionConstraints and PlanSelectionCostInputs |
| Output | PlanOutput (unchanged signature); PlanningRun { per_computation_candidates, logical_asap_candidates, physical_asap_candidates, selection, selected_physical_plan, evaluated_plans } (was Stage1ToStage3Run<Id>), with the selected round's outputs |
| PR | #1047 P7 |
Review checklist
1. Boundaries
- Orchestration (
pipeline.rs), the public API (OptimizationPass,optimize) andPlanningModelslive inasap-plan-selection;asap-plannerre-exports them. Reorganize by stage: one memo-based Pass 2, Stage 1 finishes the memo, deployment inputs split by stage, asap-planner as the only entry point #958 decision 4 moves them toasap-plannerso plan-selection is only Stage 3. [Refactor] One named type per stage boundary: PerComputationLogicalMemoCandidates → LogicalASAPMemoCandidates → PhysicalASAPMemoCandidates #1047 P7 / Reorganize by stage: one memo-based Pass 2, Stage 1 finishes the memo, deployment inputs split by stage, asap-planner as the only entry point #958: orchestration moves to the facade. -
DeploymentCapabilitiesis defined in the Stage 1 crate and read by three stages;asap-physical-asap-optimizationdepends on the Stage 1 crate only for it (Reorganize by stage: one memo-based Pass 2, Stage 1 finishes the memo, deployment inputs split by stage, asap-planner as the only entry point #958 decision 3, Deployment inputs: one bundle for cost model, accuracy model and capabilities — after #511 #525). [Refactor] One named type per stage boundary: PerComputationLogicalMemoCandidates → LogicalASAPMemoCandidates → PhysicalASAPMemoCandidates #1047: split by stage (table above). - Supported summaries default to the executor's list in two places (
installable.rs, devtoolsstage_pipeline.rs).
2. Naming and over-abstraction
-
OptimizationPass+PassRegistrywith one pass ([Design] Discussing the API Expose for the Pluggable Optimization Pass #465, [Design] Support a Separate Optimization Pass Besides the Current One For Grey-Box Users #430, [Design] Give lib User One Single lib-func Abstraction #429, [Design] Freezing Input and Output with Rust trait #441, [Design] Materializing the Logical Input / Output and Workflow Picture Onto lib Functions Visible to Users #453 discuss the pluggable-pass API). - "pass" means Stage 1 Pass 1/2 and also the whole optimizer (
OptimizationPass,Stage1ToStage3Pass) ([Refactor] Repo-wide naming survey: overloaded and misleading names #948). -
pub use search as pass;(crates/plan-selection/src/lib.rs:297) gives the API two paths.
3. Stale code
-
e2e_planvsplan_installable: decide whether both stay public. -
UserInput::validaterejects every MetricsQL input (e2e_plan cannot plan MetricsQL: UserInput::validate rejects every FrontendInput::Metricsql #534).
4. "memo" / "cache"
- none at this level.
5. Rules over memo groups
- The facade should not know about memo groups; check that devtools (
stage_pipeline) reach every stage's input and output through the facade or the stage crates, not through Stage 3 internals. [Refactor] One named type per stage boundary: PerComputationLogicalMemoCandidates → LogicalASAPMemoCandidates → PhysicalASAPMemoCandidates #1047:PlanningRuncarries each stage's output; the facade passes them on without reading groups.
Correctness
- The output stays aligned with
QueryWorkload::entries()(one root per entry, one query text per root). - Planner snapshot tests in CI (Add planner snapshot tests to CI #941).
Input/output check with the DAG viewer
stage_pipeline(crates/devtools/src/bin/stage_pipeline.rs) drives the stages for the viewer (--promql,--sql,--workload,--deployment-capabilities,--cost-model,--exhaustive,--milp-time-limit-s). stage_pipeline: write the memo after each step, the selected choice and the enumeration #1003 adds the memo after each step to its output.- Check: the viewer's result for a workload equals what
plan_installablereturns for it.
Related issues
Sub-issues: #525, #534, #941, #465, #453, #441, #430, #429.
Referenced: #958 (layout, under #1013), #949 (compile check, under #1015), #685 (end-to-end use cases), #671 (planning roadmap).
🤖 Generated with Claude Code
Activity