Skip to content

memo-milp misses rh8-q1h60 at 280 s (1.34e-05 vs 3.83e-06): suspected decode cut of a valid stricter site threshold #928

Open

Description

@zzylol

Problem, by example

rh8-q1h60: RangeHeavy 8 users, the quantile queries over 1h at 60 s steps. Five queries share one SummaryMerge; query 4 is a ratio of two quantiles (q0.9 / q0.5) over that shared state. Fargate prices, raw data retained, guarantees enforced:

stage_pipeline --workload rh8-q1h60.json \
  --deployment-capabilities '{"raw_data_retained":true,"query_time_results_retained":false}' \
  --cost-model aws-fargate-us-east-1 --enforce-accuracy-guarantees \
  --search memo-milp --milp-time-limit-s 280
cost per s rounds gap
legacy reference 3.832852e-06 – –
memo-milp on main (1800 s limit) 3.832840e-06 97 0 (772 s)
memo-milp on the integration branch 04151ce (280 s limit) 8.53e-06 (other runs 1.34e-05, the warm start) 110–153 0.71, time limit
memo-milp after the PR-B merge, 0f8bf09 (280 s limit) 1.342e-05 162 0.814, time limit

Expected: the 3.83e-06 plan, certified, well within 280 s.

Suspected cause, and what the diagnosis found

Suspicion: solution decoding evaluates one site threshold per choice set, and when Stage 3 rejects it, the no-good cut removes the whole choice set, including the same choices at a stricter (larger) threshold that would be valid. That would also allow a wrong certificate: a valid plan cut away, then a bound that no longer covers it.

Diagnosis so far (restricted-model runs on 04151ce):

  • The optimum's choice set is feasible in the model at 3.832831e-06 <= Stage 3 3.832840e-06.
  • On a neighbouring set, round 0 picks site threshold 0 at 2.3959e-06 (Stage 3 rejects: AccuracyTargetMissed on query 4, relative bound 0.105); round 1 picks threshold 1 of the same set at 3.832831e-06, which Stage 3 accepts. So thresholds are separate versions and the no-good cut is per version there: that case does not lose the valid plan.
  • The time is spent elsewhere: many choice sets at objective 2.3959e-06 to 2.4904e-06 (different pane candidates under the ratio) each fail the ratio's accuracy target at the smaller threshold and are cut one per round, ~1.8 s each. The pair accuracy rule (pair_guarantee_rules, model.rs) only fires when every threshold version fails, so it rules none of them out.

The decode-threshold mechanism is not confirmed. This issue stays open until a test shows either that it cannot happen or that it does and is fixed.

Proposal

  1. A failing-first test: a choice set rejected at one site threshold and valid at a stricter one; assert the loop still returns (and certifies) the stricter plan.
  2. The pair accuracy rule checks each child choice pair at its own site thresholds plus the largest size, ruling out a version only if it fails at its own and the largest size, tied to the site threshold variable u, not to the choice set.

Scope and acceptance

  • rh8-q1h60 certifies at <= 3.832852e-06 within 280 s.
  • The fixed regression set keeps its costs (q24h60, w840*, i840, j839 raw/no raw); lower_bound tests and the 20-seed three-way pass.
  • The new test from step 1.

Status and related work

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingoptimizerWorkload optimization and plan selectionsearchCandidate-plan search space and algorithms

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions