Skip to content

KMV insert and merge are O(k) and O(k²) in asap_sketchlib; unusable at the k the planner sizes #709

Open

Description

@zzylol

Problem

asap_sketchlib's KMV insert_by_hash does a linear scan for duplicates on every insert, and merge is O(k²). At the k the planner sizes (kmv_k_99: 40,002 at ε = 0.05, about 1e6 at ε = 0.01), building or merging a KMV is impractically slow. Found while reviewing the KMV kernel (#704).

To do (upstream, not a local copy)

  • In asap_sketchlib, check against the heap root first (a hash ≥ the current k-th smallest can be dropped immediately), and use a hash set for duplicate detection. Merge then becomes O(k log k).
  • Move the pin once it's merged or on a pin branch.

Related: #697, #704, #694.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bindingLogical-to-physical binding and implementation selectionbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions