Skip to content

Core: primitive.benchmark(), warmUp() and kernelLabels(); misc/node_device.mjs for Node scripts - #86

Merged
jowens merged 1 commit into
mainfrom
benchmark-node-device
Oct 27, 2026
Merged

jowens merged 1 commit into
mainfrom
benchmark-node-device

Conversation

@jowens

@jowens jowens commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

The last item on the "move it into core" list: benchmarking a primitive, and setting up a device in Node.

Why

  • Every timing script rewrote the same steps: warm up, time several executes, work out which per-kernel time belongs to which kernel. They were copied across regression_node.mjs, the MultisplitPerm and histogram benchmark scripts, the sort diagnostic and the paper's bench. The sort diagnostic left out the warm-up, and the V100's idle clock-down made it look 10× slow.
  • Every Node script also set up its own device, with its own set of pitfalls:
    • keep create()'s result referenced, or Dawn tears the device down;
    • Dawn's timestamp rounding;
    • requesting features and limits.

Core

  • primitive.kernelLabels(): the kernels' labels, in the order of getTimingResult()'s GPU times, nested primitives included.

  • primitive.warmUp(args, warmMs = 200): runs the primitive back to back, doubling trials until one execute keeps the GPU busy for 20 ms.

  • primitive.benchmark({ ...executeArgs, warmMs, repeats, trials }): warms up, times repeats executes, and returns medians per trial, per kernel with GPU timestamps (or CPU timing without them):

    const r = await primitive.benchmark({ inputBuffer, outputBuffer });
    // { ns, timing: "gpu", trials, repeats, kernels: [{ label, ns }, ...] }

Node (misc/, not part of the library: src/ also runs in browsers)

  • misc/node_device.mjs: nodeGPU() and createNodeDevice(). They give full-resolution timestamps, keep the instance referenced, request subgroups and timestamps when available along with the adapter's largest limits, and set WebGPU's globals.
  • misc/benchmark_node.mjs [primitives] [log2 sizes] [--kernels]: benchmark() over DLDFScan, HierarchicalScan, OneSweep, Histogram and MultisplitPerm. It replaces the one-off benchmark scripts.
  • Existing scripts: regression_node.mjs now uses primitive.warmUp(), whose executes aren't timed, and nodeGPU(). test_wgsl_functions_node.mjs and list_adapters.mjs use nodeGPU().

Checked (lavapipe)

  • Full regression suite with timing and large tests: everything passes except the existing sort failures at 8 lanes, and all 135 timings are recorded.
  • WGSL function tests: the same three subgroup failures as on main.
  • list_adapters and benchmark_node.mjs: both work. Every primitive is benchmarked, with per-kernel labels that include nested kernels (HierarchicalScan's partials scan, MultisplitPerm's DLDFScan).
  • Package test: passes.

Independent of #84 and #85.


Generated by Claude Code

…evice.mjs for Node scripts

Timing a primitive well (warm up so the GPU is at full clock, time several
executes, label the per-kernel times) was rewritten in every script, and
the one that left out the warm-up made a V100 look 10x slower. Creating a
device in Node had its own pitfalls (keep create()'s result referenced,
timestamp rounding, features, limits), copied in each script.

- BasePrimitive.kernelLabels(): the kernels' labels, in the order of
  getTimingResult()'s GPU times (nested primitives included).
- BasePrimitive.warmUp(args, warmMs = 200): back to back, doubling trials
  until one execute keeps the GPU busy for 20 ms.
- BasePrimitive.benchmark({ ...executeArgs, warmMs, repeats, trials }):
  warm up, time `repeats` executes, return medians per trial, per kernel
  with GPU timestamps (else CPU timing).
- misc/node_device.mjs: nodeGPU() and createNodeDevice() (full-resolution
  timestamps, instance kept referenced, subgroups and timestamps when
  available, the adapter's largest limits, WebGPU globals).
- misc/benchmark_node.mjs: benchmark() over DLDFScan, HierarchicalScan,
  OneSweep, Histogram and MultisplitPerm, with --kernels for per-kernel times.
- regression_node.mjs uses primitive.warmUp() (its executes are untimed)
  and nodeGPU(); test_wgsl_functions_node.mjs and list_adapters.mjs use
  nodeGPU(). Types and docs/timing-strategy.md ("Benchmarking a primitive").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kfNC6id65zEHzuAAH4jtJ
@jowens
jowens merged commit f39035b into main Oct 27, 2026
4 checks passed
@jowens
jowens deleted the benchmark-node-device branch October 27, 2026 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants