Skip to content

test(benches): time guards, imports and render - #553

Merged
LKSNDRTMLKV merged 5 commits into
mainfrom
test/criterion-benches
Oct 30, 2026
Merged

LKSNDRTMLKV merged 5 commits into
mainfrom
test/criterion-benches

Conversation

@LKSNDRTMLKV

Copy link
Copy Markdown
Member

Stacked on #550 (the wasmtime advisory fix), so the audit job here is green. The base is fix/wasmtime-49-0-5; once #550 merges, retarget this to main (gh pr edit --base main) before that branch is deleted.

Adds Criterion benches for engine paths that had none, a weekly workflow, and two just recipes. Three commits: the benches that need no infrastructure, the ones that need Docker, and the review fixes.

Benches that need no infrastructure (just bench)

Crate Target Measures
dpp-common request_guards the outbound webhook URL guard (accepted and refused URLs, the private-range switch, validate_public_https_url, ip_is_disallowed) and the idempotency policy_for lookup
dpp-integrator import_parse CSV and XLSX parsing at several row counts, validate_row, and the CPU side of a 1000-row import
dpp-render render_page the public passport page for a textile, a battery and a 50-field unmodelled passport, with and without the QR code, the continuity snapshot, and build_qr_svg

Benches that need Docker (just bench-infra)

Crate Target Measures
dpp-dal pg_repo the Postgres repositories against a real server: list, count and find_by_identity at 100, 1,000 and 10,000 passports; create, update, the primary-key reads; audit append; API-key lookup
dpp-vault publish_serve create, publish, public read and authenticated read over HTTP, and the list at 100 and 1,000 passports, with the router, auth, service and Postgres real. floor_info (no database) is the loopback cost to subtract
dpp-node nats_throughput already there; now part of the recipe
  • Both new benches use dpp_dal::test_harness::start_pg, the one Postgres harness, so just harness-check stays green. just bench-infra runs them through scripts/shared-test-pg.sh, as just test-integration does. All three are required-features = ["integration-tests"].
  • publish_serve includes the vault's tests/helpers/mod.rs with #[path] rather than copying its wiring. Its signer is the harness's MockIdentity, so publish leaves out Ed25519 (that is dpp-core's crypto bench).
  • Sweeps run before the write benches, which insert thousands of rows while they are measured.

What the first run showed

One machine, reduced sampling: orders of magnitude, not baselines.

100 1,000 10,000
find_by_identity, hit / miss 1.3 / 1.1 ms 9 / 8.7 ms 84 / 83 ms
list, first page of 20 2.3 ms 2.5 ms 4.7 ms
list, search matching nothing 3.1 ms 26 ms 246 ms
count 0.26 ms 0.39 ms 0.95 ms

Point operations are 0.15 to 3 ms; over HTTP the floor is 79 µs, create 4 ms, publish 10.5 ms (mock signer), reads 0.53 ms; NATS publish 90 µs.

find_by_identity grows linearly with the table, hit or miss. Every passport in this fixture is one product (one GTIN) differing only in batch, the worst shape for it. idx_passport_identity is keyed by product group, identifier, then doc->>'batchId', and the query compares the batch with IS NOT DISTINCT FROM, which a btree does not search. That is my reading, not confirmed with EXPLAIN. The import delta-matcher is built on this lookup (ops/pg/0019_passport_identity_index.sql). Nothing here changes it.

New dependency: criterion

It is already a workspace dependency (since July, used by dpp-node and dpp-plugin-host); this PR adds it as a dev-dependency to dpp-common, dpp-integrator, dpp-render, dpp-dal and dpp-vault. No new package enters Cargo.lock, only five dependency edges.

  • Maintenance: 0.8.2 (released 2026-02-04), Apache-2.0 OR MIT, upstream criterion-rs/criterion.rs is not archived and was last pushed on 2026-10-13.
  • Shipped graph: it is in no normal dependency graph: cargo tree -p dpp-node -e normal -i criterion prints nothing.
  • Untrusted input: none. The import bench builds its own CSV and XLSX in memory, the URL guard runs on literals (its DNS-resolving half is not benchmarked), the page bench renders fixed JSON.
  • Secrets: none. The Postgres container uses the harness's throwaway test password, the API keys are rows with a placeholder hash, the signer is a mock, no key material is read.
  • Network: loopback to containers the benches start themselves (postgres:17, nats:2); nothing is called out to.

Left out, on purpose

  • The real-plugin benches in dpp-plugin-host (battery_*) build the battery plugin from a sibling dpp-core working tree, so on a runner they skip with a warning and only the WAT floor runs. Checking dpp-core out beside this repository would bench its default branch, not the release this workspace pins, and the plugin is not a published crate. Nothing here reads from a dpp-core working tree; the lockfile resolves every dpp-* core crate from crates.io at 0.22.0.
  • A resolver and Redis bench: Cache is a 68-line adapter and the resolver has no container harness.

The workflow

bench.yml runs just bench then just bench-infra, weekly and on demand, never on a push or a pull request. A measurement, not a gate; 60-minute ceiling; actions pinned to the SHAs ci.yml and fuzz.yml use.

  • The baseline is the criterion-reports artifact, not a cache. The job downloads the last successful run's artifact (actions: read on the job). A cache entry unused for seven days is evicted and a weekly run that starts late can land past that; an artifact is kept 30 days. This repository has no scheduled run yet; the same design is on dpp-core's workflow, where it replaced a cache after a reviewer pointed out the gap.
  • The summary tells outcomes apart: a bench step that did not finish, no earlier results to compare with, and a complete comparison with nothing slower (the only plain "none").
  • Six lines of context before the verdict reach a benchmark's name: a name over 23 characters is printed on its own line and a throughput adds thrpt:, change:, time:, thrpt:.

Review findings addressed (third commit)

  • The snapshot-banner assertion matched snapshot-note, which the page's stylesheet defines on every render; it now checks class="snapshot-note" and that a live render lacks it.
  • The remainder-table assertion checks all 50 cells (<td>value N</td>), not only the last.
  • allow_private is asserted to accept before it is timed; the XLSX and CSV benches check a cell of the first row, not only the row count; find_by_id and find_published_by_id assert the passport is there; the pg_repo sweep holds a published passport and asserts the draft filter leaves it out.
  • A doc comment on each bench function that lacked one.

Checked

  • just check exits 0 on the branch tip: clippy with -D warnings over every target including the Docker benches, 1636 of 1636 tests, harness-check, the audit register, cargo deny.
  • cargo test --benches runs every new bench once in Criterion's test mode, the Docker ones against a real Postgres.
  • just bench-infra's command, with reduced sampling added, exits 0 and runs all three Docker benches in release mode.
  • The report step's script, run against synthetic logs shaped like Criterion's output, for six combinations of step outcomes.

Not run: the workflow itself (it needs a runner; Run workflow on this branch exercises it), gh run download, and a full-length cargo bench for baselines.

@LKSNDRTMLKV LKSNDRTMLKV changed the title Test/criterion benches test(benches): time guards, imports and render Oct 30, 2026
@LKSNDRTMLKV
LKSNDRTMLKV merged commit 7a8f647 into main Oct 30, 2026
26 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant