Skip to content

feat: Add a performance benchmark suite with CI regression checks - #56

Merged
rent-a-developer merged 2 commits into
mainfrom
feature/issue-53-add-benchmarks
Oct 26, 2026
Merged

rent-a-developer merged 2 commits into
mainfrom
feature/issue-53-add-benchmarks

Conversation

@rent-a-developer

@rent-a-developer rent-a-developer commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Closes #53

What does this change?

Adds a performance benchmark suite and a CI gate on allocation regressions. ResXLocalization had no
performance numbers until now. Nothing caught a change that made lookups slower or allocate more, and
consumers had no way to see what the library costs their app.

Refs #53. The spec is the OpenSpec change openspec/changes/add-benchmarks/ (proposal, design, delta spec, tasks).

Benchmarks (benchmarks/, BenchmarkDotNet, .NET 10, JIT only). Each one is compared with the code an
application would write without the library: the .resx designer class and hand-written code over it.

  • ResXLocalization.Core.Benchmarks: Localizer.Current.Get with typed, scoped, search-all and enum keys;
    search-all across 1, 5 or 20 resource managers; MessageArguments (FromValues, FromTuples,
    FromObject) against string.Format; and ICU MessageFormat against calling the library directly.
  • ResXLocalization.Avalonia.Benchmarks (headless) and ResXLocalization.WPF.Benchmarks (Windows only):
    • The cost per element of {l:Localize}, {l:Localize} with <l:Argument> children and
      {l:LocalizeEnum}, measured on 100 elements, against {x:Static}.
    • Opening 100 or 10,000 elements, against {x:Static}.
    • Switching the culture with 100 or 10,000 live bindings, against a hand-written TranslationSource
      indexer.
    • Every view is compiled XAML shown in a real window.
  • Shared/ is compiled into all three suites. Each class first checks that every benchmark produces the
    expected text, then measures.

CI (.github/workflows/ci.yml)

  • The three suites run as a matrix in parallel with the build jobs, on PRs and on pushes to main that
    change code. A benchmark-changes job skips them for every other change.
  • benchmark-report merges the results (scripts/convert-benchmark-results.ps1) and compares them with
    main using benchmark-action/github-action-benchmark, pinned by SHA:
    • Allocating more than 125% of main's bytes, or allocating anything where main allocated nothing,
      fails the check.
    • A time above 150% of main is an alert only, shown in the job summary and as a PR comment.
    • The label accept-allocation-increase turns the allocation failure into an alert.
  • On main, results are pushed to a new benchmark-data branch: the history of the last 200 commits, plus
    the latest reports.
  • deploy-docs copies that branch into the Pages site under benchmarks/latest/ and benchmarks/history/.
  • The benchmark job gets contents: write and pull-requests: write.

Docs

  • New page docs/guides/benchmarks.md, with an entry in docs/toc.yml.
  • A "Running the benchmarks" section in CONTRIBUTING.md.
  • A row for benchmarks/ in the AGENTS.md layout table.
  • A CHANGELOG.md entry under [Unreleased].

No shipped package, public API or runtime behaviour changes. BenchmarkDotNet is referenced only by the
benchmark projects.

Checklist

Always

  • pwsh -File scripts/pre-commit-gate.ps1 passes.
  • The Release build produces zero warnings.
  • Branch name follows <type>/issue-<number>-<slug>: feature/issue-53-add-benchmarks.

If the change adds behaviour or fixes a bug

Does not apply: no runtime behaviour changes. The benchmarks check themselves before they measure.
CHANGELOG.md has an entry anyway, because the results page is user-facing.

If the change touches one UI package

Does not apply: src/ is untouched. The Avalonia and WPF benchmark suites mirror each other.

If the change touches resource lookup, the generated keys, or the packaging that carries them

Does not apply: the only new dependency (BenchmarkDotNet) is referenced by the benchmark projects only, so
no package's dependencies change.

If the change touches the public API

Does not apply.

After merging

  • Create the accept-allocation-increase label if it does not exist yet.
  • The first push to main creates the benchmark-data branch with the first entry.
  • The deployed site serves benchmarks/latest/ and benchmarks/history/.
  • The next PR's job summary compares against that entry, with both the Time and Allocations series.

OpenSpec change add-benchmarks: proposal, design, delta spec and tasks.

Refs: #53
Core, Avalonia and WPF benchmarks against the designer class and
{x:Static}, a CI job that fails a pull request on allocation regressions,
the published history on GitHub Pages, and the benchmarks docs page.

Refs: #53
@rent-a-developer rent-a-developer self-assigned this Oct 26, 2026
@rent-a-developer rent-a-developer added documentation Improvements or additions to documentation enhancement New feature or request labels Oct 26, 2026
@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@rent-a-developer
rent-a-developer merged commit 0dcf57d into main Oct 26, 2026
22 of 35 checks passed
@rent-a-developer
rent-a-developer deleted the feature/issue-53-add-benchmarks branch October 26, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a performance benchmark suite with CI regression checks

1 participant