Skip to content

Run all BenchmarkDotNet cases in parallel CI jobs - #4

Merged
KSemenenko merged 2 commits into
mainfrom
codex/mcpgateway-benchmark-ci
Nov 3, 2026
Merged

KSemenenko merged 2 commits into
mainfrom
codex/mcpgateway-benchmark-ci

Conversation

@KSemenenko

@KSemenenko KSemenenko commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

The serial GitHub Actions job hit its 180-minute limit after ten of 23 cases. The ordinary 10,000-tool catalog took 32m 31s to initialize. CI now derives ten independent jobs from BenchmarkDotNet discovery and checks the identity, timing and allocations of every case. Each job uploads its reports. The suite uses the original fixtures, DefaultJob and MemoryDiagnoser.

This PR also adopts the public ManagedCode.MarkdownLd.Kb 0.3.1 package. The NuGet DLL matches the GitHub release asset and the consumer's resolved dependency. The owner repair preserves the complete ordered graph relations and reduces redundant work during catalog construction.

Ordinary 10,000-tool scenario Original serial Linux run Linux with 0.3.1
Cold catalog setup 32m 31s 12m 27s
Cumulative setup allocation 93.56 GB 82.74 GB
Managed memory after setup GC 7.27 GB 7.17 GB

The two measurements used different hosted Linux runners. The new warm discovery, schema and invocation cycle averaged 37.57 seconds and allocated 38.17 GiB per operation. The owner review isolates the SPARQL evidence and provenance cost.

Verification includes 456 passing tests locally and on Linux, 92.48% Linux line coverage, a clean formatting check, CodeQL, and a vulnerability scan with zero advisories. The exact Linux CI run completed in about 61 minutes. All 23 unique full JSON reports across ten jobs contain numeric timings and allocations. The original timed-out job and the new ordinary 10,000-tool job retain the original measurements.

@KSemenenko
KSemenenko merged commit b88b1e0 into main Nov 3, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant