Skip to content

perf: share the TypeScript worker between compilers checking the same project - #390

Closed
alexander-akait wants to merge 4 commits into
mainfrom
perf/ts-share-program
Closed

alexander-akait wants to merge 4 commits into
mainfrom
perf/ts-share-program

Conversation

@alexander-akait

Copy link
Copy Markdown
Member

Stacked on #389: this branch contains its commit. Please merge #389 first; this diff then shrinks to its own commit (7f62c3d).

Summary

Each compiler of a MultiCompiler kept its own worker and its own program for its TypeScript check. Runs that overlap in time were already shared (sharedRuns in check.js). Compilers that build one after another, though (parallelism: 1, or dependencies between configs), each built and held the whole program.

  • Shared workers:
    • Workers are kept in a module-level map keyed by sharingKey("typescript", options), the same key concurrent runs are shared by. It is now exported from check.js.
    • Each worker counts the compilers using it, and ends once the last one lets go: on shutdown or watchClose, or in cleanup() for a one-shot build.
    • Options that cannot be written down fall back to today's per-compiler key.
  • Per-run cancellation (on top of perf: stop a TypeScript run nothing will report once the next rebuild starts #389): with a shared worker, "stop every run issued so far" could stop another compiler's live run.
    • createSolo now writes a run's number into one of 64 slots of the shared Int32Array.
    • cancel(answer) stops exactly the run that promise belongs to.
  • The in-process path (threads: false) still keeps a program per compiler, so it is garbage-collected with the compiler.

Measurements

Setup: 600 .ts files, a two-compiler MultiCompiler with the same TypeScript check, a watch with the first build plus one rebuild after an edit.

wall CPU peak RSS
main, parallelism: 1 10.24 / 9.68 / 10.27 s 18.7–20.0 s 803–853 MB
this PR, parallelism: 1 5.68 / 5.80 / 5.59 s 9.8–10.4 s 467–483 MB
main, parallel 5.25 s 9.9 s 481 MB
this PR, parallel 5.45 s 9.9 s 468 MB

One compiler alone takes 5.5 s, 10 CPU-s and about 450 MB, so two sequential compilers now cost about what one does.

One-shot sequential builds are unchanged by this PR. A one-shot compiler ends its worker when its build is done, or the thread would keep the process alive. #383's persisted builder state is what makes the second compiler cheap there.

Tests

  • test/typescript/share.test.js:
    • A TypeScript mock records whether each builder program was built fresh or from the previous one.
    • Two compilers with parallelism: 1 in watch mode must record ["fresh", "reused"].
    • With the key forced back to per compiler, the test fails (fresh, fresh).
  • test/typescript/cancel.test.js from perf: stop a TypeScript run nothing will report once the next rebuild starts #389 still passes with per-run cancellation.
  • Lint, tsc, schema lint, prettier, cspell and the full suite (332 tests) pass locally.

Generated by Claude Code

… starts

A check that reportAs keeps off the compilation runs after a watch
rebuild. When the next rebuild starts, its report is dropped, yet the
TypeScript worker kept checking the whole program before it would
start the run that matters. The plugin now tells a check to stop a run
of an older compilation as soon as it takes over from it, through an
optional cancel() on the check instance. The TypeScript check passes a
CancellationToken backed by a SharedArrayBuffer to its worker, so the
run stops between one file and the next and the builder keeps what it
got through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01App18sYbe6Vf6HDCjCXbvf
… project

Each compiler of a MultiCompiler kept a worker and a program of its own
for its TypeScript check. Runs that overlapped were already shared, but
compilers building one after another built and held the program twice.
Workers are now kept by what their program is built from, the same key
concurrent runs are shared by, and ended once no compiler uses them.
A shared worker numbers its runs, so a compiler stopping its own stale
run no longer stops another compiler's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01App18sYbe6Vf6HDCjCXbvf
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f45e0d2a-95ce-4760-b0db-d5afe51fe199
📥 Commits

Reviewing files that changed from the base of the PR and between 0abded0 and bfa4584.

📒 Files selected for processing: 21
  • .changeset/ts-share-program-across-compilers.md
  • .changeset/ts-stop-stale-runs.md
  • README.md
  • src/check.js
  • src/checks/index.js
  • src/checks/typescript-program.js
  • src/checks/typescript-worker.js
  • src/checks/typescript.js
  • src/threads.js
  • test/store.test.js
  • test/typescript/cancel.test.js
  • test/typescript/fixtures/cancel/index.js
  • test/typescript/mock/typescript-counter/index.js
  • test/typescript/mock/typescript-stall/index.js
  • test/typescript/share.test.js
  • types/check.d.ts
  • types/checks/index.d.ts
  • types/checks/typescript-program.d.ts
  • types/checks/typescript-worker.d.ts
  • types/checks/typescript.d.ts
  • types/threads.d.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

CLA Not Signed

One or more co-authors of this pull request were not found. You must specify co-authors in commit message trailer via:

Co-authored-by: name <email>

Supported Co-authored-by: formats include:

  1. Anything <id+login@users.noreply.github.com> - it will locate your GitHub user by id part.
  2. Anything <login@users.noreply.github.com> - it will locate your GitHub user by login part.
  3. Anything <public-email> - it will locate your GitHub user by public-email part. Note that this email must be made public on Github.
  4. Anything <other-email> - it will locate your GitHub user by other-email part but only if that email was used before for any other CLA as a main commit author.
  5. login <any-valid-email> - it will locate your GitHub user by login part, note that login part must be at least 3 characters long.

Alternatively, if the co-author should not be included, remove the Co-authored-by: line from the commit message.

Please update your commit message(s) by doing git commit --amend and then git push [--force] and then request re-running CLA check via commenting on this pull request:

/easycla

Two compilers linting the same files at once share one run. Stopping it
for the compiler whose compilation is stale would have handed the other
one an empty answer. A run another compiler joined is now left to
finish, and a run that is stopped is taken out of the shared runs, so a
compiler coming later starts its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01App18sYbe6Vf6HDCjCXbvf

Copy link
Copy Markdown
Member Author

EasyCLA fails because the commits carry a Co-Authored-By: Claude trailer that maps to no GitHub account. It is not a code failure. Whether to drop the trailer or accept the check is for the maintainers to decide, so I haven't rewritten history.

This PR now also carries #389's review fix (84b755e: never stop a run another compiler joined), merged in from that branch.


Generated by Claude Code

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.65%. Comparing base (0abded0) to head (bfa4584).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #390    +/-   ##
========================================
  Coverage   99.64%   99.65%            
========================================
  Files          19       19            
  Lines        5618     5750   +132     
========================================
+ Hits         5598     5730   +132     
  Misses         20       20            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants