Repository navigation
perf(store): which mail writer holds the shared write lock - #277
Merged
Merged
Conversation
One mail save per transaction costs about 300 ms when it queues text (the mentions drain rebuilds the name index); slices of 50 with a 100 ms pause cut the worst wait of another process from seconds to a few hundred ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lices of 50 The real saveThreads drains mentions once per slice: 6 drains for 300 emails, a worst wait of 329 ms, no failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leemour
added a commit
to WireCatLabs/cli-memo
that referenced
this pull request
Oct 31, 2026
## Why `memo sync` runs every ~5 minutes. The mail header backfill and the HTML text repair saved one email per transaction. Each save is a write lock, so other CLIs that share the SQLite store got "database is locked". cli-core 0.506.0 added `MailStore.saveThreads(inputs, states?)`, which saves several threads and sync state in one transaction ([cli-core PR 2250](WireCatLabs/cli-core#2250)). Its own mail sync now saves 50 per transaction with a 100 ms pause. The measurement is in [cli-testing PR 277](WireCatLabs/cli-testing#277). ## What changed - `src/mail/backfill-headers.ts`: saves in slices of 50 with `saveThreads`. The cursor is saved in the same transaction as each slice, so a failed run resumes after the last saved slice. - `src/mail/repair.ts`: same slices. A converted email no longer looks like HTML, so a rerun skips what earlier slices saved. The done marker is saved with the last slice. - `src/mail/slices.ts`: slice size (50), pause (100 ms), and a small chunk helper shared by both. - Tests in `src/mail/slices.test.ts`: progress saved per slice, and resume after a failure in the second slice. - `CHANGELOG.md`: unreleased entry. ## cli-core version `main` is already on cli-core 0.509.0, which includes 0.506.0. No bump is needed. The 0.506.0 to 0.509.0 "may break callers" changes were already adapted in 0.130.0. ## Checks `pnpm lint` (only existing warnings), `pnpm typecheck`, `pnpm test` (94 files, 707 tests) all pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why:
mail syncfailed with "database is locked" next tomemo sync. This measures, on an offline copy of the owner's real store, how long mail saves hold the write lock and how long another process waits.Result: a new email costs about 300 ms of lock time, 95% of it the mentions drain (it rebuilds the known people and name index at the end of every write that queued text). Saving 300 new emails one per transaction took 76 s and made a waiter fail on 13 of 25 tries. Slices of 50 with a 100 ms pause took 2.1 s, worst wait 329 ms, no failures. The header backfill alone (re-saves, no queued text) holds about 8 ms per save; its worst wait was 4.3 s with SQLite's own wait, 1.4 s sliced. The 1 ms retry helped on re-saves but not on new emails.
Page: https://github.com/WireCatLabs/cli-testing/blob/perf/lock-hold/performance/store/lock-hold/README.md
Script: https://github.com/WireCatLabs/cli-testing/blob/perf/lock-hold/performance/store/lock-hold/bench.mjs
Only totals are published; the store copy is private and stays local. Not merged on purpose.
🤖 Generated with Claude Code