Skip to content

test: add multithreaded tests, fix two thread-safety bugs - #42

Merged
wdconinc merged 3 commits into
mainfrom
claude/multithreaded-tests
Oct 29, 2026
Merged

wdconinc merged 3 commits into
mainfrom
claude/multithreaded-tests

Conversation

@wdconinc

Copy link
Copy Markdown
Contributor

Stacked on #41.

Summary

New core/tests/multithreading.cpp tests concurrent use as in multithreaded frameworks:

  • process() on one shared algorithm instance from 8 threads;
  • constructing and configuring algorithms concurrently;
  • reading properties concurrently;
  • logging from many threads, both plain/formatted and stream messages;
  • first access of the same service, and of different services, concurrently;
  • a Generator shared between threads, and per-thread generators sharing one engine. Full-range draws must all be distinct.

The AlgorithmSvc lifecycle test also creates, lists and adds algorithms concurrently. The tsan job from #41 runs all of this to catch races that do not change results.

The tests found two thread-safety bugs, fixed here in separate commits:

  1. Service registration was not thread-safe. When different services were first accessed from several threads at once, ServiceSvc::registerService modified the registry concurrently, which corrupted it (TSan: data races, heap-use-after-free). The registry is now guarded by a recursive mutex. It is recursive because initializers can access, and so register, other services.
  2. Logger streams mixed up concurrent messages. LoggerMixin returned one shared stream per log level and locked each <<, not whole messages. Threads logging through one logger produced fragments like "6stream 0". critical() ... trace() now return a new stream per call:
    • Its buffer is only created above threshold, so suppressed messages stay cheap.
    • Output without endmsg is reported at the end of the statement, instead of being prepended to the next message.
    • LoggerMixin stays non-copyable.
    • EICrecon only uses x() << ... << endmsg, which is unaffected.

Test plan

  • Locally in eic_xl:nightly: 95/95 tests pass with TSan, ASan+UBSan, clang++ Debug, g++ Debug and g++ Release
  • Without the two fixes, TSan reports the registry race and the logger stream test fails
  • CI

🤖 Generated with Claude Code

@wdconinc
wdconinc requested a balanced review from Copilot October 29, 2026 00:50
@wdconinc
wdconinc force-pushed the claude/multithreaded-tests branch from 751f205 to 0dc7bfd Compare October 29, 2026 00:52
@wdconinc
wdconinc force-pushed the claude/multithreaded-tests branch from 0dc7bfd to 0b6d3ec Compare October 29, 2026 00:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The concurrency rework of the shared registry and framework-wide logger redesign depend on sanitizer/TSan verification that has not yet passed in CI, leaving correctness unconfirmed.

No open issues 🎉

What changed in this PR

This PR adds a dedicated multithreading test suite that exercises the algorithms core framework as it would be used by concurrent frameworks (shared process(), concurrent construction/configuration, concurrent property reads, concurrent logging, concurrent first-access of services, and shared/parallel random generators). Running these under the thread sanitizer surfaced two real races, which are fixed in the same PR: non-thread-safe service registration and interleaved logger stream messages.

Changes:

  • Guard the ServiceSvc registry with a mutable std::recursive_mutex (recursive because initializers can register other services) across all registry reads/writes.
  • Redesign LoggerMixin/LoggerStream so each critical()/…/trace() call returns a fresh stream by value with a lazily-created buffer (suppressed messages stay cheap) that flushes unflushed output on destruction; LoggerMixin is explicitly non-copyable.
  • Add core/tests/multithreading.cpp, a concurrent block in algorithm_svc.cpp, a new unflushed-stream test in logger.cpp, and link Threads::Threads.
File Description
core/​include/​algorithms/​service.h Adds recursive-mutex locking to registry mutators/accessors; documents services() as unprotected.
core/​include/​algorithms/​logger.h Per-call LoggerStream returned by value with lazy buffer and destructor flush; LoggerMixin made explicitly non-copyable; level() simplified.
core/​tests/​multithreading.cpp New suite covering concurrent algorithms, logging, properties, services, and RNG.
core/​tests/​logger.cpp New test for reporting unflushed stream output at statement end.
core/​tests/​algorithm_svc.cpp Adds concurrent create/list/add lifecycle checks.
core/​tests/​CMakeLists.txt Finds and links Threads::Threads for all test executables.

Review effort: Balanced

@wdconinc
wdconinc changed the base branch from claude/sanitizers to main October 29, 2026 01:02
wdconinc and others added 3 commits October 28, 2026 20:02
Services register themselves in the ServiceSvc on first access. When
different services were first accessed from several threads at the same
time, the registry (keys, services and initializers) was modified
concurrently, which corrupted it (data races, use after free). Guard the
registry with a recursive mutex; recursive, as initializers can access,
and thereby register, other services.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LoggerMixin returned one shared stream per log level, so messages that
several threads composed concurrently on the same logger (e.g. in an
algorithm shared between threads) got mixed up: the stream only locked
each insertion, not a whole message. critical(), error(), ..., trace()
now return a new stream per call. Its output buffer is only created for
messages above threshold, so suppressed messages stay cheap, and output
that is not flushed with endmsg is reported at the end of the statement
instead of being prepended to the next message.

LoggerMixin stays non-copyable, as it was through its stream members.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test concurrent use as in multithreaded frameworks: processing with one
algorithm instance from several threads, constructing and configuring
algorithms, reading properties, logging, first access of services,
sharing generators and their engine, and creating algorithms from the
AlgorithmSvc. The thread sanitizer job runs these tests to detect races
that do not change results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wdconinc
wdconinc force-pushed the claude/multithreaded-tests branch from 0b6d3ec to 41f156b Compare October 29, 2026 01:02
@wdconinc
wdconinc merged commit 44e1cec into main Oct 29, 2026
8 checks passed
@wdconinc
wdconinc deleted the claude/multithreaded-tests branch October 29, 2026 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants