Skip to content

feat(linux): support frame-pointer-only recording - #12

Open
not-matthias wants to merge 8 commits into
codspeedfrom
cod-3861-support-samply-fp-unwinding
Open

not-matthias wants to merge 8 commits into
codspeedfrom
cod-3861-support-samply-fp-unwinding

Conversation

@not-matthias

@not-matthias not-matthias commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

TLDR: Add samply record --unwinding-mode fp to use kernel frame-pointer callchains without copying user stacks or registers (Linux only).

  • Keep dwarf as the default with the existing kernel-FP fallback; preserve FP mode for launch, PID attach, and software-event fallback.
  • Fix perf flag ordering so kernel callchains remain excluded when kernel sampling is disabled.
  • Add eight behavior tests for CLI selection, frame ordering, context transitions, and leaf/empty callchains. cargo test -p samply --locked: 28 passed, one existing ignored test.
  • Live recording of a frame-pointer-enabled C workload verified launch, attach, software fallback, and default recording. Syscall traces confirmed FP mode omits stack/register capture; caller chains had no duplicate frames.
  • Fix the macOS startup race exposed by the existing dsymutil CI test: keep accepted processes suspended until their initial threads and libraries have been read.

FP stacks require preserved frame pointers throughout the call chain and remain subject to kernel/architecture support and callchain depth limits.

Refs COD-3861.

When a binary is found through the binary lookup dirs, the resolved path
uses the host's separators. Splitting it at the last '/' kept the whole
path as the library name on Windows. Path::file_name handles both.
tools/stack-cache-repro is a small program whose stacks are deeper than
samply's 32000-byte user stack copy. Phase A recurses and works at every
depth, then phase B recurses to the same depth through a different function
and works only in its leaf, so phase B's frames above the copy sit where
phase A's were seen earlier.

replay.sh records it once with perf and imports the same perf.data with each
given samply binary, so every build sees identical samples. stats.py counts
stitched stacks (a phase B leaf under phase A's recursion, or the reverse)
and exits non-zero if there are any.

make-fixture.sh records a short run as a static binary into
fixtures/other/stack-read-cache/, so a replay test can check the
converter without perf permissions.
…shot

Samples copy 32000 bytes of the user stack, starting at the sampled SP.
The stack read cache answered unwinder reads above that copy with the word
that some earlier sample of the process had seen at the same address. The
thread has since returned and called other code at those addresses, so the
unwinder followed stale return addresses and spliced an old call chain onto
the current stack, e.g. RocksDB and serde frames under a pure in-memory
recursive function in jazz profiles.

Keep the last 64 captured windows per thread instead of a per-process
address -> word map. A read above the current window is served by an
earlier snapshot only if that snapshot starts inside the window and every
word from its stable start (the lowest slot its own unwind read; below that
are the leaf's live locals) up to the end of the window matches the current
sample, over at least 512 words. Past that snapshot the same check chains
to an older one, so each part of the stack comes from one earlier sample.
Otherwise the stack is truncated.

This is still a heuristic: a snapshot whose overlap is identical but whose
callers above it differ is accepted.

Tests:
- unit tests on StackSnapshot::continues cover the 512-word threshold and
  that only words from the stable start upward must match;
- tests/stack_read_cache_replay.rs imports the fixtures/other/stack-read-cache
  recording of tools/stack-cache-repro and checks that no stack is spliced
  across phases while phase A is still completed to its full depth. The old
  cache splices all 14 phase B stacks of this fixture.
The stack snapshots are keyed by tid, so a new thread that reuses an exited
thread's tid (and possibly its stack mapping) could otherwise continue
through the old thread's windows. Remove them when a non-main thread exits
or execs; main-thread exit and exec already recreate the whole process.
Rust 1.98 adds useless_borrows_in_formatting and a format-width check, and
making the samply modules public enabled the new_without_default,
should_implement_trait and result_unit_err lints on their APIs.
Add --unwinding-mode fp to record kernel callchains without copying user stacks or registers. Keep the existing DWARF recording behavior as the default, including on Android.

Apply kernel callchain exclusion after initializing perf flags. Cover CLI selection and callchain-only stack conversion, and document frame-pointer build requirements.
Wait for the sampler to finish task initialization before releasing the preload handshake or resuming an attached task. Otherwise short-lived root processes can exit while their initial threads and libraries are still being read, causing the sampler to panic.

Reuse one rendezvous channel for task initialization. Release the accepter after each initialization attempt, including failures, so processes do not remain suspended when sampling stops.
The user stack copied per sample for DWARF unwinding was fixed at 32000 bytes. Expose it as --unwinding-stack-size (default unchanged) so callers can trade profile size for deeper stacks. Values are validated against the kernel's sample_stack_user constraints (8-byte aligned, below 65536).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant