Repository navigation
fix(cache): follow symlinks in the warm-build watch set - #1765
Merged
Merged
Conversation
A PlatformIO project reaches a local library through a symlink (`lib/FastLED -> ~/dev/fastled2`). `WalkDir` skipped symlinked entries, so every file under the link target was missing from the fast-path fingerprint: editing a header there left `build_fingerprint.json` unchanged, the warm path reused the previous `firmware.bin`, and `fbuild deploy` flashed byte-identical firmware until `--clean` (#1753). The two watch walks (content-hash and stamp-cache) now share one `watched_files` helper that follows links; traversal cycles are dropped (walkdir reports them as errors) so a self-referential link cannot hang the build, and the exclude list still prunes by directory name (`.git` under a link target stays out).
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
`mod.rs` crossed the 1000-LOC policy limit and the new tests used
`std::os::unix::fs::symlink`, which the platform-boundary scanner counts as a
host mechanic in engine code. Move the tests to `symlink_tests.rs` (933 LOC in
`mod.rs`) and create links through `fbuild_core::platform::fs::{symlink_dir,
symlink_file}`.
Dylint `ban_std_pathbuf` fires on `PathBuf` in type position outside the legacy allowlist; build the fixture paths with `Path::join` instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1753.
Root cause
The warm-build fast path hashes a watch set of project files and reuses the
previous
firmware.binwhen the hash is unchanged. Both walks(
hash_watch_set,hash_watch_set_stamps) usedWalkDir::new(root)withoutfollow_links, and walkdir reports a symlink entry as neither a file nor adirectory. So for
the entire library was invisible to the fingerprint. A
git checkoutthatrewrote
src/platforms/esp/32/core/fastspi_esp32_idf.hleftbuild_fingerprint.jsonbyte-identical, the fast path hit, and the deployshipped the previous build's artifacts — the two branches measured
byte-for-byte identical firmware until
--cleanforced a full build. That isthe exact symptom in #1753, including why a plain
lib/(non-symlinked)library was unaffected: its headers were already in the watch set.
check_with_stampsis the production branch (cancel... inputs.compiler_cacheis
Nonefor every orchestrator), so this is the path that produced the stalefirmware.
Change
watched_files(watch)helper for both hashes, with.follow_links(true)..flatten()dropsit; covered by
test_hash_watch_set_stamps_tolerates_a_symlink_loop..gitinside a link targetstays out of the hash.
Tests (fail before the fix, pass after)
Added to
crates/fbuild-build-engine/src/build_fingerprint/mod.rs:test_hash_watch_set_stamps_sees_edits_under_a_symlinked_librarytest_hash_watch_set_sees_edits_under_a_symlinked_librarytest_hash_watch_set_stamps_hashes_a_symlinked_source_filetest_hash_watch_set_stamps_tolerates_a_symlink_loopAll three "sees edits" tests fail on
origin/mainwith identicalbefore/after hashes and pass with the change.
Gates
soldr cargo test -p fbuild-build-engine --lib→ 504 passed, 0 failed, 4 ignoredsoldr cargo test -p fbuild-build-esp --lib→ 171 passed, 0 failed, 1 ignoredsoldr cargo clippy --workspace --all-targets -- -D warnings→ exit 0soldr cargo fmt --all --check→ exit 0Not in scope (follow-up)
source_scanner::walk_sourceswalks the projectsrc/with the sameno-follow default, so a symlinked translation unit inside
src/is nevercompiled. Same class of bug, different enumeration; left for its own change.