Skip to content

fix(audio): offset the analysis lookup by the track's placement - #198

Merged
LeadcodeDev merged 1 commit into
mainfrom
fix/audio-analysis-track-offset
Aug 12, 2026
Merged

fix(audio): offset the analysis lookup by the track's placement#198
LeadcodeDev merged 1 commit into
mainfrom
fix/audio-analysis-track-offset

Conversation

@LeadcodeDev

Copy link
Copy Markdown
Owner

Closes #190.

What

An audio track with start: 73 played the file from its beginning at t=73 in the video. A waveform or audio_spectrum at t=73 drew the file's content at 73 seconds. Picture and sound disagreed by exactly start.

Two independent time bases:

  • encode/audio.rs places the track at track.start * TARGET_SAMPLE_RATE on the scenario timeline and copies the file from its own sample 0;
  • encode/audio_analysis.rs analyses the raw file, indexed from its own frame 0, and the painters call amplitude_at(ctx.time) with scenario time. track.start appeared nowhere in that file.

end had the mirror problem: the analysis kept running past the point where the track was cut.

Why it matters in practice

Gating a soundtrack to one scene is a natural thing to want, and the obvious way to write it — start/end — silently desynchronised the visualisation. The workaround is volume_keyframes, which leaves the time mapping intact but is not what the fields suggest.

How

AudioAnalysis carries start/end, and every accessor goes through one track_time(scenario_time) -> Option<f64>: None outside the window (so the visualisation goes flat exactly when the track is silent), Some(time - start) inside.

The three consumers — waveform, audio_spectrum, and the audio-reactive binding in box_builder — are unchanged. The cache is keyed by path and the painters never see the AudioTrack, so the placement belongs with the data rather than duplicated at three call sites.

The cache fingerprint gains the placement. The analysis content depends only on the file, but the lookup no longer does, so an entry computed for one placement must not be reused for another — otherwise opening a second scenario that reuses the same file at a different offset would silently keep the first one's.

Not in scope

volume_keyframes and the fades are a separate question — the analysis is of the source, not of the mix — and now more visible since #182 made the studio play the mixed output. Worth deciding explicitly; not decided here.

Test

a_track_start_offsets_the_analysis_lookup: 1 s of sine then 1 s of silence, placed at start: 5.0, end: 6.5. Asserts flat before 5, loud at 5.2 (0.2 s into the file), quiet at 6.2 (1.2 s in, the silence), flat past 6.5 — and that the smoothed and band accessors take the same path, or a bound component would disagree with the waveform beside it.

cargo test --workspace green, cargo fmt --check and cargo clippy --all-targets -- -D warnings clean.

An audio track with start: 73 played the file from its beginning at t=73 in
the video, while a waveform or audio_spectrum at t=73 drew the file's content
*at 73 seconds*. Picture and sound disagreed by exactly `start`.

Two independent time bases: encode/audio.rs places the track at
track.start * TARGET_SAMPLE_RATE and copies the file from its own sample 0,
while the analysis indexes the file from its own frame 0 and the painters call
amplitude_at(ctx.time) with scenario time.

Carry `start`/`end` in AudioAnalysis and apply them in one place, `track_time`,
which every accessor now goes through. The consumers are unchanged: the cache
is keyed by path and the painters never see the AudioTrack, so the knowledge
belongs with the data rather than at three call sites.

Outside [start, end) the accessors return 0, so a visualisation goes flat
exactly when the track is silent instead of drawing an envelope nobody hears.

The cache fingerprint gains the placement: the analysis content depends only
on the file, but the lookup no longer does, so an entry computed for one
placement must not be reused for another.
@LeadcodeDev
LeadcodeDev merged commit eeed5b0 into main Aug 12, 2026
3 checks passed
@LeadcodeDev
LeadcodeDev deleted the fix/audio-analysis-track-offset branch August 12, 2026 15:44
@LeadcodeDev LeadcodeDev added the bug Something isn't working label Aug 12, 2026
@LeadcodeDev LeadcodeDev self-assigned this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

track.start shifts the muxed audio but not the analysis audio-reactive components read

1 participant