Skip to content

fix(producer): HDR sources are tone-mapped for formats that cannot carry HDR - #6597

Merged
miguel-heygen merged 4 commits into
mainfrom
dframey/hdr-tonemap
Oct 27, 2026
Merged

miguel-heygen merged 4 commits into
mainfrom
dframey/hdr-tonemap

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

What

HDR video sources are now tone-mapped to SDR during frame extraction whenever the output format cannot carry HDR (anything but mp4), not only with --sdr.

Why

resolveEffectiveHdrMode already downgrades HDR to SDR for non-mp4 output (webm, mov, png-sequence, gif, hls). Frame extraction decided separately, and tone-mapped only when hdrMode === "force-sdr". So auto or force-hdr into a non-mp4 format kept the HDR-encoded (BT.2020 PQ/HLG) pixels in an SDR render, which comes out washed out.

Related work

None known.

How

One predicate, rendersSdrOnly(config) in hdrMode.ts (forced SDR or non-mp4 format), now owns "this output is SDR"; extractVideosStage uses it for toneMapHdrToSdr. The in-process and distributed-plan paths share that stage, so both are covered.

Test plan

  • Unit tests added/updated: extractVideosStage.timelineBound.test.ts gains three cases (auto+mp4 false, auto+webm true, force-hdr+mov true), run for both the in-process and distributed paths. With the call site reverted to hdrMode === "force-sdr", four of them fail; with the fix, all 17 pass.
  • Manual testing performed: not done. No real HDR clip was rendered end to end to webm/mov; the test asserts the flag handed to the extractor, and the extractor's own tone-map path already serves --sdr.
  • Documentation updated (if applicable): n/a
  • Comments follow CONTRIBUTING.md "Comments"

Typecheck, oxlint, oxfmt and the fallow audit were run on a Linux dev machine.

Size: under the 100-line floor because it is one self-contained fix with nothing else to fold it into.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES · 1 blocker · 0 important · 0 nits · head de0b89ad

HDR clips are now tone-mapped, but in a mixed HDR+SDR timeline the extractor still lifts the SDR clips to BT.2020 for a render that comes out SDR. That's a second place deciding whether the output is HDR. Details inline.

Scope

Read all 3 files and the engine extractor's tone-map and SDR→HDR paths. Ran extractVideosStage.timelineBound.test.ts locally (17 pass), plus a probe extraction against real ffmpeg (see inline). CI was still running at review time. No prior reviews.

— Genesis

outputDir: join(compiledDir, "__hyperframes_video_frames"),
format: job.config.videoFrameFormat ?? "auto",
toneMapHdrToSdr: job.config.hdrMode === "force-sdr",
toneMapHdrToSdr: rendersSdrOnly(job.config),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocker: this flag reaches only the HDR clips. In extractAllVideoFrames, the SDR→HDR preflight (videoFrameExtractor.ts:2136-2179) checks only hdrInfo.hasHdr and ignores toneMapHdrToSdr. So in an auto + webm/mov/gif/hls render with one HDR clip and one SDR clip, the SDR clip still goes through SDR_TO_HDR_COLORSPACE_FILTER and skips sdrCanvasColourFilters (:1214). It ends up as PQ/HLG-encoded pixels in an SDR render, which is the washed-out symptom this PR fixes. Now the two clips also disagree within the same render.

I reproduced it against real ffmpeg: extractAllVideoFrames([sdr, hdr], …, { toneMapHdrToSdr: true }) gives hdrPreflightCount: 1, and the SDR clip's frame differs from the same clip extracted on its own. (--sdr has had the same gap all along, since it uses this path.)

Fix: gate the preflight with if (!options.toneMapHdrToSdr && hdrInfo.hasHdr && hdrInfo.dominantTransfer). Add a mixed-timeline case (auto + webm → hdrPreflightCount 0).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in c22d33f: the SDR-to-HDR preflight is gated on !options.toneMapHdrToSdr, with a mixed SDR+HDR case in videoFrameExtractor.test.ts (hdrPreflightCount 0 when tone-mapping; 1 without the gate).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Fixed in c22d33f

@github-actions

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 135 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (2)

  • seqplainkeys-none-px-r0-nested-z100: tracking 0.07, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.07, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.07, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true
  • seqplainkeys-none-pct-r0-nested-z100: tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo false, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 1 resolved, 0 open, 0 new · de0b89ad → c22d33f6

— Genesis

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 3137924 Oct 27, 2026
124 checks passed
@miguel-heygen
miguel-heygen deleted the dframey/hdr-tonemap branch October 27, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant