Skip to content

Align whole-document PDF changes and render shared regions once - #58

Merged
cu1ch3n merged 27 commits into
mainfrom
fix/whole-document-diff
Nov 6, 2026
Merged

cu1ch3n merged 27 commits into
mainfrom
fix/whole-document-diff

Conversation

@cu1ch3n

@cu1ch3n cu1ch3n commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

This PR compares complete PDFs before cutting review views, so text edits, moved passages and visual changes use one document correspondence. Source ownership determines the relevant edits, and overlapping review regions display shared ink once.

Final validation passed at a2f3b02. The latest Windows and build-progress changes from main are integrated. Adding the engine suite to Windows CI exposed inconsistent repository path identities; the client, server and Git now use the same native canonical paths. The original restart test remains enabled, with an additional directory-alias regression.

Changes

  • Preserve reading flow across page headers, floats and physical word breaks. Shared segmentation retains physical glyph footprints and ordinary word boundaries; overlapping extracted rows no longer duplicate glyphs.
  • Compare glyph appearance and relative placement using actual TeX box structure. Horizontal box offsets preserve raised/lowered and nested formula changes, while vertical flow allows ordinary paragraph rewrapping.
  • Compare non-text paths, images, masks and shading in their shared PDF clip frames. Detect relative drawing movement and text/graphic occlusion, retain margin and cross-column crops, and ignore coherent reflow of independently clipped figures.
  • Keep unchanged context in its original colours. Crop only truly blank rows, preserving faint coloured ink and fractional raster coordinates.
  • Verify one sufficient edit combination for each optional indirect effect against its final appearance and position. The combination need not be minimal or exhaustive; accept/reject applies only to the selected edit.
  • Remove time-based partial comparison results and the sixteen-variant cutoff. Update settings, translations and documentation to match.
  • Run the engine suite in Node 22/24 and Windows CI, with real TeX integration and browser checks on Node 22.

Validation

For the final source:

  • 457 engine tests passed with no skips on Node 22, including real TeX integration; engine typecheck and production build passed.
  • Application tests: 746 passed, 21 platform skips locally; final Node 22 CI passed 745 with 22 platform skips. Application/desktop typechecks, application build and translation checks passed. Twelve appearance combinations and build-progress browser checks passed after integration with main.
  • All 18 engine browser tests passed in CI after integration with main.
  • Independent reading-flow checks passed 426 combinations, 852 glyph-conservation checks and 450 incorrect-variant controls.
  • Actual indirect publishes cover separated scripts and nested fractions/text boxes; negative controls cover ordinary reflow inside minipages and paragraphs.
  • The final independent drawing pass passed 12 scenarios, retained all 47,992 changed test pixels in the reported regions, and verified the actual SVG output for faint margin ink.
  • All 172 saved-PDF rounds rendered without errors or alignment fallbacks, with zero overlapping strip regions. Synthetic preservation has no new omission flags. The three historical formula coverage flags were independently classified: all actual changed glyphs are retained and correctly marked; every raw pixel outside change crops belongs to unchanged translated glyphs. Original scores remain 27/30.
  • The raw manuscript preservation check flags 173 words across nine cases. All 173 were independently classified using source, physical glyphs, raster checks and delivered SVGs: no actual new omission or incorrect annotation was found. Unchanged reflowed text remains in context, which the raw preservation score excludes. The original raw assertion remains failed and is preserved alongside this classification.
  • The merged SyncTeX parser produces byte-identical box/baseline arrays across 241 saved builds and 18,389 pages, plus identical results for 20,770 source queries.
  • Native Windows engine CI passed 406 tests with 51 platform/TeX skips. Directory-alias recognition and the original detached-server environment restart test both passed; the restart completed in 2.8 seconds. The alias regression fails against the old probe comparison. Final-head CI passed Node 22/24, Windows, all three desktop platforms and the required gate; CodeQL also passed.

The saved-PDF corpus includes 103 manuscript rounds, 39 synthetic rounds and 30 formula cases. Historical raw preservation flags and three formula coverage flags are retained as diagnostics, not relabelled as passing checks. A source-root adapter correction affects four historical build inputs and two rounds; only those two baseline rounds were rerendered with the corrected adapter. Two existing missing non-TeX metadata inputs are recorded without inventing replacements or dropping their units.

Comparison contract

Repeated text can admit multiple valid correspondences. A printed line-end hyphen can be indistinguishable from a discretionary typesetting hyphen. Unscoped drawing primitives share a page frame because PDF does not always retain the author's figure boundaries. Glyph sampling and 2x raster checks have finite resolution; this is not a proof of arbitrary PDF equivalence. Font ink can slightly overhang a highlight rectangle while remaining fully visible inside the crop.

The optional attribution contract is one verified sufficient combination, not all causal explanations. Renderer/compilation failures remain failures and never become successful truncated comparisons. The saved corpus covers direct crops; synthetic actual-publish tests separately cover indirect effects.

All manuscript inputs, screenshots and raw diagnostics remain private and local.

A card's crops came from SyncTeX on each side separately and were compared
alone, so the old and new crops often covered different text: unchanged
words were marked, a heading edit could mark a whole section, and text that
moved or a float that settled elsewhere read as rewritten.

The two PDFs are now aligned once per round, word by word in reading order:
runs of eight words that occur once in each pin them together, and the
words between are compared. Marks are only words this comparison changed;
removed and added runs that share runs of four words moved, shown in light
blue on both sides. Each edit owns the lines its source printed (SyncTeX
records, by majority; a record over many lines, such as a float's box, does
not vote); text that only moved is an edit's only where its source printed
it. A change of words is cut at the same equal words on both sides, so the
context strips with the old crop read as the old document and with the new
crop as the new one; words of the old crop the context shows fade. A change
of layout alone is cut at the same equal lines, less those that print the
same. A line across a crop's edge stays outside unless it reaches the
crop's text, so a display above is not swallowed; empty strips are not
written.

Pages with two columns are cropped one column at a time, in reading order,
with bands across the gutter (titles, wide tables) found from the glyphs,
shapes and images that cross it. A page of one column with a few lines side
by side stays one column.

Also: word extraction joins hyphens in one pass instead of splicing, and a
block of moved text is not labelled a reflow.
A card's crops came from SyncTeX on each side separately and were compared
alone, so the old and new crops often covered different text: unchanged
words were marked, a heading edit could mark a whole section, and text that
moved or a float that settled elsewhere read as rewritten.

The two PDFs are now aligned once per round, word by word in reading order:
runs of eight words that occur once in each pin them together, and the
words between are compared. Marks are only words this comparison changed;
removed and added runs that share runs of four words moved, shown in light
blue on both sides. Each edit owns the lines its source printed (SyncTeX
records, by majority; a record over many lines, such as a float's box, does
not vote); text that only moved is an edit's only where its source printed
it. A change of words is cut at the same equal words on both sides, so the
context strips with the old crop read as the old document and with the new
crop as the new one; words of the old crop the context shows fade. A change
of layout alone is cut at the same equal lines, less those that print the
same. A line across a crop's edge stays outside unless it reaches the
crop's text, so a display above is not swallowed; empty strips are not
written.

Pages with two columns are cropped one column at a time, in reading order,
with bands across the gutter (titles, wide tables) found from the glyphs,
shapes and images that cross it. A page of one column with a few lines side
by side stays one column.

Also: word extraction joins hyphens in one pass instead of splicing, and a
block of moved text is not labelled a reflow.
The old crop shows whole rows, and words of a row before or after the
change faded even when their counterparts printed in the new crop, which
the context strips do not show: the old version of such a row could not
be read, and a changed word of a neighbouring change could fade. A word
now fades only when it equals a word outside the new crop.
@cu1ch3n cu1ch3n changed the title Cut both sides of a change at the same words of one whole-document diff Align whole-document PDF changes and render shared regions once Nov 6, 2026
@cu1ch3n
cu1ch3n marked this pull request as ready for review November 6, 2026 12:43
@cu1ch3n
cu1ch3n merged commit fdc0662 into main Nov 6, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant