Skip to content

cmd/loopvec-toolexec: give the same archive on every build - #11

Open
thesyncim wants to merge 1 commit into
mauri870:masterfrom
thesyncim:toolexec-reproducible
Open

thesyncim wants to merge 1 commit into
mauri870:masterfrom
thesyncim:toolexec-reproducible

Conversation

@thesyncim

@thesyncim thesyncim commented Nov 1, 2026 •

Copy link
Copy Markdown

The wrapper wrote rewritten files to a fresh os.MkdirTemp directory under the go command's work directory. File names end up in the export data, and the go command only trims each compile's own object directory. So every build of a rewritten package gave a different archive, with a random absolute path in it.

That breaks builds where the go command compiles one package twice with the same inputs. With PGO, go test can compile a dependency once per test binary, then compile an importer against one copy and link the other. In goav1, whose cmd/qualitybench has a default.pgo, go test ./... failed with master's wrapper:

link: fingerprint mismatch: github.com/thesyncim/goav1 has 9bcecb47ca806e38, import from github.com/thesyncim/goav1/cmd/qualitybench expecting 224b81cee36e8b4e

The rewritten files now go in loopvec/ inside the object directory, next to the importcfg the go command writes there. That's still a separate place for each compile, and the -trimpath "$WORK/bNNN=>" that the go command always passes reduces it to loopvec/x.go. After this, the two archives in that build have the same content ID, and the test binary links.

toolexec_reproducible.txt builds a package with two empty caches and compares the archives. It fails without the change. -trimpath builds were already fine.

The wrapper wrote the rewritten files to a new os.MkdirTemp directory
under the go command's work directory. The compiler records file names
in the export data, and the go command trims only the compile's own
object directory, so every build of a rewritten package produced a
different archive, with a random absolute path in it.

That fails builds where the go command compiles a package twice with
the same inputs. With PGO, go test can compile a dependency once for
each test binary that needs it, then compile an importer against one
copy and link the other. In github.com/thesyncim/goav1, whose
cmd/qualitybench has a default.pgo, go test ./... fails with:

  link: fingerprint mismatch: github.com/thesyncim/goav1 has
  9bcecb47ca806e38, import from github.com/thesyncim/goav1/cmd/qualitybench
  expecting 224b81cee36e8b4e

The rewritten files now go in a loopvec directory inside the object
directory, next to the importcfg the go command writes there. That is
still a separate place for each compile, and the -trimpath the go
command passes reduces it to loopvec/x.go.
@thesyncim
thesyncim force-pushed the toolexec-reproducible branch from 6a5db5b to 0eb9b61 Compare November 1, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant