Repository navigation
Conversation
A read at a negative constant offset in a loop with a constant limit, such as dst[i] = a[i-20] + 1 in a loop that stops before 16, was rewritten into code that does not compile: the length check of the vector loop is _ = a[16-21], a negative constant index. -split wrote that file, and the toolexec wrapper fell back to the original package. Such a loop panics as soon as it runs, as does one whose first index is a negative constant (a[i-1] in a loop from 0). Both are now left alone with their own reason, the way a constant limit of zero or less already is. A read whose indexes can all be zero or more (a[i-1] from 1) is still rewritten. The checks read the start and limit as text, so a named constant or a constant expression (const s0 = 0; for i := s0; ...) slipped past them. A constant start or limit is now spelled as its value, which also lets the short-loop rule see a named constant trip count: goav1's for x := range restorationExtraHorz, three iterations, is no longer rewritten. The vectorized set of std and gonum does not change.
… capture
The rewritten code adds names that begin with an underscore (_i,
_count, _w1, broadcast vectors declared before the loop, ...) and relies
on predeclared names (len, min, max, copy, uint64, the element types) and
on the simd and unsafe packages. Each can mean something else in the
function, and then the vector loop silently computes something else.
All of these already go wrong on master:
func F(dst, a []int32, _i int) { for i := 0; i < _i; i++ { ... } }
reads loopvec's _i in the loop condition and never runs; with a
package-level func len(s []int32) int the vector loop takes its window
length from that len; and a reduction written as for n := range n, or
for n := n - 1; n >= 0; n--, finishes with for n := _i; n < n; n++,
which never runs.
A loop that would otherwise be rewritten is now left alone, each with
its own reason, when its function uses an identifier that begins with an
underscore, other than _; when one of the names the code relies on,
looked up from the loop, is declared again around it, in the package, or
by an import; or when the loop declares such a name itself, or a name
its range operand or initial values use. The names the overlap helper
uses (uintptr, int, bool, any, false) are checked at package level,
where loopvec declares it. The lookup needs scopes, so the toolexec
wrapper records them. No loop in std, gonum or
github.com/thesyncim/goav1 stops being vectorized.
fixBlankParams works around a compiler bug that rejects _ parameters in
functions with simd code by renaming them _p0, _p1, ..., but it renamed
them in every function of a rewritten file. In a function that reads a
package-level _p0, the parameter then hid it:
var _p0 = 5
func Offset(_ int) int { return _p0 }
became func Offset(_p0 int) int { return _p0 } and returned its
argument. Only the functions loopvec rewrote are renamed now. They
mention no name that begins with an underscore, so the new names cannot
capture anything.
…an prove Every element-wise loop advanced one partial vector at a time: LoadPart and StorePart on every iteration, each working out how many elements remain. On arm64, which has no masked loads, that costs more than the operation itself (about 50 instructions per 4 lanes of AccumRowInt32s), and the bounds checks of slicing each operand at _i stay in the loop on every target. The loop now cuts every slice it touches to its window before it starts: the elements from the first iteration at that slice's offset, as many as the loop runs, with a capacity of as many, indexed from zero. _count is the first window's length, so the compiler knows it is not negative. Under conditions of the form _i <= _count-k*_lanes it then proves every slice the loop takes of a window in bounds and drops all bounds and length checks from the loop. The main loop runs four copies of the body per iteration over consecutive vectors, each copy all the statements in order, so the chunks run in the same order as with one vector per iteration; each window is cut to the iteration's four vectors, so every copy addresses its vector at a constant offset. Whole vectors then run one at a time, and one partial vector takes what is left. Integer reductions get the four accumulators float reductions have: an integer vector add takes two cycles on arm64, where they make SumInt32s three times as fast, and integer folds wrap, so regrouping them gives the same result. Because a partial load now only ever sees fewer elements than a vector holds, the emulated Load*Part of golang/go#82001, which returns len(s), returns the right count, and GODEBUG=simd=0 builds produce the scalar result.
Every rewritten loop with two or more slices calls _loopvecOverlap for each pair before it runs. The helper took the windows as ...int and inlining it cost 114, over the compiler's budget of 80, so each call was a real call: AddFloat32s made two before it loaded anything, which for a short loop costs about as much as the loop. The helper now takes the windows as four plain arguments, and a whole slice is passed as 0 and its length. It inlines (cost 55), and the comparison is the same.
…nt offset A loop that stores at an offset from the index, such as a row of a 2-D slice (b[i*ldb+j] = a[i*lda+j], c[row+j] += alpha*x[j]) or a fill that starts part way into a slice (line[off+x] = v), was rejected: loads could be at an offset, but every store had to be at exactly the loop index. A store may now be at the same offsets a load may: i+e, e+i and i-e for a loop-invariant integer e. Every store to one slice must be at the same offset, so each iteration still writes its own element; two offsets are rejected with their own reason. A load of the stored slice at that offset is the same element, ordered after the stores before it as at the index. The dependence test for a load at another offset measures the distance from the store offset instead of from the index, so a constant positive distance in a forward loop is still loaded before any store of the chunk, and a negative or variable distance is rejected as before. An if that selects between stores, and the copy builtin, take the offset too. The emitted store is capped at the limit plus the offset, so the vector loop never writes an element the scalar loop does not, and the length check and the overlap window cover the elements stored. A constant offset that puts every stored index below zero is left alone, as for a load. The vectorized set gains one loop in std (an image/jpeg fill), sixteen in gonum (dlacpy, dlaset, dlascl and dlarfb rows, and testlapack helpers) and five in github.com/thesyncim/goav1; TSVC s423 and s424 now vectorize and match their goldens.
thesyncim
force-pushed
the
offset-store-on-windows
branch
from
November 1, 2026 13:36
526d14a to
4649b62
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5; this PR's own changes.
Loads can already be at
i+e,e+iori-efor a loop-invariante. This allows the same for stores, which covers row-wise code like:Every store to a given slice has to use the same offset, so each iteration still writes its own element. Two different offsets are rejected with a new reason. For loads of a stored slice, the distance is now measured from the store offset instead of from
i. The same offset means the same element. A constant positive distance in a forward loop is loaded before the chunk's stores, as today. A negative distance, a non-constant one, or a loop that counts down is rejected. The store gets its own window, cut at the offset, so the vector loop never writes an element the scalar loop doesn't; the length check and the overlap window include the offset. The if-select and copy-builtin paths take the offset too, and a constant offset that puts the first stored index below zero is rejected, as #2 does for loads.With
loopvec -json -methods(I read every added loop), std goes from 28 to 29 vectorized loops (animage/jpegfill), gonum from 161 to 177 (dlacpy,dlaset,dlasclanddlarfbrows, plus some testlapack helpers), and goav1 from 81 to 86. TSVC s423 and s424 now vectorize and match their goldens. The generated code for loops that were already vectorized doesn't change.offset_store.txtruns 9 shapes over 14 sizes, plus short slices that must panic and aliased views that must fall back, and both builds have to print the same output.offset_store_no_rewrite.txtpins the rejected cases.