Skip to content

Fix 32-bit overflow in exponent-vector indexing - #355

Open
dcposch wants to merge 1 commit into
algebraic-solving:masterfrom
dcposch:fix-int32-exp-overflow
Open

Fix 32-bit overflow in exponent-vector indexing#355
dcposch wants to merge 1 commit into
algebraic-solving:masterfrom
dcposch:fix-int32-exp-overflow

Conversation

@dcposch

@dcposch dcposch commented Aug 23, 2026

Copy link
Copy Markdown

On inputs whose total number of terms times the number of variables exceeds INT32_MAX, msolve segfaults while parsing: (pos+j)*nvars is evaluated in 32-bit signed arithmetic, so store_exponent() is called with a negative offset (gdb: pos=-2147477296 at iofiles.c:58, called from get_coefficient_ff_and_term_from_line).

This patch uses 64-bit indexing for exponent-vector offsets and allocations (parser, F4 import, and the variable-order swap).

Minimized reproducer (10000 variables, 214750 terms of x0 over a prime field, ~700KB): unpatched 0.10.1 dies with the backtrace above; patched computes a Groebner basis. make check: 64/64 pass.

Co-Authored-By: Grok via the JC2 campaign
@dcposch

dcposch commented Aug 23, 2026

Copy link
Copy Markdown
Author

Minimized reproduction example, Python generator:

nvars, nterms = 10000, 214750  # last index × nvars just exceeds INT32_MAX
open("repro.ms","w").write(
    ",".join(f"x{i}" for i in range(nvars)) + "\n105337\n"
    + "+".join(["x0"] * nterms) + "\n")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant