Skip to content

feat(v18): accept instantiateWasm and locateFile in loadModule() - #191

Closed
Gldywn wants to merge 1 commit into
constructive-io:mainfrom
Gldywn:feat/load-module-options
Closed

Gldywn wants to merge 1 commit into
constructive-io:mainfrom
Gldywn:feat/load-module-options

Conversation

@Gldywn

@Gldywn Gldywn commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Summary

libpg-query can only get its WASM through Emscripten's default loader, which fetches libpg-query.wasm (or reads it from disk under Node). A host that cannot fetch its own files has no way in: the wrapper starts that load at import time, and loadModule() takes no options.

We hit this in Gatekeeper, a Beekeeper Studio plugin that parses SQL with libpg-query. Plugin pages are served from Electron's custom plugin:// scheme, which Beekeeper registers without supportFetchAPI, so fetch on the plugin's own files fails (beekeeper-studio/beekeeper-studio#4864, item 5). The generated factory already reads Module.instantiateWasm and Module.locateFile (it does not read Module.wasmBinary), and instantiating the module from bytes bundled in the page's JS works in Beekeeper. The only thing missing is a way to give the factory that hook.

This PR adds it, for v18 (the full-API template):

await loadModule({
  instantiateWasm: async (imports) =>
    (await WebAssembly.instantiate(wasmBytes, imports)).instance,
});
  • Options: loadModule(options?) accepts instantiateWasm(imports) => Promise<instance> and locateFile(path, scriptDirectory). locateFile is forwarded to Emscripten unchanged. An unknown key next to a hook, or a hook that is not a function, rejects with a TypeError.
  • No load at import: nothing loads when the package is imported. The first loadModule() or async call (parse, scan, ...) starts the load, so its options apply to it.
  • Late options: options are only accepted by the call that starts the first load. After that, a call that sets a hook rejects with initialization already started, and every other call joins. Late options are never silently ignored and never restart the load.
  • Reloads: the options are reused for every load. That includes retrying a failed load and the fresh instance fix(wasm): replace the module after an exception escapes a WASM call #175 loads after a WASM call throws. Without that, a fetch-less host would lose its module on the first crash, because the reload would go through the default loader.
  • Fresh instance per call: the hook must therefore create a new instance from the imports it receives each time. An instance it already returned is rejected. A cached instance would bring back the one the exception corrupted, with its imports still bound to the previous module; the result looks recovered but returns empty results.
  • Failing hooks: Emscripten's instantiateWasm path has no failure callback, so the wrapper races the factory against the hook. A hook that rejects, throws, or resolves without exports rejects loadModule() instead of leaving it pending forever.
  • Callbacks: loadModule keeps arity 0 and a zero-argument overload, so before(loadModule), beforeAll(loadModule) and promise.then(loadModule) work and type-check as before. An argument that sets neither hook (a test runner's context, a promise value, {}) counts as no options.
  • Types: the public hook types are structural, so wasm/index.d.ts still compiles without the dom lib.

Behaviour change. Because nothing loads at import, a sync call (parseSync, scanSync, ...) made without ever awaiting loadModule() or an async method now always throws WASM module not initialized. Call loadModule() first. Before, such a call worked once the import-time load happened to finish.

  • Affected: the README's parseSync example and the @pgsql/parser docs for its version exports (v18, @pgsql/parser/v18) both used that pattern. Both are fixed here.
  • Not affected: the Parser class, which awaits loadModule().
  • Release notes: this belongs there.
  • Side effect: importing the package no longer allocates the module's initial memory.

Versions 13 to 17 (slim template) are unchanged.

Evidence

Node 24.22.1, TypeScript 5.8.3, run against the published 18.1.5 libpg-query.{js,wasm}. The WASM was not rebuilt here; CI rebuilds it.

$ cd versions/18 && node --test test/parsing.test.js test/pg18.test.js test/fingerprint.test.js test/normalize.test.js test/plpgsql.test.js test/scan.test.js test/errors.test.js test/input-encoding.test.js test/wasm-recovery.test.js test/load-module.test.js
ℹ tests 162
ℹ pass 162
ℹ fail 0

The 162 tests are the 92 existing ones, input-encoding (11), wasm-recovery (40, including its node -e cases) and the new load-module (19).

  • Against main's wrapper: 14 of the 19 new tests fail. The 5 that pass are the compatibility cases: concurrent calls, parse() without loadModule(), arguments without a hook, and before(loadModule) in two suites.
  • The Gatekeeper case: after parse() throws on deeply nested SQL, the rebuild goes through the same instantiateWasm hook (2 hook calls, 2 factory calls). A hook that returns its cached instance is refused, instead of leaving a module that returns "" from fingerprintSync('SELECT 1').
  • Hook race: removing the race against the hook makes the 3 tests where the hook fails fail.
  • Publish before start: starting the factory before publishing the pending load makes the re-entrant locateFile test fail.
  • Synchronous start: deferring the start by a microtask makes 13 wasm-recovery cases fail, because they require a failing sync call to start its fresh instance at once.
  • Types: checked with tsc --noEmit, strict and skipLibCheck: false, on small consumers of wasm/index.d.ts.
    • With lib: ["es2022"] and types: [], it compiles.
    • With dom, hooks typed with the real WebAssembly.Imports and WebAssembly.Instance compile, and so does the README example; the { instance, module } pair is a type error.
    • With @types/node, before(loadModule) and .then(loadModule) compile.
  • Not run: pnpm test:mutations. By reading, all of its patterns still match the compiled index.cjs.

Hosts that cannot fetch their own files (an Electron custom protocol
registered without supportFetchAPI, as for Beekeeper Studio plugins)
had no way to hand libpg-query its WASM: the module started loading
at import and loadModule() took no options.

- Nothing loads at import; the first loadModule() or async call starts
  the load, so its options apply.
- loadModule(options?) accepts instantiateWasm, adapted to Emscripten's
  callback form and raced against the hook's failure, and locateFile,
  forwarded unchanged. A call that sets a hook after the first load
  started rejects with "initialization already started".
- The options are reused for every load, including retries and the
  fresh instance loaded after a WASM exception; an instance the hook
  already returned is rejected.
- loadModule keeps arity 0 and a zero-argument overload, so it can
  still be passed as a callback; the hook types are structural, so the
  declarations need no dom lib.
- Docs in the v18 README, LOADING_WASM.md and the @pgsql/parser README;
  tests in versions/18/test/load-module.test.js.
@devin-ai-integration

Copy link
Copy Markdown

Thanks @Gldywn, and thanks for the clear Beekeeper write-up. We reproduced the fetch-less plugin:// failure and shipped a fix in #193, which is merged and credits you as co-author. It covers v13–18 and keeps the current import-time loading. A host that can't fetch the file passes the bytes, or a compiled module, once:

await loadModule({ wasm }); // ArrayBuffer, typed array, or WebAssembly.Module

Every later instance is built from it, including the one created after a failed parse, so nothing fetches the .wasm again. Closing this in favour of #193. It'll be in the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant