Skip to content

feat: accept the .wasm bytes or a compiled module in loadModule() - #193

Merged
pyramation merged 2 commits into
mainfrom
devin/1793298512-load-module-wasm-main
Oct 29, 2026
Merged

pyramation merged 2 commits into
mainfrom
devin/1793298512-load-module-wasm-main

Conversation

@pyramation

Copy link
Copy Markdown
Collaborator

Summary

Some hosts can't fetch or read the package's own libpg-query.wasm, for example an Electron custom protocol registered without supportFetchAPI (Beekeeper Studio plugins) or a bundle that inlines its assets. Until now those hosts couldn't load the parser at all, because loadModule() ignored its arguments. With this change they hand the module over once:

import { loadModule, parseSync } from 'libpg-query';

await loadModule({ wasm }); // ArrayBuffer, typed array, or a compiled WebAssembly.Module
parseSync('SELECT 1');
  • The bytes are compiled once into provided. Each later instance is created from it through Emscripten's instantiateWasm, using async WebAssembly.instantiate and no fetch. That includes the replacement instance after a failed call (createModule() in instantiate()).
  • If an instance is already loaded from the file, or one is loading, it is swapped for one built from wasm.
  • Nothing else changes: import-time loading, parseSync and the stale-instance recovery all behave as before. loadModule.length stays 0, via a rest parameter.
  • The change goes in templates/index.ts (v13–17) and templates/full/index.ts (v18), and the generated versions/*/src/index.ts files were regenerated with copy:templates.

Builds on the approach in #191 by @Gldywn, who is credited as co-author.

Tests

versions/*/test/load-module.test.js runs the wrapper from a copy that has no libpg-query.wasm next to it:

  • On main: 3 of 6 fail.
  • With this change: all pass.

Every version's WASM was built locally and each full suite passed with 0 failures: v18 149/149, v13–17 78 pass / 20 skipped (the full-API tests, which are skipped in the slim builds).

I also checked it in Electron 37 on a plugin:// scheme without supportFetchAPI. A parse succeeded, a deeply nested query threw, and the next parse succeeded again: {"ok":true,"first":1,"threw":true,"afterRecovery":1}.

Link to Devin session: https://app.devin.ai/sessions/9bbf3e7a34e9477ea812446ccf8a5fb0
Open in Devin Desktop: https://app.devin.ai/desktop/session/9bbf3e7a34e9477ea812446ccf8a5fb0?variant=devin
Requested by: @pyramation

Hosts that cannot fetch or read libpg-query.wasm (a custom protocol without
fetch support, a bundle that inlines assets) can pass it once with
loadModule({ wasm }). Every later instance, including replacements after a
failed call, is created from it. Applies to v13-v18.

Co-authored-by: Gldywn <14254051+Gldywn@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@tenki-reviewer

tenki-reviewer Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Review complete. 🟠 1 high · 🟡 1 medium

💬 Inline comments (2)


The change threads a caller-provided WASM module through Emscripten's instantiateWasm hook and adds a module-wide load/recovery singleton (loadModule, stale handling, failure counters), mirrored byte-for-byte in templates/, templates/full/, and versions/13-18. READMEs and LOADING_WASM.md document the new option, and each version's package.json gains a load-module test that spawns a fresh node subprocess to exercise OOM/reload behavior.

Files Change
templates/index.ts, templates/full/index.ts, versions/*/src/index.ts Add loadModule accepting wasm bytes/WebAssembly.Module, wiring them into instantiateWasm and the stale-recovery loop
versions/*/package.json Add the test/load-module.test.js test target to each versioned package
versions/*/test/load-module.test.js New node:test harness spawning a subprocess to verify loadModule recovery under memory pressure
versions/*/README.md, LOADING_WASM.md Document the loadModule({ wasm }) option and its constraints

Note: the new wasm-supplied load path has a high-severity defect (the Emscripten receive callback gets the instantiate result record instead of the WebAssembly.Instance), so loads with a caller-provided module currently fail to initialize.

Reviewed commit: 4640ae9

@tenki-reviewer tenki-reviewer Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds a loadModule({ wasm }) API letting callers supply pre-compiled WASM bytes or a WebAssembly.Module, copied identically across all six versioned packages, plus a subprocess-based test harness for WASM load recovery.

Key findings

Comment thread versions/15/src/index.ts
Comment thread versions/13/test/load-module.test.js Outdated
@pyramation
pyramation merged commit 2b4178e into main Oct 29, 2026
58 checks passed
devin-ai-integration Bot pushed a commit that referenced this pull request Oct 29, 2026
Provided wasm sets compiled, so the sync replaceInstance recovery path reuses it without fetching; load-module OOM test asserts the repeated-limit error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant