Skip to content

Run scenarios in parallel workers on Windows #427

Open

Description

@schlessera

Part of #394. Follow-up to #390.

bin/run-behat-tests turns parallel workers off on Windows (MINGW*|MSYS*|CYGWIN*) PARALLEL="";;), and the Docker mode is unavailable there as well, so the Windows Behat job runs every scenario one after the other. Since the Linux jobs run in 4 parallel workers, the Windows job is now the longest job of most large suites. In the full-matrix runs of 2026-11-02:

Package Longest Linux job macOS job (4 workers) Windows job (serial)
entity-command 9.8 min 23.2 min 53.0 min
extension-command 8.4 min 12.9 min 28.7 min
core-command 5.3 min 4.7 min 11.4 min
media-command 4.9 min 6.9 min 11.0 min
scaffold-command 5.4 min 3.4 min 9.1 min
language-command 4.4 min 3.4 min 8.8 min

The Windows job is continue-on-error, so it does not block a pull request. It still sets how long the Testing workflow takes to finish, though. In entity-command, the job took 53 minutes for 498 scenarios. The macOS job already runs in parallel workers on the runner and went from about 26 to 23 minutes.

The Windows job uses SQLite, so the worker databases are not an obstacle. #405 does not say why Windows was left out. Things to check before turning it on:

  • process handling in utils/behat-parallel.php (starting the workers, collecting their output and exit codes, and signals, see Remove the Docker containers of runs that were stopped or killed #418) with Git Bash and the Windows PHP build;
  • per-worker temporary directories and TMPDIR, as Windows uses TEMP/TMP, and the runner moves temporary files to the D: drive;
  • run directories are still removed right away on Windows (Trim fixed per-scenario overhead #403), which may make workers slower than expected;
  • scenarios that use fixed ports or background processes, which already run in the first worker.

Measuring: run the Windows job of entity-command and extension-command with WP_CLI_TEST_PARALLEL=4 on a draft branch, and compare Behat time and results with the serial runs of 2026-11-02. Check the first parallel pass, not only the result after composer behat-rerun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions