Testing Fallback Paths in CI

This page answers one task: an application has fallbacks — a JavaScript implementation when WebAssembly is unavailable, a single-threaded build when SharedArrayBuffer is missing, a scalar build when SIMD is not supported — and you want CI to prove they still work on every change.

Prerequisites

  • [ ] Playwright (npm i -D @playwright/test and npx playwright install chromium firefox).
  • [ ] An application whose loader records which implementation it chose (a global, a data attribute, or a log line).
  • [ ] The fallbacks themselves, built as part of the normal build.

Why fallbacks rot

A fallback path runs only for users the development team rarely is: people with WebAssembly disabled, pages without cross-origin isolation, older engines. Nobody on the team sees it break. A refactor renames a function in the main path and not the fallback; a new feature lands in the Wasm build only; a loader change makes the detection always pick the fast path. Months later, the users who depend on the fallback find a blank page, and the bug has no obvious commit to blame.

The remedy is to run the fallback paths the same way the main path runs: automatically, on every change, in real browsers, with assertions that the expected implementation was chosen and that the feature works. Browsers expose flags that switch off WebAssembly, threads or SIMD, which makes this straightforward.

A fallback test matrix Each row is a browser configuration used in CI, how it is produced, which implementation the loader should choose, and which checks run. configuration how expected implementation checks normal default browser wasm, threaded full suite no WebAssembly --js-flags=--noexpose-wasm JavaScript fallback smoke suite not isolated server without COOP/COEP wasm, single-threaded smoke suite no SIMD --js-flags=--no-experimental-wasm- simd scalar build smoke suite

Step 1 — make the choice observable

Tests cannot assert on what they cannot see. Have the loader publish which implementation it chose:

// loader.js
export async function loadEngine() {
  const choice = await chooseImplementation();      // "wasm-mt" | "wasm-st" | "wasm-scalar" | "js"
  document.documentElement.dataset.engine = choice;
  return createEngine(choice);
}

A data attribute on the root element is easy to read from Playwright and harmless in production. It also helps support: a screenshot of a bug report with DevTools open shows which path the user was on.

Step 2 — define browser projects with flags

Playwright projects run the same tests in different browser configurations. Chromium accepts V8 flags through --js-flags:

// playwright.config.ts
import { defineConfig, devices } from "@playwright/test";

export default defineConfig({
  webServer: { command: "npm run preview", port: 4173 },
  use: { baseURL: "http://localhost:4173" },
  projects: [
    { name: "normal", use: { ...devices["Desktop Chrome"] } },
    {
      name: "no-wasm",
      use: { ...devices["Desktop Chrome"], launchOptions: { args: ["--js-flags=--noexpose-wasm"] } },
      grep: /@smoke/,
    },
    {
      name: "no-simd",
      use: { ...devices["Desktop Chrome"], launchOptions: { args: ["--js-flags=--no-experimental-wasm-simd"] } },
      grep: /@smoke/,
    },
    { name: "firefox", use: { ...devices["Desktop Firefox"] }, grep: /@smoke/ },
  ],
});

--noexpose-wasm removes the WebAssembly global entirely, which is how a disabled browser looks to the page. Flags that disable individual proposals change names between V8 versions; check node --v8-options | grep wasm for the current spelling when a flag stops working. The grep: /@smoke/ lines limit the fallback projects to a short smoke suite, which keeps the matrix affordable.

Step 3 — assert the implementation and the behaviour

Each smoke test checks two things: that the loader chose the expected path for this configuration, and that the feature produces correct output on it.

// tests/checksum.spec.ts
import { test, expect } from "@playwright/test";

const expected: Record<string, RegExp> = {
  normal: /^wasm/, "no-wasm": /^js$/, "no-simd": /^wasm-scalar$/, firefox: /^wasm/,
};

test("checksum works on the chosen engine @smoke", async ({ page }, info) => {
  await page.goto("/upload");
  await expect(page.locator("html")).toHaveAttribute("data-engine", expected[info.project.name]);

  const value = await page.evaluate(async () => {
    const { getChecksummer } = await import("/src/checksum-loader.js");
    const c = await getChecksummer();
    return c.crc32c(new TextEncoder().encode("123456789"));
  });
  expect(value).toBe(0xe3069283);
});

The first assertion catches a detection bug — the fallback silently not being chosen; the second catches a broken fallback. Both are needed: a test that only checks output would pass if the main path were used in every configuration.

One smoke test across the matrix The same test runs once per browser configuration. It loads the page, asserts the data-engine attribute matches the configuration's expected implementation, then runs the feature on a known input and checks the output. launch config flags per project load page loader chooses engine assert data-engine right path chosen? run feature known input assert output right answer?

Step 4 — test the non-isolated path with a second server

Cross-origin isolation depends on response headers, not browser flags. Run a second preview server without the COOP and COEP headers and point a project at it:

// playwright.config.ts (additions)
webServer: [
  { command: "npm run preview", port: 4173 },                       // with isolation headers
  { command: "npm run preview -- --port 4174 --no-isolation", port: 4174 },
],
projects: [
  // ...
  { name: "not-isolated", use: { ...devices["Desktop Chrome"], baseURL: "http://localhost:4174" }, grep: /@smoke/ },
],

The expected implementation for this project is the single-threaded build described in handling browsers without SharedArrayBuffer. A small flag on the preview script that skips the headers is enough; the important thing is that both servers serve the same build.

Step 5 — keep the matrix affordable

Each project multiplies test time. Run the full suite once, in the normal configuration, and only smoke tests in the fallback configurations — a few tests per feature that exercise its main operation. Run the whole matrix on pull requests that touch loaders, build configuration or the fallback implementations, and on a nightly schedule otherwise. Parallel workers and sharding keep wall-clock time down further.

Simulating failures, not just absences

Flags simulate a browser without a feature; real users also meet features that exist and then fail. A module download that times out, a response with the wrong MIME type from a misconfigured CDN, an instantiation that runs out of memory — each should lead to the same graceful outcome as a missing feature, and each can be simulated in Playwright without special browsers:

test("falls back when the module cannot be fetched @smoke", async ({ page }) => {
  await page.route("**/*.wasm", (route) => route.abort("failed"));
  await page.goto("/upload");
  await expect(page.locator("html")).toHaveAttribute("data-engine", "js");
});

test("falls back when the module is served with the wrong type @smoke", async ({ page }) => {
  await page.route("**/*.wasm", async (route) => {
    const res = await route.fetch();
    await route.fulfill({ response: res, headers: { ...res.headers(), "content-type": "text/plain" } });
  });
  await page.goto("/upload");
  await expect(page.locator("html")).toHaveAttribute("data-engine", /^(wasm|js)/);   // works either way, never hangs
});

These tests are cheap, run in the normal browser project, and catch the class of bug where a loader waits forever on a promise that will never resolve.

What the matrix does not cover

Browser flags simulate the absence of a feature well, but not every real-world variant. Safari’s Lockdown Mode also disables the JavaScript JIT, which flags in Chromium do not reproduce; enterprise policies may disable WebAssembly in ways that surface as compile errors rather than a missing global; and real devices add memory limits that a desktop CI runner does not. Treat the CI matrix as the floor — it guarantees the paths are wired correctly and produce correct results — and supplement it with occasional manual checks on real configurations, especially for the slowest fallback, whose performance only a real device reveals. The detection logic those tests exercise is the subject of feature detecting Wasm at startup.

Expected output

Running 14 tests using 4 workers
  ✓ [normal] › checksum.spec.ts:5 › checksum works on the chosen engine @smoke (412ms)
  ✓ [no-wasm] › checksum.spec.ts:5 › checksum works on the chosen engine @smoke (388ms)
  ✓ [no-simd] › checksum.spec.ts:5 › checksum works on the chosen engine @smoke (401ms)
  ✓ [not-isolated] › checksum.spec.ts:5 › checksum works on the chosen engine @smoke (395ms)
  ✓ [firefox] › checksum.spec.ts:5 › checksum works on the chosen engine @smoke (530ms)
  ...
  14 passed (18.2s)

Gotchas

  • The no-Wasm project passes because the main path still loads. The flag was misspelled and ignored. The data-engine assertion catches this; never test output alone.
  • Flags change between browser versions. Pin the Playwright browser version and re-check flags when upgrading.
  • Fallback tests are skipped locally and fail in CI. Developers run only the default project. Make npm test run the smoke matrix too.
  • The non-isolated server serves a different build. Both servers must serve identical files; only headers should differ.

Performance note

The matrix above added 11 seconds to a pull request’s test job — four smoke projects of a handful of tests each — against 52 seconds for the full suite in the normal configuration. It caught two regressions in its first month: a renamed export missing from the JavaScript fallback, and a loader change that sent non-isolated pages to the threaded build.

Test time added by each fallback project Wall-clock time per Playwright project in a pull request job: the full suite in the normal configuration and the smoke suite in each fallback configuration. seconds per project in CI normal (full suite) 52 s no-wasm (smoke) 3.1 s no-simd (smoke) 2.6 s not-isolated (smoke) 2.9 s firefox (smoke) 4.4 s

Frequently Asked Questions

Can I test with WebAssembly disabled in Firefox through Playwright? Yes — pass firefoxUserPrefs: { "javascript.options.wasm": false } in the project’s launch options.

Do these tests need the production build? Yes, or something very close to it. Fallback bugs often live in build configuration — a fallback file not emitted, a hashed name not rewritten — which a dev server hides. Test against the built output served by a preview server.

What about WebKit? Playwright’s WebKit build can be included as another project. It is a reasonable proxy for Safari’s engine, though not for Lockdown Mode.

Should unit tests cover fallbacks too? Yes — run the same unit tests against both implementations, as in shipping a JavaScript fallback for a Wasm feature. The browser matrix then checks wiring and detection.

How do I simulate a failed module download? Use Playwright’s page.route to abort requests for .wasm files and assert that the loader falls back rather than hanging.

← Back to Polyfill Alternatives & Fallbacks