Handling Input and Audio in a Wasm Game

This page answers one task: a game whose logic runs in WebAssembly needs responsive keyboard, mouse, touch and gamepad input, and sound that plays on every browser — including mobile Safari, which blocks audio until the player interacts.

Prerequisites

  • [ ] A game loop driven by requestAnimationFrame calling into the module once per frame, as in porting a C game loop to Emscripten.
  • [ ] A canvas element that hosts the game.
  • [ ] Familiarity with Web Audio’s AudioContext.

Events arrive between frames; the game reads them once per frame

Browsers deliver input as events — keydown, pointermove, pointerdown — whenever they happen, between animation frames. The game simulation, inside Wasm, advances in discrete frames. Calling into the module from every event handler works, but it couples the simulation to event timing and makes input handling order-dependent. The robust pattern is to buffer input in JavaScript (or in a small shared buffer in Wasm memory) as events arrive, and let the game read a snapshot at the start of each frame: which keys are down, which were pressed or released since the last frame, where the pointer is, how far it moved.

Gamepads are different: there are no events for button presses, so the game polls navigator.getGamepads() once per frame. Audio is different again: it must be started from inside a user-gesture event handler, or the browser keeps it suspended.

Input flowing into a Wasm game loop Keyboard and pointer events are recorded into an input buffer as they arrive. Once per animation frame, JavaScript polls gamepads, copies the buffered state into Wasm memory and calls the module's frame function, which reads a consistent snapshot. The buffer's per-frame edges are then cleared. DOM events keydown, pointermove … input buffer down set, pressed, released, deltas rAF: poll gamepads navigator. getGamepads() frame(dt) Wasm reads snapshot clear edges ready for next frame

Step 1 — buffer keyboard input by physical key

Use event.code (the physical key, such as KeyW or ArrowLeft), not event.key (the produced character), for game controls: WASD stays in the same place on AZERTY and Dvorak layouts. Map codes to small integers and keep three sets: keys currently down, keys pressed this frame, keys released this frame.

const KEYS = { KeyW: 0, KeyA: 1, KeyS: 2, KeyD: 3, Space: 4, ShiftLeft: 5, Escape: 6 };
const down = new Uint8Array(32), pressed = new Uint8Array(32), released = new Uint8Array(32);

addEventListener("keydown", (e) => {
  const k = KEYS[e.code]; if (k === undefined) return;
  if (!down[k]) pressed[k] = 1;
  down[k] = 1;
  e.preventDefault();                                      // stop Space from scrolling the page
});
addEventListener("keyup", (e) => { const k = KEYS[e.code]; if (k !== undefined) { down[k] = 0; released[k] = 1; } });
addEventListener("blur", () => down.fill(0));              // avoid stuck keys when focus leaves

Clearing all keys on blur prevents the classic stuck-key bug: a key held while the player alt-tabs never sends keyup. Ignore e.repeat for actions that should trigger once per press.

Step 2 — pointer input and pointer lock

Use Pointer Events, which unify mouse, pen and touch. Accumulate movement deltas between frames; for first-person controls, request pointer lock on a click so the cursor is hidden and movement is unbounded:

let dx = 0, dy = 0, buttons = 0;
canvas.addEventListener("pointermove", (e) => { dx += e.movementX; dy += e.movementY; });
canvas.addEventListener("pointerdown", (e) => { buttons = e.buttons; canvas.setPointerCapture(e.pointerId); });
canvas.addEventListener("pointerup", (e) => { buttons = e.buttons; });
canvas.addEventListener("click", () => canvas.requestPointerLock?.({ unadjustedMovement: true }));
canvas.style.touchAction = "none";                          // stop the browser from panning on touch

unadjustedMovement asks for raw mouse input without OS acceleration where supported. For touch controls, track pointers by pointerId to support multiple fingers — a virtual joystick on the left, buttons on the right.

Step 3 — copy the snapshot into Wasm once per frame

function frame(t) {
  const pads = navigator.getGamepads?.() ?? [];
  const input = new Int32Array(wasm.memory.buffer, wasm.input_ptr(), 16);
  input[0] = packBits(down); input[1] = packBits(pressed); input[2] = packBits(released);
  input[3] = dx; input[4] = dy; input[5] = buttons;
  const gp = pads[0];
  if (gp) { input[6] = Math.round(gp.axes[0] * 32767); input[7] = Math.round(gp.axes[1] * 32767); input[8] = gp.buttons[0].pressed ? 1 : 0; }
  wasm.frame(t);
  pressed.fill(0); released.fill(0); dx = dy = 0;
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);

One small write into a fixed input struct in linear memory per frame replaces dozens of boundary calls. Apply a dead zone to analogue sticks (ignore values below about 0.15) in the game code; raw axes rarely rest at exactly zero.

The per-frame input struct in linear memory A fixed 64-byte struct in Wasm memory holds bitmasks for keys down, pressed and released, pointer deltas and buttons, and the first gamepad's axes and buttons. JavaScript fills it once per frame before calling the frame function. input struct (16 × i32) at input_ptr() keys down pressed released dx dy buttons gamepad spare +0 +12 +64

Step 4 — unlock audio on the first gesture

Browsers create AudioContexts in the suspended state unless they were created or resumed during a user gesture. Show a start screen and resume audio in its click handler:

const audio = new AudioContext({ latencyHint: "interactive" });
startButton.addEventListener("click", async () => {
  await audio.resume();                                      // must happen inside the gesture
  startGame();
}, { once: true });

iOS Safari is the strictest: audio stays silent until resumed in a touchend or click handler, and the silent switch may mute Web Audio. Resume again on visibilitychange when the page returns to the foreground, since some browsers suspend contexts in the background.

Step 5 — play sound effects with low latency

Decode sound effects once into AudioBuffers at load time and trigger them with AudioBufferSourceNodes — cheap, one-shot nodes designed for exactly this:

const buffers = await Promise.all(["jump", "hit", "coin"].map(async (n) =>
  audio.decodeAudioData(await (await fetch(`/sfx/${n}.ogg`)).arrayBuffer())));

export function playSfx(id, gain = 1) {                    // imported by the Wasm module
  const src = audio.createBufferSource();
  src.buffer = buffers[id];
  const g = audio.createGain(); g.gain.value = gain;
  src.connect(g).connect(audio.destination);
  src.start();
}

The module calls playSfx through an import when game events happen. For music, use a streaming <audio> element routed through a MediaElementAudioSourceNode, or a decoded buffer for short loops. Games that synthesise audio in Wasm should run the synthesiser in an AudioWorkletProcessor, as in running a DSP kernel in an AudioWorklet.

Fixed timesteps and input timing

Many games simulate at a fixed rate — 60 or 120 steps per second — independent of the display’s refresh rate, running zero, one or several simulation steps per animation frame. Input snapshots must fit that model. Pressed and released edges belong to the first simulation step after they occurred, or a quick tap that falls between steps is lost; held state applies to every step. Pointer deltas should be divided among the steps of a frame or applied once to the camera outside the simulation. On high-refresh displays at 120 or 144 Hz, requestAnimationFrame fires more often than the simulation steps, so some frames run no step at all — keep accumulating input until a step consumes it. Recording the per-step input snapshots also gives replays and deterministic debugging almost for free: feed the recorded snapshots back into the same module and the game reproduces exactly, which is one of the pleasant properties of a deterministic Wasm simulation.

Input across devices and accessibility

A game that only supports one input method loses players. Map actions rather than keys in the game code — “jump”, “fire”, “move left” — and keep a binding table from keys, pointer buttons and gamepad buttons to actions in JavaScript or in the module, so players can rebind controls and every device drives the same logic. Show on-screen touch controls only when touch is detected (pointerType === "touch" on the first event), and hide them when a gamepad connects. Listen for gamepadconnected to switch prompts from keyboard glyphs to controller buttons. Respect accessibility settings where possible: prefers-reduced-motion can disable screen shake, and offering hold-to-toggle alternatives helps players who cannot hold buttons. None of this needs to live in Wasm; keeping the device layer in JavaScript and the action layer in the game keeps both simple.

Expected output

Keyboard, mouse, touch and an Xbox or PlayStation controller all control the game with no perceptible lag; WASD works on an AZERTY keyboard via physical key codes; no keys stick after alt-tabbing; and sound effects play within about 20 ms of their trigger on desktop, after the start screen unlocks audio on iOS.

Gotchas

  • Using event.key for movement. Layout-dependent. Use event.code.
  • Stuck keys after focus loss. Clear key state on blur and visibilitychange.
  • Silent audio on mobile. Resume the AudioContext inside a user gesture.
  • Page scrolling or zooming on input. Call preventDefault for game keys and set touch-action: none on the canvas.
  • One boundary call per event. Buffer events and copy a snapshot once per frame.

Performance note

Copying the 64-byte input snapshot into Wasm cost under 1 µs per frame. Dispatching each input event as its own Wasm call cost more and, more importantly, made input timing depend on event order within a frame. Measured input-to-photon latency was about one frame (16.7 ms) plus the display’s latency.

Sound-effect trigger latency by approach Milliseconds from a game event to audible output on desktop Chrome for AudioBufferSourceNode with an interactive latency hint, an HTML audio element, and an AudioContext with the default latency hint. ms from trigger to sound AudioBufferSource, interactive 18 ms AudioContext default hint 42 ms HTML audio element 95 ms

Frequently Asked Questions

Should input be handled in a worker? Input events only reach the main thread. Buffer them there and pass snapshots to a worker-hosted simulation through shared memory if needed.

How do I support controller vibration? Use gamepad.vibrationActuator.playEffect("dual-rumble", …) where supported.

Why does the gamepad not show up? Browsers expose gamepads only after a button press on that page, for privacy.

Can Emscripten’s SDL handle all of this? SDL2 and GLFW ports wrap these browser APIs, including gamepads and audio; the same browser rules about gestures and focus still apply.

How do I handle keyboard input in text fields inside the game? Use a hidden <input> element focused while typing, so the browser handles IME and composition, and pass the resulting text to the module.

← Back to Graphics, Games & Simulation