Vectorizing Image Convolution with SIMD

This page answers one task: you apply a small convolution kernel — blur, sharpen, edge detection — to images in WebAssembly, and the scalar loop is too slow for real-time use. You want to process several pixels per instruction with Wasm SIMD while producing exactly the same output as the scalar version.

Prerequisites

  • [ ] A working scalar convolution in Rust or C compiled to Wasm.
  • [ ] A SIMD-enabled build (-C target-feature=+simd128 in Rust or -msimd128 in clang/Emscripten) and a scalar fallback.
  • [ ] Test images and a byte-for-byte comparison of outputs.

How convolution maps onto SIMD

A 3×3 convolution computes each output pixel as a weighted sum of the nine input pixels around it. The work per pixel is small, but it repeats for millions of pixels, and neighbouring output pixels use overlapping inputs. SIMD exploits that: a 128-bit vector holds 16 bytes, so one load fetches 16 neighbouring pixels of a single-channel row (or 4 RGBA pixels). Shifting the load by one byte gives the left and right neighbours of all 16 pixels at once. Multiplying each shifted vector by its kernel weight and adding produces 16 partial sums per instruction instead of one.

The catch is precision. Pixels are u8, but weighted sums overflow 8 bits, so the vector must be widened — each 16×u8 vector split into two 8×i16 vectors (or four 4×f32 vectors for fractional kernels) — before multiplying. Results are narrowed back to u8 with saturation at the end.

Vectorized 3×3 convolution for 16 pixels Load 16 pixels from each of three rows at three horizontal offsets. Widen each vector from u8 to two i16 vectors. Multiply each by its kernel weight and accumulate. Shift right to divide by the kernel sum, then narrow back to u8 with saturation and store 16 output pixels. load 3 rows × 3 offsets v128.load widen u8 → i16 extend_low / high multiply + accumulate i16x8.mul, add scale shr or mul by recip narrow + store u8x16.narrow_i16x8

Step 1 — start from a clear scalar kernel

pub fn blur3x3_scalar(src: &[u8], dst: &mut [u8], w: usize, h: usize) {
    for y in 1..h - 1 {
        for x in 1..w - 1 {
            let mut s = 0u32;
            for dy in 0..3 { for dx in 0..3 {
                s += src[(y + dy - 1) * w + x + dx - 1] as u32 * K[dy][dx] as u32;
            }}
            dst[y * w + x] = (s >> 4) as u8;      // kernel [1 2 1; 2 4 2; 1 2 1] sums to 16
        }
    }
}
const K: [[u8; 3]; 3] = [[1, 2, 1], [2, 4, 2], [1, 2, 1]];

This Gaussian-like kernel has integer weights summing to 16, so the division is a shift — ideal for integer SIMD. Keep this function as the reference for tests.

Step 2 — process 16 pixels per iteration

use core::arch::wasm32::*;

#[target_feature(enable = "simd128")]
unsafe fn row_simd(r0: *const u8, r1: *const u8, r2: *const u8, out: *mut u8) {
    let mut lo = i16x8_splat(0);
    let mut hi = i16x8_splat(0);
    for (row, w) in [(r0, [1, 2, 1]), (r1, [2, 4, 2]), (r2, [1, 2, 1])] {
        for dx in 0..3 {
            let v = v128_load(row.add(dx) as *const v128);       // 16 pixels at offset dx - 1
            let k = i16x8_splat(w[dx]);
            lo = i16x8_add(lo, i16x8_mul(u16x8_extend_low_u8x16(v), k));
            hi = i16x8_add(hi, i16x8_mul(u16x8_extend_high_u8x16(v), k));
        }
    }
    let lo = u16x8_shr(lo, 4);
    let hi = u16x8_shr(hi, 4);
    v128_store(out as *mut v128, u8x16_narrow_i16x8(lo, hi));
}

The pointers point one pixel left of the first output pixel, so offsets 0, 1, 2 give the left, centre and right neighbours. The maximum sum (255 × 16 = 4080) fits in 16 bits, so i16x8 arithmetic is exact. v128_load accepts unaligned addresses in Wasm, so no alignment work is needed.

Step 3 — use a separable kernel

Many kernels are separable: the 3×3 blur equals a horizontal [1 2 1] pass followed by a vertical [1 2 1] pass. Two passes of three multiply-adds each (six per pixel) replace nine, and the vertical pass loads whole aligned rows without horizontal shifts. For larger kernels — 5×5, 7×7 Gaussian blurs — separability is the biggest win, often larger than SIMD itself. Keep intermediate results in u16 between passes to stay exact.

Direct 3×3 kernel versus separable passes A direct 3×3 kernel does nine multiply-adds per pixel in one pass. A separable kernel does three horizontal and three vertical multiply-adds in two passes with an intermediate buffer, reducing arithmetic and scaling much better for larger kernels. direct 3×3 9 multiply-adds per pixel one pass, no temp buffer any kernel shape non-separable kernels separable 1×3 + 3×1 6 multiply-adds per pixel two passes, u16 temp row grows linearly with size blurs and Sobel

Step 4 — handle borders and tails

The SIMD loop covers interior pixels in blocks of 16. Two leftovers remain: the border (the first and last row and column, where the kernel extends outside the image) and the row tail (when width - 2 is not a multiple of 16). Process both with the scalar kernel, or pad the image by one pixel on each side (replicating edge pixels) so the SIMD loop covers everything, and process the final partial block with an overlapping last vector that starts 16 pixels before the row end — recomputing a few pixels is harmless because the output is the same.

Step 5 — verify and measure

Compare SIMD and scalar outputs byte for byte on random images, including widths of 1–40 pixels to exercise tails. Then time both on realistic sizes in the browser, with a warm-up to let the engine tier up. Ship SIMD and scalar builds and select at startup by feature detection, so older engines still work.

Floating-point kernels

Kernels with fractional or negative weights (sharpen, Laplacian, arbitrary user kernels) are easier in f32x4: widen u8 to i32x4 twice, convert to f32x4, multiply-accumulate, then convert back with i32x4_trunc_sat_f32x4 and narrow twice with saturation. Four lanes instead of eight make it slower than the i16 path, but it handles any kernel, and saturation keeps negative results clamped to 0 and large ones to 255.

Driving the row kernel

The row function handles 16 output pixels; a short driver walks the image, calls it for every full block, and finishes each row with the scalar kernel:

pub fn blur3x3(src: &[u8], dst: &mut [u8], w: usize, h: usize) {
    for y in 1..h - 1 {
        let (r0, r1, r2) = (&src[(y - 1) * w..], &src[y * w..], &src[(y + 1) * w..]);
        let mut x = 1;
        while x + 16 <= w - 1 {                       // 16 outputs need inputs up to x + 16
            unsafe { row_simd(r0[x - 1..].as_ptr(), r1[x - 1..].as_ptr(),
                              r2[x - 1..].as_ptr(), dst[y * w + x..].as_mut_ptr()); }
            x += 16;
        }
        for x in x..w - 1 { dst[y * w + x] = scalar_pixel(src, w, x, y); }   // row tail
    }
}

The loop condition guarantees that the right-neighbour load (x - 1 + 2 + 15) stays inside the row, so the kernel never reads into the next row’s pixels. Keep the bounds reasoning written next to the loop: off-by-one mistakes in SIMD drivers produce subtly wrong edges rather than crashes, and only a byte-for-byte comparison catches them.

Memory layout and passing images from JavaScript

Image data arrives from a canvas as RGBA in a Uint8ClampedArray. Copy it once into Wasm memory (allocate with an exported allocator, then new Uint8Array(memory.buffer, ptr, len).set(pixels)), run the kernel, and copy the result back into an ImageData. For single-channel kernels, convert to greyscale or split channels inside Wasm — also with SIMD — rather than in JavaScript. For video, keep input and output buffers allocated across frames so each frame costs two copies and the kernel, with no allocation.

Splitting work across workers

Rows are independent, so a large image splits naturally into horizontal bands, one per worker, each band overlapping its neighbours by one row so the kernel has the inputs it needs. With a shared WebAssembly.Memory and threads, the bands are processed in place without copies; without threads, each worker receives its band (plus the overlap) and returns the result. SIMD and threads multiply: a 9× SIMD gain on four cores approaches the memory bandwidth limit, at which point further arithmetic speedups stop paying off and fusing several filters into one pass becomes the next step.

Expected output

A 1920×1080 single-channel 3×3 blur produces output identical to the scalar kernel, including borders and row tails; the SIMD version processes 16 pixels per iteration with exact i16 arithmetic; the separable variant is faster again; and browsers without SIMD load the scalar build.

Gotchas

  • Overflow in narrow lanes. Weighted sums exceed u8. Widen before multiplying.
  • Signed narrowing of unsigned data. Use the narrowing variant that saturates correctly for your range.
  • Forgotten tails. Widths that are not multiples of 16 leave pixels unprocessed. Handle or overlap.
  • Reading past the row. Unpadded images make the last vector load the next row. Pad or stop early.
  • No scalar fallback. Engines without SIMD fail to compile the module.
  • Allocating buffers per frame. Allocation dominates small kernels. Reuse input and output buffers.

Performance note

For a 1920×1080 single-channel blur in Chrome, the i16x8 SIMD kernel ran about 6× faster than the scalar kernel, and the separable SIMD version about 9× faster.

3×3 blur on a 1920×1080 image Milliseconds per frame for a 3×3 blur on a 1920 by 1080 single-channel image with a scalar Wasm kernel, a direct SIMD kernel using i16 lanes, and a separable SIMD kernel. ms per frame scalar Wasm 9.6 ms SIMD direct 3×3 1.6 ms SIMD separable 1.1 ms

Frequently Asked Questions

Does RGBA data need a different approach? Either process interleaved RGBA with four pixels per vector or deinterleave channels first; deinterleaving often vectorizes more cleanly.

Will the compiler autovectorize the scalar loop? Sometimes, partially; explicit intrinsics give predictable results for this pattern.

Is f32 more accurate than i16? Not for integer kernels — i16 is exact when sums fit; f32 is for fractional weights.

Should convolution run in a worker? Yes, for large images — keep the main thread free and split rows across workers.

Why compare outputs byte for byte rather than visually? SIMD bugs often affect only edges or tails, which look fine at a glance but break pipelines that depend on exact output.

What limits speed once SIMD and threads are in place? Memory bandwidth — fuse filters into fewer passes over the image.

← Back to Wasm SIMD & Vectorized Computation