Generating Video Thumbnails in the Browser

This page answers one task: users pick video files — for upload, a media library, an editor timeline — and the application needs thumbnails or a strip of preview frames immediately, before any upload, without freezing the page and without decoding the whole video.

Prerequisites

  • [ ] Video files from a file input or drag-and-drop.
  • [ ] A Wasm image encoder (JPEG/WebP, for example from a codec library) or OffscreenCanvas.convertToBlob.
  • [ ] For WebCodecs paths, a Wasm or JavaScript demuxer (MP4/WebM parsing).

Three ways to get frames

There are three practical approaches, in increasing order of control. The video element approach loads the file into a <video> (via an object URL), seeks to each timestamp, waits for the seeked event and draws the frame onto a canvas. It is simple and uses the browser’s decoders, but seeking is slow and imprecise, and it only works for formats the browser can play. The WebCodecs approach demuxes the container yourself (with a Wasm or JavaScript MP4/WebM parser), feeds encoded chunks to a VideoDecoder, and receives VideoFrames — precise and fast, especially when decoding only keyframes, but more code. The full Wasm approach decodes video entirely in WebAssembly (for example with an FFmpeg build); it handles formats browsers do not support at the cost of a large download and much slower decoding.

For thumbnails, keyframes are usually good enough, and decoding only keyframes avoids decoding everything in between — the single biggest speed-up available.

Video element seeking versus WebCodecs keyframe decoding Seeking a video element is simple and uses browser decoders but each seek is slow and lands near rather than exactly on the requested time. Demuxing in Wasm and decoding only keyframes with WebCodecs is fast and precise, but needs a container parser and works only for codecs the browser can decode. <video> seek + canvas minimal code slow, imprecise seeks browser-playable formats only a few thumbnails Wasm demux + WebCodecs keyframes only, fast exact timestamps needs a demuxer strips, many files

Step 1 — the simple path with a video element

async function thumbnailAt(file, seconds, width = 320) {
  const video = Object.assign(document.createElement("video"), { muted: true, preload: "metadata" });
  video.src = URL.createObjectURL(file);
  await new Promise((r) => video.addEventListener("loadedmetadata", r, { once: true }));
  video.currentTime = Math.min(seconds, video.duration - 0.1);
  await new Promise((r) => video.addEventListener("seeked", r, { once: true }));
  const h = Math.round((video.videoHeight / video.videoWidth) * width);
  const canvas = new OffscreenCanvas(width, h);
  canvas.getContext("2d").drawImage(video, 0, 0, width, h);
  URL.revokeObjectURL(video.src);
  return canvas.convertToBlob({ type: "image/webp", quality: 0.8 });
}

This is fine for one poster frame per file. For strips of many frames, each seek can take tens to hundreds of milliseconds, and the approach must run on the main thread because <video> is a DOM element.

Step 2 — demux in Wasm and decode keyframes with WebCodecs

In a worker, read the file, parse the container to find keyframe samples and their timestamps, and decode only those:

// thumbs-worker.js
import { demux } from "./pkg/demux.js";               // Wasm MP4/WebM parser: returns codec config + samples

self.onmessage = async ({ data: { file, count } }) => {
  const info = demux(new Uint8Array(await file.arrayBuffer()));   // for large files, parse incrementally
  const keyframes = info.samples.filter((s) => s.key);
  const picks = pickEvenly(keyframes, count);
  const frames = [];
  const decoder = new VideoDecoder({ output: (f) => frames.push(f), error: (e) => self.postMessage({ error: String(e) }) });
  decoder.configure(info.config);                       // codec string + description from the container
  for (const s of picks) decoder.decode(new EncodedVideoChunk({ type: "key", timestamp: s.ts, data: s.data }));
  await decoder.flush();
  // frames: VideoFrame objects → resize + encode (Step 3), then frame.close()
};

Because each keyframe decodes independently, there is no need to decode the frames between them. For long videos, read only the parts of the file needed (the index and the chosen samples) using file.slice() rather than loading the whole file into memory.

Step 3 — resize and encode thumbnails

Draw each VideoFrame onto an OffscreenCanvas at thumbnail size, or copy its pixels (frame.copyTo) into Wasm memory and resize with a high-quality filter there. Encode with convertToBlob (JPEG or WebP, where supported) or with a Wasm encoder when you need consistent output across browsers or formats the browser cannot encode. Close every VideoFrame as soon as it is drawn or copied — they hold decoder memory, and decoders stall when too many frames are open.

A thumbnail strip from keyframes The worker parses the container in Wasm to list keyframes with timestamps, picks evenly spaced ones, and feeds only those to a WebCodecs VideoDecoder. Each decoded VideoFrame is resized, encoded to WebP and closed immediately. The encoded thumbnails are posted back to the page. parse container (Wasm) keyframe list pick N keyframes evenly spaced VideoDecoder keyframes only resize + encode WebP / JPEG close frames post thumbnails

Step 4 — handle formats the browser cannot decode

VideoDecoder.isConfigSupported(config) tells you before decoding whether the browser supports the codec. Common gaps include HEVC in some browsers, ProRes and many professional or legacy codecs. For those, fall back to a Wasm decoder (an FFmpeg build with only the needed decoders), accepting slower decoding and a larger download, or show a generic placeholder and generate thumbnails server-side after upload. Decide based on how common such files are among your users.

Step 5 — keep memory and the UI under control

Process files one at a time in the worker, keep at most a few frames open, and read files in slices. Report progress per file. For a batch of hundreds of videos (a media library import), queue them and generate a single poster frame first for each, then strips in the background.

Choosing frame positions

Keyframes may be several seconds apart, so “the frame at exactly 10.0 s” may not be available without decoding the frames after the preceding keyframe. For thumbnails this rarely matters; for an editor’s frame-accurate scrubbing, decode from the preceding keyframe up to the target frame and discard the rest. Avoid the very first frame as a poster — it is often black or a fade-in — and pick a frame a few percent into the video, or the keyframe with the highest visual variance if you compute a cheap measure in Wasm.

Rotation, aspect ratio and colour

Phone videos often store frames sideways and carry a rotation flag in the container metadata; a thumbnail drawn without honouring it appears rotated by 90 degrees. The video element applies rotation automatically, but frames from WebCodecs do not — read the rotation (the track’s transformation matrix in MP4) during demuxing and rotate when drawing. Non-square pixels in some formats need the display aspect ratio, not the coded size, to avoid squashed thumbnails. HDR and wide-gamut videos decode to frames whose colour space the canvas converts when drawing; most thumbnails look fine, but if colours appear washed out or oversaturated, check the frame’s colorSpace and the canvas’s colour settings. These details are where home-grown thumbnailers most often look worse than the operating system’s, and handling them is mostly a matter of reading metadata the demuxer already has.

Caching thumbnails

Generating thumbnails is cheap per file but adds up across a media library. Store generated thumbnails with a key derived from the file’s identity — name, size and modification time, or a hash of its first and last few megabytes — in the Cache API, OPFS or IndexedDB, so reopening the library shows them instantly. When files are uploaded, send the client-generated poster frame along, so the server can use it immediately while its own processing runs, and other users see a preview without waiting.

Accessibility of thumbnail strips

Strips used for scrubbing should be operable with the keyboard and expose the timestamp of each frame as text, so the feature is not purely visual.

Expected output

Selecting 20 phone videos produces a poster thumbnail for each within a second or two and a 10-frame strip per video shortly after, all in a worker; HEVC files in browsers without HEVC support fall back to a Wasm decoder or a placeholder; memory stays under 200 MB; and nothing is uploaded.

Gotchas

  • Decoding every frame for thumbnails. Wasteful. Decode keyframes only.
  • Forgetting frame.close(). Decoders stall and memory grows. Close promptly.
  • Loading whole files into memory. Large videos exhaust memory. Read slices.
  • Assuming codec support. Check isConfigSupported and fall back.
  • Many seeks on a <video> element. Slow and main-thread bound. Use WebCodecs for strips.
  • Ignoring container rotation. Phone thumbnails appear sideways. Apply the rotation from metadata.

Performance note

For a 4-minute 1080p phone video, a 10-frame strip took about 3.8 s with video-element seeking and about 0.35 s with Wasm demuxing plus WebCodecs keyframe decoding.

Time to generate a 10-frame thumbnail strip Seconds to produce ten thumbnails from a four-minute 1080p video by seeking a video element and by demuxing in Wasm and decoding only keyframes with WebCodecs. seconds per strip video element seeks 3.8 s Wasm demux + WebCodecs keyframes 0.3 s

Frequently Asked Questions

Is WebCodecs available everywhere? In current Chromium-based browsers and Safari, and in Firefox in recent versions; check support and keep the video-element fallback.

Can FFmpeg in Wasm do all of this? Yes, but it is a large download and decodes on the CPU; use it as a fallback.

How do I get the video duration quickly? From the container metadata, parsed in Wasm, or from loadedmetadata on a video element.

Should thumbnails be generated on upload instead? Server-side generation remains useful for storage; client-side thumbnails make the UI immediate.

Why are some phone video thumbnails sideways? The container stores a rotation flag that WebCodecs frames do not apply; read it during demuxing and rotate when drawing.

Should thumbnails be cached between visits? Yes — key them by file identity and store them, so reopening a library shows previews instantly.

Can the client-generated poster be sent with the upload? Yes — upload it alongside the video so the server and other users have a preview immediately.

← Back to Media Processing & Codecs in Wasm