Visitar URL original
GitHub - stepancar/html-video-element-webworker-polyfill: Allows to use htmlVideoElement like interface for processing frames in webworker · GitHub
Skip to content

About

Allows to use htmlVideoElement like interface for processing frames in webworker

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

html-video-element-webworker-polyfill

CI npm

HTMLVideoElement for Web Workers. Live demo · npm package

npm install html-video-element-webworker-polyfill

1. Get video frames in a worker and draw them on a canvas

main.js

import { createVideoHost } from "html-video-element-webworker-polyfill/host";

const worker = new Worker("./worker.js", { type: "module" });
createVideoHost(worker);

const canvas = document.querySelector("canvas").transferControlToOffscreen();
worker.postMessage({ canvas }, [canvas]);

worker.js

import { polyfillDocument } from "html-video-element-webworker-polyfill/worker";
import { polyfill2dCanvasApi } from "html-video-element-webworker-polyfill/canvas2d";

polyfillDocument();
polyfill2dCanvasApi();

self.addEventListener("message", ({ data }) => {
  if (!data.canvas) return;
  const ctx = data.canvas.getContext("2d");
  const video = document.createElement("video");
  video.muted = true;
  video.src = "movie.mp4";
  video.play();
  video.requestVideoFrameCallback(function draw() {
    ctx.drawImage(video, 0, 0, ctx.canvas.width, ctx.canvas.height);
    video.requestVideoFrameCallback(draw);
  });
});

Open in StackBlitz

2. Connect to an existing video and manage it from a worker

index.html

<video id="player" src="movie.mp4"></video>

main.js

import { createVideoHost } from "html-video-element-webworker-polyfill/host";

createVideoHost(new Worker("./worker.js", { type: "module" }));

worker.js

import { connectToHostVideo } from "html-video-element-webworker-polyfill/worker";

const video = await connectToHostVideo("#player");
video.currentTime = 42;

Open in StackBlitz

3. React to a video created by the worker and add it to the page

main.js

import { createVideoHost } from "html-video-element-webworker-polyfill/host";

createVideoHost(new Worker("./worker.js", { type: "module" }), {
  onVideoCreated(video) {
    if (video.id === "preview") document.body.append(video);
  },
});

worker.js

import { polyfillDocument } from "html-video-element-webworker-polyfill/worker";

polyfillDocument();

const video = document.createElement("video");
video.id = "preview";
video.muted = true;
video.src = "movie.mp4";
video.play();

Open in StackBlitz

4. Step through frames without <video>: CanvasVideoElement

CanvasVideoElement has the HTMLVideoElement API but decodes with WebCodecs onto its own canvas, so it needs no host and works the same in a worker and on the main thread. One decoder stays warm across seeks: setting currentTime to the next frame runs at the speed of the decoder itself, and seeking does not wait for a visible tab. MP4 and WebM are demuxed by Mediabunny from HTTP byte ranges, so only the bytes that are needed are downloaded.

import { CanvasVideoElement } from "html-video-element-webworker-polyfill/canvas-video";

const video = new CanvasVideoElement({ canvas: offscreenCanvas });
video.src = "movie.mp4";
await new Promise((resolve) => video.addEventListener("loadeddata", resolve, { once: true }));

for (let i = 0; i < frameCount; i++) {
  video.currentTime = i / fps;
  await new Promise((resolve) => video.addEventListener("seeked", resolve, { once: true }));
  process(video.currentFrame);
}

video.currentFrame is the VideoFrame on screen; it belongs to the element, so clone() it to keep it. play() plays with sound through Web Audio. In a worker, create the audio output on the main thread and pass its port:

import { createAudioOutput } from "html-video-element-webworker-polyfill/canvas-video";

const audio = createAudioOutput();
worker.postMessage({ audio: audio.port }, [audio.port]);
const video = new CanvasVideoElement({ audio: data.audio });

Open the demo

Why not WebCodecs?

WebCodecs gives you a decoder, not a player: you demux containers yourself, configure codecs, feed chunks and find keyframes to seek. HTMLVideoElement already handles every format the browser can play and seeks frame-accurately. It just does not exist in workers. This package makes it available there.

API

Import Exports
html-video-element-webworker-polyfill/host createVideoHost(worker, { container?, onVideoCreated?, canConnect? }) — main thread
html-video-element-webworker-polyfill/worker createVideoElement(), connectToHostVideo(selector), polyfillDocument() (document.createElement), WorkerVideoElement
html-video-element-webworker-polyfill/canvas2d polyfill2dCanvasApi() — drawImage, createPattern
html-video-element-webworker-polyfill/webgl polyfillWebglAPI() — texImage2D and friends, WebGL 1 and 2
html-video-element-webworker-polyfill/webgpu polyfillWebgpu() — copyExternalImageToTexture, importExternalTexture
html-video-element-webworker-polyfill/image-bitmap polyfillCreateImageBitmap() — createImageBitmap, polyfillVideoFrame() — new VideoFrame
html-video-element-webworker-polyfill/auto installs every worker polyfill
html-video-element-webworker-polyfill/canvas-video CanvasVideoElement({ canvas?, audio?, readAhead?, frameQueueSize?, hardwareAcceleration?, requestInit? }), createAudioOutput()

The worker element implements the HTMLVideoElement API, plus currentFrame (the VideoFrame on screen) and dispose().

Notes

  • Cross-origin media needs CORS.
  • The package shares the worker's postMessage channel with your code, so check your own messages, as in example 1.
  • Requires VideoFrame and requestVideoFrameCallback: Chrome/Edge 94+, Firefox 132+, Safari 17+.
  • CanvasVideoElement plays only the codecs the browser's WebCodecs decoders support, and changes pitch with playbackRate (preservesPitch is not supported).

License

Apache-2.0

About

Allows to use htmlVideoElement like interface for processing frames in webworker

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages