Add narration to canvas recordings with CanvasTake

1. Prepare the canvas and audio

Install @grundyjs/canvas-take 0.2.0 or later. This code runs in a browser. Pass a canvas with the first frame already drawn and narrationBytes, an ArrayBuffer from File.arrayBuffer() or fetch().arrayBuffer(). Set canvas dimensions before recording and keep them fixed until stop().

2. Pass the destination track

Connect an AudioBufferSource to a MediaStreamDestination and pass its getAudioTracks() as audioTracks. Invoke the function from a button handler so audio.resume() can unlock playback. This records audio without sending it to the speakers; also connect source to audio.destination if monitoring is needed.

import { createCanvasRecorder } from "@grundyjs/canvas-take";

async function recordNarration(canvas, narrationBytes) {
    const audio = new AudioContext();
    let recorder;
    let destination;
    try {
        await audio.resume();
        const buffer = await audio.decodeAudioData(narrationBytes);
        destination = audio.createMediaStreamDestination();
        const source = audio.createBufferSource();
        source.buffer = buffer;
        source.connect(destination);
        recorder = createCanvasRecorder(canvas, {
            fps: 30,
            mimeType: "video/mp4",
            audioTracks: destination.stream.getAudioTracks(),
        });
        const finished = new Promise(resolve => {
            source.addEventListener("ended", resolve, { once: true });
        });
        await recorder.start();
        source.start();
        // Run your canvas animation while the narration plays.
        await finished;
        return await recorder.stop();
    } finally {
        recorder?.dispose();
        destination?.stream.getTracks().forEach(track => track.stop());
        await audio.close();
    }
}

3. Start narration after recording starts

Await recorder.start(), then call source.start() and start your animation. CanvasTake does not draw frames or start your player. Replace the Run your canvas animation comment with that integration. This example stops when narration ends; use another stopping condition if the animation is longer. Recording is real time, without a frame-perfect synchronization guarantee.

4. Check container and codec support

An explicit video/mp4 request fails if unsupported; it does not silently fall back to WebM. MP4 does not guarantee H.264/AAC. If required by your publishing destination, request video/mp4;codecs=avc1,mp4a.40.2 and handle support errors. Check the returned mimeType and extension. Omit mimeType to use automatic format selection.

5. Release resources and review output

Use finally even when rendering or encoding fails. CanvasTake stops its owned tracks and clones; your application owns the original audioTracks and AudioContext. Ending an AudioBufferSource does not end its destination track. Mix multiple sounds into one destination. Listen to the beginning and end, check narration against frames and verify playback on the destination platform. Each recorder is single-use.