Skip to main content

视频处理

您可以使用 drawImage() API 将 <OffthreadVideo><Video> 的帧绘制到 <canvas> 元素上。

note

在预览期间,使用 requestVideoFrameCallback() API。
浏览器支持:Firefox 130(2024 年 8 月),Chrome 83,Safari 15.4。

基本示例

在此示例中,一个 <OffthreadVideo> 被渲染并设置为不可见。
每发出的帧都会绘制到 Canvas 上,并应用灰度 filter

0:00 / 0:05

tsx
export const VideoOnCanvas: React.FC = () => {
const video = useRef<HTMLVideoElement>(null);
const canvas = useRef<HTMLCanvasElement>(null);
const { width, height } = useVideoConfig();
 
// Process a frame
const onVideoFrame = useCallback((frame: CanvasImageSource) => {
if (!canvas.current ) {
return;
}
const context = canvas.current.getContext("2d");
 
if (!context) {
return;
}
 
context.filter = "grayscale(100%)";
context.drawImage(frame, 0, 0, width, height);
}, [height, width]);
 
return (
<AbsoluteFill>
<AbsoluteFill>
<OffthreadVideo
// Hide the original video tag
style={{ opacity: 0 }}
onVideoFrame={onVideoFrame}
src="http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"
/>
</AbsoluteFill>
<AbsoluteFill>
<canvas ref={canvas} width={width} height={height} />
</AbsoluteFill>
</AbsoluteFill>
);
};
tsx
export const VideoOnCanvas: React.FC = () => {
const video = useRef<HTMLVideoElement>(null);
const canvas = useRef<HTMLCanvasElement>(null);
const { width, height } = useVideoConfig();
 
// Process a frame
const onVideoFrame = useCallback((frame: CanvasImageSource) => {
if (!canvas.current ) {
return;
}
const context = canvas.current.getContext("2d");
 
if (!context) {
return;
}
 
context.filter = "grayscale(100%)";
context.drawImage(frame, 0, 0, width, height);
}, [height, width]);
 
return (
<AbsoluteFill>
<AbsoluteFill>
<OffthreadVideo
// Hide the original video tag
style={{ opacity: 0 }}
onVideoFrame={onVideoFrame}
src="http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"
/>
</AbsoluteFill>
<AbsoluteFill>
<canvas ref={canvas} width={width} height={height} />
</AbsoluteFill>
</AbsoluteFill>
);
};

绿幕示例

在此示例中,我们遍历图像缓冲区中的每个像素,如果是绿色,则将其透明化。拖动下面的滑块以使视频透明化。

0:00 / 0:05

Slide to adjust transparency:

tsx
export const Greenscreen: React.FC<{
opacity: number;
}> = ({ opacity }) => {
const canvas = useRef<HTMLCanvasElement>(null);
const { width, height } = useVideoConfig();
 
// Process a frame
const onVideoFrame = useCallback(
(frame: CanvasImageSource) => {
if (!canvas.current ) {
return;
}
const context = canvas.current.getContext("2d");
 
if (!context) {
return;
}
 
context.drawImage(frame, 0, 0, width, height);
const imageFrame = context.getImageData(0, 0, width, height);
const { length } = imageFrame.data;
 
// If the pixel is very green, reduce the alpha channel
for (let i = 0; i < length; i += 4) {
const red = imageFrame.data[i + 0];
const green = imageFrame.data[i + 1];
const blue = imageFrame.data[i + 2];
if (green > 100 && red < 100 && blue < 100) {
imageFrame.data[i + 3] = opacity * 255;
}
}
context.putImageData(imageFrame, 0, 0);
},
[height, width]
);
 
 
return (
<AbsoluteFill>
<AbsoluteFill>
<OffthreadVideo
style={{ opacity: 0 }}
onVideoFrame={onVideoFrame}
src="https://remotion-assets.s3.eu-central-1.amazonaws.com/just-do-it-short.mp4"
/>
</AbsoluteFill>
<AbsoluteFill>
<canvas ref={canvas} width={width} height={height} />
</AbsoluteFill>
</AbsoluteFill>
);
};
tsx
export const Greenscreen: React.FC<{
opacity: number;
}> = ({ opacity }) => {
const canvas = useRef<HTMLCanvasElement>(null);
const { width, height } = useVideoConfig();
 
// Process a frame
const onVideoFrame = useCallback(
(frame: CanvasImageSource) => {
if (!canvas.current ) {
return;
}
const context = canvas.current.getContext("2d");
 
if (!context) {
return;
}
 
context.drawImage(frame, 0, 0, width, height);
const imageFrame = context.getImageData(0, 0, width, height);
const { length } = imageFrame.data;
 
// If the pixel is very green, reduce the alpha channel
for (let i = 0; i < length; i += 4) {
const red = imageFrame.data[i + 0];
const green = imageFrame.data[i + 1];
const blue = imageFrame.data[i + 2];
if (green > 100 && red < 100 && blue < 100) {
imageFrame.data[i + 3] = opacity * 255;
}
}
context.putImageData(imageFrame, 0, 0);
},
[height, width]
);
 
 
return (
<AbsoluteFill>
<AbsoluteFill>
<OffthreadVideo
style={{ opacity: 0 }}
onVideoFrame={onVideoFrame}
src="https://remotion-assets.s3.eu-central-1.amazonaws.com/just-do-it-short.mp4"
/>
</AbsoluteFill>
<AbsoluteFill>
<canvas ref={canvas} width={width} height={height} />
</AbsoluteFill>
</AbsoluteFill>
);
};

v4.0.190 之前

在 v4.0.190 之前,<OffthreadVideo><Video>onVideoFrame 属性不受支持。
您只能使用 requestVideoFrameCallback API 操纵 <Video>
点击这里查看此页面的旧版本。