Comment by chrismorgan
14 hours ago
What’s that like for performance and power usage? I understand normal videos can generally be entirely hardware-accelerated so that the video doesn’t even touch the CPU, and are passed straight through to the compositor. I’m guessing with this you’re stuck with only accelerating individual frames, and there’ll be more back and forth so that resource usage will probably be a fair bit higher?
An interesting and unpleasant side-effect of rendering to canvas: it bypasses video autoplay blocking.
It's all hardware accelerated, assuming the VideoDecoder has hardware support for the codec. VideoFrame is available in WebGL and WebGPU as a texture or gpu-buffer. We're only rendering after a`requestAnimationFrame` callback so decoded frames may get automatically skipped based on the display frame rate.
I don't think the performance would be any worse than the <video> tag. The only exception would be browser bugs. It definitely sounds like the black bars are a browser rendering bug given it's fine when recorded.
Unfortunately canvas (rgb'ish) can't overlay as efficiently as <video> (yuv'ish), so there is some power cost relative to the lowest power video overlays.
It really only matters in long form content where nothing else on the page is changing though.
> It really only matters in long form content where nothing else on the page is changing though.
Did you not just describe at least 99% of all web video?
Oh and the autoplay restrictions for <video> don't apply when muted.
Depends on your configuration. Firefox has a “block audio and video” option. Which this bypasses.