Comment by jjcm
13 hours ago
A few things - this is one step in a long, LONG path. AV2 is currently unusable in its current state (the encoder typically runs at around 1fps on good hardware), and likely will remain so til ~2028 when the first av2 hardware accelerated chips start dropping. Even then, I wouldn't expect AV2 streams to be common til 2030.
IMO, if it were just the efficiency gains on the table (which are substantial - ~20-30% over AV1), I'd say that AV2 isn't worth it. The biggest thing it does add though is multi-stream support, which will be a big win for VR and live sports. The other fun thing is you can send an alpha channel as a separate stream, which the file will then composite for proper transparent video support.
Based on AV1's trajectory, hardware encode isn't necessary (though it is nice). The current encoder is a reference encoder. Now that the spec is finalized, expect significant speed improvements from production encoders (realtime likely won't happen until we get it in hardware though)
Hardware encode is required if you want things like video calls, camera recording and such to use it.
It isn’t required for content distribution platforms which aren’t realtime and the cost of encode is easily made up by hundreds of thousands of streams.
One of the interesting usage of AV1 was specifically for low bitrate calls, and software encoding was perfectly fine, even on mobile.
With low enough resolution, framerate and bitrate, you can get a quality stream without significant encoding artifacts compared to any other codec. It is in production right now and has been for a while.
The tradeoff CPU / bandwidth is quite advantageous in situations like this. And no, AV1 HW encoders cannot usually be used, they are not designed for a tight bitrate control or realtime communications like software encoding is usually.
6 replies →
[dead]
Anything running on a battery will need hardware acceleration
> The biggest thing it does add though is multi-stream support
I would have thought this would be a part of the container format rather than the video codec?
Google H264 SVC
The way things are going, we can pretty much forget about AV2 hardware encoders in PCs anytime soon. All the newest, best chip capacity is being completely hogged by Apple and AI companies.
Unless chipmakers port the AV2 design to older, cheaper nodes, it’s just not happening for average users. We’ll probably see some Chinese TV chip makers throw in an AV2 decoder just to check a box, but as an actual encoder? I wouldn't count on it anytime soon.
I wouldn't be so pessimistic, Intel and AMD aren't going to stop making CPUs, and if their integrated graphics adds AV2 it will be motivation enough for others to follow.
PCs don’t need hardware encoders. There are no realtime jobs on these. You can let it encode in however long it takes.
You need hardware encoders for things like cameras because they need to encode in real time since the buffer would quickly overflow otherwise.
Use cases for hardware encoders in PCs:
- Video calls
- Screen/webcam recording
- Live streaming
- Real-time transcoding for media servers (don’t know much about this but I’ve heard it’s a thing)
- Game streaming
- Video editing (making exporting less frustrating)
2 replies →
Looking at how GPU development has been sidetracked for NPU, I worry that this is 2035 target at best. Manufacturers will push for maximising matrix operation silicon area. In the era of trillion dollar investments into datacenters, traffic cost is afterthought. The only benefitors might be YouTube, Netflix and such, but on their scale investment into ISP level caches might be cheaper.
> enabling high-quality video delivery at significantly lower bitrates
> likely will remain so til ~2028 when the first av2 hardware accelerated chips start dropping
This might sound dumb, but whats the point if its intended for slower devices, but those slower devices don't even exist yet?
It's not for slower devices, it's for lower data transfer bills for providers like YouTube.
It's a win for consumers as well if you can get better video quality or more reliable calls on a slower connection.
so that new devices can adopt it
They can't adopt it if it doesn't exist.
And what about old devices? I'm sure someone out there is still using an s5 as a daily driver... Future proofing is great and all, but 240p on modern devices looks like trash, even worse than tube tv.
5 replies →
In HW accelerated chips, what part of the calculations they usually accelerate? Could it be possible to repurpose old HW?
Essentially all of the processing of the video data, barring the container format which the CPU uses to know what part of the data to send to the GPU or the Audio chip or codec.
And HW acceleration is generally a preset baked in version of the encoder or decoder. These are mostly codec specific.
So, no using hardware from previous versions.
Now, you can see some software that tries to use the GPU itself, instead of the dedicated hardware acceleration, to decode, but that isn't the HW accelerated, and may not operate in real time.
At the same time, that will consume much more power, eliminating some of the advantages or the pure HW rendition, especially important for mobile.
I could see an argument being made for encoding, if it is 2x or faster than the CPU, but I haven't looked at any in a while, so don't know the speeds.
One of the biggest gains of having dedicated hardware is that the computation doesn’t happen on the general hardware.
This is what makes it viable on mobile devices where system responsiveness and power efficiency are high priority.
Generally these hardware decoders haven’t been retoolalble.
Typically things like quantization and motion estimation.
I feel like in 2030 it's more likely that we send 480p and just upscale with ai on the other end
480p ? More like 200i. There's a race to the bottom driven by those "up to" codecs.
> The biggest thing it does add though is multi-stream support
This was supported in H.264 MVC but only saw real use for 3D movies on physical BluRays. With almost no content available outside that.
That's fine and not anything new for codecs, they always take a long time before mass adoption.
Take a look at AV1 itself, you can't even say it's really ubiquitous on all hardware. It's quite well along in adoption compared to early days, but some mobile devices are still lacking hardware acceleration for it.
It is still ironic to me that my Steam Deck has decode AV1 acceleration, on a really old CPU/GPU combo.
AMD added AV1 encoding only in later SoCs though. Next Steam Deck will have both.
>........I'd say that AV2 isn't worth it.
Unless they have hardware encoder and decoder design done in parallel, otherwise it would be 2028 before a hardware block design is done and 2030 for the earliest product to ship with it. In reality I think 2031 or 2032 is more likely.
And I have been saying the same for quite some time that 20-30% for a generational codec improvement isn't worth it. I think they originally aimed at 50%, and then 40% and then 30%.
Where do you see information about the efficiency gains over AV1?
It's at the end on the conclusion slide @24:40:
https://www.youtube.com/watch?v=Se8E_SUlU3w&t=1480
I feel like these encoders that require acceleration are a big reason why good hardware obsoletes so quickly.
> The biggest thing it does add though is multi-stream support, which will be a big win for VR and live sports.
AV1 supports it too ?
Does anyone know what is so costly to calculate in AV2?
In general they just increase the numbers on everything when they go up a generation.
e.g. if you check in 4 directions to see if you can reuse a chunk then make it check in 8 or 16.
Faster encoders will have smart heuristics on when to use these new abilities and when to skip them but the reference encoder will try everything in a dumb way to eke out a tiny win to maximize a theoretical advantage and map out the extreme best case.