Comment by Daiz
24 days ago
A potentially major issue I have with AVIF is that because it is based on a video format, any hardware decoding support AVIF will get is likely to be restrained to common video scenarios. This can result in eg. only 4:2:0 YUV being supported by hardware decoders, as that's the upper limit of AV1 Main Profile (and thus the limit of AVIF Baseline Profile). And 4:2:0 is just a poor fit for certain categories of images (like various kinds of illustrations, screenshots, etc). I would not be very happy about a web where lossy 4:4:4 image compression would be heavily discouraged with modern formats.
Also, good lossless compression absolutely does matter for the web as well. Lossy images just cannot be used in all circumstances - like when comparing compression quality (especially of videos), for example!
EDIT: To give an additional example, pixel art is an entire field of art (which is very much still active today) where both lossless compression is a must and where 4:2:0 would be absolutely catastrophic for quality.
> And 4:2:0 is just a poor fit for certain categories of images (like various kinds of illustrations, screenshots, etc).
I'd go even further and say that 4:2:0 is unsuitable for still images, period. Even for cases where it would be a suitable tradeoff for video. And even for video there's a significant chunk of content where it's undesirable.
In general I think we've come to accept some pretty gnarly limitations on image quality that really should not be such a given. 8 bits per channel is another one. You actually really do notice the banding with the naked eye in a fairly large range of situations, and if everything is 8 bits per channel then it gets really easy to exacerbate the problem on edit. Dithering in 8 bits per channel isn't really a solution because it's hard to do and it gets lost in compression quickly.
4:2:0 is the main reason why most games look like crud once they've passed through any mainstream video encoding pipeline. That being said, the bitrates used for streaming games and VoDs are so low everything looks like mush anyway and millions still watch that.
Do you have an example comparison?
Luckily, 4:4:4 AVIF is supported in Firefox, Chrome, and Safari, right now. This won't ever regress.
Browsers already fully support 4:4:4 AVIF and are not going to regress on this, as it would be a huge compat issue.
It isn't clear whether AVIF hardware decoding will ever be an advantage, but for video formats, Firefox and Chrome already pick between hardware and software decoders to ensure the content plays correctly. Safari is much flakier here, but again, for images, they're not going to regress on 4:4:4 AVIF support.
Lossless AVIF encoding is inferior to JXL though, the article oddly dismisses lossless as a valid case for the web so it ignores the core argument that lossless is critical for some type of content.
I find the hardware support argument far more compelling that diminishing the value of lossless publishing.
I think the need for lossless images within a web page is extremely niche. I've used them before when comparing image codecs, but that's about it.
In cases where you need lossless, WebP is there. It's close to JPEG XL's performance, sometimes beats it, and is orders of magnitude faster to decode.
3 replies →
There have been a number of experiments to use hardware decoders for images in browsers, all of which have fallen flat; not even Safari does it for AVIF or WebP. Thus, 4:4:4 AVIF is supported absolutely everywhere. Feel free to try it now. I can see how hwdec is compelling for JPEG XL in theory, given how slow decoding is.
In the modern age image decoders need to handle untrusted inputs, and the big players rarely use hardware image decoding, even when it's available, because it's so hard to rely on its memory safety.
Several 'zero click exploits' have involved image/document format decoder bugs, as messaging apps and file browsers will often generate preview images the moment an image arrives.
All Browsers do use hardware decoding for video though, I doubt safety is a major factor.
The factor is latency. Software decoding takes milliseconds.
Depending on CPU and the video/image resolution, performance drops means the eye perceives lags when the 50ms threshold is reached. For videos, especially at 60fps that matters.
For still images it doesn't.
If it ain't broke why fix it.
Isn't the same true for video decoders, though?
Wait, DAIZ!? I genuinely did not expect to see you here chiming in. Still appreciate much of your 10 bit anime advocacy from way back.
I don’t think hardware decoders are used for images. They are light enough you can do it in software.
AVIF's support for non-photographic images is covered in the article. It supports lossless.
Yes, I know the format supports it. The question is, will hardware decoders built primarily around video use cases? And if those kind of hardware decoders end up forming the majority of hardware decoders for AVIF, will we end up in a future where using the full range of the format is heavily discouraged because it won't be supported by hardware decoders?
For comparison, it was possible to do 10-bit video with H.264, but 10-bit video didn't see any mainstream adoption with it because hardware decoders generally didn't support 10-bit H.264 video. Only when newer formats came around and specced 10-bit support as a baseline necessity did we start to see wider adoption for it. And well, the baseline specs (main profile) for AV1, which AVIF is based on, are limited to 4:2:0...
AFAIK WebP has never used hardware decoding and no browser uses it for AVIF now.
It's not lossless if you have to convert to YUV first.
You are correct. However, AVIF does not require this conversion. For lossless it supports an "identity" matrix coefficient, which means no conversion.
It isn't particularly efficient in terms of file size, but as others have said, lossless images within a web page is extremely niche.
1 reply →
I'm not super familiar with AV1/AVIF but I do have extensive experience using h264 for desktop streaming, and all major implementations support 444 in hardware and software, and I doubt more advanced codecs/image formats have a limitation like this. Asked ChatGPT and it also claimed there's no problem with 444 and AVIF/AV1.
> I'm not super familiar with AV1/AVIF but I do have extensive experience using h264 for desktop streaming, and all major implementations support 444 in hardware and software, and I doubt more advanced codecs/image formats have a limitation like this.
This is a good helpful comment containing relevant information from personal experience.
> Asked ChatGPT and it also claimed there's no problem with 444 and AVIF/AV1.
This part is useless, there's no one who cares what a LLM thinks that can't just prompt it themselves.
Then chatgpt missed an obvious issue, nvidia definitely have had shortcomings in the hardware implementation of coder/decoders only supporting 420 in nvenc/nvdec. One obvious problem with that was that it affected premiere.
We have been using NVENC with 444 and h264 in production going back years and HW as old as Turing.
But it turns out you are right, 444 decoding with H264 doesn't seem to be supported up until recently:
https://developer.nvidia.com/video-encode-decode-support-mat...
Never noticed, seems like a strange omission, but in any case doesn't seem to apply to newer codecs like AV1.
I'm pretty sure 444 h264 fails to decode on iOS Safari due to poor support in the hardware decoder.