The post doesn't mention hardware or silicon implementations. That's a key requirement for modern market adoption: if a mobile device can't hardware-decode it (which saves precious power!), that's going to be a significant blocker.
I know Google provided a free hardware decode block for VP8, and I assume also for VP9 (since there appears to have been some, though not universal, hardware acceleration for that according the the Wikipedia article). I'm hoping they'll soon (or already) have the same for VP1.
I see Apple, Nvidia, Intel, Allegro, AMD, and a bunch of other silicon vendors are also members of the Alliance for Open Media, so I expect to see hardware acceleration within the next 2-3 years. I'll be impressed if it's earlier.
Edit: maybe hardware acceleration earlier. If the standard was released recently with the collaboration of the silicon industry, it's likely the current in-design/early-production silicon already has an implementation of the important parts of the codec, so we can expect consumer products with at least partial acceleration within 6 months.
There is some interesting Q&A at the end of Pascal's talk. My schoolboy French is not fast enough to do a proper translation unfortunately. He was asked about parallel (software) decoding and I think the gist was he thought it would be a bit difficult because of the dependencies between region images.
Came here to comment this. Seriously, HN, relax. This is like the introductory blogpost, how is there supposed to be a meatspace hardware implementation yet?
I'm not sure that you need mobile hardware encoding to drive initial adoption. As long as streaming services and the like can start using it to save bandwidth, you have a clear win for a big segment of the market. Older codecs can serve just fine for mobile encoding until the hardware catches up. But if most existing phones can't decode it efficiently, it's going to be pretty stuck until that situation changes.
Hardware acceleration for encoding is also pretty important. Entry-level game streamers need it (Twitch recently proved that this isn't a negligible market[1]). Playing back AV1 on a phone implies receiving it - which means that users need to store their videos to AV1 directly.
Adoption is forced by pirated stuff. If scene groups start to use it, people will want to play it. And those groups don't care if you can play whole movies on your phone. Because thats not what a phone is mainly for.
You can tell that they do if you read the scene standards.
They care much much more about this than most people do. Even a lot of people that work with video professionally don’t care as much about codecs and codec support as these guys do.
So no, I don’t believe that “real” scene groups would start using a new codec unless it was widely supported. The scene standards, then, would not include such codecs. Failure to follow scene standards will have your releases nuked in the scene.
As for the torrent sites, if someone puts up videos using codecs that cause the videos to stutter or plain fail to play then the torrent users will learn to associate whoever encoded those videos with videos that don’t work and they will avoid torrents with that person or group name in them.
The data from YouTube or Netflix dwarfs what you'd get from a torrent distributor of a show or whatever.
Some pirate/niche markets may adopt a codec solely for some arbitrary reason, but those reasons (say, maximum compression, good open source tools) are normally orthogonal to what a streaming business usually needs (compression, but also hardware/software compatibility for all sorts of form factors, and commercial unencumberment).
Movie and music piracy has taken a prodigal nosedive to the point that I don't think this would be true at all anyway. In addition streaming video and short form content dwarfs film and TV consumption, making high mobile efficiency paramount.
For the record. Yes, a lot of money was spent developing this codec. However, a lot of creativity and risk-taking was involved as well.
It was far from clear 15 years ago when xiph released Theora (aka vp3, ancestor of av1) that going up against mighty MPEG, with their enormous war chest from a decade of royalties, and dozens of patents (and the money to defend them) that success in the end was even possible much less assured.
It's really a wonderful story of the triumph of open source philosophy over the IP licensing model.
That was a pretty nice write up about it. I don't know very much about video encoding, and usually when I've tried reading about it I very quickly am out of my depth, but this bit seemed pretty reader friendly to a not-even-close-to-an-expert like myself. How did it read for those who actually know about video encoding?
Luc Trudeau, who wrote most of the deeper technical papers and specifications on CfL at Xiph, has a real gift too. I'm riding his technical prose in this post.
If you liked my update, you'll like his papers linked in 'Additional Resources'. They're some of the best and clearest technical documents I've ever read.
>> Monty" Montgomery, has a real gift for technical explanation.
He does. I love his posts. Some say you don't really understand something until you can explain it to someone else. By that metric he's probably in the top 5 codec folks in the world today.
What I appreciate most of all in that video is that he has carefully considered his words, not just to make a lay audience able to follow, but also so that what he says is _precisely_ correct.
The next two will be on the Constrained Directional Enhancement Filter and the Daala Entropy Coder, which AV1 adopted as it EC system. Beyond that, there will be several more though I've not yet planned out a specific order.
Can we somehow convince you to part-time more technical blogs and videos? Like others here said, you have a knack for writing really accessible technical articles without them feeling dumbed down.
I don't know if you're just more into doing the hands-on work of developing the codec (I can imagine that's exciting), or if it's a question of technically not being hired to do that.
In the latter case, maybe "The People" could vote with their wallet to convince the higher-ups otherwise?
Like, Ubuntu lets us decide what we want them to spend our donations on. If we had that with xiph.org and it had a section that said "more technical blogs!", or maybe even something like patreon a set-up where you can pledge to automated repeat donations for each blog entries, wouldn't be surprised if there were enough nerds willing to send donations specifically to justify to the managers that you may spend more time on outreach.
Considering Intel, AMD, ARM, Nvidia, Apple, and other hardware companies are all members of the Alliance for Open Media - which is the consortium behind AV1 - I'd expect that work to support it in hardware is already well underway.
Upon purchasing On2, It was Google's intention to release a new codec every 18-24 months or so. That would mean hardware would never catch up. However, Google didn't keep that pace. VP8 came out in 2010 and VP9 was 2013. AV1 was just released now. I would estimate that high end hardware launching in 2020 will have hardware decoding support for AV1. If AV2 doesn't launch until at least 2022, we'll have 2 good years :).
To what extent are the set of intrinsics used for VP9/h265 decoding unreusable for AV1 decoding? My guess is that there is large amount of reusable intrinsics.
But equally x265 likes to refer to MSU comparisons when the results make x265 look good: http://www.x265.org/
An x265 post referencing another paper which says HEVC outperforms AV1. The post insinuates AV1 might not be royalty-free. Reads like an attempt to spread fear, uncertainty, and doubt: http://www.x265.org/excited-av1-closing-doors/
MSU actually in the meantime updated their description. According to them they used a more recent version – see the Facebook comment thread below the streamingmedia.com link you shared.
But indeed it's a little bit ironic for someone advocating for HEVC to be concerned about patent royalties and licensing.
30% smaller than h26_4_ is an often heard statement. If you know how much more efficient h265 is in comparison with h264 you can do the math :)
At transloadit we’ve been testing the reference implementation but to encode a video it’s about 20x slower than h264 still :) there will be many improvements to speed up but it’s expected to remain a lot slower (up to 4x as slow) or so I’ve read.
Current reference implementation's encoding speed is much worse.
That is definitely going to change when real optimized encoders are written, especially since all the major CPU/GPU manufacturers are part of the Alliance for Open Media.
You could argue not only is it an ergonomic/usability mess but also a security risk, because if future flaws are found in an AVI container code path, it could be used to trick users into executing the video that they normally wouldn’t play. I know that’s a stretch, but there’s just no reason to even have the any of these issues.
That said, I hate to criticize them too much for this detail, in fact, congratulations are in order. It’s been a great engineering effort so far, and hopefully this will continue with the reference implementation evolving into some highly optimized players and encoders.
Aside from the engineering accomplishment you can imagine that it took quite a bit of business effort and coordination to get all these players together and on the same page.
I’d make a bet whoever first proposed the extension name has never been involved with user experience design. But again it’s relative. An excellent chef made a great meal, if just the garnish on top is not perfect it’s not a lot to complain about.
Hopefully, it doesn't ever become the file extension. Current containers for AV1 include Matroska (.mkv), WebM (.webm), ISO base media file format (.3gpp), and WebRTC.
It's a video codec, the average user will never give a shit or even realize they are using it. C is a stupid name for a programming language. No one cares.
I very frequently deal with .h264 files, containing just a raw h264 bitstream. Granted, that's more of a special case, because I deal with video encoding and decoding a lot through work.
All the more reason for AV2 to be created more quickly (like when those "lapping approach and frequency-domain techniques" mentioned in the article reach technical maturity)
Fun fact - "DivX ;-)" was a cracked version of the Microsoft MPEG4 decoder which was originally restricted to only play in ASF containers and has nothing to do with DivX the company which came later.
Everyone here is asking about hardware encoding/decoding, which is obviously important.
My question is, at what level do these encoders work at? Are they basically specialized SIMD instructions, or are they fully featured chips that take raw data as input and produce byte streams in the format of the protocol? Or somewhere in between?
Typically it's a dedicated hardware block on the die of a larger chip, like AMD's UVD [1] and VCE [2] inside their GPU/APUs, or Nvidia's PureVideo [3] and NVENC [4]. Sometimes the line is even blurrier, like the Broadcom SOC in the Raspberry Pi, where various more general-purpose processors can work together to decode [5].
Then you have APIs that try to disassemble all the typical stages of video codec processing that applications can call and GPU drivers can implement, which serve as a bridge between hardware-assisted decode and application code. These are APIs like DXVA or one of the several in use with Linux [6].
In between, but more like the latter conceptually. It’ll be a collection of bulk data processing blocks that are glued together by software. Generally, you want to implement the large bulk operations in HW, with SW handling all the option parsing and control logic.
And this opens up the question of how easy it is to re-target exiting hardware to new techniques.
For example, if I understand the Chroma-from-Luma prediction; then the maths involved is just 2-d linear regression (once for U vs. L once for V vs L.). That's a pretty generic task. Even when with the domain-specialisation that it is done over pixels in an encoding block, we are still talking about concepts common to all codecs.
So maybe there existing acceleration hardware can already do it. But even if it can, the require primitive needs to be exposed to software if new protocols are to benefit from it. So my question (and maybe Boxxed's too) is whether the hardware interface is low-level enough for such adaptability.
Does anyone know when h264 goes patent-free? It’s truly ubiquitous, and it first came out in around 2003. It’ll be very exciting when it becomes totally libre.
And both standards are “good enough” for most use cases and are supported by an overwhelming majority of hardware devices. Once h264 goes off patent, the interest in more advanced formats will dwindle.
Yes. We still have internal investment is many pieces of Daala that didn't get adopted in AV1. I don't know the liklihood that Daala will rise again as a standalone codec project... possible, wouldn't bet on it.
Additional background testing and perspective can be found in the following sponsored industry article - note BitMovin and Harmonic are both discussing this at the ongoing NAB conference in Las Vegas - http://www.streamingmedia.com/Articles/ReadArticle.aspx?Arti...
You can simply reencode the result of a low bitrate test using high bitrate H.264. It would take more bandwidth of course but the visual result would be more or less the same.
every day I come on here and read about these groups of people who are smarter than me doing cooler things than I will ever get to do and I just want to give up.
> The AV1 format is and will always be royalty-free
Not sure how they can make this statement.
VP8 was patent encumbered all over the place and eventually required licensing from MPEG-LA. Surely AV1 would be infringing on at least some of HEVC's expansive patent pool. Unless of course Google is willing to indemnify user's against patent infringement lawsuit. Which would be really big news.
Apparently Google does IP diligence for AV1 to navigate around patents.
Matt Frost of Google is quoted as saying:
"Obviously, if we have an open source codec, we need to take very strong steps, and be very diligent in making sure that we are in fact producing something that's royalty free. So we have an extensive IP diligence process which involves diligence on both the contributor level – so when Google proposes a tool, we are doing our in-house IP diligence, using our in-house patent assets and outside advisors – that is then forwarded to the group, and is then again reviewed by an outside counsel that is engaged by the alliance. So that's a step that actually slows down innovation, but is obviously necessary to produce something that is open source and royalty free."
The never do. It's like the mob: "pay us or you'll be in trouble". You can either pay the MPEGLA (and by the way, you will never know who inside the MPEGLA decided to sue you), or face consequences (and discover which patents are supposedly infringed)
Why should Google indemnify users when MPEG-LA isn't willing to do so for their licensees? (see the mess that HEVC is in now, or even good old MP3 with Sisvel)
AOMedia Video 1 (AV1), is an open, royalty-free video coding format designed for video transmissions over the Internet. It is being developed by the Alliance for Open Media (AOMedia), a consortium of firms from the semiconductor industry, video on demand providers, and web browser developers, founded in 2015.
Guessing (maybe generously?) at OP's meaning, claims were made in the past about VP8 being royalty free etc, and that turned out to be wrong. It's not clear how much we can trust them this time around.
Well I recently watched an old laserdisc rip of star wars and I found it so terrible that I stopped watching. To each his own, but I for one welcome high quality codecs.
The post doesn't mention hardware or silicon implementations. That's a key requirement for modern market adoption: if a mobile device can't hardware-decode it (which saves precious power!), that's going to be a significant blocker.
I know Google provided a free hardware decode block for VP8, and I assume also for VP9 (since there appears to have been some, though not universal, hardware acceleration for that according the the Wikipedia article). I'm hoping they'll soon (or already) have the same for VP1.
I see Apple, Nvidia, Intel, Allegro, AMD, and a bunch of other silicon vendors are also members of the Alliance for Open Media, so I expect to see hardware acceleration within the next 2-3 years. I'll be impressed if it's earlier.
Edit: maybe hardware acceleration earlier. If the standard was released recently with the collaboration of the silicon industry, it's likely the current in-design/early-production silicon already has an implementation of the important parts of the codec, so we can expect consumer products with at least partial acceleration within 6 months.
Yes, Google is developing a free AV1 HW decoder, as discussed near the end of this (interesting!) deck: https://news.ycombinator.com/item?id=16699000)
There is some interesting Q&A at the end of Pascal's talk. My schoolboy French is not fast enough to do a proper translation unfortunately. He was asked about parallel (software) decoding and I think the gist was he thought it would be a bit difficult because of the dependencies between region images.
https://parisvideotech.com/pvt-4-les-formats-et-codecs-du-fu...
It shouldn't surprise anyone to learn that a brand new just released video codec does not yet have hardware decode support.
Came here to comment this. Seriously, HN, relax. This is like the introductory blogpost, how is there supposed to be a meatspace hardware implementation yet?
3 replies →
> if a mobile device can't hardware-decode it (which saves precious power!), that's going to be a significant blocker.
A mobile device needs to be able to hardware-encode it as well - countless hours of 4k/60 video are taken on phones.
I'm not sure that you need mobile hardware encoding to drive initial adoption. As long as streaming services and the like can start using it to save bandwidth, you have a clear win for a big segment of the market. Older codecs can serve just fine for mobile encoding until the hardware catches up. But if most existing phones can't decode it efficiently, it's going to be pretty stuck until that situation changes.
9 replies →
Hardware acceleration for encoding is also pretty important. Entry-level game streamers need it (Twitch recently proved that this isn't a negligible market[1]). Playing back AV1 on a phone implies receiving it - which means that users need to store their videos to AV1 directly.
[1]: https://www.forbes.com/sites/insertcoin/2018/04/07/ninjas-ne...
Adoption is forced by pirated stuff. If scene groups start to use it, people will want to play it. And those groups don't care if you can play whole movies on your phone. Because thats not what a phone is mainly for.
The scene cares a lot about codec support.
You can tell that they do if you read the scene standards.
They care much much more about this than most people do. Even a lot of people that work with video professionally don’t care as much about codecs and codec support as these guys do.
https://en.wikipedia.org/wiki/Standard_(warez)#Video_standar...
So no, I don’t believe that “real” scene groups would start using a new codec unless it was widely supported. The scene standards, then, would not include such codecs. Failure to follow scene standards will have your releases nuked in the scene.
As for the torrent sites, if someone puts up videos using codecs that cause the videos to stutter or plain fail to play then the torrent users will learn to associate whoever encoded those videos with videos that don’t work and they will avoid torrents with that person or group name in them.
13 replies →
That's not really true.
The data from YouTube or Netflix dwarfs what you'd get from a torrent distributor of a show or whatever.
Some pirate/niche markets may adopt a codec solely for some arbitrary reason, but those reasons (say, maximum compression, good open source tools) are normally orthogonal to what a streaming business usually needs (compression, but also hardware/software compatibility for all sorts of form factors, and commercial unencumberment).
1 reply →
Movie and music piracy has taken a prodigal nosedive to the point that I don't think this would be true at all anyway. In addition streaming video and short form content dwarfs film and TV consumption, making high mobile efficiency paramount.
2 replies →
I am always blown away when reading about codecs. Video compression seems to be the perfect intersection of math, programming, and psychology.
And money. This is what happens when a lot of money is thrown at a problem.
For the record. Yes, a lot of money was spent developing this codec. However, a lot of creativity and risk-taking was involved as well.
It was far from clear 15 years ago when xiph released Theora (aka vp3, ancestor of av1) that going up against mighty MPEG, with their enormous war chest from a decade of royalties, and dozens of patents (and the money to defend them) that success in the end was even possible much less assured.
It's really a wonderful story of the triumph of open source philosophy over the IP licensing model.
19 replies →
This is what CAN happen when a lot of money is thrown at a problem.
1 reply →
That was a pretty nice write up about it. I don't know very much about video encoding, and usually when I've tried reading about it I very quickly am out of my depth, but this bit seemed pretty reader friendly to a not-even-close-to-an-expert like myself. How did it read for those who actually know about video encoding?
I'm no video codec expert either, but the author of this article, "Monty" Montgomery, has a real gift for technical explanation. '
Here's a video he created a few years ago, taking apart some popular misconceptions about how digital audio works: https://xiph.org/video/vid2.shtml
Luc Trudeau, who wrote most of the deeper technical papers and specifications on CfL at Xiph, has a real gift too. I'm riding his technical prose in this post. If you liked my update, you'll like his papers linked in 'Additional Resources'. They're some of the best and clearest technical documents I've ever read.
>> Monty" Montgomery, has a real gift for technical explanation.
He does. I love his posts. Some say you don't really understand something until you can explain it to someone else. By that metric he's probably in the top 5 codec folks in the world today.
1 reply →
What I appreciate most of all in that video is that he has carefully considered his words, not just to make a lay audience able to follow, but also so that what he says is _precisely_ correct.
I knew I recognized his name! That digital audio video (series?) was so fantastic and I loved the hardware setups that he used to make his points.
Site seems to be down. Google cache link: https://webcache.googleusercontent.com/search?q=cache:https:...
This made me realise how much I enjoyed and have missed the Daala blog updates :)
Any idea how many more articles are planned, so that can adjust my expectations a bit?
The next two will be on the Constrained Directional Enhancement Filter and the Daala Entropy Coder, which AV1 adopted as it EC system. Beyond that, there will be several more though I've not yet planned out a specific order.
Thanks, looking forward to them!
Can we somehow convince you to part-time more technical blogs and videos? Like others here said, you have a knack for writing really accessible technical articles without them feeling dumbed down.
I don't know if you're just more into doing the hands-on work of developing the codec (I can imagine that's exciting), or if it's a question of technically not being hired to do that.
In the latter case, maybe "The People" could vote with their wallet to convince the higher-ups otherwise?
Like, Ubuntu lets us decide what we want them to spend our donations on. If we had that with xiph.org and it had a section that said "more technical blogs!", or maybe even something like patreon a set-up where you can pledge to automated repeat donations for each blog entries, wouldn't be surprised if there were enough nerds willing to send donations specifically to justify to the managers that you may spend more time on outreach.
What does this mean for hardware accellerated decoding?
Intel implemented VP9/h265 decoding in its processors just recently.
Considering Intel, AMD, ARM, Nvidia, Apple, and other hardware companies are all members of the Alliance for Open Media - which is the consortium behind AV1 - I'd expect that work to support it in hardware is already well underway.
https://aomedia.org
Ah...
So we have to buy new processors.
22 replies →
Same as when every new format comes out -- give a few years for the hardware folks to iterate on the hardware.
Upon purchasing On2, It was Google's intention to release a new codec every 18-24 months or so. That would mean hardware would never catch up. However, Google didn't keep that pace. VP8 came out in 2010 and VP9 was 2013. AV1 was just released now. I would estimate that high end hardware launching in 2020 will have hardware decoding support for AV1. If AV2 doesn't launch until at least 2022, we'll have 2 good years :).
3 replies →
To what extent are the set of intrinsics used for VP9/h265 decoding unreusable for AV1 decoding? My guess is that there is large amount of reusable intrinsics.
Intrinsics aren't used. The decoding is generally done by a fixed function hardware block.
6 replies →
How much more effective is this than HEVC/x265 (especially at 4K HDR) resolutions?
Bitmovin's comparison: https://bitmovin.com/av1-multi-codec-dash-dataset/
MSU's comparison: http://www.streamingmedia.com/Articles/News/Online-Video-New...
x265 complained about the recent MSU test: http://www.x265.org/x265-incorrectly-represented-msus-2017-c...
But equally x265 likes to refer to MSU comparisons when the results make x265 look good: http://www.x265.org/
An x265 post referencing another paper which says HEVC outperforms AV1. The post insinuates AV1 might not be royalty-free. Reads like an attempt to spread fear, uncertainty, and doubt: http://www.x265.org/excited-av1-closing-doors/
MSU actually in the meantime updated their description. According to them they used a more recent version – see the Facebook comment thread below the streamingmedia.com link you shared.
But indeed it's a little bit ironic for someone advocating for HEVC to be concerned about patent royalties and licensing.
2 replies →
30% smaller than h26_4_ is an often heard statement. If you know how much more efficient h265 is in comparison with h264 you can do the math :)
At transloadit we’ve been testing the reference implementation but to encode a video it’s about 20x slower than h264 still :) there will be many improvements to speed up but it’s expected to remain a lot slower (up to 4x as slow) or so I’ve read.
h26_4_? What do the underscores mean (excuse my ignorance).
4 replies →
It's not more effective technically, or perhaps only slightly in some metrics. In some ways it's much worse, like encoding speed.
People like it because it has better licensing.
Current reference implementation's encoding speed is much worse.
That is definitely going to change when real optimized encoders are written, especially since all the major CPU/GPU manufacturers are part of the Alliance for Open Media.
AV1 is a poor name for a video codec because it is too easy to visually confuse with AVI.
I know how you feel. But they didn't like my suggestion of "Deep Space VP9".
what about .. av2
3 replies →
You could argue not only is it an ergonomic/usability mess but also a security risk, because if future flaws are found in an AVI container code path, it could be used to trick users into executing the video that they normally wouldn’t play. I know that’s a stretch, but there’s just no reason to even have the any of these issues.
That said, I hate to criticize them too much for this detail, in fact, congratulations are in order. It’s been a great engineering effort so far, and hopefully this will continue with the reference implementation evolving into some highly optimized players and encoders.
Aside from the engineering accomplishment you can imagine that it took quite a bit of business effort and coordination to get all these players together and on the same page.
I’d make a bet whoever first proposed the extension name has never been involved with user experience design. But again it’s relative. An excellent chef made a great meal, if just the garnish on top is not perfect it’s not a lot to complain about.
Hopefully, it doesn't ever become the file extension. Current containers for AV1 include Matroska (.mkv), WebM (.webm), ISO base media file format (.3gpp), and WebRTC.
16 replies →
It's a video codec, the average user will never give a shit or even realize they are using it. C is a stupid name for a programming language. No one cares.
Not just visually. For a long time, googling for "AV1" would produce mostly results about AVI.
At first I thought this was a late April Fool's prank. The next headline might be a new FOSS font called C0mic S4ns.
No need for that now that we have Comic Sans Neue: http://www.comicneue.com/ (FOSS-licensed)
1 reply →
At least it's not a container - you won't have *.av1 filenames.
I very frequently deal with .h264 files, containing just a raw h264 bitstream. Granted, that's more of a special case, because I deal with video encoding and decoding a lot through work.
That's a very important distinction.
I immediately thought this was like, a joke article RE: AVI format.
..So there you have it from an "unbiased^, first impression" -standpoint.
^ Unbiased in terms of any exposure / prior knowledge of this project.
All the more reason for AV2 to be created more quickly (like when those "lapping approach and frequency-domain techniques" mentioned in the article reach technical maturity)
The first is a video codec, the second a container format. You won't see the raw .av1 much.
Finally, a codec name stupider than "DivX ;-)".
Fun fact - "DivX ;-)" was a cracked version of the Microsoft MPEG4 decoder which was originally restricted to only play in ASF containers and has nothing to do with DivX the company which came later.
For more history on this, see https://en.wikipedia.org/wiki/DivX#History
1 reply →
I can't agree with you enough. I am astonished this didn't occur to anyone at Xiph.
Everyone here is asking about hardware encoding/decoding, which is obviously important.
My question is, at what level do these encoders work at? Are they basically specialized SIMD instructions, or are they fully featured chips that take raw data as input and produce byte streams in the format of the protocol? Or somewhere in between?
Typically it's a dedicated hardware block on the die of a larger chip, like AMD's UVD [1] and VCE [2] inside their GPU/APUs, or Nvidia's PureVideo [3] and NVENC [4]. Sometimes the line is even blurrier, like the Broadcom SOC in the Raspberry Pi, where various more general-purpose processors can work together to decode [5].
Then you have APIs that try to disassemble all the typical stages of video codec processing that applications can call and GPU drivers can implement, which serve as a bridge between hardware-assisted decode and application code. These are APIs like DXVA or one of the several in use with Linux [6].
[1] https://en.wikipedia.org/wiki/Unified_Video_Decoder [2] https://en.wikipedia.org/wiki/Video_Coding_Engine [3] https://en.wikipedia.org/wiki/Nvidia_PureVideo [4] https://en.wikipedia.org/wiki/Nvidia_NVENC [5] https://github.com/hermanhermitage/videocoreiv [6] https://wiki.archlinux.org/index.php/Hardware_video_accelera...
In between, but more like the latter conceptually. It’ll be a collection of bulk data processing blocks that are glued together by software. Generally, you want to implement the large bulk operations in HW, with SW handling all the option parsing and control logic.
And this opens up the question of how easy it is to re-target exiting hardware to new techniques.
For example, if I understand the Chroma-from-Luma prediction; then the maths involved is just 2-d linear regression (once for U vs. L once for V vs L.). That's a pretty generic task. Even when with the domain-specialisation that it is done over pixels in an encoding block, we are still talking about concepts common to all codecs.
So maybe there existing acceleration hardware can already do it. But even if it can, the require primitive needs to be exposed to software if new protocols are to benefit from it. So my question (and maybe Boxxed's too) is whether the hardware interface is low-level enough for such adaptability.
> AV1 leapfrogs the performance of VP9 and HEVC
Does anyone have any evidence of this claim?
For decoders. Encoder now is very very slow.
I believe that in that context "performance" refers to encoding efficiency (aka compression factor), not the encoding or decoding speed.
So not a good format for live streaming then?
24 replies →
Does anyone know when h264 goes patent-free? It’s truly ubiquitous, and it first came out in around 2003. It’ll be very exciting when it becomes totally libre.
2027
http://www.osnews.com/story/24954/US_Patent_Expiration_for_M...
That is actually a good question. Mp3 is now patents free as well.
And both standards are “good enough” for most use cases and are supported by an overwhelming majority of hardware devices. Once h264 goes off patent, the interest in more advanced formats will dwindle.
2 replies →
Probably 2020 or later.
Hopefully it will go the way copyrights are going and never be patent-free... /s
Is Daala development abandoned now?
They said a while ago they will be using Daala as a developmental testbed.
Yes. We still have internal investment is many pieces of Daala that didn't get adopted in AV1. I don't know the liklihood that Daala will rise again as a standalone codec project... possible, wouldn't bet on it.
4 replies →
Additional background testing and perspective can be found in the following sponsored industry article - note BitMovin and Harmonic are both discussing this at the ongoing NAB conference in Las Vegas - http://www.streamingmedia.com/Articles/ReadArticle.aspx?Arti...
Where's the demo? It's not unreasonable to expect to see an actual video if the article is about a next generation video.
This has been available for a while: https://demo.bitmovin.com/public/firefox/av1/
(although might not be up to spec which was published only recently: https://aomedia.org/the-alliance-for-open-media-kickstarts-v... )
How would we play it? The only decoders that exist are experiments or tests, and they are not currently optimised enough for mainstream use.
You can simply reencode the result of a low bitrate test using high bitrate H.264. It would take more bandwidth of course but the visual result would be more or less the same.
Also a webassembly decoder could probably do short clips.
1 reply →
Congratulations Chris!
every day I come on here and read about these groups of people who are smarter than me doing cooler things than I will ever get to do and I just want to give up.
Are you hiring?
Mandatory xkcd comic strip ;-) https://xkcd.com/927/
> The AV1 format is and will always be royalty-free
Not sure how they can make this statement.
VP8 was patent encumbered all over the place and eventually required licensing from MPEG-LA. Surely AV1 would be infringing on at least some of HEVC's expansive patent pool. Unless of course Google is willing to indemnify user's against patent infringement lawsuit. Which would be really big news.
Apparently Google does IP diligence for AV1 to navigate around patents.
Matt Frost of Google is quoted as saying:
"Obviously, if we have an open source codec, we need to take very strong steps, and be very diligent in making sure that we are in fact producing something that's royalty free. So we have an extensive IP diligence process which involves diligence on both the contributor level – so when Google proposes a tool, we are doing our in-house IP diligence, using our in-house patent assets and outside advisors – that is then forwarded to the group, and is then again reviewed by an outside counsel that is engaged by the alliance. So that's a step that actually slows down innovation, but is obviously necessary to produce something that is open source and royalty free."
The original source is a video, but this quote transcript is on Wikipedia: https://en.wikipedia.org/wiki/AV1#cite_note-frost-sme2017-16
How did we end up at this point? It's like a self-enforced ceiling on evolution of technology. The tar pit gets more tarry every year.
7 replies →
> VP8 was patent encumbered all over the place
Was it? Can you name a single patent? MPEG-LA never did.
https://arstechnica.com/information-technology/2013/03/googl...
> MPEG-LA never did
The never do. It's like the mob: "pay us or you'll be in trouble". You can either pay the MPEGLA (and by the way, you will never know who inside the MPEGLA decided to sue you), or face consequences (and discover which patents are supposedly infringed)
Why should Google indemnify users when MPEG-LA isn't willing to do so for their licensees? (see the mess that HEVC is in now, or even good old MP3 with Sisvel)
> Surely AV1 would be infringing on at least some of HEVC's expansive patent pool
Yes, it probably would, but if these patents are owned by members of AOM it is not an issue
When did VP8 require licensing?
From wikipedia:
AOMedia Video 1 (AV1), is an open, royalty-free video coding format designed for video transmissions over the Internet. It is being developed by the Alliance for Open Media (AOMedia), a consortium of firms from the semiconductor industry, video on demand providers, and web browser developers, founded in 2015.
https://en.wikipedia.org/wiki/AV1
Guessing (maybe generously?) at OP's meaning, claims were made in the past about VP8 being royalty free etc, and that turned out to be wrong. It's not clear how much we can trust them this time around.
http://www.streamingmedia.com/Articles/News/-76781.aspx
http://carlodaffara.conecta.it/an-analysis-of-webm-and-its-p...
http://www.fosspatents.com/2013/03/patent-clouds-remain-over...
http://www.fosspatents.com/2013/03/next-patent-infringement-...
5 replies →
Not sure what this comment is supposed to mean.
All of that was covered in the article as well.
1 reply →
You know I just watched an old (and much degraded) VHS of a TV broadcast of the 80s superman. I missed zero HD 4K whatever.
Something about the medium and the message here.
I could survive just fine without glasses. That doesn't make it noble to do so.
Well I recently watched an old laserdisc rip of star wars and I found it so terrible that I stopped watching. To each his own, but I for one welcome high quality codecs.