You might also want to mention AMWA NMOS, which is increasingly used alongside SMPTE 2110 in setups like this. NMOS (Networked Media Open Specifications) defines open, vendor-neutral APIs for device discovery, registration, connection management, and control of IP media systems. In practice, it's what lets 2110 devices automatically find each other, advertise their streams, and be connected or reconfigured via software.
The specs are fully open source and developed in the open, with reference implementations available on GitHub (https://github.com/AMWA-TV)
The specs define REST API's, JSON schemas, certificate provisioning, and service discovery mechanisms (DNS-SD / mDNS), providing an open control framework for IP-based media systems.
There’s also AES70, or OCA (https://news.ycombinator.com/item?id=46934318). More popular in audio than video, something of a competitor to NMOS (although there are parts of NMOS that were very much inspired by OCA). There are open source C++, Python, JavaScript and Swift implementations as well as some commercial ones.
Seems the classic legacy overengineered thing that costs 100x production costs because it's a niche system, is 10x more complex than needed for to unnecessary perfectionism and uses 10-100x more people than needed due to employment inerta.
A more reasonable thing is to just use high quality cameras, connect to the venue fiber Internet connection, use normal networked transport like H.265 with MPEG-TS over RTP (sports fans certainly don't care about recompression quality loss...), do time sync by having A/V sync and good clocks on each device and aligning based on audio loud enough to be recorded by all devices, then mix, reencode and distribute on normal GPU-equipped datacenter servers using GPU acceleration
The sort of systems which demand 100% reliability tend to be like that. "Disruption" in the middle of live sports broadcast is unpopular with customers.
The engineer on the truck seemed to have the most annoyance with the PTP aspect of 2110, but it seemed nobody questioned the move to 2110, and at least as far as broadcast equipment goes, they're all in on 2110. As a small(ish) YouTuber, NDI is more exciting to me, but I'm not mixing dozens or hundreds of sources for a real time production, and can just re-record if I get a sync issue over the network.
Perfect is the enemy of the good, as always—reading through that site, it seems like no solution is perfect, and the main tradeoff from that authors perspective is bandwidth requirements for UHD.
It looks like most places are only hitting 1080p still, however. And the truck I was looking at could do 1080, but runs the NHL games at 720p.
> do time sync by having A/V sync and good clocks on each device and aligning based on audio loud enough to be recorded by all devices
Why do you need good clocks? For audio, even with simultaneously playing speakers, you only need to synchronize within a couple of ms unless you need coherence or are a serious audiophile. If if want to maintain sync for an hour I suppose you need decently good clock.
But as long as you have any sort of wire, basically any protocol can synchronize well enough. Although synchronizing based on visual and audible sources is certainly an interesting idea. (Audio only is a completely nonstarter for a sporting event: the speed of sound is low and the venues are large. You could easily miss by hundreds of ms.)
> then mix, reencode and distribute on normal GPU-equipped datacenter servers using GPU acceleration
Really? Even ignoring latency, we’re talking quite a few Gbps sustained. A hiccup would suck, and if you’re not careful, you could easily spend multiple millions of dollars per day in egress and data handling fees if you use a big cloud. Just use a handful of on-site commodity machines.
Frame sync. In order to reduce latency, these systems tend to be unbuffered, which means that the frames have to arrive at a very specific time, and you can't afford significant jitter or (worse) phase drift. If you have one source at 25.000FPS and one at 25.001FPS eventually you're going to be a frame out between them.
Most 2110 kit relies on narrow timing. That means packets arriving and leaving in a window on the order of 10 microseconds. Doing that in software reliably for your typical 100gbit interface is challenging.
PipeWire had some decent AES67 support for network audio. Some really fun interesting hardware already tested. Afaik no SMPTE 2110 (which is video) but I don't really know.
I know it's not the use case but I do wish compressed formats were more supported. Not really necessary for production, but these are sort of the only defacto broadly capable network protocols we have for AV, so it would expand the potential uses a lot IMO. There may be some very proprietary JPEG-XS compression, but generally the target seems to be uncompressed.
Actually SMPTE 2110 can host video (2110-20), audio (2110-30) and ancillary data (2110-40) essences, and each essence can be delivered independently of the others.
ST 2110-22 standardizes compressed video using JPEG XS. While there is a patent pool for XS, otherwise the format is standardized and open.
It would be nice to see an essence type defined for AVC, but the quality tradeoffs of AVC/HEVC are really not appropriate for the domain that ST 2110 is aiming for: which is the contribution side video network of a broadcast operation.
There are alternative "consumer grade" and "prosumer grade" IP video solutions out there.
There is Teleport which is growing up in the OBS space, but is quite capable (we've used it in production for quite awhile)
Really it's a bit odd to wish that ST 2110 had compressed video when it's really just a specific profile of RTP with some broadcast specific bits on top while RTP itself does support lots of payloads.
ST 2110-10 which provides the timing just standardizes PTP and the meaning of the RTP timestamps (notably a specific epoch), but there's nothing stopping you from using PTP based timestamps for your RTP payloads otherwise.
ST 2110 is not a "plug and play" system by itself. There is a whole standard that adds in such capabilities which is the NMOS IS standards, but none of that is attempting to make "peer to peer" (so to speak) ST 2110 a thing, so actually using it for anything other than a broadcast system is far from trivial, and you'd be better off using something else. NMOS goal is to make auto configuration of ST2110 flows a thing, which they have broadly succeeded in doing.
ST 2110-22 is codec agnostic. It just standardises CBR compression, for which JPEG-XS is a good fit today.
For plug-and-play IPMX (https://ipmx.io/about/) is looking to be a pretty promising approach that combines ST 2110 with NMOS, auth, encryption and other useful features. It's targetted at the ProAV market but IMO should be mostly suitable for consumer use.
Just to add options, there is also the relatively new OpenMediaTransport ( https://www.openmediatransport.org/ ) that aims to be a licensing free, open alternative to NDI. At the moment, there are a number of programs supporting it, but sadly not many cameras, stand alone converters, nor audio gear. If line to see that change.
Generally NDI is not widely used in professional AV. There's a fair bit in prosumer, and a _little_ bit in the low end of pro. But the fact that it's a proprietary protocol (they can claim it's “open” all they want, but the SDK is closed source, there is no spec and they sue people who make open reimplementations), has poor image quality (it's roughly MPEG-2 intraframe, i.e., not very good), has poor latency and isn't very reliable makes it a no-go for most larger installations.
I'm amused but not entirely surprised to see that live video production hasn't meaningfully progressed since I was involved 30+ years ago.
Yes, the technology has evolved – digital vs analog (partly – for example analog comms here because digital (optical) "isn't redundant" (lol, what?)); higher resolution; digital overlays and effects, etc. But the basic process of a bunch of humans winging it and yelling to each other hasn't changed at all.
This is an industry ripe for massive disruption, and the first to do it will win big.
You might also want to mention AMWA NMOS, which is increasingly used alongside SMPTE 2110 in setups like this. NMOS (Networked Media Open Specifications) defines open, vendor-neutral APIs for device discovery, registration, connection management, and control of IP media systems. In practice, it's what lets 2110 devices automatically find each other, advertise their streams, and be connected or reconfigured via software.
The specs are fully open source and developed in the open, with reference implementations available on GitHub (https://github.com/AMWA-TV)
The specs define REST API's, JSON schemas, certificate provisioning, and service discovery mechanisms (DNS-SD / mDNS), providing an open control framework for IP-based media systems.
There’s also AES70, or OCA (https://news.ycombinator.com/item?id=46934318). More popular in audio than video, something of a competitor to NMOS (although there are parts of NMOS that were very much inspired by OCA). There are open source C++, Python, JavaScript and Swift implementations as well as some commercial ones.
Seems the classic legacy overengineered thing that costs 100x production costs because it's a niche system, is 10x more complex than needed for to unnecessary perfectionism and uses 10-100x more people than needed due to employment inerta.
A more reasonable thing is to just use high quality cameras, connect to the venue fiber Internet connection, use normal networked transport like H.265 with MPEG-TS over RTP (sports fans certainly don't care about recompression quality loss...), do time sync by having A/V sync and good clocks on each device and aligning based on audio loud enough to be recorded by all devices, then mix, reencode and distribute on normal GPU-equipped datacenter servers using GPU acceleration
The sort of systems which demand 100% reliability tend to be like that. "Disruption" in the middle of live sports broadcast is unpopular with customers.
Sounds like you've got it made then: produce the equivalent that fits in a minivan and laugh all the way to the bank.
While I think you are oversimplifying the timing issue, you are not the first to think that about 2110.
https://stop2110.org/
The engineer on the truck seemed to have the most annoyance with the PTP aspect of 2110, but it seemed nobody questioned the move to 2110, and at least as far as broadcast equipment goes, they're all in on 2110. As a small(ish) YouTuber, NDI is more exciting to me, but I'm not mixing dozens or hundreds of sources for a real time production, and can just re-record if I get a sync issue over the network.
Perfect is the enemy of the good, as always—reading through that site, it seems like no solution is perfect, and the main tradeoff from that authors perspective is bandwidth requirements for UHD.
It looks like most places are only hitting 1080p still, however. And the truck I was looking at could do 1080, but runs the NHL games at 720p.
1 reply →
> do time sync by having A/V sync and good clocks on each device and aligning based on audio loud enough to be recorded by all devices
Why do you need good clocks? For audio, even with simultaneously playing speakers, you only need to synchronize within a couple of ms unless you need coherence or are a serious audiophile. If if want to maintain sync for an hour I suppose you need decently good clock.
But as long as you have any sort of wire, basically any protocol can synchronize well enough. Although synchronizing based on visual and audible sources is certainly an interesting idea. (Audio only is a completely nonstarter for a sporting event: the speed of sound is low and the venues are large. You could easily miss by hundreds of ms.)
> then mix, reencode and distribute on normal GPU-equipped datacenter servers using GPU acceleration
Really? Even ignoring latency, we’re talking quite a few Gbps sustained. A hiccup would suck, and if you’re not careful, you could easily spend multiple millions of dollars per day in egress and data handling fees if you use a big cloud. Just use a handful of on-site commodity machines.
Frame sync. In order to reduce latency, these systems tend to be unbuffered, which means that the frames have to arrive at a very specific time, and you can't afford significant jitter or (worse) phase drift. If you have one source at 25.000FPS and one at 25.001FPS eventually you're going to be a frame out between them.
1 reply →
That rack cabling is a bit rough. Appreciate it's a live event (I've worked on them myself) but come on :)
nice to see an article from my industry! st2110 is such a complex standard which a lot of the hardware mentioned has been molded to deal with.
Most 2110 kit relies on narrow timing. That means packets arriving and leaving in a window on the order of 10 microseconds. Doing that in software reliably for your typical 100gbit interface is challenging.
Challenging but certainly doable with kernel bypass technologies and dedicated CPU cores.
1 reply →
It’s SMPTE. Not SMTPE
Doh! Will fix on my article at least.
Unfortunate typo in the headline, reproduced here and once in the article. This is not about email or spam.
> like why they use bundles of analog copper wire for audio instead of digital fiber
Good article. Got me to read the article because I was curious why...
Neat post. I wonder what the drift is on those clocks.
Fun to see.
PipeWire had some decent AES67 support for network audio. Some really fun interesting hardware already tested. Afaik no SMPTE 2110 (which is video) but I don't really know.
I know it's not the use case but I do wish compressed formats were more supported. Not really necessary for production, but these are sort of the only defacto broadly capable network protocols we have for AV, so it would expand the potential uses a lot IMO. There may be some very proprietary JPEG-XS compression, but generally the target seems to be uncompressed.
https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/AES...
Actually SMPTE 2110 can host video (2110-20), audio (2110-30) and ancillary data (2110-40) essences, and each essence can be delivered independently of the others.
ST 2110-22 standardizes compressed video using JPEG XS. While there is a patent pool for XS, otherwise the format is standardized and open.
It would be nice to see an essence type defined for AVC, but the quality tradeoffs of AVC/HEVC are really not appropriate for the domain that ST 2110 is aiming for: which is the contribution side video network of a broadcast operation.
There are alternative "consumer grade" and "prosumer grade" IP video solutions out there.
There is Teleport which is growing up in the OBS space, but is quite capable (we've used it in production for quite awhile)
https://github.com/fzwoch/obs-teleport
And of course the underpinning of 2110 itself is RTP, which is a standard network protocol, which does have AVC defined as a payload type in RFC 6184
https://www.rfc-editor.org/rfc/rfc6184
Really it's a bit odd to wish that ST 2110 had compressed video when it's really just a specific profile of RTP with some broadcast specific bits on top while RTP itself does support lots of payloads.
ST 2110-10 which provides the timing just standardizes PTP and the meaning of the RTP timestamps (notably a specific epoch), but there's nothing stopping you from using PTP based timestamps for your RTP payloads otherwise.
ST 2110 is not a "plug and play" system by itself. There is a whole standard that adds in such capabilities which is the NMOS IS standards, but none of that is attempting to make "peer to peer" (so to speak) ST 2110 a thing, so actually using it for anything other than a broadcast system is far from trivial, and you'd be better off using something else. NMOS goal is to make auto configuration of ST2110 flows a thing, which they have broadly succeeded in doing.
ST 2110-22 is codec agnostic. It just standardises CBR compression, for which JPEG-XS is a good fit today.
For plug-and-play IPMX (https://ipmx.io/about/) is looking to be a pretty promising approach that combines ST 2110 with NMOS, auth, encryption and other useful features. It's targetted at the ProAV market but IMO should be mostly suitable for consumer use.
1 reply →
It's worth mentioning NDI (Network Device Interface) as well, which is widely used in the Pro-AV for transporting compressed video and audio over IP.
Just to add options, there is also the relatively new OpenMediaTransport ( https://www.openmediatransport.org/ ) that aims to be a licensing free, open alternative to NDI. At the moment, there are a number of programs supporting it, but sadly not many cameras, stand alone converters, nor audio gear. If line to see that change.
Generally NDI is not widely used in professional AV. There's a fair bit in prosumer, and a _little_ bit in the low end of pro. But the fact that it's a proprietary protocol (they can claim it's “open” all they want, but the SDK is closed source, there is no spec and they sue people who make open reimplementations), has poor image quality (it's roughly MPEG-2 intraframe, i.e., not very good), has poor latency and isn't very reliable makes it a no-go for most larger installations.
2 replies →
I'm amused but not entirely surprised to see that live video production hasn't meaningfully progressed since I was involved 30+ years ago.
Yes, the technology has evolved – digital vs analog (partly – for example analog comms here because digital (optical) "isn't redundant" (lol, what?)); higher resolution; digital overlays and effects, etc. But the basic process of a bunch of humans winging it and yelling to each other hasn't changed at all.
This is an industry ripe for massive disruption, and the first to do it will win big.