Comment by atonse
5 hours ago
I don't quite understand this sort of thing happening. Wasn't the whole point of the internet to be a self-healing network where we route around severed cables, etc?
Or is it that these ATC networks are their own air-gapped network with less redundancy? That just doesn't add up. Or maybe there was only one line going to the ATC, with no multiple "ISPs" like a datacenter would have?
* Yes, TRACON have their own dedicated links. You can look into ASTERIX and STARS to learn some of the cursed ways the data processing and dataflow work.
* There is supposed to be a primary and a secondary link, in this case the primary failed and the fail-over also failed. It's unclear from the reporting if they were damaged in the same incident or if the failover was not tested or monitored adequately.
The Philadelphia TRACON site has been notoriously unreliable and was supposedly improved in 2025, it's also unclear if these issue actually could stem from that implementation.
Reporting is saying that the backup was cut a while ago and they only discovered it when the primary failed. Apparently nobody was ping testing the backup link.
I’ve had enough double-redundant links fail - and I mean pretty thoroughly double-redundant, different provider, different media, different physical path - that I’m quite surprised something like the air traffic control system only has two links.
Contractually dual-path, or actually dual-path? My understanding is that there's enough infrastructure horse-trading going on behind the scenes that it's very difficulty to be certain that two circuits between points A and B don't share the same infrastructure somewhere in between.
First big oops of this form that I remember:
"In December 1986, the ARPANET had 7 dedicated trunk lines between NY and Boston, except that they all went through the same conduit -- which was accidentally cut by a backhoe. "
https://www.csl.sri.com/~neumann/insiderisks06.html
2 replies →
One thing that needs to be reviewed is if they are dual links or redundant links.
In any system where both sides of redundancy always carry traffic then loss of one link can cause congestion failure if any link fails. This is a very common means of failure in electrical networks that requires load shedding. Well, you can't load shed air traffic.
A more complex system that I like, but comes with it's own set of constraints and implementation issues is a system where both lines carry all the traffic at all times. This way the default state of the system is always working and your first failure isn't invisibly critical.
But this is very hard as we see in TCP when things get out of order and high latency creeps in. You have to manage a lot more state at the data level.
It is, but you need to set it up correctly.
Do you want air traffic control to be on the general internet?
That seems like an extremely foolhardy thing to do.
As a tertiary backup: Yeah, maybe I do want that.
It seems like it would present a less-chaotic solution than that provided by having no data communications at all.
You certainly don’t need to run this over public Internet to get resiliency over multiple paths.
Why not? Assuming it's competently set up - serious encryption, details not blabbered about (to attracted DDoS or whatever), etc.
Competency appears to be the limited resource here.