Comment by petcat
2 hours ago
The NSA collects and archives all internet traffic it can access for future analysis. It's the purpose of the Utah Data center.
2 hours ago
The NSA collects and archives all internet traffic it can access for future analysis. It's the purpose of the Utah Data center.
There’s no way a single datacenter costing a couple of billion dollars can store “all Internet traffic the NSA can access”, unless the traffic the NSA can access is a microscopic fraction of the total Internet traffic.
Think about it. The Internet runs on tens of thousands of massive datacenters. Thousands are being built as we speak. Obviously a single datacenter cannot hold an appreciable fraction of that.
BTW, the total budget of the NSA is less than the R&D budget of a FAANG company, so if you find yourself believing that they might have alien-level technology far beyond Google and AWS, you’re watching too much TV.
People said/wrote in the past that what NSA and other intelligence agencies did was to gather data and store the meta data. The actually traffic got processed for the meta data, and small amount of the traffic got sorted out and also stored.
I would suspect that today they also process the traffic for llms and thus store a bit more of the traffic as weight and biases. All that can be done distributed and to different degrees based on how much access they got and under what operational conditions.
https://en.wikipedia.org/wiki/Fairview_%28surveillance_progr...
https://en.wikipedia.org/wiki/Room_641A
Public information shows that the NSA has been active intercepting as much data as possible.
It doesn't require the budget of a FAANG to peek through a significant volume of internet data.
It doesn't need to store the payloads itself. It stores the metadata of connections and probably the fingerprints of the content passing through the Internet. "The headers of the whole Internet" could be physically stored in a single datacenter.
Regarding storage, a single data center using only slow but dense storage (magnetic tapes) can store far more data than a data center providing regular web services.
> The Internet runs on tens of thousands of massive datacenters.
"The Internet" would require 1000 times less servers if it wasn't running off Python scripts in Docker containers in VMs in a virtual overlay network. (I'm exaggerating these numbers only slightly.)
“They’re not as good at violating your civil liberties as you think they are” isn’t very comforting. They are still doing it at scale and i’m not too keen on opening the back door for them myself.