Comment by datageek_31
10 months ago
Since you are generating hundreds of terabytes of data every day, can we get some insight into what the data platform is being used here to handle such scale?
10 months ago
Since you are generating hundreds of terabytes of data every day, can we get some insight into what the data platform is being used here to handle such scale?
That's cumulative for all data collected like satellites, weather radars, and surface observing systems. Soundings are relatively "lightweight" compared to most sets. The major difficulty is pushing the data through our low-power satellite backhaul - in practice we average ~10B/s
I would be interested in this as well. I hope they start an engineering blog at some point.