Comment by yongjik
9 days ago
It's even worse. According to other articles [1], the total data of "G drive" was 858 TB.
It's almost farcical to calculate, but AWS S3 has pricing of about $0.023/GB/month, which means the South Korean government could have reliable multi-storage backup of the whole data at about $20k/month. Or about $900/month if they opted for "Glacier deep archive" tier ($0.00099/GB/month).
They did have backup of the data ... in the same server room that burned down [2].
[1] https://www.hankyung.com/article/2025100115651
[2] https://www.hani.co.kr/arti/area/area_general/1221873.html
(both in Korean)
AWS? Linus Tech Tips has run multiple petabyte servers in their server closet just for sponsor money and for the cool of it. No need to outsource your national infrastructure to foreign governments, a moderate (in government terms) investment in a few racks across the country could've replicated everything for maybe half a year's worth of Amazon subscription fees.
But then they will depend on the security people at both sides to agree on the WAN configuration. Easier to let everything burn in a fire and rebuild from scratch.
Exactly, everyone here on hackernews is talking about Azure/AWS/GCP as if it was the only correct way to store data. Americans are too self centered, it's quite crazy.
Yeah the comments here are slightly surreal; the issue was that they didn’t have an off-site backup at all, not that it wasn’t on AWS or whatever.
I made an 840TB storage server last month for $15,000.
840TB before or after configuring RAID?
840TB raw unformatted.
>AWS S3 has pricing of about $0.023/GB/month, which means ... about $20k/month
or outright buying hardware capable of storing 850TB for the same $20K one time payment. Gives you some perspective on how overpriced AWS is.
Where are you getting 850TB of enterprise storage for $20k?
I had 500TB of object storage priced last year and it came out closer to $300k
That's including the enterprise premium for software, hardware support, and licenses. Building this in-house using open source software (e.g. Ceph) on OEM hardware will be cheaper by an order of magnitude.
You of course need people to maintain it -- the $300k turnkey solution might be the better option depending on current staff.
136tb for $3k (used gen 2 epyc hardware and refurb <1 hour 16tb hdd's) they're zero risk after firmware validation and 1 full drive read and write.
Priced out by whom? What kind of object storage? Were you looking at the price of drives, or the price of a complete solution delivered by some company?
Couldn’t even be bothered to do a basic 3-2-1! Wow
Did you expect government IT in a hierarchical respect-your-superiors-even-when-wrong society to be competent?
South Korea isn't some sort of backwards nation and I'm sure it's chaebols share the same culture.
Having had unfortunate encounters with government IT in other countries I can bet that the root cause wasn't the national culture. It was the internal culture of "I want to do the same exact same thing I've always done until the day I retire."
Absent outside pressure, civil services across the word tend advance scientifically - one funeral (or retirement) at a time.
1 reply →
I mean...I feel you but holy hell dude. Nothing? Boggles the mind.
Edit: my bad backups in the room is something, somehow just forgot about that part
2 replies →
I have almost 10% of that in my closet RAID5'd with large part of it backing up constantly to Backblaze for 10$/month, running on 10 year old hardware, with basically only the hard drives having any value ... Used a case made of cardboard till I wanted to improve the cooling, and got a used Fractal Design case for 20€.
_Only_ the kind of combination of incompetence and bad politics here can lead to the kind of % of how much data has been lost here, given the policy was to only save stuff on that "G-drive" and avoid local copies. The "G-drive" they intentionally did not back up because they couldn't figure out a solution to at least store a backup across the street ...
How does this even make sense business wise for AWS?
Is their cost per unit so low?
This is just the storage cost. That is they will keep your data on their servers, nothing more.
Now if you want to do something with the data, that's where you need to hold your wallet. Either you get their compute ($$$ for Amazon) or you send it to your data centre (egress means $$$ for Amazon).
When you start to do math, hard drive are cheap when you go for capacity and not performance.
0.00099*1000 is 0.99. So about 12$ a year. Now extrapolate something like 5 year period or 10 year period. And you get to 60 to 120$ for TB. Even at 3 to 5x redundancy those numbers start to add up.
S3 does not spend 3x drives to provide redundancy. Probably 20% more drives or something like that. They split data to chunks and use erasure coding to store them in multiple drives with little overhead.
5 replies →
They charge little for storage and upload, but download, so getting your data back, is pricey.
Mate, this is better than an entire nation's data getting burned.
Yes its pricey but possible.
Now its literally impossible.
I think that AWS Glacier at that scale should be the thing preferred as they had their own in house data too but they still should've wanted an external backup and they are literally by the govt. so they of all people shouldn't worry about prices.
Have secure encrypted backups in aws and other possibilities too and try to create a system depending on how important the treat model is in the sense that absolutely filter out THE MOST important stuff out of those databases but that would require them to label it which I suppose would make them gather even more attention to somehow exfiltrate / send them to things like north korea/china so its definitely a mixed bag.
my question as I said multiple times, why didn't they build a backup in south korea only and used some other datacentre in south korea only as the backup to not have to worry about encryption thing but I don't really know and imo it would make more sense for them to actually have a backup in aws and not worry about encryption personally since I find the tangents of breaking encryption a bit unreasonable since if that's the case, then all bets are off and the servers would get hacked too and that was the point of phrack with the advanced persistent threat and so much more...
are we all forgetting that intel has a proprietory os minix running in the most privileged state which can even take java bytecode through net and execute it and its all proprietory. That is a bigger security threat model personally to me if they indeed are using that which I suppose they might be using.
1 reply →
It's expensive if you calculate what it would cost for a third party to compete with. Or see e.g. this graph from a recent HN submission: https://si.inc/posts/the-heap/#the-cost-breakdown-cloud-alte...
That's unfortunate.
It's incompetent really.
No. Fortuna had nothing to do with this, this is called bad planning.