← Back to context

Comment by eCa

9 days ago

Agree completely that it's absolute wild to run such a system without backups. But at this point no government should keep critical data on foreign cloud storage.

Good thing Korea has cloud providers, apparently Kakao has even gone...beyond the cloud!

https://kakaocloud.com/ https://www.nhncloud.com/ https://cloud.kt.com/

To name a few.

  • They are overwhelmingly whitelabeled providers. For example, Samsung SDI Cloud (the largest "Korean" cloud) is an AWS white label.

    Korea is great at a lot of engineering disciplines. Sadly, software is not one of them, though it's slowly changing. There was a similar issue a couple years ago where the government's internal intranet was down a couple days because someone deployed a switch in front of outbound connections without anyone noticing.

    It's not a talent problem but a management problem - similar to Japan's issues, which is unsurprising as Korean institutions and organizations are heavily based on Japanese ones from back in the JETRO era.

    • I spent a week of my life at a major insurance company in Seoul once, and the military style security, the obsession with corporate espionage, when all they were working on was an internal corporate portal for an insurance company… The developers had to use machines with no Internet access, I wasn’t allowed to bring my laptop with me lest I use it to steal their precious code. A South Korean colleague told me it was this way because South Korean corporate management is stuffed full of ex-military officers who take the attitudes they get from defending against the North with them into the corporate world; no wonder the project was having so many technical problems-but I couldn’t really solve them, because ultimately the problems weren’t really technical

      13 replies →

    • That doesn't seem accurate at all. The big 3 Korean clouds used inside Korea are NHN Cloud, Naver Cloud and now KT. Which one of these is whitelabeled? And what's the source on Samsung SDI Cloud being the "largest Korean cloud"? What metric?

      NHN Cloud is in fact being used more and more in the government [1], as well as playing a big part in the recovery effort of this fire. [2]

      No, unlike what you're suggesting, Korea has plenty of independent domestic cloud and the government has been adopting it more and more. It's not on the level of China, Russia or obviously the US, but it's very much there and accelerating quickly. Incomparable to places like the EU which still have almost nothing.

      [1] https://www.ajunews.com/view/20221017140755363 - 2022, will have grown a lot now [2] https://www.mt.co.kr/policy/2025/10/01/2025100110371768374

    • I am very happy with the software that powers my Hyundai Tuscon hybrid. (It's a massive system that runs the gas and electric engines, recharging, shifting gears, braking, object detection, and a host of information and entertainment systems.) After 2 years, 0 crashes and no observable errors. Of course, nothing is perfect: maps suck. The navigation is fine; it's the display that is at least 2 decades behind the times.

      4 replies →

    •     > Korea is great at a lot of engineering disciplines. Sadly, software is not one of them
      

      I disagree. People say the same about Japan and Taiwan (and Germany). IMHO, they are overlooking the incredible talents in embedded programming. Think of all of the electronics (including automobiles) produced in those countries.

      4 replies →

Encrypted backups would have saved a lot of pain here

  • Any backup would do at this point. I think the most best is: encrypted, off-site & tested monthly.

You don’t need cloud when you have the data centre, just backups in physical locations somewhere else

  • Others have pointed out: you need uptime too. So a single data center on the same electric grid or geographic fault zone wouldn’t really cut it. This is one of those times where it sucks to be a small country (geographically).

    • > so a single data center on the same electrical grid or geographic...

      Yes, but your backup DC's can have diesel generators and a few weeks of on-site fuel. It has some quakes - but quake-resistant DC's exist, and SK is big enough to site 3 DC's at the corners of an equilateral triangle with 250km edges. Similar for typhoons. Invading NK armies and nuclear missiles are tougher problems - but having more geography would be of pretty limited use against those.

> no government should keep critical data on foreign cloud storage

Primary? No. Back-up?

These guys couldn’t provision a back-up for their on-site data. Why do you think it was competently encrypted?

  • They fucked up, that much is clear but the should not have kept that data on foreign cloud storage regardless. It's not like there are only two choices here.

    • > the should not have kept that data on foreign cloud storage regardless. It's not like there are only two choices here

      Doesn't have to be an American provider (Though anyone else probably increases Seoul's security cross section. America is already its security guarantor, with tens of thousands of troops stationed in Korea.)

      And doesn't have to be permanent. Ship encrypted copies to S3 while you get your hardenede-bunker domestic option constructed. Still beats the mess that's about to come for South Korea's population.

      11 replies →

    • They should have kept encrypted data somewhere else. If they know how to use encryption, it doesn’t matter where. Some people use stenographic backup on YouTube even.

It's 2025. Encryption is a thing now. You can store anything you want on foreign cloud storage. I'd give my backups to the FSB.

  • > I'd give my backups to the FSB.

    Until you need them - like with the article here ;) - then the FSB says "only if you do these specific favours for us first...".

  • There's certifications too, which you don't get unless you conform to for example EU data protection laws. On paper anyway. But these have opened up Amazon and Azure to e.g. Dutch government agencies, the tax office will be migrating to Office365 for example.

Why not? If the region is in country, encrypted, and with proven security attestations validated by third parties, a backup to a cloud storage would be incredibly wise. Otherwise we might end up reading an article about a fire burning down a single data center

  • Microsoft has already testified that the American government maintains access to their data centres, in all regions. It likely applies to all American cloud companies.

    America is not a stable ally, and has a history of spying on friends.

    So unless the whole of your backup is encrypted offline, and you trust the NSA to never break the encryption you chose, its a national security risk.

    • > France spies on the US just as the US spies on France, the former head of France’s counter-espionage and counter-terrorism agency said Friday, commenting on reports that the US National Security Agency (NSA) recorded millions of French telephone calls.

      > Bernard Squarcini, head of the Direction Centrale du Renseignement Intérieur (DCRI) intelligence service until last year, told French daily Le Figaro he was “astonished” when Prime Minister Jean-Marc Ayrault said he was "deeply shocked" by the claims.

      > “I am amazed by such disconcerting naiveté,” he said in the interview. “You’d almost think our politicians don’t bother to read the reports they get from the intelligence services.”

      > “The French intelligence services know full well that all countries, whether or not they are allies in the fight against terrorism, spy on each other all the time,” he said.

      > “The Americans spy on French commercial and industrial interests, and we do the same to them because it’s in the national interest to protect our companies.”

      > “There was nothing of any real surprise in this report,” he added. “No one is fooled.”

      2 replies →

    • > America is not a stable ally, and has a history of spying on friends

      America is a shitty ally for many reasons. But spying on allies isn’t one of them. Allies spy on allies to verify they’re still allies. This has been done throughout history and is basic competency in statecraft.

      7 replies →

    • There are no stable allies. No country spies on its friends because countries don't have friends, they have allies. And everybody spies on their allies.

    • Spies play one of the most important roles in global security.

      People who don’t know history think spying on allies is bad.

  • Exactly.

    Like, don't store it in the cloud of an enemy country of course.

    But if it's encrypted and you're keeping a live backup in a second country with a second company, ideally with a different geopolitical alignment, I don't see the problem.

    • The problem is money,

      you are seeing the local storage decision under the lens of security, that is not the real reason for this type of decision.

      While it may have been sold that way, reality is more likely the local DC companies just lobbied for it to be kept local and cut as many corners as they needed. Both the fire and architecture show they did cut deeply.

      Now why would a local company voluntary cut down its share of the pie by suggesting to backup store in a foreign country. They are going to suggest keep in country or worse as was done here literally the same facility and save/make even more !

      The civil service would also prefer everything local either for nationalistic /economic reasons or if corrupt then for all kick backs each step of the way, first for the contract, next for the building permits, utilities and so on.

    • From the perspective of securing your data, what's the practical difference between a second country and an enemy country? None. Even if it's encrypted data, all encryption can be broken, and so we must assume it will be broken. Sensitive data shouldn't touch outside systems, period, no matter what encryption.

      10 replies →

  • And which organization has every file, from each of their applications using the cloud, encrypted *before* it is sent to the cloud?

If you can’t encrypt your backups such that you could store them tatooed on Putin’s ass, you need to learn about backups more.

  • Governments need to worry about

    1. future cryptography attacks that do not exist today

    2. Availability of data

    3. The legal environment of the data

    Encryption is not a panacea that solves every problem

And yet here is an example where keeping critical data off public cloud storage has been significantly worse for them in the short term.

Not that they should just go all in on it, but an encrypted copy on S3 or GCS would seem really useful right about now.

  • You can do a bad job with public or private cloud. What if they would have had the backup and lost the encryption key?

    Cost wise probably having even a Korean different data center backup would not have been huge effort, but not doing it exposed them to a huge risk.

    • Then they didn't have a correct backup to begin with; for high profile organizations like that, they need to practice outages and data recovery as routine.

      ...in an ideal world anyway, in practice I've never seen a disaster recovery training. I've had fire drills plenty of times though.

  • We’ve had Byzantine crypto key solutions since at least 2007 when I was evaluating one for code signing for commercial airplanes. You could put an access key on k:n smart cards, so that you could extract it from one piece of hardware to put on another, or you could put the actual key on the cards so burning down the data center only lost you the key if you locked half the card holders in before setting it on fire.