AWS stops billing Middle East cloud customers as repairs to war damage drag on

5 hours ago (arstechnica.com)

Data centers are such great targets in modern warfare. A few cheap drones can inflict billions in damage with low direct casualties (if the attacker even cares). I have heard AWS in particular is secretive about the exact location of their data centers, but no doubt every major country knows exactly where they are.

  • Just look on satellites for giant buildings with no cars or semi trailers parked in the parking lots.

    I wonder if data centers will have to start doubling as automobile junkyards to conceal themselves.

    • Some data centers are more valuable as targets than others. For example, those comprising us-gov-east-1 and us-gov-west-1 or, god forbid, us-east-1. I don’t expect it is a difficult task to find them and other critical infrastructure for a state, but probably more involved than popping open google maps.

      2 replies →

    • Some Paris data centers are disguised as apartment buildings with the classic Hausmannian facade, and then you open up Google maps and see a ton of AC units stacked on the roof. These aren’t likely major cloud data centers mind you, and the motivation for concealing them has more to do with the city’s aesthetic codes than military defense.

      1 reply →

  • I'm surprised they did not build them in a bunker or other hardened location. The region is not exactly known for its political stability.

  • That's why the SWIFT backup data centers in Belgium are camouflaged as posh villas (or so i've heard)

  • Big tech's love for cheap labor is a great mechanism for finding where all their most valuable assets are and mapping out any and all vulnerabilities. I imagine state actors are applying to any and all low paying jobs that have seemingly juicy job requirements and feeling out details during interviews. Even better if you offer to accept a salary far below standard rates and actually get the job.

    While probably not a state agent, I've personally done online interviews with some people that were clearly lying about everything and trying to feel out details about the company. People claiming to live in our country and being citizens but having little ability with the language, saying they would love to come to our city but it's a bit far, saying they graduated from a major university but being unable to describe anything about the town (with their resume mentioning graduating from a different university, and their LinkedIn a different university from either), random people moving around and arguing in the background, all their work was with random crypto businesses that shut down within months. I had to stop my coworkers from saying too much. I had to convince them why hiring that person for remote work and giving them access to our servers was a bad idea. There are without a doubt companies giving similar people physical access to their hardware. And there are undoubtedly people who practice interviewing to better deceive companies.

  • I wonder if you can uncover where the data center is just by using ping command.

    • >> I wonder if you can uncover where the data center is just by using ping command.

      Not exactly, but you can uncover cloud providers like Google and Azure, who forget to tell you, their "availability zones" are in the same data center ;-)

      1 reply →

    • That's some of how geolocation works. Ping can't go faster than the speed of light, so that gives you a circle for where something is. Ping from enough places and you can get a good enough idea, if you're the Iranian Guard or otherwise.

  • You can be secretive all you want, but it's extremely difficult to hide massive heat exchanging systems and/or generators from aerial/space photography. Particularly at the scale of an AWS-like datacenter.

    Building a fully camouflaged datacenter could be done at much greater cost, but you still can't hide its thermal emissions from infrared. Basically every watt hour used in a datacenter environment turns into waste heat ultimately rejected into the atmosphere (except for the 0.000000001% that leaves the facility as photons down a fiber), so if you have N megawatts of waste heat from a rectangular shaped building located on a 300 x 400 meter sized plot of land, it's going to stand out.

    • Wouldn't it be possible to pipe away the heat to the next city and use it as heating there? That way the heat emissions wouldn't be as noticeable

    • > except for the 0.000000001% that leaves the facility as photons down a fiber

      Realistically you're getting photons returned too.

    • Geothermal exists, but you would have to take care to design accordingly and even then there are plenty of other ways for a state actor to locate you. It probably doesn’t make much sense to spend money trying to hide from state actors; it’s probably better to (1) avoid conflict prone areas to the extent possible and (2) make it expensive for an attacker to shut you down (use more smaller data centers within a sensitive region, put some of them underground, etc) or (3) accept the risk of data center disruption.

"Stops billing" makes it sound generous. If those regions can't run customer apps, not charging for them is just the minimum.

  • IDK your S3 data may be fine, they're still incurring the cost to store it on those drives - even if they're buried in rubble /s

    • Also you have to remember the basics of statuspage messages: Its always just elevated error rates. Even when the error rate is elevated to 100%.

      "We are observing elevated error rates when accessing objects stored in the affected region. Impacted customers may experience increased latency or intermittent failures while retrieving debris adjacent data." /s

I'm surprised this reportedly only affected 19 server racks. Some of the small FPV quadcopter strikes I've seen videos of have collapsed entire homes. Even if the structure is more resilient than a fragile home, I would have expected the blast from a larger long-range drone like a Shahed to damage more server racks than that.

  • Yea it's hard to reconcile such a small number of affected racks with such a widespread impact though, so this must not be the whole story. They're talking about a half a year to restore the data center. It must be more than a roof repair and 19 racks.

  • It's either 100lb or 100Kg, with a direct hit on a dense centre, it would damage a lot of racks, but if it's oblique, or indirect, impartial, the damage could be less pronounced. They could also be misrepresenting by diminishing the damage as there's a lot of information suppression going on.

When I was working at AWS, which was a new service at the time, the example we often heard was a natural disaster or comet strike; would be what we were making our data centers redundant for. I don't think we were ever considered to be targeted during war and I'm sure they considered that they just didn't want to that affect that morale cost on the staff.