Comment by anarcat
6 years ago
I understand people might be upset because they lost data, but as a sysadmin, my reaction is "ooh shit, poor guys, that must be a horrible week"...
And honestly, if you don't keep data of stuff you host on a server provider like this, you kind of get what you deserve...
No you don't. While agree everyone should have their own backups, you should expect your hosting company to properly replicate and backup their datacenters.
I don't, actually, expect them to do so. But even if I would, and Gandi, here, were doing backups and replications, no one is immune from errors and catastrophes.
Pretending that the cloud is permanent in infallible is extremely dangerous. I would seriously question the competence of any sysadmin relying on this as a base principle.
Sure, they screwed up, but this stuff happens. We should actually be happy it happens "only" on a "small-ish" provider like Gandi and not an entire AZ at Amazon.
Can't wait for that shoe to drop, I'll bring the popcorn, if there's anything left of civilization then...
> Gandi, here, were doing backups and replications
As far as I understand correctly they only made snapshots on the same machine, which is why there's trouble to begin with.
Considering they're currently "reminding" customers that backups are an industry standard right after losing data due to missing backups I wouldn't just shrug it off.
2 replies →
That is not the industry standard for web hosting. Never has been, never will be.
Backups aren't free. Replication isn't free. DR isn't free. If a customer isn't paying a premium for them, they aren't getting them. Read the terms of service.
In this case, the customer did pay for it: https://twitter.com/andreaganduglia/status/12152083871699804...
See full thread. Snapshots are marketed as backups.
10 replies →
AWS recently had data loss https://news.ycombinator.com/item?id=20875489
I'd be interested in more on this claim.
- Was this mostly a power loss or a data loss?
- If data loss, did this affect EBS (which has had a claimed annual failure rate of 0.2% - 0.5% or so if I remember) or S3 (much lower failure rate). Remember, EBS WILL have volumes go bad - that's in the docs, they recommend snapshots, aws backup manager etc if you need higher durability.
The sysadmins over there probably have a whole list of stuff that should actually have been done, but management never gave them time to do. Then this happened and they were proven right. Their reward? Working a lot of overtime probably.
> And honestly, if you don't keep data of stuff you host on a server provider like this, you kind of get what you deserve...
While I agree that everyone should have their own off-site backups, this does come across as incredibly crass victim blaming.
At least then now we know what kind of service we may expect from Gandi... Shit happens to everyone, it is in the cleaning up you learn who you're dealing with, is my personal view on that.
> You get what you deserve
Sure, let's blame the victims here; that's effective and helpful.
culpability isn't zero-sum, everyone can have some. some entities deserve a lot, others deserve just a teeny tiny little bit.
for purposes of keeping your data safe, your cloud provider is just one, single, copy of your data. all of their redundancies and backups and whatnot are for _their_ convenience, not yours, regardless of the marketing copy.
(they can decide to intentionally delete your data because they think you didn't pay. no amount of RAID and georedundant backups on their part will help you then.)
Oh god, the victims, really? You host your data on someone else's computer to save on costs and get rid of the burden of dealing with metal and stabbing yourself with screwdrivers , and you're the victim when they fuckup?
Give me a break... It's not like anyone died here. There's a reason I host my own shit. Problems happen, errors are made, and data is lost. It's also your responsibility to deal with data permanence, even if your provider has all the promises in the world.
Well yea that’s why you pay them, to do a job. That payment comes with certain expectations and when they aren’t met you incur cost. In this case downtown and effort and time to restore from your own backup. Victim may be a bit strong but of course it’s Gandi’s fault and not their customers’.
> You host your data on someone else's computer to save on costs and get rid of the burden of dealing with metal and stabbing yourself with screwdrivers , and you're the victim when they fuckup?
A company violates their agreement with you in a way that costs you time, money, and potentially business, and you're not the victim?
Exactly, its like you somehow give your original private keys to a cloud hosting provider or a service like Gandi, they have a problem and lose your mission critical data and you later blame them for their responsibility.
They are fools on their side for failing to preserve user data, but you end up being the bigger fool for trusting them to do this for you without preserving a backup plan yourself.
Shit does happen, but pretending like it's not a big deal and not providing a solid RCA seems to be what's really annoying about their reaction.
Even if it is a bad practice not to have your own backups, no one is at fault here but Gandi