Comment by davedx
2 years ago
Utter insanity. So much cost and complexity, and for what? Startups don’t think about costs or runway anymore, all they care about is “modern infrastructure”.
The argument for RDS seems to be “we can’t automate backups”. What on earth?
Is spending time to make it reliable worth it vs working on your actual product? Databases are THE most critical things your company has.
I see this argument a lot. Then most startups use that time to create rushed half-assed features instead of spending a week on their db that'll end up saving hundreds of thousands of dollars. Forever.
For me that's short-sighted.
Startups are in the job of earning millions. If they can spend $100k on a managed DB now and just spend every braincell on getting their product right, it's a win for their investors.
3 replies →
All that infra doesn’t integrate itself. Everywhere I’ve worked that had this kind of stack employed at least one if not a team of DevOps people to maintain it all, full time, the year round. Automating a database backup and testing it works takes half a day unless you’re doing something weird
Setting up a multi-az db with automatic failover, incremental backups and PiTR, automated runbooks and monitoring all that doesn't take half a day, not even with RDS.
3 replies →
A startup sized company using this many tools? They're for sure doing something weird (and that's not a compliment :) )
Totally on your side with this one - but alas, people associate value with complexity.
> Automating a database backup and testing it works takes half a day unless you’re doing something weird
True story bro
I'm sure that's possible if you're storing the backup on the same server you're restoring on and everything is on top of the line nvme storage. Otherwise your backup just started to run and will need another few days to finish. And that's only if you're running single master.
You're massively underestimating the challenge to get that kind of automation done in a stable manner - and the maintenance required to keep it working over the years.
10 replies →
So investing in a critical part of my business is the bad thing to do?
> The argument for RDS seems to be “we can’t automate backups”. What on earth?
I can automate backups and I'm extremely happy they with some extra cost in RDS, I don't have to do that.
Also, at some size automating the database backup becomes non-trivial. I mean, I can manage a replica (which needs to be updated at specific times after the writer), then regularly stop replication for a snapshot, which is then encrypted, shipped to storage, then manage the lifecycle of that storage, then setup monitoring for all of that, then... Or I can set one parameter on the Aurora cluster and have all of that happen automatically.
The argument for RDS (and other services along those lines) is "we can't do it as good, for less".
And, when factoring in all costs and considering all things the service takes care of, it seems like a reasonable assumption that in a free market a team that specializes in optimizing this entire operation will sell you a db service at a better net rate than you would be able to achieve on your own.
Which might still turn out to be false, but I don't think it's obvious why.
I agree but also I'm not entirely sure how much of this is avoidable. Even the most simple web applications are full of what feels like needless complexity, but I think actually a lot of it is surprisingly essential. That said, there is definitely a huge amount of "I'm using this because I'm told that we should" over "I'm using this because we actually need it"
As the famous quote goes, "If I'd had more time, I would've written a shorter letter".
Also does primary / secondary global clusters with automated failover. Saves a ton of time not to manage that manually
Everyone who says they can run a database better than Amazon is probably lying or Has a story about how they had to miss a family event because of an outage.
The point isn’t that you can’t do it, the point is that it’s less work for extremely high standards. It is not easy to configure multi region failover without an entire network team and database team unless you don’t give a shit about it actually working. Oh yea, and wait until you see how much SOC2 costs if you roll your own database.
One don’t necessarily need to run a DB better than Amazon. Just sufficiently good for the product/service you’re are working on. And depending on specifics it may costs much less (but your mileage may vary).
There are other providers with better value for service within AWS or GCP, like Crunchy.