Comment by gaoshan
1 day ago
Anyone that says, "they don’t have to manage infrastructure" I would invite them to deal with a multi-environment terraform setup and tell me again that about what they don't have to manage.
1 day ago
Anyone that says, "they don’t have to manage infrastructure" I would invite them to deal with a multi-environment terraform setup and tell me again that about what they don't have to manage.
While terraform is not ideal it is much much more easy to deal with managed services in AWS than to deal with on premises baremetal servers.
Most are biased because they like dealing with the kind of issues in on premises.
They like dealing with the performance regressions, heat maps, kernel issues etc. Because why not? You are a developer and you need some way to exercise your skills. AWS takes that away and makes you focus on the product. Issues arising from AWS only requires you talking to support. Most developers get into this industry for the love of solving these problems and not actually solving product requirements.
AWS takes away what devs like and brings in more "actual" work.
> AWS takes that away and makes you focus on the product. Issues arising from AWS only requires you talking to support.
Not my experience at all. e.g. NLBs don't support ICMP which has broken some clients of the application I work on. When we tried to turn on preserve-client-ip so we could get past the ephemeral port limit, it started causing issues with MSS negotiation, breaking some small fraction of clients. This stuff is insanely hard to debug because you can't get onto the loadbalancer to do packet captures (nor can AWS support). Loadbalancing for long-lived connections works poorly.
Lambda runs into performance issues immediately for a web application server because it's just an entirely broken architecture for that use-case (it's basically the exact opposite of user-mode threads to scale: let's use an entire VM per request!). For some reason they encourage people to do it anyway. Lord help you if you have someone with some political capital in your org that wants to push for that.
RDS also runs into performance issues the moment you actually have some traffic. A baremetal server is orders of magnitude more capable.
ipv6 support is still randomly full of gaps (or has only very recently been fixed, except you might have to do things like recreate your production EKS cluster, oops) which leads to random problems that you have to architect around. Taken with NAT gateway being absurdly expensive, you end up having to invert sensible architectures or go through extra proxy layers that just complicate things.
AWS takes basic skills around how to build/maintain backend systems and makes half of your knowledge useless/impossible to apply, instead upgrading all of your simple tuning tasks into architectural design problems. The summary of my last few years has basically been working around problems that almost entirely originate from trying to move software into EKS and dealing with random constraints that would take minutes to fix baremetal.
I agree that building your backend on Lambda is terrible for many reasons: slow starts, request / response size restrictions, limitations in "layer" sizes, etc.
RDS, however, I have found to be rock solid. What have you run into?
6 replies →
> performance regressions, heat maps, kernel issues etc.
> AWS takes that away and makes you focus on the product.
ha ha ha no. Have been dealing with kernel issues on my AWS machines for a long time. They lock up under certain kinds of high load. AWS support is useless. Experimenting with kernel version leads to performance regressions.
AWS is great if your IT/purchasing department is inefficient. Getting a new AWS machine is instant, compared to getting purchasing to approve new machine and IT allocating it to you. But all the low-level stuff is still there.
>AWS is great if your IT/purchasing department is inefficient
Fwiw, I think a lot of companies have this problem.
2 replies →
We have both AWS and colocated servers. The server workload mostly scales with the machine count not the user count. And you can get a lot done with very few servers these days.
> While terraform is not ideal it is much much more easy to deal with managed services in AWS than to deal with on premises baremetal servers.
If you have one or two baremetal servers it is not, but yes once you start having a lot of infra it is way better.
But you can get really, really far with one or two baremetal servers and some SaaS solutions...
That was barely true a decade ago. It's total nonsense today when it's trivial to ensure all your servers have IPMI or similar and it cann all be automated apart from the couple-of-times a year component swap outs.
But it's also the wrong comparison: there's rarely a reason to go on premises, and need to take responsibility for the hardware yourself - renting bare metal servers is usually the sweet spot and means someone else does the annoying bits for you but you still have the simplicity and lower cost.
As someone contracted to manage systems for people, I consistently make more money from people who overengineer their cloud setups than from people with bare metal servers. It tends to require far more maintenance to keep an AWS setup same, secure, and not bankrupting you.
I have literally never met a dev who wanted to deal with any kind of infrastructure.
Uh, no I don't in fact like dealing with any of that. And I've never had to in 20 years of managing some mid scale service on bare metal. (Though of course there have been issues of various sorts.)
I think you may have it backwards: people like tinkering with complex cloud stuff, even if they don't need it.
Those are the ones that also usually tell you you can just stitch together a few SaaS products and it's magic.
It's much the same mindset as: "Vibe-coding can do it for you so you don't have to program"
Yep. Low-effort, shallow knowledge, risk-taking guys.
You are an outdate Boomer!!! I have 37 agents doing that for me!!!!!11^
LOL
I've certainly done some things where outsourcing hosting meant I didn't have to manage infrastructure. For services running on vm instances in gcp vs services running on bare metal managed hosts, there's not a whole lot of difference in terms of management IMHO.
But any infrastructure that the product I support use is infrastructure I need to manage; having it outside my control just makes it that much harder to manage. If it's outside my control, the people who control it better do a much better job than I would at managing it, otherwise it's going to be a much bigger pain.