Comment by danielheath
8 hours ago
I'm one of those people, and I don't agree.
Before I drop 5 figures on a single server, I'd like to have some confidence in the performance numbers I'm likely to see. I'd expect folk who are experienced with on-prem have a good intuition about this - after a decade of cloud-only work, I don't.
Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem.
I've estimated our IT workload would roughly double if we were to add physically racking machines, replacing failed disks, monitoring backups/SMART errors etc. That's... not cheap in staff time.
Moving things on-prem starts making financial sense around the point your cloud bills hit the cost of one engineers salary.
> Also, cloud networking offers a bunch of really nice primitives which I'm not clear how I'd replicate on-prem.
Like what?
IAM comes to mind, with fine grained control over everything.
S3 has excellent legal and auditory settings for data, as well as automatic data retention policies.
KMS is a very secure and well done service. I dare you to find an equivalent on-prem solution that offers as much security.
And then there's the whole DR idea. Failing over to another AWS region is largely trivial if you set it up correctly - on prem is typically custom to each organization, so you need to train new staff with your organizations workflows. Whereas in AWS, Route53 fail-over routing (for example) is the same across every organization. This reduces cost in training and hiring.
I've worked at many enterprises that have done and do these very things. Some for fixed workloads at scale, some for data creation/use locality issues, some for performance. I think there is about a 15 year knowledge gap in on-prem competence and what the newest shiniest is on prem for some people. Yes, some of the vendors and gear are VERY bad, but not all, and there's always eBPF :)
The biggest one for me is the way AWS security groups & IAM work.
In AWS, it's straightforward to say e.g. "permit traffic on port X from instances holding IAM role Y".
You can easily e.g. get the firewall rules for all your ec2 instances in a structured format.
I really would not look forward to building something even 1/10th as functional as that.
And you think just anyone can set that up? No sys admin/infra guy needed? Seems pretty risky.
2 replies →
I would probably just build the infra in crossplane which standardizes a lot of features across the board and gives developers a set of APIs to use / dashboard against. Different deployments and orgs have different needs and desire different features though.
BGP based routing is a major pain in the ass to do on-prem. If you want true HA in the datacenter you are going to need to utilize BGP.
I mean, BGP EVPN is the datacenter standard. (Linux infra / k8s / networking guy)