Maybe you are right about kubernetes, I don't have enough experience to have an opinion. I disagree about containers though, especially the wider docker toolchain.
It is not that difficult to understand a Dockerfile and use containers. Containers, from a developer pov, solve the problem of reliably reproducing development, test and production environments and workloads, and distributing those changes to a wider environment. It is not perfect, its not 100% foolproof, and its not without its quirks or learning curve.
However, there is a reason docker has become as popular as it is today (not only containers, but also dockerfiles and docker compose), and that is because it has a good tradeoff between various concerns that make it a highly productive solution.
I can maybe make a case for running in containers if you need some specific security properties but .. mostly I think the proliferation of 'fucked up piles of shit' is the problem.
Containers are just processes plus some namespacing, nothing really stops you from running very huge tasks on Kubernetes nodes. I think the argument for containers and Kubernetes is pretty good owing to their operational advantages (OCI images for distributing software, distributed cron jobs in Kubernetes, observability tools like Falco, and so forth).
So I totally understand why people preemptively choose Kubernetes before they are scaling to the point where having a distributed scheduler is strictly necessary. Hadoop, on the other hand, you're definitely paying a large upfront cost for scalability you very much might not need.
Time to market and operational costs are much higher on kubernetes and containers from many years of actual experience. This is both in production and in development. It’s usually a bad engineering decision. If you’re doing a lift and shift, it’s definitely bad. If you’re starting greenfield it makes sense to pick technology stacks that don’t incur this crap.
It only makes sense if you’re managing large amounts of large siloed bits of kit. I’ve not seen this other than at unnamed big tech companies.
99.9% of people are just burning money for a fashion show where everyone is wearing clown suits because someone said clown suits are good.
Containers are too low-level. What we need is a high-level batch job DSL, where you specify the inputs and the computation graph to perform on those inputs, as well as some upper limits on the resources to use, and a scheduler will evaluate the data size and decide how to scale it. In many cases that means it will run everything on a single node, but in any case data devs shouldn't be tasked with making things run in parallel because the vast majority aren't capable and they end up with very bad choices.
And by the way, what I just described is a framework that Google has internally, named Flume. 10+ years ago they had already noticed that devs aren't capable of using Map/Reduce effectively because tuning the parallelism was beyond most people's abilities, so they came up with something much more high-level. Hadoop is still a Map/Reduce clone, thus destined to fail at useability.
Are you saying that running your application in a pile of containers somehow helps that problem ..? It's the same problem as CPU scheduling, we just don't have good schedulers yet.. Lots of people are working on it though
Not really? At the moment it's done by some user-land job scheduler. That could be something container based like k8s, something in-process like ray, or a workload manager like slurm.
Yeah this. As I explain many times to people, processes are the only virtualisation you need if you aren’t running a fucked up pile of shit.
The problem we have is fucked up piles of shit not that we don’t have kubernetes and don’t have containers.
Maybe you are right about kubernetes, I don't have enough experience to have an opinion. I disagree about containers though, especially the wider docker toolchain.
It is not that difficult to understand a Dockerfile and use containers. Containers, from a developer pov, solve the problem of reliably reproducing development, test and production environments and workloads, and distributing those changes to a wider environment. It is not perfect, its not 100% foolproof, and its not without its quirks or learning curve.
However, there is a reason docker has become as popular as it is today (not only containers, but also dockerfiles and docker compose), and that is because it has a good tradeoff between various concerns that make it a highly productive solution.
I suggest you read my comment here, which I'd rather not repeat as it's quite a long one https://news.ycombinator.com/item?id=46676676
> problem of reliably reproducing development, test and production environments and workloads
Then again so does a tar file.
Some people might disagree that the problem is "solved" but there you go.
Hahhah, yuuuup.
I can maybe make a case for running in containers if you need some specific security properties but .. mostly I think the proliferation of 'fucked up piles of shit' is the problem.
Containers are just processes plus some namespacing, nothing really stops you from running very huge tasks on Kubernetes nodes. I think the argument for containers and Kubernetes is pretty good owing to their operational advantages (OCI images for distributing software, distributed cron jobs in Kubernetes, observability tools like Falco, and so forth).
So I totally understand why people preemptively choose Kubernetes before they are scaling to the point where having a distributed scheduler is strictly necessary. Hadoop, on the other hand, you're definitely paying a large upfront cost for scalability you very much might not need.
Time to market and operational costs are much higher on kubernetes and containers from many years of actual experience. This is both in production and in development. It’s usually a bad engineering decision. If you’re doing a lift and shift, it’s definitely bad. If you’re starting greenfield it makes sense to pick technology stacks that don’t incur this crap.
It only makes sense if you’re managing large amounts of large siloed bits of kit. I’ve not seen this other than at unnamed big tech companies.
99.9% of people are just burning money for a fashion show where everyone is wearing clown suits because someone said clown suits are good.
12 replies →
Containers are too low-level. What we need is a high-level batch job DSL, where you specify the inputs and the computation graph to perform on those inputs, as well as some upper limits on the resources to use, and a scheduler will evaluate the data size and decide how to scale it. In many cases that means it will run everything on a single node, but in any case data devs shouldn't be tasked with making things run in parallel because the vast majority aren't capable and they end up with very bad choices.
And by the way, what I just described is a framework that Google has internally, named Flume. 10+ years ago they had already noticed that devs aren't capable of using Map/Reduce effectively because tuning the parallelism was beyond most people's abilities, so they came up with something much more high-level. Hadoop is still a Map/Reduce clone, thus destined to fail at useability.
Disagree.
Different processes can need different environments.
I advocate for something lightweight like FreeBSD jails.
Yes, Sun had the marketing message "The network is the computer" already in the 1980's, we were doing microservices with plain OS processes.
Containers solve:
1. Better TCP port administration with networking layer
2. Clusterfuck that is glibc versions
3. Shipping a Python venv
Can't really speak to (1), but (2) and (3) definitely qualify as 'fucked up piles of shit', which is what he's saying the real problem is.
Its all fun and games, until the control plane gets killed by the OOMkiller.
Naturally, that detaches all your containers. And theres no seamless reattach for control plane restart.
Or your CNI implementation is made of rolled up turds and you lose a node or two from the cluster control plane every day.
(Large EKS cluster)
Until you need to schedule GPUs or other heterogenous compute...
Are you saying that running your application in a pile of containers somehow helps that problem ..? It's the same problem as CPU scheduling, we just don't have good schedulers yet.. Lots of people are working on it though
Not really? At the moment it's done by some user-land job scheduler. That could be something container based like k8s, something in-process like ray, or a workload manager like slurm.