Comment by vkazanov
8 months ago
Yeah...
It took 4 years to properly integrate Kafka into our pipelines. Everything, like everything is complicated with it: cluster management, numerous semi-tested configurations, etc.
My final conclusion with it is that the project just doesn't really know what it wants to be. Instead it tries to provide everything for everybody, and ends up being an unbelievably complicated mess.
You know, there are systems that know what they want to be (Amazon S3, Postres, etc), and then there are systems that try to eat the world (Kafka, k8s, systemd).
> that the project just doesn't really know what it wants to be
It's a distributed log? What else is it trying to do?
Calling it a distributed log may just be a Reductio ad absurdum
How?
> You know, there are systems that know what they want to be (Amazon S3, Postres, etc), and then there are systems that try to eat the world (Kafka, k8s, systemd).
I am not sure about this taxonomy. K8s, systemd, and (I would add) the Linux kernel are all taking on the ambitious task of central, automatic orchestration of general purpose computing systems. It's an extremely complex problem and I think all those technologies have done a reasonably good job of choosing the right abstractions to break down that (ever-changing) mess.
People tend to criticize projects with huge scope because they are obviously complex, and complexity is the enemy, but most of the complexity is necessary in these cases.
If Kafka's goal is to be a general purpose "operating system" for generic data systems, then that explains its complexity. But it's less obvious to me that this premise is a good one.
systemd knows very well what it wants to be, they just don't tell anyone.
it's real goal is to make Linux administration as useless as windows so RH can sell certifications.
tell me the output of systemctl is not as awful as opening the windows service panel.
Tell me systemctl output isn't more beneficial than per distro bash-mess
Well, systemd IS useful, the same way Kafka is. I don't want to back to crappy bash for service management, and Kafka is a de fact standard event streaming solution.
But both are kind of hard to understand end-to-end, especially for an occasional user.
I have no dog in the systemd wars. The bash output is what? Distro use other open source software.
not really. both requires that you know obscure and badly documented stuff.
systemd whole premise is "people will not read the distro or bash scripting manual"...
then nobody read systemd's (you have even less reason, since it's badly written, ever changing in conflicting ways, and a single use tool)
so you went from complaining your coworkers can't write bash to complaining they don't know they have to use EXEC= EXEC=/bin/x
because random values, without any hint, are lists of commands instead of string values.
2 replies →
There are 2 service panels in Windows since 8 and they are quite different...
both (and systemctl) will show two to three times the same useless info on the screen. e.g. service x - start the service x.