Comment by sidkshatriya
3 months ago
Can anybody explain to me again why systemd is so bad ? Genuinely I'm not sure anymore: It is chock full of features and it gets the job done. Since it is used in a lot of big distributions it gets a lot of fixes, updates, testing and feature improvements regularly.
Yes, it is maybe monolithic (but so is the Linux kernel). Its philosophy may differ from unix's "get one thing done well" too but integration of various functionalities comes with its benefits.
Some people say it is bloated. The substitutes to systemd are lightweight but less featureful. Maybe some of them will get bloated as they achieve feature parity.
People have a right to build substitutes and replacements -- I believe in the "Let a hundred flowers bloom" philosopy. However I can't understand why systemd is the point of so much disagreement.
I think this is as good a time as any to reread "The Rise of Worse is Better".
https://www.dreamsongs.com/RiseOfWorseIsBetter.html
The article compares the Unix/C philosophy with the Lisp philosophy and concludes that Lisp is "better" (it is The Right Thing), but that Unix/C will probably end up dominating. Obviously, he was right.
The article is pretty famous around the suckless, minimalist, crowd. But I think everyone who reads it takes a slightly wrong lesson from that article. The point isn't that Unix/C won because it was simple. The point is that Unix/C was simple enough to use, simple enough to implement on most available hardware and that it did everything you needed it to do.
The minimalist people have turned software simplicity into the modern version of The Right Thing, and are surprised when they found out that the "worse" (more complex) thing wins. But The Right Thing will never win. The Good Enough always wins.
Is Linux better than *BSD? Well *BSDs are more simple, easier to understand and reason about etc. But, at least OpenBSD, doesn't have mount --bind, so I can't use it. Case closed. And I think that's the case for most people. Linux has thing X which I need so I will use Linux, even if ideologically *BSD is imo better.
Is D-Bus The Right Thing? No; but it is The Good Enough. polkit, selinux, ACLs, and so on.
The most recent example I can think of is Hyprland. It is basically the only viable Wayland window manager (aside from sway). Not because it is the best designed or the simplest, or anything like that. But simply because it is the only one Good Enough to use.
SystemV init was not good enough for complex service management. systemd is. systemd simply does everything you need it to do well enough. So systemd wins. Simple as. It will only be replaced when it is no longer Good Enough.
Exactly, for some values of "good enough". (Unfortunately IMNSHO) Systemd cannot be toppled by all-out assault. It must be subverted. Looking forward to reimplementations and/or shims which provide the Systemd API but are reimplementations.
Something like Wine, where in the analogy systemd is the Microsoft Win32 API and we want to be able to run systemd "enabled" software on something else. Or at least re-compile it.
Wine also started with an incredible amount of stubs which did nothing, but that was often enough for many programs to work anyway.
> Looking forward to reimplementations and/or shims which provide the Systemd API but are reimplementations.
We've seen the launch of GNU's systemd equivalent, which seems to introduce more complexity to the simplest levels and is more difficult for users to understand and configure. Given that every service file is its own erlang program/DSL/whatever, understanding erlang is critical to being able to write or reason about service files.
In essence, it seems to be trying to re-implement what systemd does, but badly.
Reimplementing systemd's API but using other discrete components would also be a bad idea. The biggest reason systemd took off is that it's all the tools a system needs, built in a way that they work well together and can be interacted with and configured consistently. Providing systemd-compatible replacement shim APIs would mean trying to tie a dozen (or more) discrete OSS projects, each with their own philosophies, developers, licenses, goals, etc. together, generating their config files consistently, and trying to mush all that together in a way that doesn't make things vastly more complex than what systemd already provides.
In short: the reason systemd is successful is that it got rid of a giant mess of unrelated or overlapping services and replaced it with something simple, coherent, and functional, built out of components that you can (in most cases) choose to use or not. Most people who hate systemd seem to hate it for largely ideological reasons ("it's too big!") and are more interested in solving those ideological problems at the expense of practical solutions to real user issues. I've yet to see someone argue against systemd because it's bad for end users; only because they don't like the way that it provides benefits for users.
7 replies →
Yeah. Exactly. It's also how Wayland and Pipewire managed to win: by being backwards compatible with X and Pulseaudio, while also overcoming their shortcomings.
The problem is that if the only complaint with systemd is that it's too complex, any backwards compatible alternative will necessarily be as complex as systemd + the new thing (at least in terms of complexity of interface).
If there are actual technical deficiencies of systemd, then sure, maybe such a backwards compatible alternative might be in order.
Also, everything expands in time. Wine may have started out with many stubs, but now we're at the point of implementing Windows-y APIs in the kernel with the sole goal of improving wine performance (NTSYNC).
1 reply →
Subversion is key. Remember when "performance" was the key reason for moving to systemd (scripts are sooo slow). To subvert, you need some lever and the chink in the armor to break in.
D-Bus is good enough for all sorts of things. But you absolutely don't need a behemoth like systemd to use D-Bus. A D-Bus communication option can be added to existing programs and utilities, or new ones could be fashioned for which it is a central mode of operation; and still there would have been no an ever-growing pile of centrally-maintained and inter-dependent artifacts.
As for SystemV... as I mentioned in another comment - in hindsight, that was not the issue. There were and are SystemV alternatives [1]. One can even write one that uses D-Bus in an opt-in fashion, to foster other facilities' use of it.
The init system excuse is very much like the stone in the fable of the stone soup: https://en.wikipedia.org/wiki/Stone_Soup - it's what gets the vagabond's foot in the door, and is the first ingredient for the soup. But actually, the stone carried no significance, and the resulting soup is pretty much the same thing as without the stone part.
[1] : ruint, OpenRC, launchd, s6, nosh, finit, procd, upstart etc. See brief description and links at https://alternativeto.net/software/sysvinit/ for example.
s/SystemV/sysvinit/
and one more thing about that: For PC users and "vanilla" machine setups - sysvinit, weirdly, still works fine. Startup is quick, and developers don't live in pain making it work. Of course it is kind of lame design-wise, but it's not even that we just _had_ to replace it - we've still not reached even that point.
> The most recent example I can think of is Hyprland. It is basically the only viable Wayland window manager (aside from sway). Not because it is the best designed or the simplest, or anything like that. But simply because it is the only one Good Enough to use.
I'd wager the vast majority of (non-embedded / single-purpose) Wayland deployments are gnome and KDE.
They might be Raspberry Pi which doesn't use gnome or kde! :-)
Obviously, but I'm talking about bare window managers, not DEs. Alternatives to i3, awesomewm etc. Different niche.
2 replies →
cosmic is going to be a great wayland windows manager.
I think a lot of the arguments I’ve seen stem from “Unix philosophy” style arguments. Also, historically the systemd project has been quite hostile to user requests in some cases, which broke existing workflows for people.
I personally think the basic “service manager” functionality works pretty well and I like systemd overall.
However, the same is not true for a bunch of the more peripheral services (e.g. resolved, networkd, timesyncd). What’s particularly annoying is that there exist mature open source projects that can provide better functionality in a lot of cases which could be used, rather than bundling a half-assed attempt with systemd (eg. unbound, chrony).
> resolved, networkd, timesyncd
None of these are mandatory though. It's up to the distros whether to use them. For example at this point resolved is pretty commonly enabled by default, networkd not at all, and timesyncd is perhaps 50-50 with chrony.
Yes, but not using those seems to defeat the point of using systemd.
The most convincing advocacy I have seen for systemd (by a BSD developer, its fairly well known) is that it provides an extra layer of the OS (on top of the kernel) to standardise things.
9 replies →
So, systemd is like "doing Linux services the Microsoft way"
Your remark may suggest that everything MS does is “wrong”. This is of course an extreme overstatement and all of their approaches should be evaluated on their own.
13 replies →
It's more like doing Linux services the UNIX(TM) way since it's more similar to other UNIX service managers like SMF from Solaris or SRC from AIX in the integration -- NT's service manager requires an active event loop which responds to messages.
As an aside, the reason I don't like systemd is because it's inferior to its UNIX counterparts -- especially SMF -- for system management.
1 reply →
Given that it was based off of the design of OS X's launchd, I think it's probably more accurate to say it's "doing Linux services the Apple way".
1 reply →
resolved and timesyncd are intended to be removed by the end user of the operating system. Sort of like a transparent peel is intended to be removed by the buyer of a new shiny device.
I haven't seen anything worse than those two from systemd crowd.
Resolved is useful if you frequently change networks (e.g. a laptop), particularly if you’re also using an overlay such as Tailscale. Unfortunately, like anything related to systemd it is flakey and buggy and the solution to most things is to restart it.
I started using Linux because I liked stability. Systemd makes a Linux system as dynamic as a Windows one (which is nice) at the price of making it as stable as a Windows one (which is not).
I believe that part of the problem is that it is written in C, which is an absolutely terrible language for reasoning in. Writing it in Lisp would have been smart. Writing it in Go nowadays would probably be okay. But C is just an unmitigated disaster.
Also, bringing INI to Linux is unforgivable.
4 replies →
It's worth pointing out that Lennart Poettering simply rubbed some people the wrong way in his communication, and that ended up reflecting on systemd, irrespective of the software itself. (I am not making a case that this is good or bad, right or wrong. Just pointing it out.)
It was absolutely the way it was done and the arguments use to shut up the opposition. Then the distros all just adopted it and it was case closed and &*@#$#@ you. This happened with GNOME turning into some attempt at a tablet/phone UI and that upset enough of us to result in Cinnamon and MATE.
You have the people who like GNOME as it is now just as people accept systemd but you might as well be talking to Mac fans - to people who have to learn to like it because they have no choice.
Technically I think launchd might be the inspiration for systemd and for people who like the way the Mac works it's "yay" but it's not necessary.
Anyhow, I like dinit. It's declarative with a simple config language and not a monolith - absolutely perfect.
And there is the track record:
https://security-tracker.debian.org/tracker/source-package/s...
That looks not too bad considering what it does and the time range of CVEs
Maybe you can do a comparison to another well known open source project and see how well systemd fares: https://security-tracker.debian.org/tracker/source-package/l...
2 replies →
some?
Let's rephrase the question the other way: is there anyone who thinks that LP's attitude is fine?
> People have a right to build substitutes and replacements -- I believe in the "Let a hundred flowers bloom" philosopy.
It's a blessing and a curse. Look at package managers, they more or less all do the same thing, with one primary job of "go download some binary so I can run it", yet there's so many to choose from. Every time you read some Linux guide they have to list 7 different ways of installing the same package. It's stupid, probably even more so for the maintainers of those packages because they have to distribute their package 7 different ways. At least I'm glad systemd has mostly become the standard, so you don't have to also see 7 different ways of creating a service.
Usually, it's the distributions problem to package software. You as a software developer publish documentation on how to build your application and then simply wait for other people to do the packaging for you. The creation of services is the same, you can maybe create a recommendation, but the service definition is part of the package file and thus not your problem.
In practice, though, the packagers quite liked systemd, because it a) makes service definition easier than any other system, and b) it significantly increases the likelihood that the developer has already written a suitable service file (and developers will like that that is used, because it reduces the chance that a packager makes a mistake and increases their support burden).
And as an end user of multiple distros, I really appreciate it because I also have to make services on occasion and it's nice that there's one way to do it and it's pretty easy to do correctly.
Usually, this is not good enough. I as a software developer often make use of the package manager built into the language of choice and use that to distribute my software. I also commonly make use of package managers of languages that I don't use to install software.
We are overdue to package manager interop and common interfaces.
I think you answered your own question:
> monolithic
> It's philosophy may differ from unix's "get one thing done well"
> it is bloated.
Besides all this, the main issue, for me, is how it managed to spread and ingrain itself into distributions making them dependent on it.
If you want to use an alternative to systemd on those distributions, you are usually on your own, constantly trying to fix it whenever there are breaking changes.
It's good to have options which are simple to replace.
> It is chock full of features and it gets the job done.
So is Windows :)
>Besides all this, the main issue, for me, is how it managed to spread and ingrain itself into distributions making them dependent on it.
I don't think the phrasing is correct. Your choice of word (spread/ingrain itself) seems to imply there is malicious intent. Software do not sneak itself into distribution by themselves. It is the other way around. Distribution creators have total freedom on what components/software they find useful to build their distributions on. If a majority of distros decided to use systemd, that mean a majority of people maintaining distributions found the positive outcomes of using systemd were worth dealing with any disadvantage it may had over using another solution.
> Your choice of word (spread/ingrain itself) seems to imply there is malicious intent. Software do not sneak itself into distribution by themselves.
No, you're right: It's people that do that. And those definitely can have intent (often benevolent, sometimes malicious, other times just so misguided as to be in-effect-malicious).
So let's go with "the main issue is how some people managed to spread and ingrain it into distributions making them dependent on it."
Does that make it much better?
This isn't true. A lot of higher level stuff (e.g. GNOME) assumes systemd.
1 reply →
> seems to imply there is malicious intent
There is. Well, sort of.
For example, we had cron working just fine for decades . We had sshd listen on its port for decades. We had fstab for decades. No one wanted systemd-timesyncd.
In my opinion, all these aux systemd projects came to life purely out of psychological reasons. Can we label them malicious?
5 replies →
> If a majority of distros decided to use systemd, that mean a majority of people maintaining distributions found the positive outcomes of using systemd were worth dealing with any disadvantage it may had over using another solution.
This is overall fairly weak evidence that users actually find the software to be of quality. Surely there's got to be a stronger signal that this is a positive way forward, like users enthusiastically saying "wow this is an improvement".
3 replies →
How it managed to spread is no surprise. Linux desktop was a complete mess with consolekit and unmaintained stuff. Then they supported cgroups which distros wanted to use and since all the unmaintained stuff didn't there wasn't a lot of options.
Booting up a system is a complex domain. If you randomly cut a complex domain into pieces, you will have the exact same complexity PLUS a huge amount of additional complexity for all the communication/error handling between the different parts - what other complex domain uses million tiny tools? Does chrome use curl and then pipe it into a html renderer and whatnot? Sure, there are libraries (that's a different architectural layer though with less complexity to break, and functions don't decompose arbitrarily either). The unix's philosophy is more of a sounds good on paper, and there are certain cases where it applies - it's definitely not universal.
Also, the core of systemd is not even particularly big. People often mix into completely optional modules that run under the wider systemd project, but that's a false conclusion that "systemd eats the world".
> How it managed to spread
You mean that individual distributions voted/decided separately, multiple times to choose the better tool? Debian has the most democratic voting system and unanimously voted for systemd.
And yeah, if I want to use my own display manager protocol instead of X or Wayland I would also be similarly stranded. Options are good, but standards and compatibility are just as important - a million incompatible options only give rise to chaos.
I am, for example, very happy that Linux applications are finally not as distribution-dependent and there is a good chance to run that .deb file on anything else running systemd without much trouble. I remember the times when it was not so.
> You mean that individual distributions voted/decided separately, multiple times to choose the better tool? Debian has the most democratic voting system and unanimously voted for systemd.
They are influenced by other distros decisions. Debian's justification of adopting systemd starts "Systemd is becoming the de facto standard init system for Linux." https://wiki.debian.org/Debate/initsystem/systemd
I do not think this looks like a unanimous vote: https://www.debian.org/vote/2019/vote_002#outcome
> Booting up a system is a complex domain.
systemd goes far beyond this domain.
> You mean that individual distributions voted/decided separately, multiple times to choose the better tool?
This is a mystery to me. Given the LP attitude is known to be hostile to people with expertise and given the cancerous nature of systemd projects, I really wonder how did people choose to be treated that way.
Maybe they have voted for systemd-as-PID1, which is incomparably better than sysvinit, but this is the way systemd crowd got its foot in the door and before you know nothing works without systemd metastasis present.
3 replies →
> Also, the core of systemd is not even particularly big. People often mix into completely optional modules
So how "optional" are those modules in practice -- how many systems run just the init core of systemd and not the whole shebang?
> that run under the wider systemd project, but that's a false conclusion that "systemd eats the world".
Isn't the very fact that there is what you call a "wider systemd project" at least a fairly good indication that systemd indeed is attempting to "eat the [Linux] world"?
>Besides all this, the main issue, for me, is how it managed to spread and ingrain itself into distributions making them dependent on it.
Because it's better than the alternatives?, don't remember systemd buying distro maintainers lol
>If you want to use an alternative to systemd on those distributions, you are usually on your own, constantly trying to fix it whenever there are breaking changes.
Of course they aren't being paid to support everything under the sun and http://islinuxaboutchoice.com/
>So is Windows :)
Then use it if it's better for you
> Can anybody explain to me again why systemd is so bad ?
It's huge, messy, has a poor bug and security history, obfuscates things that don't need obfuscating, and is just generally IMO not a clean or efficient implementation. It's a very Windows style solution, very different from the lean and minimal stuff I like to run generally.
The advantages it provides are questionable, and are dwarfed by the issues it has had in the past IMO.
OpenRC perfectly meets my needs at present and my system boots incredibly quickly. When s6 is finished that situation will only be improved.
It would be good to know if it's as ergonomic as systemd. I know it's not perfect, and sometimes the docs are confusing, but it is pretty simple to create a service that starts on boot etc. No scripts required, really.
It's pretty easy, you can add stuff to a 'local' script that runs last, or just create a new service and rc-update add new-service to start it on boot.
4 replies →
Half the things you said are straight up lies and the OpenRC is a good example how miniscule your world is
> Half the things you said are straight up lies
lol, no.
> the OpenRC is a good example how miniscule your world is
Weird attempt at an insult, but ok.
> OpenRC perfectly meets my needs
What distro are you running?
Alpine. It's wonderful. I've always liked minimal distros, and Alpine actively facilitates that. It doesn't get in the way, has everything needed to run a modern desktop, and the focus on security and musl compatibility are huge pluses.
4 replies →
I gave up systemd for openRC on Alpine. Much better suited to my own needs, and uptime is vastly improved.
1 reply →
I've noticed the folks who like it tend to want to get other stuff done, don't care about the details, and want things to 'just work'. The ones who don't like it seem to be the folks who do low-level stuff and have to interact with more than it's "happy path" abstractions. i.e. the ones who actually have to care how it was architected.
Sysadmins that have to care how the system is running and being organized behind the scenes gonna kill someone before they have to went back to bash script for system management, people who complains about systemd do it because ideological reasons, don't like LP, Unix philosophy or something, etc, or never learned it and want to argue that their mess of bash scripts, that they use solely to boot their system and nothing more, are somewhat more stable or secure
From the problems and use case you're describing, that's the first category: sysadmins who just need stuff to work so they can move on.
> it gets the job done
Except exactly the opposite is true: it is sysvinit (and derivatives) who had the job done. In a very basic way, slow and perhaps not even compatible with modern requirements and expectations, but the job was done.
Systemd on the other hand does the job it thinks should be done, not you. Your opinion does not matter.
You can spend however much time you want editing /etc/ssh/sshd_config and then restarting sshd and then pulling your hair out on why tf sshd ignores certain options while still definitely parsing sshd_config. Only to later find out that systemd/ubuntu crowd decided to listen on port 22 via systemd, sort of like inetd did back in the days. It's like sshd directly listening on port was not good enough for the last 30 years, definitely had to be changed. Oh yeah, and to make your life miserable, they mask the name of the listening process with "sshd" in it, so you get confused even more.
And people can come up with countless of similar examples.
Incredible attitude and arrogance is what makes people hate systemd.
I'm sure you have other examples (and I'm not really asking for them) but your sshd example would make me dislike the choices Ubuntu made, not systemd. Like I don't blame systemd for snaps being so awful.
You are technically right.
My view is that there is an attitude/approach spillover from systemd crowd into other teams. Switching port 22 from sshd to systemd was not something that any sane person could come up with. It's a stream of incredibly hostile decisions from systemd upstream that allowed that kind of thinking.
3 replies →
Sysvinit left everyone to their own ends to write their own bespoke special ways of configuring & running services. With no security options and no consistency.
Personally I found that to be offering nothing at all & having every maintainer and sysadmin diy their services to be incredibly inefficient & waste a ton of time because every service you looked at was potentially built from a different init skeleton & modified in bespoke unique ways.
Sysvinit offers way way way too little.
Let me reiterate: sysvinit had the job done in an extremely inefficient way that it's not really feasible for long ago. Upstart tried to remedy that but failed.
sysvinit was bad, no arguing about that, but it was controllable, predictable and comprehensible. While I believe that there was a time when systemd-as-PID1 looked like an excellent solution, I am confident that systemd has none of those qualities.
2 replies →
"and it gets the job done."
Until it doesn't. Fortunately I didn't have to debug it often, but when it was a lot of pain.
I prefer the fs under /tmp to be tmpfs. On the BBB is wasn't. I changed that and on next reboot the system didn't come up again -- no way to log in. Why? And what to do? Turns out I failed to state the fs_freq and fs_passno fields -- those aren't optional, but while the old mount tool tolerated their absence, systemd chose to interpret the fstab strictly, failed to mount that fs, consequently didn't reach the local-fs.target and hence didn't offer any way to log-in. Unlike Raspberry Pis, the BBB doesn't offer the possibility to remove (and mount and debug elsewhere) the boot medium. Fortunately, I found help on-line on how to connect a serial terminal to the BBB and interrupt the boot process, forcing the use of /bin/sh as init process...
Big monolith that imposes requirements on your system and robs one of the feeling of knowing how the whole thing works?
At least that’s me. I use systemd in most of my installs for reasons similar to yours, but nothing feels more sublime to me than installing a simple init system and a few other daemons for system features I actually use.
What requirements does it impose?
Also, feel free to "rob that feeling back" by learning how systemd works
Ok, try to go off the well trodden path of a default systemd setup.
Eg. make your system "login interactive" before networking and other unneeded services (database, web server, whatever) are even attempted to be set up/started.
Use just drop-in configuration fragments (*.service.d/override.conf) without making an unmaintainable mess (ie. without making your own copies of whole, OS installed unit files).
That should get you the feel. :) And it's not something crazy, it's just basic re-ordering of dependencies.
2 replies →
> What requirements does it impose?
Numerous kernel features, reliance generally on it by itself and by third party programs (running ceph w/ openrc is hard bc it assumes systemd is present). I’m buying the full car when I use it, when what I really want is a kit that I can optimize for purpose. Honestly though, this close integration is worth it. My main OS build at the moment is pretty much just systemd and podman on top of linux.
> Also, feel free to "rob that feeling back" by learning how systemd works
I really want to! I just don’t know any good resources that can help get me familiar with all it’s doing for me in the background. All I know about it is what I have gleaned from tutorials, I don’t yet feel I know what’s possible with it yet.
>learning how systemd works
Waste of time. It's easier to get used to a systemd-free distro
1 reply →
In my experience, it doesn’t get its extremely simple job done.
I’ve tried a half dozen systemd distros. Each one hit some insane bug at some point that took a day to debug. Every single time, it was some undocumented systemd thing that has no reason for existing.
Anyway, it’s all devuan and openbsd at home these days. I’ve had zero non-hardware related issues in over a year.
Actually, that’s not true. Devuan still defaults to pulseaudio (the moral precursor to systemd), which still doesn’t support locking your audio output port in a way that survives reboot. It also crashes randomly when audio ports come and go. The upside is it randomly requires a reboot or moves audio out to an unconnected jack.
I replaced it with pipewire, bringing the number of Poettering services on my machines to zero. Now things work reasonably.
My n=1 is that I only ever had problems with it during the "early days", now I just use it and get on with things. I can't recall the last issue I had with systemd. tbf I generally only write a few scripts and rust microservices that need to be loaded/start/stop/reload... It's simple to add them to the systemd process. The rest of the system stays out of the way. I will admit I stick to more conservative systems like Redhat and Ubuntu LTS. I guess everyone's opinion is valid on this. It's definitely kind of heavy but it's next to nothing of a load on modern systems of the past 7 or 8 years.
Pipewire has pulse protocol behind the scenes, so your still using his tech you just don't know about that, also I got it now, is mostly ideological thinking and no reasoning behind systemd hate, is about not using that person tech, and if you break dozen of distros messing where you shouldn't the fault is yours not the distro
And yet, some of us manage to successfully run critical national infrastructure on stacks that include Linux distributions running systemd.
I think you are beating a somewhat dead cow here. systemd wars are over. It's in most mainstream systems nowadays, but there are also lots of cool projects out there doing different things. Everything's fine, nobody wants to go into those old pro and con flame wars any more.
Everything is fine, unless/until many developers begin to assume systemd is present and make software ports to non-systemd Linux (or *BSD) systems prohibitively expensive.
> Everything is fine, unless/until many developers begin to assume systemd is present and make software ports to non-systemd Linux
Nothing wrong with this if a system service is going to be present on 99.999% of installs and frees the developer from having to do work.
e.g. GNOME swapped its service manager for subprocesses (e.g. bluetooth) to systemd user units because it does a far better job.
6 replies →
> systemd wars are over
Well yes but actually no.
With every new Ubuntu version I have to carve out new metastasis no one asked for. For example, 24.04 (or maybe even 23.10) changed the way sshd is started up - by systemd listener instead of sshd on its own like it was for decades. This way they saved a few megabytes of RAM (solely on computers that are not exposed to the internet, of course).
While fighting against systemd-as-PID1 is futile for many years, fighting against the spread is definitely worth it.
So you are mad about Ubuntu, not upstream systemd?
2 replies →
Fair, point taken: Some people are working on systemd replacements because they want to build some cool things in that area. Just like there are multiple programming languages and no one says "Why don't you just use Java/C++" it should be OK to work on Linux systems without systemd and not think too much of it.
This doesn't make sense. I, for one (transitioning to Linux usage only gradually), haven't been aware of the systemd and its criticism, and I'm grateful for the info. Then, the informed users can vote with their feet. Things change all the time, and only mass adoption/usage is what makes a given distribution to be mainstream or niche.
So, please do continue to criticize, continue to raise awareness. That's useful.
> Then, the informed users can vote with their feet.
That's the thing, that it is quite difficult to vote with your feet. You can't remove systemd from your distribution in favor of separate independent facilities: The combination of its design, its gradual expansion, and the way some higher-level packages depend on it (especially GNOME) - typically prevent its removal.
So, you would need to switch distributions, which most users are not very inclined to do. And - even then, you look around, and you see that the big basic distros: RedHat/Fedora, Debian (and Ubuntu), Arch - use systemd. So almost all of the distributions based on them are also not an option if you want to avoid systemd.
... and the bottom line is that users will effectively not vote with their feet. And neither will system administrators, or whoever maintains OS images at organizations, because it's difficult for them to tell their superiors they want to switch everyone to, say, Slackware, or Devuan.
NixOS is great at mitigating some of systemd's sharp edges and putting its actual properties in stark relief so that they can actually be evaluated logically (and even swapped out for alternatives without having to drastically change the whole OS distribution).
Things I do not like:
Configuration splayed out amongst tons of tiny .ini files (this manifests in nixos by a mostly boilerplate ~10 line config burp for every service)
Multiple dependency types including its own enable/disable system including a whole parallel configuration tree of symlinks, as opposed to the simple/standard/traditional approach of configuration being disabled by not existing.
Network interfaces/drives/etc splayed out each into multiple services, some from concrete configuration, some implicit - seemingly to make everything visible to its uniform model.
Complex logical model making it hard to debug it without using bespoke tools to analyze dependency graphs and whatnot.
I'm not sitting around hating on systemd because sysvinit was also totally overwrought and obtuse, and BSD style init scripts weren't really scalable. But I get the hate. And I totally see the room for a drastically simpler init system (especially for NixOS). It's fantastic that someone finally dug in and attempted it.
I really also don't understand this. I started using Linux on server OS around 2010. I think this was not long before SystemD was available in Ubuntu by default. I found writing or even understanding init scripts was really difficult for me. When I first had to get a program running under SystemD it clicked immediately and it took me about 4 hours to get it run reliably and that includes learning systemd.
I find it easier and better on so many levels!
Systemd is one of the most complex and difficult to understand parts of your system. Did you know it contains a virtualization manager.
>I find it easier and better on so many levels!
Openrc is by any possible standard easier to use. Among the reasons is that it does not include a way to create virtual machines.
I did not know and I must not know that to get a service running on start.
8 replies →
> Can anybody explain to me again why systemd is so bad ?
it isn't, it's great. it is different, though.
and then come in two phenomenon:
1. people resist to change, often out of ignorance.
2. some people tend to look at tradition and consider it just better, without much reason. "unix systems in the 80ies had shell scripts to launch system services, that's what i shall do in the 2020ies".
> Yes, it is maybe monolithic (but so is the Linux kernel).
it really isn't, it's largely a collection of relatively small binaries that do one thing well but they are designed to work with each other. systemd appears to be monolithic from outside, but it really isn't.
> Maybe some of them will get bloated as they achieve feature parity.
they will, most likely. it's easy (as a piece of software) to be lean and clean when you're doing very little (with little error checking and recovery).
> However I can't understand why systemd is the point of so much disagreement.
systemd was essentially pushed down everybody's throat by Red Hat when they realized it was high time to level up the boot process for the whole ecosystem. systemd made a bunch of things not only feasible (concurrency, on-demand service activation, multiple-instances of the same service running, proper log management) but so easy they're trivial.
so 10+ years, it's definitely a net positive.
some people are still complaining but meh, some people will complain anyway, so let them talk, whatever.
>> Yes, it is maybe monolithic (but so is the Linux kernel).
> it really isn't, it's largely a collection of relatively small binaries that do one thing well but they are designed to work with each other.
So where are all small the projects trying to replace the small parts of systemd? AFAIK there are no stable or convenient APIs in all those parts, so even though technically systemd consists if many binaries, it is monolithic.
This is exactly what people call "forces distributions to use it", i. e., you must choose all or nothing.
I don't believe Linux userspace maintainers are lining up en masse, plus if your alternative implementation doesn't significantly improve the status quo, why even bother?
1 reply →
> So where are all small the projects trying to replace the small parts of systemd?
you should go ask that to "small projects" developers, not to systemd developers, i guess
3 replies →
> systemd appears to be monolithic from outside, but it really isn't.
If it looks like a duck, walks like a duck... systemd is modular solely in theory. In practice, I am grateful that I can still remove journald and resolved from current Ubuntu installs and I'm pretty sure that's not going to be possible quite soon.
Systemd itself may be modular, but practically distros are not going to bother.
> 2. some people tend to look at tradition and consider it just better, without much reason. "unix systems in the 80ies had shell scripts to launch system services, that's what i shall do in the 2020ies".
And from that the systemd crowd tend to draw the following conclusion: if something was done a certain way for decades then it is bad and should be replaced.
Seeing how good systems is and remembering how messy it was to work with init scripts, runlevels and other stuff… i guess they were/are right
1 reply →
>they will, most likely. it's easy (as a piece of software) to be lean and clean when you're doing very little (with little error checking and recovery).
Why would you even begin to think that openrc would ever want to include a whole virtualization manager. Are you serious?
>but they are designed to work with each other.
And they are not designed to work without one another. See e.g. dbus and how hard it is to get it working on non systemd systems.
Systemd binaries aren't drop in features.
Some defaults are not good for servers. For example, systemd gives up after a few tries to restart a service. This is really annoying for server administrators when they don’t know about the bad defaults.
https://michael.stapelberg.ch/posts/2024-01-17-systemd-indef...
If a service fails to startup after n times, I most certainly want to take a look at it as it might as well just end up corrupting its state even more - I don't think it's a fair assumption that most errors will just go away on their own. Nonetheless, it can be configured declaratively as a single line, and as you mention, the default can be changed.
> Can anybody explain to me again why systemd is so bad ?
See the slides of the linked talk, starting at page 7: https://cfp.cccv.de/media/38c3-community-stages/submissions/...
> Guix solves the following problem: – I love scheme and want to minimize the number of hours I spend dealing with any other programming language.
This is a pretty common feeling after writing a couple of nix packages.
systemd has done nothing but make my life easier. The people still moaning about it simply haven't used it and refuse to learn.
Nothing makes my life easier than pulling my hair on why tf sshd listens to port 22 after editing /etc/ssh/sshd_config and restarting multiple times. Oh, of course, it's now systemd that listens to port 22, not sshd.
Nothing makes my life easier than cleaning out crontabs of things that wake my box in the night and find out that something still gets launched. Oh yes, systemd now has timers no one asked for.
Nothing makes my life easier than disabling a service then masking a service then masking it's sockets, then blanking files then chattr +x those files and end up having broken ubuntu instance because there are so many ways to launch a service it's not even funny at this point.
Nothing makes my life easier than finding out multi gigabytes /var/log/journald folder that has exactly zero reasons to exist because text-based logs are still collected and properly rotated in a place where any sane person would expect them to be.
Shall I continue or do you get the gist? :-)
Anyone who tries to use complex software without really knowing what they're doing is going to have a bad time. I'm not sure that's the software's fault :-)
1 reply →
An important skill in life in general is to understand that your personal needs and requirements are not the same as everyone else's.
Some people may be at the other side of bell curve of "understnanding systemd on X axis" and "liking it on Y axis" of the graph. :)
There is a wonderful talk by Benno Rice a few years ago. Insightful and entertaining. Not focussed only on the negatives, BTW.
https://lwn.net/Articles/777595/
For the GP: This is not a talk which explains why systemd is bad, it's a talk which argues which systemd is good and misunderstood. Which I don't agree with, plus isn't what you asked.
Indeed, as the LWN reported stated: "His attempt to cast that story for the pleasure of his audience resulted in a sympathetic and nuanced look at a turbulent chapter in the history of the Linux system."
I believe "nuance" is the key word here.
In the last two years I have spun up new-ish, actively supported versions of ubuntu for pre-production rollout that were shipped with a systemd-resolvd that would occasionally just stop responding for no reason. Logs said nothing was wrong, process was UP and saying it is ready to do work. strace shows a hung process.
Stuff like this is possibly where the complaints from some people begin.
Fortunately, I can find a tracked bug that is active. New updates get pushed, as the honestly phenomenal community of people intended this process to go. But How may years have some of those people been patching bugs like this in systemd on a regular basis? Did that time have to be spent if systemd's architecture was better from the get go?
The Trail of Bugs to get to the point we are at today has truly taken a monumental effort from MANY people. Why wasn't systemd's community doing things well "enough" from the beginning before being folded into so many OSes?
Let's be real. From then till now they have put in hard work, in the face of huge community pushback, because someone had to do it.
Diehards feel it felt "foisted" upon us before it was fully baked, everyone knew sysvinit was way past its prime. Put up or shut up. Plug in a different init for your OS and move on.
But, you know, people like talking and complaining to each other. It's a very important part of solving technical problems!
> Diehards feel it felt "foisted" upon us before it was fully baked
That was the minor problem. The major meta-problems are that:
1. The baked form itself is broken. It has numerous fundamentally flawed aspects to its design (and to its existence as a project). More baking would not have helped this.
2. It is not intended as an option, nor even just the default option - but to make more and more utilities and libraries for managing various system facilities and services become unusable and incompatible, to be taken over by systemd. It's not like foisting, say, Thunderbird instead of Evolution as the default mail client; if you don't like it, you run the other thing. Or even version X of the kernel instead of version Y of the kernel and so on.
> everyone knew sysvinit was way past its prime
At this point we can say with clarity that systemd has almost nothing to do with the choice of init system. You could replace the init system with any number of things - some already existing 15 years ago, some which could be developed in the mean time. In hindsight, this is an excuse and not a reason.
Resolvd is not part of systemd's core. Would some KDE app misbehaving be a fair criticism of the whole project?
Also, I would be delighted to see that mythical program that requires no maintenance and just works^TM.
Also, what does it have to do with systemd's architecture?
The vast majority of criticism of systemd would cease if it were made out of easily mixed and matched components which worked independently of one another.
The goal of systemd is of course to be a highly integrated system which solves everything. As can easily be seen how hard it is to run e.g. dbus without systemd.
> Resolvd is not part of systemd's core
It sure feels like it because Ubuntu insist on installing this incredible piece of software even on the server installs.
I think the previous poster made it clear they were just explaining where the complaining comes from. Why are you trying to debate their description of other people's motivations for complaining?
> Would some KDE app misbehaving be a fair criticism of the whole project?
...yes? If dolphin sucked then that's a fault in KDE.
>Can anybody explain to me again why systemd is so bad ?
I only started to hate systemd when I really had to work with it in depth.
It is hard to understand unless why people hate it if you view it from the surface. On a high level there is nothing wrong with it. If you had to write a basic unit file you could easily do so, there are just a few basic concepts and keywords you need to know to get it working. From that perspective systemd is very much like openrc, which works very much in a similar matter.
Systemd is not bad because it is "monolithic" or because it violates some "UNIX principle" or because it is "bloated". Systemd is bad because it is made by people who have no idea what they want to make and who seem obsessed with solving every single problem they can imagine.
Did you know that on Ubuntu your fstab is fake? It is just a file which tells systemd which mount units it should create during boot. (https://manpages.ubuntu.com/manpages/focal/en/man8/systemd-f...)
Did you know that systemd contains an entire virtualization manager? (https://www.man7.org/linux/man-pages/man1/systemd-nspawn.1.h...)
Did you know that you can use systemd to manage network devices? (https://www.man7.org/linux/man-pages/man5/systemd.network.5....)
Why do these things need to be part of my init system? I can not even imagine what the answer could be.
Systemd is an enormous C project, which has innumerable features, many of them used extremely infrequently and which wants to perform very important tasks for your OS. It has lost any focus on being an init systemd.
People hate systemd because it is bad software and it is bad software because it has no idea what it wants to be. The systemd documentation is one of the biggest rabbit holes in computing.
These aren't part of the init system, they're optional components of the systemd monorepo.
Yes I knew all of these & have used all of these optional components. And they have provided immeasurably better administration than the historical tools that purported to do these jobs before, with far more consistency & clarity about their functioning.
Agreed that there are lots and lots of features. Buts its amazing. Oh you want to make this service secure, with temporary users & limited permissions and limits? Sure of course. Oh you want service restarting? Not perfect I agree but pretty good for me, and avoids some thundering herd problems I've seen other service managers have. There's so many things we could be doing, to make systems better, and a lot of people really are intimidated by & hate the fact that theres a much more competent cohesive system that shows such a lack with the bare bones unchanging unimproving inconsistent low-end insecure dirty past.
Having to work with systemd was the most miserable I have ever been in my life. It is genuinely terrible software and a total pain to use,
> Can anybody explain to me again why systemd is so bad ?
Mostly I seen misguided criticism from less technical people that don't understand systemd but seem to like the idea of fitting in and have identified that a certain few verbal linux users they consider to be cool hate it with passion.
It's really dumb and boring and they often resort to just make stuff up. This thread has a bunch of such comments.
Systemd solves countless real problems and helped linux grow in many ways.
The UNIX philosophy (about which there’s a book by the same name by Michael Gancarz, and which I highly recommend) Is that each tool should do one thing and do it well.
The old school init styles, whether BSD or SysV, adhere to that philosophy.
Systemd is a travesty. I think it was about a decade ago that there was a remotely exploitable root equivalent compromise in the system DS built in DNS resolver. And these days includes not just DNS but also privilege escalation and who knows what else
It’s probably fine for most people and most purposes. And by fine, I mean most people can probably use it and never see a live exploited against it.
And if you care about security, you can probably apply enough mitigating controls that it’s not gonna be a disaster for you.
But for me, defense in depth means avoiding systems to begin with and not trying to bandit over the problems with it
I think OpenRC is easier to use.
But overall, I agree on this point. Having been a Gentoo purist during Uni time, I'm now full on NixOS. NixOS has fully abstracted away any interaction I have with SystemD, so I don't think it's useful to replace it.
How much does it abstract it away? I'm not much of a big Linux user, and when I am, the random 99-something-something files always irritate me because it seems totally arbitrary. Does NixOS prevent you needing to writ "unit" files?
OpenRC might just be easier to use because it does not include a manager for virtualized machines.
> I believe in the "Let a hundred flowers bloom" philosopy. However I can't understand why systemd is the point of so much disagreement.
Your first sentence explains, to a great extent, your second. You see, systemd is designed to _prevent_ a hundred flowers from blooming. Unlike most Unix-like system components, the idea is for systemd not be an alternative replacable component, but the whole installation above the kernel. You don't choose things; you get those services and facilities which live within systemd; and those which don't - mostly won't work. That is, unless you tear out everything related to systemd. Which is difficult, since GNOME and some other packages over time have introduced dependencies on systemd, rather than on services or facilities it provides via a common protocol or interface.
This was also how the big Debian debate exploded, causing the fork of Devuan: The "opposition" did not demand "No systemd in Debian!", but rather, that the user would be able to choose not to have systemd at all, if they so please. I am paraphrasing from memory of course, but I believe that was the gist of it.
And the speaker in this video faced this situation.
I am not well-versed in NixOS and certainly not in this new project, but I imagine that if the author could easily just configure NixOS not have systemd, they might not have bothered starting this project and would more likely have published some kind of recipe/how-to page on
It's incomprehensible. I make some changes to config files and they don't apply so I have to find whatever poorly-documented systemd thing is controlling it instead. It is taking over every important area of the operating system. Init, login, boot, it just keeps spreading. I think on other Linux systems I am better able to reason through and modify aspects of the system or fix a problem I am having.
I have been trying out atomic Fedora and enjoying it. I like ostree, read only root, podman, building the distro from containers. I think I have had enough of systemd, though, and might move on because of it.
For me, it's primarily its scope. I had a system for 10 years that changed, but fundamentally worked the same. With systemd, the changes exceeded my threshold and continue to expand into something that to me, seems ugly.
I'm not a typical example. My view is inarticulate, subjective and maybe even irrational, but I remember the system I came to love and that was Linux. My overwhelming impression with systemd is that it's not Linux and it should not be referred to as Linux or even in the context thereof. It's not evil or bad or anything but itself, but it's not Linux. In my abstract, medieval view, Linux has a soul, albeit, at times, a tormented one. Systemd exorcises this soul for me.
Now despite me disclaiming the subjectivity and abstraction of these perceptions, I'm pretty certain there will be some hostile replies, possibly with valid points. If you're really trying to understand the creature, I suspect it's very improbable that it will happen here. There are existing strong, well written arguments both for, against, and even neutral. Definitely worth one's time if understanding is the true objective.
For me, it's scratching a chalkboard and smells bad. It frightens me and makes me sad. It's worse than brussel sprouts and freaks me out.
"My overwhelming impression with systemd is that it's not Linux and it should not be referred to as Linux or even in the context thereof. "
Yes, systemd is an init system/service handler (and a lot of related utilities) running on top of the OS kernel Linux. They are two different things working together to make an usable OS.
Thanks for the intel man. I might have not known without your help. And I'm sure you wouldn't insincerely leap on an overly literal interpretation just to leave a silly comment.
But yeah, you cracked a tough one there. Two separate things. Who'd have thunk it!
1 reply →
Sorry, but there many people who do not like bloat, bad practicies and daily security bugs, partially because of the aformentioned bloat. If we settle this "gets the job done" mindset we are never going to improve.
It might be just psychological not technical.
Humans have a natural tendency to change resistance. This tendency tends to be higher in sysadmins who are often deep into autistic spectrum.
Because Karen Sandler made it a hard dependency of GNOME, effectively forcing every distro to adopt it or give up GNOME. In Linux, parent processes hold dominion over all child processes, due to the way things like ptrace work. So it was a coup d'etat of the entire userspace. Then next thing you know, systemd took over the bootloader too, which gave it dominion over the kernel as well. Old school sysadmins don't like being dominated. Especially not by some useful stooge like he who shall not be named. The source code of his projects is open source in name only. Least hackable code imaginable. Almost like it's machine generated code to make the sources as opaque and inscrutable as possible. It uses binary formats to log data, exchange messages between processes, and other awful things to obfuscate its behaviors. Someone with zero gravitas shouldn't be reinventing the vision of Bell Labs and forcing their ideas on others through NGO policy changes. I'm surprised Trump hasn't signed an executive order banning it. Even the name itself, System D, expresses this arrogance, since it claims to be 100x better than UNIX System V, because D=500 and V=5 in roman numerals. Being forced to use this init system changed the workflow and daily lifestyle of every sysadmin. In one fell swoop they took an OS that's open, transparent, and owner controllable and made it into something more obtuse than Windows. And guess who the stooge works for? Microsoft! It's the sort of thing that would make people pivot their careers towards management or quit working.
"Because Karen Sandler made it a hard dependency of GNOME, effectively forcing every distro to adopt it or give up GNOME."
How did she do that? Can you dig up the git commit where that happened?
By being the executive director of the GNOME Foundation.
1 reply →
Just see `systemctl $command $unit` to discover that systemd it's not designed by sysadmins, while it's built for them. That's it's problem, like stratis/btrfs vs zfs.
These are software designed by people who have no experience in user-centric design, they design in theory, reasoning in theory, ignoring practice.
SystemdD does too much and it's like a monopoly, because it's so much work to replace, it can piss you off a lot, it needs to be very shitty for you to replace it.
>However I can't understand why systemd is the point of so much disagreement.
I think the simplest way to describe it is that it's a religious conflict.
A not insignificant segment of people who develop and/or use Linux are fervent followers of POSIX philosophy, the Unix heritage, Free Open Source Software development, and the Anti-Microsoft Church (note: Poettering is a Microsoft employee).
These people have a religious duty to hate systemd, it's essentially heresy. It's not a disagreement based on technical or even practical merits, so the conflict can't be addressed with technical debates and will continue for the foreseeable future.
You think you get to just declare the opinion that differs from yours invalid by calling it religious?
Don't the holders of that opinion get to call your position equally religious?
"not a disagreement based on technical or even practical merits" indeed, right there in the mirror.
The first few minutes of this video absolutely articulated some perfectly technical and practical merits. So have some comments here on this hn post. I don't see how it's valid to say no one has ever produced any.
I think the debate on systemd as regards to its merits was decided already years ago when all the major Linux distros adopted it. You need to go way out of your way to run Linux without systemd now, people who want that setup have clearly become a (very small) minority while the rest of the Linux world moved on.
The conflict as parent commenter witnesses it continues because that small minority continue to (very loudly) disagree not on the merits. It is their prerogative to disagree if they desire, but their basis is much more about beliefs than superiority.
It's like Polandball memes where UK crazily chants "Britannia rule the waves...!" while US goes off to freedom some oil with its boats.
3 replies →
> disagreement based on technical or even practical merits
Philosophy can't be reasoned based on merits. Of course there are technical merits as to why systemd-as-PID1 was (not today!) an extremely good piece of software compared to sysvinit, there's no question about that. Likewise there are valid technical reasons why systemd-timesyncd should be abolished and never mentioned again.
But how do you decide LP's attitude on merits? How do I argue with Red Hat who shoved systemd down everyone's throat?
So yes, it's a religious question of sorts. The religion that made Unix unix.
The usual arguments against systemd border on religious drivel. I've seen the talk and while I very much value the work that the speaker has done I did not appreciate that the reasons for doing this are the usual vitriolic cat-v talking points.
systemd is very good because it makes many things that the Linux kernel can do very easy. I would like to know how the people who swear against it implement features that I regularly use in systemd like socket/mount/dbus activation, services that dynamically create a user and group on activation and keep their service and temp directories private from other services, syscall filtering, user session services that start when I log into a graphical session (very useful when you have issues with your system tray's config, for example), network mounts that get mounted asynchronously only when you're using them, actual service management and restarting a service when it fails and service dependencies (which some init systems still don't do!), and so on and so forth...
Yes you could do all of these things by composing other programs, but there is lots of value in having them all bundled together and only having to consult one resource for documentation on them, and the fact that these are all designed to work together reduces the friction that you would get by composing other "general-purpose" tools.
On the other hand, systemd is bad because the implementation is messy, when it does fail it tends to do so in odd and obscure ways, it comes bundled with tons of components that most people won't need, and yes the fact that it's essentially the only option you're given and that it's not portable to BSDs is not very nice.
I would encourage people to read dinit's comparison page and Chimera Linux's FAQ section on systemd for good arguments that are not fueled by religious belief as to both why systemd is valuable and in which ways it is bad.
https://github.com/davmac314/dinit/blob/master/doc/COMPARISO...
http://chimera-linux.org/docs/faq#what-is-the-projects-take-...
Here's my hot take: systemd is the best thing that happened to Linux.
I genuinely believe it's the reason Linux has dominated the other operating systems in the server space in the past decade. As someone who manages servers I wouldn't even consider the other options in space right now, and the features systemd provides is a substantial part of that decision.
> I genuinely believe it's the reason Linux has dominated the other operating systems in the server space in the past decade.
I don't think that makes sense; Linux was already completely dominant before systemd came along.
As a person who manages thousands of servers I genuinely fail to see systemd as anything but a problem. I can certainly see systemd on Desktop, where the environment (network, screen resolutions, peripherals) are always changing on the go, but on server where everything is more or less fixed... systemd? The worst. Unlimited source of problems.