Comment by bjourne
3 years ago
Glibc is not Linux, and they have different backwards compatibility policies, but everyone should still read Linus Torvalds' classic 2012 email about ABI compability: https://lkml.org/lkml/2012/12/23/75 Teaser: It begins with "Mauro, SHUT THE FUCK UP!"
man it's always a trip to see how much of a jerk torvalds could be, even if exasperation is warranted in this context (i have no idea), by god, this is not how you build consensus or a high functioning team
The context, from Mauro’s previous message:
> Only an application that handles video should be using those controls, and as far as I know, pulseaudio is not a such application. Or are it trying to do world domination? So, on a first glance, this doesn't sound like a regression, but, instead, it looks tha pulseaudio/tumbleweed has some serious bugs and/or regressions.
Style and culture are certainly open for debate (I wouldn’t be as harsh as Linus), but correcting a maintainer who was behaving this way towards a large number of affected users was warranted. The kernel broke the API contract, a user reported it, and Mauro blamed the user for it.
I'd like to mention that Mauro is a very nice person. I worked briefly with him when I submitted some patches to ZBar and it was the best experience I've had contributing to open source to this day. He gave me feedback and I got to learn new stuff such as D-Bus integration.
When this comes up in conversation it is worthwhile remembering that Linux was built on the team of volunteers centered around Torvalds who was famous for not acting like a jerk. Really. The perception of him among hackers of being a good guy, you could work with, who acknowledged when linux had bugs, accepted patches and was pretty self-effacing is probably the thing that most made that project at that time take off to the stratosphere. Linus was a massive contrast to traditional bearded unix-assholery.
The nature of the work changes. The pressures change. The requirements change. We age. Also the times change too.
But yeah, it is possible to act like a jerk sometime without actually being a jerk in all things. It is also possible to be a lovely person who makes the odd mistake. Assholes can have good points. Life is nuanced.
Of the bajillion emails linus has sent to lkml how many of them can you find that you believe show evidence of being a jerk.
Compare to Theo de Raadt at OpenBSD who have also built a pretty useful thing with their community. Compare also to Larry Wall and Guido van Rossum.
None of us is above reasoned, productive criticism. Linus has done ok.
It’s not my personal style, but there’s plenty of high functioning teams in different domains headed by leaders who communicate like Torvalds. From Amy Klonuchar throwing binders (https://www.businessinsider.com/amy-klobuchar-throwing-binde...) to tons of high level folks in banking, law firms, etc.
Put differently, you can construct a high functioning team composed of certain personalities who can dish out and take this sort of communication style without burning out on it.
I've definitely seen teams that were low functioning because they were so worried about consensus and upsetting someone else that no one ever criticized any decisions any team member made even if they were both impactful and objectively terrible.
13 replies →
> but there’s plenty of high functioning teams in different domains headed by leaders who communicate like Torvalds
Maybe in the past. It is not acceptable now.
A lot of men from the "old days" are finding that their table thumping "plain talking" (obscenity shouting) ways are getting them sidelined and ignored.
Good.
11 replies →
Are the teams high functioning because of that, or despite that?
3 replies →
That probably works if people get bribed with interesting enough projects (Linux) or money (banks, lawfirms...). Most other projects probably fall apart before you can blink an eye
Speaking about consensus - there is another thread on the HN where people complain about Android 13 UI. I guess that was built with a healthy dose of consensus.
The point is - sometimes you need a jerk with a vision so that the thing you're building don't turn into amorphous blob.
You need someone with vision who enforces strict adherence to that vision. I'm not convinced you have to be a jerk to do that though.
1 reply →
The old quote: I would trust Einstein, but I wouldn't trust a committee of Einsteins.
How is Steve Jobs getting left out of this conversation
1 reply →
>by god, this is not how you build consensus or a high functioning team
Says you, while criticizing Linus Torvalds from 2012. Who has a better track record of building consensus and high functioning teams ?
Says Linus Torvalds from 2018…
> My flippant attacks in emails have been both unprofessional and uncalled for. Especially at times when I made it personal. In my quest for a better patch, this made sense to me. I know now this was not OK and I am truly sorry.
> The above is basically a long-winded way to get to the somewhat painful personal admission that hey, I need to change some of my behavior, and I want to apologize to the people that my personal behavior hurt and possibly drove away from kernel development entirely.
https://lkml.org/lkml/2018/9/16/167
He still writes very frankly, but he generally doesn’t resort to personal insults like he did in the past.
2 replies →
I think if you take it out of context (which most people do), it looks a lot worse than it is.
A very senior guy who shouldve known better was trying, fairly persistently, to break a very simple rule everybody agreed to for a very bad reason. Linus told him to shut the fuck up.
I wouldnt say that Linus's reaction was anything to look up to but I wouldnt say that calling the tone police is at all justified either.
I mean, the guy sent one short email before Torvalds flew off the handle; that’s hardly “trying persistently” to break a rule. I can think of a thousand assertive ways to tell the guy to shut up that wouldn’t have required behaving like an angry toddler.
> by god, this is not how you build a consensus or a high functioning team
I beg to differ. Linus Torvalds is an example for us all, and I’d argue he has one of the most, if not the most, highly functioning open source teams in the world. The beauty in open source is you’re not stuck with the people you do not want to work with. You can “pick” your “boss”. Plus, different people communicate differently. Linus is abrasive. That is Okay because it works for him. What is not okay is having other people policing the tone in a conversation. Linus had this same conversation with Sarah Sharp, I’ll post the relevant quote below:
Because if you want me to "act professional", I can tell you that I'm not interested. I'm sitting in my home office wearign a bathrobe. The same way I'm not going to start wearing ties, I'm also not going to buy into the fake politeness, the lying, the office politics and backstabbing, the passive aggressiveness, and the buzzwords. Because THAT is what "acting professionally" results in: people resort to all kinds of really nasty things because they are forced to act out their normal urges in unnatural ways.
> man it's always a trip to see how much of a jerk torvalds could be, even if exasperation is warranted in this context (i have no idea), by god, this is not how you build consensus or a high functioning team
True. I think Linux could've been pretty successful if someone with good management practices had been in charge from the start.
> by god, this is not how you build consensus or a high functioning team
Linus has been pretty successful so far. There's not just "one style" that works.
maybe it is how you build the world's most popular operating system?
because he did
People often think that because jerks work at successful companies, you need to be a jerk to be successful. It’s more the other way around: a successful firm can carry many people who don’t add value, like parasites.
Guarantee you Linus wasn’t this bad in the 90s.
I think he's not this bad these days? He issued some public apologies for his behaviour. He gave us Linux and Git. Yes, he used to be an asshole, but he still did way more for the betterment of humanity than most people.
1 reply →
> Guarantee you Linus wasn’t this bad in the 90s.
Guarantee you that Linux was not that big and influential in the industry in the 90s.
1 reply →
naw man, let the old git be. he is a lovely old man. one day we wont have people like this. he gave more than he took.
He's a product of a different time. Personally, I love his attitude -- wouldn't want to work under him though.
Glibc is GNU/Linux though and cannot be avoided when distributing packages to end users. If you want to interact with the userspace to do things get users, groups, netgroups, or DNS queries you have to use glibc functions or your users will hit weird edge cases like being able to resolve hosts in cURL but not your app.
Now, do I think it would make total sense for syscall wrappers and NSS to be split into their own libs (or dbus interfaces maybe) with stable ABIs to enable other libc's, absolutely! But we're not really there. This is something the BSD's got absolutely right.
There are other libc implementations that work on Linux with various tradeoffs. Alpine famously uses musl libc for a lightweight libc for containers. These alternate libc implementations implement users/groups/network manipulation via well-known files like /etc/shadow, /etc/passwd, etc. You could fully statically link one of these into your app and just rely on the extremely stable kernel ABI if you're so interested.
We're not disagreeing. You can, of course, use other libc's on Linux the kernel, but you cannot use other libc's on GNU/Linux the distro that uses glibc without some things not working. This can be fine on your own systems so long as you're aware of the tradeoffs but if you're distributing your software for use on other people's systems your users will be annoyed with you.
Even Go parses /etc/nsswitch.conf and farms out to cgo when it finds a module it can't handle. This technically doesn't work because there's no guarantee that the hosts or dns entries in nsswitch have consistent behavior, it's just the name of a library you're supposed to dlopen. On evil, but valid, distro resolv.conf points to 0.0.0.0 and hosts module reads an sqlite file.
1 reply →
> Now, do I think it would make total sense for syscall wrappers and NSS to be split into their own libs (or dbus interfaces maybe) with stable ABIs to enable other libc's, absolutely!
I worked on this a few years ago: liblinux.
https://news.ycombinator.com/item?id=28283632
But there are other "Linux"'s that are not GNU/Linux which was I think the point. Like Android, which doesn't use glibc, and doesn't have this mess. I think that was one of the things people used to complain about, that Android didn't use glibc, but since glibc seems to break ABI compatibility kinda on the regular that was probably the right call.
Solaris had separate libc, libnss, libsocket, and libpthread, I think?
Unlike many languages, Go doesn't use any libc on Linux. It uses the raw kernel API/ABI: system calls. Which is why a Go 1.18 binary is specified to be compatible with kernel version 2.6.32 (from December 2009) or later.
There are trade-offs here. But the application developer does have choices, they're just not no-cost.
If in distribution discussions Linux is name for the operating system and shell, downplaying the role of GNU, then it is also fair game to say here: Linux does not have a stable ABI because glibc changed.
Really appreciate your stuff Bjorn, this link always brings a smile to my (too young to be cynical) face.
Thanks!
I am no longer able to see this comment. It says the message body was removed.
Anyone else? I'll have to assume this is the history of how we built great things being deleted in realtime.
I think lkml.org has issues with lots of traffic: https://lore.kernel.org/lkml/CA+55aFy98A+LJK4+GWMcbzaa1zsPBR...