Comment by tadfisher
10 months ago
Do you feel the same about Make, Device Tree, KConfig, Python, the myriad of machine-specific assembly, POSIX shell, Perl, and every other non-C language currently in the code base?
10 months ago
Do you feel the same about Make, Device Tree, KConfig, Python, the myriad of machine-specific assembly, POSIX shell, Perl, and every other non-C language currently in the code base?
That's a false equivalency. All of those languages are integrated through strong semantic and concrete abstractions (e.g. file system I/O or GCC interfaces) that evolve, if at all, very slowly. Some, like assembly, are necessary concessions, and are exceptions that prove the rule--developers prefer GCC builtins if available, for example.
The problem with Rust in the kernel is that the kernel has historically eschewed strong internal abstractions. The Linux kernel has no principled abstract architecture in the same way that Windows NT does; it has evolved very organically and even some of the most foundational subsystem APIs regularly see refactorings that touch almost every corner of the kernel.
The latest dispute (if it could be called that) regarding Rust was Rust building abstractions around the DMA interface. There's nothing wrong with this, per se. For Rust it's a necessary chore. But when you build complex abstractions atop something, you're either ossifying the interface or building a sand castle. If we're being charitable, some Linux developers felt like this was pressure to ossify the existing abstractions. Rust developers, OTOH, promised that they'd take full responsibility for any future refactoring, implicitly admitting that they understood they were building a sand castle.
How do you bridge that divide? Because of the lack of hard architectural line drawing in how Linux is developed, the norm is that both redesigns and refactoring are in many respects highly cooperative (notwithstanding the bickering and friction). A subsystem maintainer contemplating a redesign takes into consideration the burdens on other users, and how a redesign might be incrementally rolled out, if at all. Conversely, users of interfaces know--or should know--to take into consideration future potential changes in an interface when relying on an interface for their subsystems. It's a constant back-and-forth, give-and-take, but also messy and chaotic. Transparency in source code is key--this kind of development would never work in commercial projects across binary interfaces. Yet in important respects that's what the interface between C and Rust is like--opaque--especially when developers on one side aren't intimately familiar with the semantics of the other and how they translate, if at all, across the boundary.
Now here comes Rust, which has spent years putting into place the infrastructure just to get some drivers going. Rust as a language demands careful architecture and line drawing. It's no surprise that much of that initial infrastructure effort, excluding the build, was expended on building towers of abstraction. Refactoring can be painful in Rust when it involves very low-level changes in semantics (the kind that are common in kernel development), and while there are tools to help address that, they don't work well, or at all, outside of Rust. There's a huge impedance mismatch between Rust and historic Linux kernel development. In user land, developers' experience is that Rust is relatively easy to interface with C libraries. But that experience is primarily interfacing with public APIs, those public APIs have always been quite stable, and user land libraries (at least the good ones) are designed to be, as much as possible, blackboxes from the outside. Abstractions that leak tend to be fixed abstractions, like file descriptors, etc, that are typical for the environment and rather predictable.
That situation with user land C FFI is utterly incomparable to how interfaces evolve in Linux. The clash between these worlds, at both a technical and cultural level, was inevitable. Heck, it was evident from day 1. This doesn't make either side right or wrong, and I'm not trying to suggest that the Linux developer community can't figure out a path forward. But it's a difficult problem on many levels.
> Refactoring can be painful in Rust when it involves very low-level changes in semantics
I don't know what this is about. In my experience, refactorings that change the semantics of APIs are much easier in Rust than in C. E.g., change assumptions about the lifetimes of pointers passed into APIs: the Rust compiler will tell you where you need to change anything; the C compiler will happily compile your code and you'll corrupt memory at runtime.
Right, and while sometimes a lot of code needs to change in Rust, it's often largely mechanical, at least in typical contexts (e.g. user land development). But Rust won't automatically detect changes in semantics that happen across FFI boundaries. And I didn't argue that refactoring is easier in C than Rust. For one thing, it's an inapt comparison in the context of a mixed C and Rust kernel codebase in which the vast majority of the code and semantics come from the C side.
And there are more subtle issues. From the perspective of C code, Rust looks and behave as if it assumes strict aliasing, a consequence of the borrowing rules--a mutable pointer can never alias. But the Linux kernel uses -fno-strict-alias (i.e. any pointer can alias, regardless of type), so a subtle change in C code which works fine for the kernel could silently break Rust code if the C-side developer wasn't aware of these subtle nuances in memory models and how they were expressed in the unsafe wrappers. This might be a totally contrived scenario[1], or it could be very real given the tower of abstractions built on the Rust side over the C APIs, which might overspecify certain semantics to fit "cleanly" (i.e. best practice) into the Rust model.
Which points at another issue: all of these hypotheticals might be (probably are?) overblown. But over the past 2-3 years, with the exception of a couple of high-profile drivers not yet mainlined (AFAIU), the vast majority of the effort on the Rust side has been building towers of abstraction. In almost any open source project, C or otherwise, a newcomer who starts writing towers of abstractions rather than concrete, usable code would be shooed away. There are reasonable justifications for why Rust has been doing this, yet its also understandable why this might draw suspicions about the practicality and utility of mixing C and Rust in the kernel. But either way it means that after all this time people are still largely arguing over hypotheticals.
[1] Or entirely wrong. Maybe kernel Rust is using flags to ensure aliasing optimizations can't happen. clang (and thus LLVM) supports -fno-strict-alias, but AFAIU alot of Rust code massaging happens on the Rust (MIR or equivalent) side.
4 replies →
"The problem with Rust in the kernel is that the kernel has historically eschewed strong internal abstractions. The Linux kernel has no principled abstract architecture in the same way that Windows NT does; it has evolved very organically and even some of the most foundational subsystem APIs regularly see refactorings that touch almost every corner of the kernel."
And this is the reason why it's always regressing and never moves beyond alpha.
No, I feel this way about the Linux kernel.
Those all exist in the Linux kernel repository. Granted they are not overlapping with c in their purpose for the most part.
One language per repo rules makes sense when you can have many smaller repos, but for something as immense as the Linux monorepo it's really limiting. Especially considering the lack of desire to add stable interfaces from which other repos could operate independently.