Author here. I'm surprised to see my hobby project on Hacker News.
I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.
That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel. You can learn how printf(3) works. You can learn what happens before main().
Especially, in Rust, it's inspiring to explore ways to write a kernel safely and expressively. One of my favorite part is the implementation of read system call [1].
Lastly, You can try this OS on SSH. Hope you enjoy :)
> I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.
Considering that this is exactly how Linux was born (just a hobby project for fun), I wouldn't assume so fast that it's useless. Moreover, you don't need to justify yourself if you just want to have fun !
I don't see why we need to pretend that every hobby has the potential for greatness. Being a hobby is a good enough end to itself.
In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. The industry is totally different now. There's much more competition than there was when Linux was released and that competition is much more mature too. Plus the bar for a production-quality kernel is a lot higher than it was when Linux was released.
Aside from "rewrite in rust" debate that I was unaware of, there seems to be this pervasive attitude among the HN hivemind that the end-goal for all projects, side- or main-, is "launch", and therefore needs a market analysis to decide the "worth" of such an idea, and it ends up being rather silly.
For example, I've written a Mandelbrot visualizer so many times I've lost count. Not because the world needs another poorly written or optimized rainbow-ladybug-simulator, but because it serves as a slightly-non-trivial hello-world. For example, it's the first end-to-end thing I made in Common Lisp. https://git.sr.ht/~amtunlimited/mandelbrot-plot
Wow, I really like the implementation of that syscall. I've got my own toy OS project in C, for the M68k, and my `read` is "a little" less clean [0, 1].
Even "rewriting Linux in Rust" is far from a waste if you make it run and if your code quality is good enough for others to join.
If only you could write an entirely new (meant to be better, for some cases at least) kernel compatible with Linux hardware drivers - this would be not waste at all but in fact fantastic.
The most challenging point is, as others said, the lack of the compatibility with Linux Driver API. I believe it would be really hard to implement and keep following changes in Linux.
An idea in my mind is to use Kerla for virtualized environments like KVM where only limited device drivers are needed (virtio for example).
In addition to its huge contribution to OS development in Rust, as a microkernel enthusiast, it sounds exciting to writing a microkernel in Rust, in the "Everything is a URL" principle. Moreover, it can run a good-looking GUI on a real machines [1]! I know it's very hard to be realized.
Aside from Redox, I'd mention Tock [2], an embedded operating system. It introduces a novel components isolation using Rust's power. I believe this field is where Rust shines and am looking forward to seeing it in production.
> That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel.
> You can learn how printf(3) works. You can learn what happens before main().
Yes! It's such a wonderful experience. This is exactly what I most enjoy doing, just seeing how things work, maybe making my own version. I hope you have lots of fun.
You're reimplementing Linux's kernel-userspace binary interface, right? The system call interface is stable and language agnostic, it's really nice. Some pointers for anyone who'd like to know more:
I honestly think there is real value here for firecracker style containers. Running memory safe code for your whole stack, with a minimal number of virtio devices? Fantastic!
The permissive license would also be interesting in some applications.
I'm new to Rust and am far less along than you are, as evidenced by this project. I noticed that in the few files I spot checked, there aren't many/any tests. Without any context or opinion, I'm curious whether unit and integration testing are hard for a project like this.
nice project! curious how the rust ownership model works in a monolithic kernel context. are the lifetimes of kernel structures associated with a process "owned" by the process itself somehow?
About 3-4 months in total: took 1.5 month to run a simple Hello World program, 1 month to implement bunch of system calls, and another 1 month to implement features essential to run Dropbear SSH server (tty/pry, bug fixes, …).
"I completely agree that rewriting existing large, feature-rich, and robust operating system kernels is not a good idea. However, a question came to my mind: what are the pros and cons of writing a practical UNIX-like OS kernel in Rust from scratch? How does it look like? This is why I started this project."[1]
Rewriting the Linux kernel in rust would be a rehash of why we ended up with Linux in the first place: rewriting an older OS from scratch.
I would love to see a solid implementation of a real micro kernel based OS based on a message passing core in Rust to become a viable alternative to the now 50 year old Unix architecture. This would give us a possibility of some real progress on the security and process scheduling front, both of which are severely lacking in the Unix model.
Why do people implicitly assume that micro kernels are the best OS design?
In fact, micro kernels have a lot of issues and limit the possibilities of what an OS can do! One of the most important issues is they prevents resource sharing in a rich and efficient way between the components of the kernel. They make building rich relationships between kernel data structures and kernel subsystems very messy, complicated, impracticable and very inefficient.
Micro kernels are delusional and people should stop implicitly assuming their superiority.
Historically, one of the main arguments against microkernels is that having a kernel that's broken up into a lot of threads that each have their own hardware-enforced memory protection and use message passing to communicate with each other is dreadfully inefficient. All those context switches are expensive, and copying all that data around when you send messages is expensive.
Rust's memory model largely makes those isolated address spaces unnecessary, and sending a message at runtime basically amounts to copying a pointer. So, with these performance issues resolved, I think it makes sense to look at microkernels again. The reasons why Linus was opposed to writing Linux as a microkernel in the 90's don't necessarily still apply now in 2021.
There may some things the Linux kernel does that can't be efficiently expressed using a message-passing style, but I don't know if that's actually true or not.
This is a limitation of the memory model, not the microkernel. If you had a single hardware-protected address space, kernel modules could share resources freely.
But this does argue against message passing, since such systems preferentially try to use messages over sharing whenever it's possible.
Mach's internals didn't follow the Unix model, and was by almost any metric a failure.
There have been several other "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.
Process scheduling in Linux in 2021 is far, far ahead of anything in any other OS, ukernel or MP-based. The idea that there hasn't been progress in this area is really completely absurd. In fact, the general idea that Linux follows "the now 50 year old Unix architecture" is also pushing the boundaries quite a lot.
> "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.
BeOS wasn't half bad. The failure was probably mostly commercial. It is true that they moved their networking stack into the kernel, but it's not entirely clear to me that they had to do that, it was just the most expedient way to get acceptable performance and stability given the (programmer-time) resourcing constraints of the project.
Mach is of course doing quite well and moving back towards being microkernel-y as hardware gets fast enough to support it and Apple needs a better security story.
Written properly, a monolithic kernel in Rust would provide many of the benefits of a microkernel, due to the memory safety. It'd be much along the same lines as Microsoft Research's Midori, with software isolation instead of hardware isolation.
It would be wide open to specially crafted drivers, it's the equivalent of client side security (or maybe that should be 'compile time security'). Memory safety is something only process space isolation can bring.
What are some ways I can increase my knowledge in this domain that the OP is very skilled at, meaning low level OS development?
I've taken an intro to OS class and am currently going through Linux From Scratch [1], which is interesting and is teaching me a lot, but it's more about how to setup a Linux distro using existing packages and not really about reading/writing the code involved.
One of the easiest ways to get into writing your own kernel is to start with the OS Dev wiki [0]. Both the wiki and their forums are a great place for getting your feet wet.
For more general information, there are also a handful of articles and/or blog posts that I've come across when I was getting started that gave a very good introduction to many of the low-level components of how computers work and what the OS needs to do to run them. One of my favorites was Many But Finite [1]. Check out a bunch of his posts about the memory map, the computer boot process [2], cpu rings, etc... Not exhaustive by any means, but good, well-presented info. Many other such sources exist; maybe others will share their favorites.
And if you'd just like to get a more gentle introduction into actual OS code than the monster that is Linux, maybe try looking at some of the BSD's. NetBSD [3] and OpenBSD [4] are remarkably easy to navigate and follow from a code flow perspective, and if you just want to see how things work in a real OS, it's both a good reference and a good place to start.
I think you're on the right track. I am similar to you and not so skilled at low level OS development. There is an educational OS called PIOS from Yale's CS department with specific parallelism goals in mind, but it boots from metal so the code is a great resource. Here's the code: https://github.com/bford/PIOS
Also, you could try reading the Plan 9 source code.
(I would say the code is what you're after now, but in case you are interested in more of the theory of why it's designed that way, you can check out the research paper here: https://dedis.cs.yale.edu/2010/det/)
Linux from scratch won't really help you. I'd say start by looking into writing drivers for Linux and looking at tiny OS examples for microcontrollers that can be run in emulators or cheap boards. Bare-metal projects are available for pi and beaglebone. You will absolutely have to do something like that at a minimum, so it's good to apply what you learned in your OS class to that stuff.
This is awesome. Imagine the potential if it was just complete enough to run Kubernetes’ kubelet on a cloud instance. Lots of security minded folks would love it.
That's IMHO how you get something like this off the ground. Make it useful for a simple use-case while offering something of value - in this case the security of Rust. It can then become a preferred solution to some problem and that will bring users and developers looking to incrementally expand it.
I wonder, if you run every container in it's own MicroVM a-la Firecracker like Fargate does (or Kerla's demo host), does the security of the kernel still matter as much?
That’s very cool! Could you share some details on how to set it up for oneself with AWS Firecracker the same way that this demo was set up?
Also, as owner of the instances are you able to see what commands people are running? First thing I did was to type hello, and it said hello world to me :)
Author here. I've made the demo system public [1] but it's written just for me so it should be painful to set up the same environment.
The mechanism is pretty simple: a Node.js server (running on genuine Linux) listens on tcp:22. Once you connect, it boots a dedicated Firecracker microVM instance and forwards network packets between your SSH client and VM.
Regarding the command history, others (including I) can't see what you type. If you could, it must be a vulnerability.
I think a killer feature would be to emulate the Linux kernel device driver model such that existing C driver modules could be used with Kerla. Further, borrow (no pun intended!) the Linux device driver Rust wrappers from the Rust for Linux kernel project to then enable writing Linux device drivers in Rust.
The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.
I think there might be reasonable trade offs that make the problem more tractable.
Having the ability to have a significantly simpler kernel with Rust’s memory and thread safety guarantees plus the ability to use even a small set of Linux’s massive set of drivers would be very compelling.
Linux drivers don't have a fixed ABI (or even API), making such a feat somewhere between hard and impossible. You'd have to constantly play catch up with the latest kernel refactorings, which would require a tremendous amount of engineering effort.
Heck, it'd probably be easier to emulate the Windows driver architecture - at least those have proper ABI compat.
I asked this question during the recent Linux kernel plumbers conference: "How often in practice do the kernel A{B,P}Is fro device drivers change ?". I got a very mixed response.
My take away was that while 'new' driver classes/subsystems do involve more churn in the core kernel support subsystems, there is a clear trend towards convergence in terms of significant change.
If the change truly was significant and continuous then that would impede productisation with the Rust kernel. I'm not saying that the A{B,P}Is are in any way stable but I personally don't think that keeping up will be an intractable problem.
In the spirit of Kerla's authors prime motive - which appears to be hacking for knowledge - this might be something to try, perhaps with a driver class that is known to be more stable.
FreeBSD’s linuxkpi - bits and pieces of Linux KAPI provided to make porting Linux drivers to FreeBSD easier - works just fine, so this assumption is wrong.
> The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.
The kernel has (deliberately) never had a simple or stable driver ABI, so I think this is potentially a huge task. And then you've also left a C-based attack surface in place.
I'm not sure what the minimal set is if you decide to only support 2021-era hardware. PCIe+SATA+framebuffer+USBhost+USBHID?
I think using the Rust-for-linux wrappers to implement the drivers in Rust would eventually help.
Also, even if initially Linux C drivers were transplanted into Kerla that could be a net benefit if the tradeoff of wide device driver availability vs increase in the trusted compute base was acceptable.
I personally think that tradeoff would be acceptable - if the transplanting ability was tractable. It's a hard one to absolutely agree on, admittedly.
Towards the minimal set of hardware, fair point! I think that one should be able to get quite far with judicious selection of device driver classes and leave the truly in-flux classes alone ?
While I completely agree that NetBSD's Rump kernel would be a great way to go (I've played with it in the past and quite like it!) I do wish there was a similar initiative bound to the Linux kernel.
This may be a naive question, but I cannot help but wonder: As C and Rust aim to be link-compatible, wouldn't it be easier to gradually replace parts of the Linux kernel with Rust code?
The only technical problem I can think of is that Rust may not be available for all CPU architectures Linux supports, but this is just speculation on my part having done no research on the matter.
I think the approach the Rust for Linux project is taking is wise: Not about outright re-writes but more about focusing on those subsystems where Rust's intrinsic safety and security properties helps the most.
Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?
That's what may happen with the Linux kernel over time, now that it allows parts written in Rust.
The limitation of that approach is that all internal interfaces (which includes most data structures) have to be C-compatible. You can get much of the safety benefits of Rust, but lose a lot of Rust's ergonomics.
Yes, gradual replacement is easier and more practical for large projects. However, typical C APIs and idioms are different from idiomatic Rust. Rust uses more type-system features, generics, iterators, RAII, and prefers tree-like data structures and far fewer pointers (no linked lists!).
C rewritten in Rust is still very C-like, and requires refactorings that you can't do until it's all Rust.
BTW: There are two GCC-based Rust implementations in the works, so compatibility with exotic platforms is going to be solved.
A practical problem is that in order to gain the benefit of linking Rust and C code, one has to give up Rust's wonderful guarantees at the interface. So until islands of Rust meet up, these interact through a C ABI, and have to expose C-like behavior.
But you still get those guarantees and benefits in the Rust implementation itself, even if not at the interface (to non-Rust code). By a similar argument, a standalone Rust program "gives up" its guarantees whenever the process makes a call to libc or to the operating system (syscalls), but this isn't really a practical problem.
The world really needs _two_ projects, a practical one that is interoperable with the here and now, to get incremental improvements in security from bits reimplemented in Rust.
And then, in the longer term, someone ought to write a book about operating system implementation in Rust that ignores Linux interoperability and focuses on readability, maintainability by showing use of Rust's idiomatic style.
It’s not naive, it’s just been speculated and talked about incessantly since Rust came out. The one naive part is underestimating how long that would take - Linux is one of the largest codebases on Earth. It would require massive coordination over years to rewrite, and in the end all you would get is the exact same product.
It’s just not an interesting or unique perspective, and pretty silly to suggest seriously.
Well, writing a kernel from scratch that supports as many CPU architectures and devices as Linux would take about as long, assuming one could attract a critical mass of developers.
Using an incremental approach at least gives us some benefits in the near future.
I'm not even saying a fresh start would be a bad idea. But incrementally replacing parts of Linux seems like a more promising approach, IMHO.
After looking at the source I find this one to be much easier to read than equivalent C, because Rust is a more expressive language. So looking at what an all-rust kernel would look like seems quite worthwhile. You can't reach anything new if you're not willing to experiment.
The type of software makes the difference - does it process untrusted input, maybe even over the network?
This here is a kernel including a memory-safe TCP/IP stack (https://github.com/smoltcp-rs/smoltcp/), and not having it crash or be full of security vulnerabilities due to preventable memory corruption is a quality beyond personal language preferences.
It is possible to overstate a valid case until it becomes meaningless... are modern OS IP stacks written in C really 'crash'ing or 'full of security vulnerabilities'? No.
Not lamenting in this case, the project is awesome and rust is made for this. I just find amusing that almost all popular rust projects I see have "in Rust" front and center
Mozilla decided to create Rust after years of using C/C++. Why?
Because they spent a fortune fixing the same problems over and over again, and they realized that no matter how clever their developers were, those problems were always going to arise as long as they kept using C/C++.
Not because C/C++ is inherently bad, but because is more flexible than strictly needed in some ways.
Rust was designed to prevent many classes of problems.
I'm ok with things rewritten in Rust, really looks awesome as a language. And the fact every rust project advertises rust front and center compounds that image.
A big problem in the Rust ecosystem is the lack of continued work and support on libraries (understandably, since many are new and aren't commercially supported).
For example, pyroute2 is still better for Linux admin that the equivalent Rust crates. Whilst the latter often fully support async, none of them cover all of the features of pyroute2 nor the easy-to-use API.
Hopefully things will improve over time, it's always a shame to see great projects be abandoned.
I just meant that even big projects posted here in the past like the Headcrab debugger: https://github.com/headcrab-rs/headcrab have been seemingly abandoned now.
Author here. I'm surprised to see my hobby project on Hacker News.
I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.
That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel. You can learn how printf(3) works. You can learn what happens before main().
Especially, in Rust, it's inspiring to explore ways to write a kernel safely and expressively. One of my favorite part is the implementation of read system call [1].
Lastly, You can try this OS on SSH. Hope you enjoy :)
[1]: https://github.com/nuta/kerla/blob/main/kernel/syscalls/read...
> I know this kind of stuff spark the ``it's meaningless to rewrite everything (especially Linux) in Rust'' debate. I agree 100% that rewriting Linux in Rust (or your favorite language) is just a waste of time and such a project won't live long.
Considering that this is exactly how Linux was born (just a hobby project for fun), I wouldn't assume so fast that it's useless. Moreover, you don't need to justify yourself if you just want to have fun !
Any rust project like this inevitably gets sardonic replies about rewrite-in-rust fanaticism.
I agree they shouldn't have to justify themselves, but it is handy to preempt that.
1 reply →
I don't see why we need to pretend that every hobby has the potential for greatness. Being a hobby is a good enough end to itself.
In fact in this instance I think it's a little disingenuous to quote Linux and say it could happen again. The industry is totally different now. There's much more competition than there was when Linux was released and that competition is much more mature too. Plus the bar for a production-quality kernel is a lot higher than it was when Linux was released.
13 replies →
Thanks! I'll keep doing Just for Fun :D
Aside from "rewrite in rust" debate that I was unaware of, there seems to be this pervasive attitude among the HN hivemind that the end-goal for all projects, side- or main-, is "launch", and therefore needs a market analysis to decide the "worth" of such an idea, and it ends up being rather silly.
For example, I've written a Mandelbrot visualizer so many times I've lost count. Not because the world needs another poorly written or optimized rainbow-ladybug-simulator, but because it serves as a slightly-non-trivial hello-world. For example, it's the first end-to-end thing I made in Common Lisp. https://git.sr.ht/~amtunlimited/mandelbrot-plot
I have a A* search algorithm and a toy compiler that I use exactly for the same purpose.
I just rewrite them all the time as means to get a feeling about programming languages.
I dumped a couple of other stuff on GitHub so that HR people are happy to get a link that they never read anyway.
Then I get back to Java and .NET at the office. :)
3 replies →
This is really cool. Ignore the haters:
* Some people bake bread even though there's a good bakery nearby.
* Some people grow gardens even though there's a farmers' market down the street.
* Some people restore old cars even though there's a good restoration professional in town.
* Some people make k8s clusters on rpi even though they could rent that for cheap.
You're making an OS that's linux compatible even though there's already linux. And that is awesome!
Yes, and some people write an OS that is similar to Minix and won't ever be ported to anything beyond 386 and AT harddisks ;)
Wow, I really like the implementation of that syscall. I've got my own toy OS project in C, for the M68k, and my `read` is "a little" less clean [0, 1].
[0]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L469
[1]: https://gitlab.com/0xTJ/mosys/-/blob/master/src/vfs.c#L907
Thanks! By the way, your MUTEX_WITH macro looks pretty interesting to me. I've never seen the idea.
Even "rewriting Linux in Rust" is far from a waste if you make it run and if your code quality is good enough for others to join.
If only you could write an entirely new (meant to be better, for some cases at least) kernel compatible with Linux hardware drivers - this would be not waste at all but in fact fantastic.
The most challenging point is, as others said, the lack of the compatibility with Linux Driver API. I believe it would be really hard to implement and keep following changes in Linux.
An idea in my mind is to use Kerla for virtualized environments like KVM where only limited device drivers are needed (virtio for example).
2 replies →
Thanks for checking in. Makes me happy to see your excitement for learning the inner workings of things.
What do you think about other Rust OS projects like Redox?
https://www.redox-os.org/
In addition to its huge contribution to OS development in Rust, as a microkernel enthusiast, it sounds exciting to writing a microkernel in Rust, in the "Everything is a URL" principle. Moreover, it can run a good-looking GUI on a real machines [1]! I know it's very hard to be realized.
Aside from Redox, I'd mention Tock [2], an embedded operating system. It introduces a novel components isolation using Rust's power. I believe this field is where Rust shines and am looking forward to seeing it in production.
[1]: https://www.redox-os.org/screens/ [2]: https://www.tockos.org
9 replies →
Spending your time how you want to is never meaningless. Especially if you’re creating value in the world.
> That said, what I'd say here is that it's fun. Really fun. Implementing ABI compatibility requires you to understand under the hood of the kernel.
> You can learn how printf(3) works. You can learn what happens before main().
Yes! It's such a wonderful experience. This is exactly what I most enjoy doing, just seeing how things work, maybe making my own version. I hope you have lots of fun.
You're reimplementing Linux's kernel-userspace binary interface, right? The system call interface is stable and language agnostic, it's really nice. Some pointers for anyone who'd like to know more:
https://man7.org/linux/man-pages/man2/syscalls.2.html
https://man7.org/linux/man-pages/man2/syscall.2.html
https://github.com/torvalds/linux/blob/master/Documentation/...
I honestly think there is real value here for firecracker style containers. Running memory safe code for your whole stack, with a minimal number of virtio devices? Fantastic!
The permissive license would also be interesting in some applications.
I'm new to Rust and am far less along than you are, as evidenced by this project. I noticed that in the few files I spot checked, there aren't many/any tests. Without any context or opinion, I'm curious whether unit and integration testing are hard for a project like this.
Good point. As you say Kerla lacks unit tests I focused on running a SSH server as soon as possible.
IMO, writing and running tests in the kernel space is pretty easy thanks to Rust’s flexible testing feature [1].
[1]: https://os.phil-opp.com/testing/
Really cool that you mentioned OSv on your github, really not enough eye's on it, i think.
It doesn't matter if it's meaningless to the hackernews crowd, it really only matters if it has meaning to you and if you're learning.
How did you set-up the ssh system? Where a new vm is spawned on ssh ?
nice project! curious how the rust ownership model works in a monolithic kernel context. are the lifetimes of kernel structures associated with a process "owned" by the process itself somehow?
I’m not sure this answers what you asked, objects that are referenced from multiple objects are simply wrapped with Arc<T>.
1 reply →
How long did it take you to write this?
About 3-4 months in total: took 1.5 month to run a simple Hello World program, 1 month to implement bunch of system calls, and another 1 month to implement features essential to run Dropbear SSH server (tty/pry, bug fixes, …).
Just to clarify goals of the project:
"I completely agree that rewriting existing large, feature-rich, and robust operating system kernels is not a good idea. However, a question came to my mind: what are the pros and cons of writing a practical UNIX-like OS kernel in Rust from scratch? How does it look like? This is why I started this project."[1]
[1] https://seiya.me/writing-linux-clone-in-rust
Rewriting the Linux kernel in rust would be a rehash of why we ended up with Linux in the first place: rewriting an older OS from scratch.
I would love to see a solid implementation of a real micro kernel based OS based on a message passing core in Rust to become a viable alternative to the now 50 year old Unix architecture. This would give us a possibility of some real progress on the security and process scheduling front, both of which are severely lacking in the Unix model.
Why do people implicitly assume that micro kernels are the best OS design?
In fact, micro kernels have a lot of issues and limit the possibilities of what an OS can do! One of the most important issues is they prevents resource sharing in a rich and efficient way between the components of the kernel. They make building rich relationships between kernel data structures and kernel subsystems very messy, complicated, impracticable and very inefficient.
Micro kernels are delusional and people should stop implicitly assuming their superiority.
Historically, one of the main arguments against microkernels is that having a kernel that's broken up into a lot of threads that each have their own hardware-enforced memory protection and use message passing to communicate with each other is dreadfully inefficient. All those context switches are expensive, and copying all that data around when you send messages is expensive.
Rust's memory model largely makes those isolated address spaces unnecessary, and sending a message at runtime basically amounts to copying a pointer. So, with these performance issues resolved, I think it makes sense to look at microkernels again. The reasons why Linus was opposed to writing Linux as a microkernel in the 90's don't necessarily still apply now in 2021.
There may some things the Linux kernel does that can't be efficiently expressed using a message-passing style, but I don't know if that's actually true or not.
11 replies →
This is a limitation of the memory model, not the microkernel. If you had a single hardware-protected address space, kernel modules could share resources freely.
But this does argue against message passing, since such systems preferentially try to use messages over sharing whenever it's possible.
Message passing at first seems interesting and neat and useful. It is for some cases. But it comes at a cost of complexity and latency.
Just in case you don't know about it: https://redox-os.org/ could be interesting to you :)
Mach's internals didn't follow the Unix model, and was by almost any metric a failure.
There have been several other "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.
Process scheduling in Linux in 2021 is far, far ahead of anything in any other OS, ukernel or MP-based. The idea that there hasn't been progress in this area is really completely absurd. In fact, the general idea that Linux follows "the now 50 year old Unix architecture" is also pushing the boundaries quite a lot.
Meanwhile, the world runs on QnX. The fact that you don't realize it and don't see it is testimony to its success.
2 replies →
> "real micro kernel based OS based on a message passing core", and they too, by almost any metric, have been a failure.
BeOS wasn't half bad. The failure was probably mostly commercial. It is true that they moved their networking stack into the kernel, but it's not entirely clear to me that they had to do that, it was just the most expedient way to get acceptable performance and stability given the (programmer-time) resourcing constraints of the project.
6 replies →
Mach is of course doing quite well and moving back towards being microkernel-y as hardware gets fast enough to support it and Apple needs a better security story.
Written properly, a monolithic kernel in Rust would provide many of the benefits of a microkernel, due to the memory safety. It'd be much along the same lines as Microsoft Research's Midori, with software isolation instead of hardware isolation.
It would be wide open to specially crafted drivers, it's the equivalent of client side security (or maybe that should be 'compile time security'). Memory safety is something only process space isolation can bring.
Fuchsia and QNX provide a lot of what you want.
https://en.wikipedia.org/wiki/QNX
https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
https://lobste.rs/s/p8bizb/getting_know_fuchsia_google_s_ope...
I think I'm aware of QnX ;) (I wrote a clone).
I believe someone is building something like that on top of sel4, which gives even stronger correctness guarantees than Rust, even though it's C.
What are some ways I can increase my knowledge in this domain that the OP is very skilled at, meaning low level OS development?
I've taken an intro to OS class and am currently going through Linux From Scratch [1], which is interesting and is teaching me a lot, but it's more about how to setup a Linux distro using existing packages and not really about reading/writing the code involved.
Any recommendations?
[1] https://www.linuxfromscratch.org/
One of the easiest ways to get into writing your own kernel is to start with the OS Dev wiki [0]. Both the wiki and their forums are a great place for getting your feet wet.
For more general information, there are also a handful of articles and/or blog posts that I've come across when I was getting started that gave a very good introduction to many of the low-level components of how computers work and what the OS needs to do to run them. One of my favorites was Many But Finite [1]. Check out a bunch of his posts about the memory map, the computer boot process [2], cpu rings, etc... Not exhaustive by any means, but good, well-presented info. Many other such sources exist; maybe others will share their favorites.
And if you'd just like to get a more gentle introduction into actual OS code than the monster that is Linux, maybe try looking at some of the BSD's. NetBSD [3] and OpenBSD [4] are remarkably easy to navigate and follow from a code flow perspective, and if you just want to see how things work in a real OS, it's both a good reference and a good place to start.
[0] https://wiki.osdev.org/Main_Page
[1] https://manybutfinite.com/archives
[2] https://manybutfinite.com/post/how-computers-boot-up
[3] http://netbsd.org/
[4] https://www.openbsd.org/
Thank you!
I think you're on the right track. I am similar to you and not so skilled at low level OS development. There is an educational OS called PIOS from Yale's CS department with specific parallelism goals in mind, but it boots from metal so the code is a great resource. Here's the code: https://github.com/bford/PIOS
Also, you could try reading the Plan 9 source code.
(I would say the code is what you're after now, but in case you are interested in more of the theory of why it's designed that way, you can check out the research paper here: https://dedis.cs.yale.edu/2010/det/)
Thanks!
Linux from scratch won't really help you. I'd say start by looking into writing drivers for Linux and looking at tiny OS examples for microcontrollers that can be run in emulators or cheap boards. Bare-metal projects are available for pi and beaglebone. You will absolutely have to do something like that at a minimum, so it's good to apply what you learned in your OS class to that stuff.
This is awesome. Imagine the potential if it was just complete enough to run Kubernetes’ kubelet on a cloud instance. Lots of security minded folks would love it.
That's IMHO how you get something like this off the ground. Make it useful for a simple use-case while offering something of value - in this case the security of Rust. It can then become a preferred solution to some problem and that will bring users and developers looking to incrementally expand it.
I wonder, if you run every container in it's own MicroVM a-la Firecracker like Fargate does (or Kerla's demo host), does the security of the kernel still matter as much?
A minimal interactive demo running busybox is here:
That’s very cool! Could you share some details on how to set it up for oneself with AWS Firecracker the same way that this demo was set up?
Also, as owner of the instances are you able to see what commands people are running? First thing I did was to type hello, and it said hello world to me :)
Author here. I've made the demo system public [1] but it's written just for me so it should be painful to set up the same environment.
The mechanism is pretty simple: a Node.js server (running on genuine Linux) listens on tcp:22. Once you connect, it boots a dedicated Firecracker microVM instance and forwards network packets between your SSH client and VM.
Regarding the command history, others (including I) can't see what you type. If you could, it must be a vulnerability.
[1] https://github.com/nuta/kerla-demo
1 reply →
I'm not the project author, maybe you can ask via e-mail, see https://seiya.me/
1 reply →
is there not an opportunity to implement the posix filesystems interface on top of something other than a block device system. That would be fun.
Very cool!
I think a killer feature would be to emulate the Linux kernel device driver model such that existing C driver modules could be used with Kerla. Further, borrow (no pun intended!) the Linux device driver Rust wrappers from the Rust for Linux kernel project to then enable writing Linux device drivers in Rust.
The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.
I think there might be reasonable trade offs that make the problem more tractable.
Having the ability to have a significantly simpler kernel with Rust’s memory and thread safety guarantees plus the ability to use even a small set of Linux’s massive set of drivers would be very compelling.
Linux drivers don't have a fixed ABI (or even API), making such a feat somewhere between hard and impossible. You'd have to constantly play catch up with the latest kernel refactorings, which would require a tremendous amount of engineering effort.
Heck, it'd probably be easier to emulate the Windows driver architecture - at least those have proper ABI compat.
I asked this question during the recent Linux kernel plumbers conference: "How often in practice do the kernel A{B,P}Is fro device drivers change ?". I got a very mixed response.
My take away was that while 'new' driver classes/subsystems do involve more churn in the core kernel support subsystems, there is a clear trend towards convergence in terms of significant change.
If the change truly was significant and continuous then that would impede productisation with the Rust kernel. I'm not saying that the A{B,P}Is are in any way stable but I personally don't think that keeping up will be an intractable problem.
In the spirit of Kerla's authors prime motive - which appears to be hacking for knowledge - this might be something to try, perhaps with a driver class that is known to be more stable.
FreeBSD’s linuxkpi - bits and pieces of Linux KAPI provided to make porting Linux drivers to FreeBSD easier - works just fine, so this assumption is wrong.
>Heck, it'd probably be easier to emulate the Windows driver architecture - at least those have proper ABI compat.
And better drivers.
1 reply →
> The question is whether emulating the Linux device driver model (by which I kind of mean mimicking the set of core kernel services needed to support drivers - kernel threads, sync primitives, allocators etc) ends up being a massive task akin to writing the core kernel itself.
The kernel has (deliberately) never had a simple or stable driver ABI, so I think this is potentially a huge task. And then you've also left a C-based attack surface in place.
I'm not sure what the minimal set is if you decide to only support 2021-era hardware. PCIe+SATA+framebuffer+USBhost+USBHID?
Towards the C-based attack surface:
I think using the Rust-for-linux wrappers to implement the drivers in Rust would eventually help.
Also, even if initially Linux C drivers were transplanted into Kerla that could be a net benefit if the tradeoff of wide device driver availability vs increase in the trusted compute base was acceptable.
I personally think that tradeoff would be acceptable - if the transplanting ability was tractable. It's a hard one to absolutely agree on, admittedly.
Towards the minimal set of hardware, fair point! I think that one should be able to get quite far with judicious selection of device driver classes and leave the truly in-flux classes alone ?
Maybe starting with the rumpkernel interface is easier and one could already use some of the NetBSD drivers: https://en.wikipedia.org/wiki/Rump_kernel
That's a very good point indeed.
While I completely agree that NetBSD's Rump kernel would be a great way to go (I've played with it in the past and quite like it!) I do wish there was a similar initiative bound to the Linux kernel.
The closest I've seen is Octavian Purdila's Linux-Kernel-Library project: https://lkml.org/lkml/2015/11/3/706
Not sure how far they've come though. Perhaps Kerla could be a prime mover for them ?
Dunno: just a hobby, won't be big and professional like linux.
:)
This may be a naive question, but I cannot help but wonder: As C and Rust aim to be link-compatible, wouldn't it be easier to gradually replace parts of the Linux kernel with Rust code?
The only technical problem I can think of is that Rust may not be available for all CPU architectures Linux supports, but this is just speculation on my part having done no research on the matter.
The Rust for Linux kernel project aims to enable writing Linux kernel device drivers in Rust.
See: https://lwn.net/Articles/862018/
The issue of Rust's LLVM based compiler toolchain not targeting all CPU archs is intended to be solved by the gcc-rs project.
See: https://lwn.net/Articles/871283/
I think the approach the Rust for Linux project is taking is wise: Not about outright re-writes but more about focusing on those subsystems where Rust's intrinsic safety and security properties helps the most.
Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?
23 replies →
That's what may happen with the Linux kernel over time, now that it allows parts written in Rust.
The limitation of that approach is that all internal interfaces (which includes most data structures) have to be C-compatible. You can get much of the safety benefits of Rust, but lose a lot of Rust's ergonomics.
I don't think that the Linux kernel has decided (yet) to allow rust code into the kernel proper, has it?
5 replies →
Yes, gradual replacement is easier and more practical for large projects. However, typical C APIs and idioms are different from idiomatic Rust. Rust uses more type-system features, generics, iterators, RAII, and prefers tree-like data structures and far fewer pointers (no linked lists!).
C rewritten in Rust is still very C-like, and requires refactorings that you can't do until it's all Rust.
BTW: There are two GCC-based Rust implementations in the works, so compatibility with exotic platforms is going to be solved.
A practical problem is that in order to gain the benefit of linking Rust and C code, one has to give up Rust's wonderful guarantees at the interface. So until islands of Rust meet up, these interact through a C ABI, and have to expose C-like behavior.
But you still get those guarantees and benefits in the Rust implementation itself, even if not at the interface (to non-Rust code). By a similar argument, a standalone Rust program "gives up" its guarantees whenever the process makes a call to libc or to the operating system (syscalls), but this isn't really a practical problem.
1 reply →
The world really needs _two_ projects, a practical one that is interoperable with the here and now, to get incremental improvements in security from bits reimplemented in Rust.
And then, in the longer term, someone ought to write a book about operating system implementation in Rust that ignores Linux interoperability and focuses on readability, maintainability by showing use of Rust's idiomatic style.
It’s not naive, it’s just been speculated and talked about incessantly since Rust came out. The one naive part is underestimating how long that would take - Linux is one of the largest codebases on Earth. It would require massive coordination over years to rewrite, and in the end all you would get is the exact same product.
It’s just not an interesting or unique perspective, and pretty silly to suggest seriously.
The Linux kernel is massive, so writing any part of it in Rust is an enormous undertaking. Even a few functions at a time, we're talking decades.
Well, writing a kernel from scratch that supports as many CPU architectures and devices as Linux would take about as long, assuming one could attract a critical mass of developers.
Using an incremental approach at least gives us some benefits in the near future.
I'm not even saying a fresh start would be a bad idea. But incrementally replacing parts of Linux seems like a more promising approach, IMHO.
Kudos, it works and thee source code is a beautiful read.
Thanks so much for sharing!
For each piece of software, there just be a version in Rust, and that version must advertise it's rustiness
After looking at the source I find this one to be much easier to read than equivalent C, because Rust is a more expressive language. So looking at what an all-rust kernel would look like seems quite worthwhile. You can't reach anything new if you're not willing to experiment.
The type of software makes the difference - does it process untrusted input, maybe even over the network? This here is a kernel including a memory-safe TCP/IP stack (https://github.com/smoltcp-rs/smoltcp/), and not having it crash or be full of security vulnerabilities due to preventable memory corruption is a quality beyond personal language preferences.
It is possible to overstate a valid case until it becomes meaningless... are modern OS IP stacks written in C really 'crash'ing or 'full of security vulnerabilities'? No.
3 replies →
For each rust submission, there shall be a comment at top lamenting the gradual rusting of hackernews.
Not lamenting in this case, the project is awesome and rust is made for this. I just find amusing that almost all popular rust projects I see have "in Rust" front and center
Must be an awesome language to work on.
I mean honestly who cares? its their time. I use rust over c/c++ these days for practical reasons.
1. the toolchain is better. (cargo ala)
2. maintenance of the software is better. (static linked binaries, easier deploys)
3. memory safety.
4. resource consumption (cpu/ram)
when I don't need 4 I use golang (most cases) because of 1, 2, and 3.
and I choose tools (all other things being equal) written in rust for the same reasons.
Me too, but in my case it's despite of point 2.
Now ask yourself why.
Mozilla decided to create Rust after years of using C/C++. Why?
Because they spent a fortune fixing the same problems over and over again, and they realized that no matter how clever their developers were, those problems were always going to arise as long as they kept using C/C++.
Not because C/C++ is inherently bad, but because is more flexible than strictly needed in some ways.
Rust was designed to prevent many classes of problems.
I'm ok with things rewritten in Rust, really looks awesome as a language. And the fact every rust project advertises rust front and center compounds that image.
A star has been born. A Linux alternative needs to be compatible with Linux binaries and also support modern architectures.
If you want Linux to survive, it needs to be re-written from scratch, optimised in Rust and compatible with the existing Linux binaries or drivers.
This project might have a chance.
From: [0]
> TL;DR: I'm writing a Linux clone in Rust just for fun. It does NOT aim to replace the Linux kernel.
A certain someone thought that their hobby OS Minix clone won't be 'big or professional like GNU'.
It's time for 'change'.
[0] https://seiya.me/writing-linux-clone-in-rust
You think Linux, which the vast majority of web servers run today, is going to die out if it's not rewritten from scratch in Rust?
How on earth have you come to that one?
sigh...
Linux is going to survive. And no it does not need to be re-written from scratch in Rust. Someone lied to you at the church of rust.
But its a fun thing to do, and that's a good enough reason to do it. After all is that not exactly how Linux came into being anyway.
4 replies →
Sometimes, just sometimes, I wished one could post soyjaks on HN.
Absolute codswallop. Linux will be here for many, many years to come yet.
Absolutely! But in Rust.
Why do you think the Linux developers are bolting on Rust in the Linux kernel themselves? [0]
It has already been admitted.
[0] https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@ke...
2 replies →
Not with Apache v2 License.
Post it when it's production ready.
A big problem in the Rust ecosystem is the lack of continued work and support on libraries (understandably, since many are new and aren't commercially supported).
For example, pyroute2 is still better for Linux admin that the equivalent Rust crates. Whilst the latter often fully support async, none of them cover all of the features of pyroute2 nor the easy-to-use API.
Hopefully things will improve over time, it's always a shame to see great projects be abandoned.
> Post it when it's production ready.
This is Hacker News, not Enterprise IT News.
I just meant that even big projects posted here in the past like the Headcrab debugger: https://github.com/headcrab-rs/headcrab have been seemingly abandoned now.
Even Rocket https://github.com/SergioBenitez/Rocket seems to have greatly slowed development unfortunately.
3 replies →
The project author asks for help at the end of the blog post¹ announcing it, if you want to have a production ready kernel, maybe invest some time?
¹ https://seiya.me/writing-linux-clone-in-rust