Comment by Animats
3 years ago
> I have long wondered... How was it possible to construct this? Is it just extreme self-discipline, omitting everything not strictly essential? Or is there an underlying formalism that makes a very small amount of code do a very great deal?
Two really good designers, Gorden Bell and Dan Dodge.
Here are the key design decisions:
- All the kernel does is manage memory, dispatch processes, handle interrupts and timers, and pass messages from process to process. No device drivers, no file systems. Everything else is in user space. The kernel is small (about 60KB of code in some versions), very well written, and rarely changed. It's so small, so heavily used, and so rarely changed that it reached an essentially bug-free state.
- All non-preemptable kernel operations have fixed upper bounds on how long they can take, worst case. That upper bound is in microseconds. That's why real time works.
- Message passing and CPU scheduling are integrated and very efficient. In particular, the case of sending a message to another process and getting a reply message is not only low overhead, but does not involve losing your turn for the CPU. The two systems are designed together. So calling another process can be used almost like a subroutine call. Most interprocess message systems botch this, and calling another process means two or more trips through the CPU scheduler and may cost you your CPU quantum. Which means a "microservice architecture" runs too slowly. Which means people work around making interprocess calls, putting too much in one process.
- There is no swapping or paging. This is essential for real-time. It simplifies other things. Message passing is copying user space to user space, and can't page fault. The destination area has to be in memory. So there is no need for kernel data buffers.
- At boot time, the boot program loads the kernel, a user space utility process called "proc", plus any user space drivers or programs you want at startup. That's how it gets started with no device drivers in the kernel. More drivers can be loaded later, if desired. Having a file system or disk is optional. The minimal configuration is a CPU, boot ROM with the software, and memory. That's a common configuration for small embedded systems.
- File systems, networking, and drivers are all user programs. Some have the privilege of writing to device space. The kernel turns interrupts into interprocess messages.
It's a nice architecture for small and medium real-time systems that have to Just Work.
This sounds almost exactly like erlang at a lower and even more time critical level which is, super cool.
The idea that the kernel does so little is really cool. I don’t know if it’d work at the true microcontroller level where 128k is precious memory but I can definitely see the larger microcontrollers that exist now or even stuff like fpga socs being useful with this sort of setup. Sounds cool.
> So calling another process can be used almost like a subroutine call.
Any numbers on the overhead of a message send/reply round trip compared to a subroutine call? I always assumed it was just axiomatic that the difference in latency between those two options would be orders of magnitude.
I don't have numbers, but the key ingredient is really minimizing context switches. This also results in priority inheritance while message passing which means that a high priority process gets the answer with it's own high priority, even if sending a message to a low priority process. Very smart.
Teading the old open sourced sources can be quite interesting...
https://github.com/vocho/openqnx/tree/master/trunk/lib/qnx43...
Here are some numbers from QNX. These are old.[1]
[1] http://support.qnx.com/developers/docs/6.5.0/index.jsp?topic...
I learned a lot from this comment, thank you. If you could add - what would be a good architecture for large RT systems?
This explains (thank you) how they can provide real-time service and reliability, but does not explain how they cram all the GUI and browser functionality into 1.4MB.
Is there anything about L4 to make make it worse in the role of that kernel?