Comment by maleldil
5 years ago
Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?
5 years ago
Last I checked, one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel. Is there any progress on that?
Out of the box Rust doesn't provide any way to allocate heap memory, so, you can't run out if it.
Your ordinary userspace apps use std (the standard library) which relies on the alloc crate, and that provides heap allocation which indeed panics if the allocation fails. Because it correctly guesses that your "clever" strategy to handle allocation failure actually isn't and will just triple fault anyway so it should cut to the chase.
The kernel obviously doesn't have std, and Rust for Linux implements its own alloc crate.
Linus' requirement that you can fail memory allocation just means all the calls in alloc that can actually allocate memory now return Result to indicate whether the allocation was successful. This isn't how you'd do things in userspace, but, this isn't userspace so fine.
Can you elaborate as to why "his isn't how you'd do things in userspace, but, this isn't userspace so fine" holds?
Naive me - not a kernel dev at all - would argue that returning Result<Memory, AllocationError> is always better, even for userspace because it would allow me to additionally log something or gracefully deal with this.
Even if I don't want to deal with it, I could just `.unwrap()` or `.expect('my error message')` it.
Note: I am not trying to be snarky here, I genuinely don't know and would like to.
If answering this is too complex, maybe you can point me in the right direction so I can ask the right questions to find answers myself? Thanks in any case!
> it would allow me to additionally log something
If you don't have any memory your allocations are all failing. When you assemble the log message, the allocation needed to do that fails. Bang, double fault.
Now, often people don't really mean they want allocations to be able to fail generally, they're just thinking about that code they wrote that reads an entire file into RAM. If it was a 100GB file that would be a bad idea. But the best answer is: Guard the allocation you're actually worried about, don't ladle this into the fast path everybody has to deal with on every allocation.
6 replies →
Tialaramex answered this in their post already, and you almost answered the question yourself:
> I could just .unwrap() or .expect('my error message') it.
Panicking can allocate. Allocating can fail. Failing can panic. Panicking can allocate. Allocating can fail. You can bite yourself in the ass like a real Ourobouros.
IMO, a prerequisite to using fallible allocation APIs should be attempting to write your own allocator, handling the weird and wacky problem of initialising a data structure (for the heap) in such a way that if it fails, it fails without allocating but leaves some hint as to what went wrong.
8 replies →
AFAIK one other thing to note is that in Linux userspace, malloc (or fork) might succeed, but accessing the memory later can fail because of memory overcommit.
Yes, this is called "fallible allocations." You add methods with the "try_" prefix that work like the existing methods, except they return a Result which fails if it's out of memory instead of panicking.
We have a light / temporary fork of the Rust stdlib allocator with fallible allocation support: https://github.com/Rust-for-Linux/linux/tree/rust/rust/alloc
See e.g. https://github.com/Rust-for-Linux/linux/commit/487d7578bd036...
Yes a lot. https://rust-for-linux.github.io/docs/alloc/alloc/index.html
>> one of the big problems was the Rust panics when you run out of memory, which is unacceptable in the kernel.
If the kernel is running out of memory, IMHO that's a bug. The kernel is ultimately responsible for memory management right? It needs to prioritize itself over everything else or the system is in trouble.
This sounds like a design choice that was made decades ago and permeated everything else in Linux. I don't think it's changeable at this point, if they wanted to.