Comment by jeffbee
6 hours ago
This is why memory allocators should be linked into applications and should not be considered operating system components.
6 hours ago
This is why memory allocators should be linked into applications and should not be considered operating system components.
If an OS update gives me a better allocator, I want every app to take advantage of it. Same idea with other libraries, if I update a library for reading zip files to make it go faster, or to fix a vulnerability, I want every application that uses that library to take advantage of it.
This is a debatable position. Linux distros usually do this, but a lot of modern applications ship with all their dependencies. I'd say the former is more efficient and generally better from a user perspective and the latter is more stable and is generally better from a developer/maintainer perspective. Open source encourages the former, proprietary software encourages the latter.
That isn't how history has unfolded. What has actually happened is that operating systems ship with extremely bad allocators unsuitable for any application and due to some kind of emergent collective problem they are never improved.
Userland system DLLs need to allocate memory too.
Whenever I write I library I don't allocate memory. If memory needs to be allocated, the program calling into the library allocates memory and passes a pointer (or reference I suppose) into the library call.
Clearly this can't be enforced, but I've always found it bad practice to allocate memory inside a library.
How to you manage the API when caller doesn't know the right size to allocate? I mean you could pass an allocator object (ala zig/C++ stdlib etc), though I'm not sure I like the extra arg on functions... I guess a quasi-global-variable approach (e.g. a "context" object, passed to all functions, which can hold an allocator, or equivalently implicit arguments in a functional language) would be one way to achieve that without the "noise" of explicit allocators passed everywhere.
1 reply →
How does the memory allocator statically linked into the application request a new memory allocation into it's address space (sbrk or VirtualAlloc etc.)? Allocating memory will in most cases involve calling into the kernel, it's kinda a core function of a kernel. It has to have some link externally to the programme, directly or indirectly.
The smallest amount of memory you can request from the OS is a page (typically 4K, sometimes 16K), and there’s a lot of additional stuff that goes on in malloc().
And when you are running more than one application at the same time how is that supposed to work?
It works fine. The processes get a fenced off piece of memory and are allowed to portion it out to themselves any way they prefer. Syscalls are designed to be mostly neutral about how the app wants to proceed.
On a system with an MMU, applications have their own address space that the allocator draws from; pages need to be centrally allocated, but individual allocations can be handled in userspace
Windows NT (NT - Windows 11) keeps applications in their own private memory space by design. Windows 1,2,3,95,98,Me didn't do this which is why it was more prone to blue screens.
True, but this design decision (to include memory allocation API in the OS) predates the process separation/MMU.
In 20 years we'll have finally reached the logical conclusion to all of this that there shouldn't be separate os components you link against and your "OS" will just be manager of lightweight VMs.
Now you can just ship the user your machine and software can theoretically be immortal.