Comment by gpderetta
3 years ago
I wonder if it is possible to avoid OOM by making sure that all allocations are done from a named (on disk, not shm) memory file. This way in principle is always possible to swap to disk and never overcommit.
I guess in practice the kernel might be in such dire straits that it is not able to even swap to disk and might need to kill indiscriminately.
That would have to be all allocations in all processes (and the kernel and drivers)
In extreme circumstances, the OOM killer can decide to kill your process even if it barely uses any memory (a simple way to get there is by fork-bombing copies of such processes)
Also, using oom_score_adj (https://www.baeldung.com/linux/memory-overcommitment-oom-kil...) is a lot easier.
You would also need to prevent overcommit of disk; you'd typically mmap to a sparse file, and then you've got the same problem of overcommit on disk as you did in memory.
If you're going to do drastic things, you can configure Linux's memory overcommit behavior, although strictly avoiding overcommit usually results in trouble from software not written with that in mind.