Comment by newnewpdro
7 years ago
You're being downvoted, but it's true about linux for at least two reasons to this day:
1. dm-crypt threads are unfair to the rest of the system's processes [1]. On dmcrypt systems, regular user processes can effectively raise their process scheduling priority in a multithreaded fashion by generating heavy IO on dmcrypt storage.
2. Under memory pressure, the VM system in linux will enter a thrashing state even when there is no swap configured at all. I don't have a reference on hand, but it's been discussed on lkml multiple times without solution. I suspect the recent PSI changes are intended as a step towards a solution though. What happens is clean, file-backed pages for things like shared libraries and executable programs become a thrashing set resembling anonymous memory swapping under memory pressure. As various processes get scheduled, they access pages which were recently discarded from the page cache as evictable due to their clean file-backed status when other processes ran under pressure, and now must be read back in from the backing store. This ping-ponging continues dragging everything down until either an OOM occurs or pressure is otherwise relieved. This often manifests as a pausing/freezing desktop with the disk activity light blazing, and it's only made worse by the aforementioned dmcrypt problem if these files reside on such volumes.
This behaviour - and perhaps other causes with similar issues for desktop users, is what drove me away from helping to make Linux ‘ready for the desktop’. The kernel philosophy was incompatible with the needs of desktop users. Those developing for the desktop (like myself) didn’t have the required expertise to make the kernel do the Right a Thing, and couldn’t find enough people willing or able to help.
Things may have changed over the years - I’ve been running a Linux desktop recently and haven’t seen this kind of issue yet (the kind where you need to use magic keys to ask the kernel to kill all, sync and reboot) but reading your post, perhaps this is because RAM is much more plentiful these days.
Its probably the ram plentiful thing. I haven't looked recently into if the Arch mainline kernel kconfig is just bad or not but the oomkiller is trash for me. Used to have the "recommended" swap == ram size but then the memory manager never even tried to cull pages until it was OOM and froze up trying to swap constantly. Currently running a 16/4 spread and probably going to drop to 16/1 because any time I hit memory limits everything just freezes permanently rather than the oomkiller getting invoked. I've hit it twice this week trying to render in Kdenlive and run a debug build of Krita...
Yeah, I still observe this from time to time, with SSD-only storage and 32 GB main memory.
If more people reproduced the dmcrypt issue and commented in the bugzilla issue it'd put more pressure on upstream to revert the known offending commit.
For some reason they seem to be prioritizing the supposedly improved dmcrypt performance over fairness under load, even though it makes our modern machines behave like computers from the 90s; unable to play MP3s and access the disk without audio underruns.
I assume it's because they're not hearing enough complaints.
Is there no way to set a minimum page cache size?