Comment by bjourne
2 years ago
> Fix the apps. Nobody expect magical perf if you do that when writing to files,
We write to files line-by-line or even character-by-character and expect the library or OS to "magically" buffer it into fast file writes. Same with memory. We expect multiple small mallocs to be smartly coalesced by the platform.
If you expect a POSIX-y OS to buffer write(2) calls, you're sadly misguided. Whether or not that happens depends on nature of the device file you're writing to.
OTOH, if you're using fwrite(3), as you likely should be actual file I/O, then your expectation is entirely reasonable.
Similarly with memory. If you expect brk(2) to handle multiple small allocations "sensibly" you're going to be disappointed. If you use malloc(3) then your expectation is entirely reasonable.
Whether buffering is part of POSIX or not is beside the point. Any modern OS you'll find will buffer write calls in one way or the other. Similarly with memory. Linux waits until accesses page faults before reserving any memory pages for you. My point is that various forms of buffering is everywhere and in practice we do rely on it a whole lot.
> Any modern OS you'll find will buffer write calls in one way or the other.
This is simply not true as a general rule. It depends on the nature of the file descriptor. Yes, if the file descriptor refers to the file system, it will in all likelihood be buffered by the OS (not with O_DIRECT, however). But on "any modern OS", file descriptors can refer to things that are not files, and the buffering situation there will vary from case to case.
1 reply →
True to a degree. But that is a singular platform wholly controlled by the OS.
Once you put packets out into the world you're in a shared space.
I assume every conceivable variation of argument has been made both for and against Nagles at this point but it essentially revolves around a shared networking resource and what policy is in place for fair use.
Nagles fixes a particular case but interferes overall. If you fix the "particular case app" the issue goes away.
Yes, your libraries should fix that. The OS (as in the kernel) should not try to do any abstraction.
Alas, kernels really like to offer abstractions.