Comment by rwmj
1 day ago
I'm surprised the article didn't also mention MSG_MORE. On Linux it hints to the kernel that "more is to follow" (when sending data on a socket) so it shouldn't send it just yet. Maybe you need to send a header followed by some data. You could copy them into one buffer and use a single sendmsg call, but it's easier to send the header with MSG_MORE and the data in separate calls.
(io_uring is another method that helps a lot here, and it can be combined with MSG_MORE or with preallocated buffers shared with the kernel.)
you can already send fragmented data in one system call without copying it to a single buffer.
Indeed you can, but we've found it useful to use MSG_MORE when using state machines, where different states are responsible for different parts of the reply. (Plenty of examples in states*.c here: https://gitlab.com/nbdkit/libnbd/-/tree/master/generator?ref...)
Doing more system calls isn't really a good idea for performance.
Also if you're doing asynchronous writes you typically can only have one write in-flight at any time, you should aggregate all other buffers while that happens.
Though arguably asynchronous writes are often undesired due to the complexity of doing flow-control with them.
1 reply →
Consider using a user-space buffer instead, syscalls are slow so bonus points for limiting them are on the table. If you want to avoid resizing an array have a look at the vectored io syscalls (readv etc.).