Comment by batmanthehorse
2 years ago
Does anyone know of a good way to enable TCP_NODELAY on sockets when you don't have access to the source for that application? I can't find any kernel settings to make it permanent, or commands to change it after the fact.
I've been able to disable delayed acks using `quickack 1` in the routing table, but it seems particularly hard to enable TCP_NODELAY from outside the application.
I've been having exactly the problem described here lately, when communicating between an application I own and a closed source application it interacts with.
Would some kind of LD_PRELOAD interception for socket(2) work? Call the real function, then do setsockopt or whatever, and return the modified socket.
> Would some kind of LD_PRELOAD interception for socket(2) work?
That would only work if the call goes through libc, and it's not statically linked. However, it's becoming more and more common to do system calls directly, bypassing libc; the Go language is infamous for doing that, but there's also things like the rustix crate for Rust (https://crates.io/crates/rustix), which does direct system calls by default.
And go is wrong for doing that, at least on Linux. It bypasses optimizations in the vDSO in some cases. On Fuchsia, we made direct syscalls not through the vDSO illegal and it was funny the hacks to go that required. The system ABI of Linux really isn't the syscall interface, its the system libc. That's because the C ABI (and the behaviors of the triple it was compiled for) and its isms for that platform are the linga franca of that system. Going around that to call syscalls directly, at least for the 90% of useful syscalls on the system that are wrapped by libc, is asinine and creates odd bugs, makes crash reporters heuristical unwinders, debuggers, etc all more painful to write. It also prevents the system vendor from implementing user mode optimizations that avoid mode and context switches when necessary. We tried to solve these issues in Fuchsia, but for Linux, Darwin, and hell, even Windows, if you are making direct syscalls and it's not for something really special and bespoke, you are just flat-out wrong.
28 replies →
Depending on the specifics, you might be able to add socat in the middle.
Instead of: your_app —> server
you’d have: your_app -> localhost_socat -> server
socat has command line options for setting tcp_nodelay. You’d need to convince your closed source app to connect to localhost, though. But if it’s doing a dns lookup, you could probably convince it to connect to localhost with an /etc/hosts entry
Since your app would be talking to socat over a local socket, the app’s tcp_nodelay wouldn’t have any effect.
Attach debugger (ptrace), call setsockopt?
opening `/proc/<pid>/fd/<fd number>` and setting the socket option may work (not tested)
LD_PRELOAD.
Thank you, found this: https://github.com/sschroe/libnodelay
Is it possible to set it as a global OS setting, inside a container?
you could try ebpf and hook on the socket syscall. might be harder than LD_PRELOAD as suggested by other commenters though