Comment by jchw
3 years ago
Actually if you're sending a file or something, do you really need Nagle's algorithm? It seems like the real mistake might be not using a large enough buffer for writing to the socket, but I could be speaking out my ass.
There's actually a lot of prevailing wisdom that suggests disabling Nagle's algorithm is (often) a good idea. While the problem with latency is caused by delayed ACKs, the sender can't do anything about that, because it's the receiver side that controls this.
Not saying that it's good the standard library defaults this necessarily... But this post paints the decision in an oddly uncharitable light. That said, I can't find the original thread where this was discussed, if there ever was one, so I have no idea why they chose to do this, and perhaps it shouldn't be this way by default.
It's often a good idea when the application has its own buffering, as is common in many languages and web frameworks which implement some sort of 'reader' interface which can alternate symbols of "chunks" and "flushes" or only emit entire payloads (a single chunk). With scatter-gather support for IO, it's generally OK for the application to produce small chunks followed by a flush. Those application layer frameworks want Nagle's algorithm turned off at the TCP layer to avoid double-buffering and incurring extra latency.
Go however is disabling Nagle's by default as opposed to letting it be a framework level decision.
This is a great point. Why is Git LFS uploading a large file in 50 byte chunks?
Ideally large files would upload in MTU sized packets, which Nagle's algorithm will often give you, otherwise you may have a small amount of additional overhead at the boundary where the larger chunk may not be divisible into MTU sized packets.
Edit: I mostly work in embedded (systems that don't run git-lfs), perhaps my view is isn't sensible here.
Dividing packets into MTUs is the job of the tcp stack - or even the driver or NIC in the case of offloads. Userspace software shouldn’t deal with MTUs and always use buffer sizes that make sense for the application - eg 64kB or even more. Otherwise the stack wouldn’t be very efficient with every tiny piece of data causing a syscall and independent processing by the networking stack
2 replies →
I do not know Go. But what if there are so many high level abstractions in the Go language that it operates on streams directly?
The standard convention is to slap bufio.Reader/bufio.Writer on streams to make them more performant.
Though how LFS ends up with ~50 byte chunks is probably something very, very, dumb in the LFS code itself. Better to fix that mistake than to paper over it.
1 reply →
The footnote has a brief note about delayed ACKs but it's not like the creator of the socket can control whether the remote is delaying ACKs or not. If ACKs are delayed from the remote, you're eating the bad Nagle's latency.
The TCP_NODELAY behavior is settable and documented here [1]. It might be better to more prominently display this behavior, but it is there. Not sure what's up with the hyperbolic title or what's so interesting about this article. Bulk file transfers are far from the most common use of a socket and most such implementations use application-level buffering.
[1]: https://pkg.go.dev/net#TCPConn.SetNoDelay
The title is hyperbolic because a real person got frustrated and wrote about it, the article is interesting because a real person got frustrated at something many of us can imagine encountering but not so many successfully dig into and understand.
“Mad at slow, discovers why slow” is a timeless tale right up there with “weird noise at night, discovers it was a fan all along”, I think it’s just human nature to appreciate it.
> There's actually a lot of prevailing wisdom that suggests disabling Nagle's algorithm is (often) a good idea.
Because even in mediocre networks it is a good idea.
Don’t write a small amount of data if you want (or in this case even need) to send a large amount of data!