Comment by Animats
3 years ago
The basic problem, as I've written before[1][2], is that, after I put in Nagle's algorithm, Berkeley put in delayed ACKs. Delayed ACKs delay sending an empty ACK packet for a short, fixed period based on human typing speed, maybe 100ms. This was a hack Berkeley put in to handle large numbers of dumb terminals going in to time-sharing computers using terminal to Ethernet concentrators. Without delayed ACKs, each keystroke sent a datagram with one payload byte, and got a datagram back with no payload, just an ACK, followed shortly thereafter by a datagram with one echoed character. So they got a 30% load reduction for their TELNET application.
Both of those algorithms should never be on at the same time. But they usually are.
Linux has a socket option, TCP_QUICKACK, to turn off delayed ACKs. But it's very strange. The documentation is kind of vague, but apparently you have to re-enable it regularly.[3]
Sigh.
[1] https://stackoverflow.com/questions/46587168/when-during-the...
Gotta love HN. The man himself shows up to explain.
Imagine being on a math forum discussing Fermat’s theorem and the guy shows up.
This is such a cool aspect of CS being a young field: influent people are still alive!
It happens in math too! for example: https://mathoverflow.net/questions/81960/the-dzhanibekov-eff...
Andrew Wiles showing up would probably be the next best thing.
1 reply →
Fermat would probably get banned for too much trolling around the proof of his last theorem...
Can I show up for the future?
For those like me who didn't know, GP designed Nagle's Algorithm in 1984 working at Ford Aerospace.
i thought you mistyped your comment and wanted to reply to rsc ... then i clicked on animats profile. Yeah HN is becoming a treasure trove for CS.
Yeah, it's pretty cool. I'm gonna start saving these moments every time they happen. Last time I witnessed something like this was:
https://news.ycombinator.com/item?id=24455758
> The documentation is kind of vague, but apparently you have to re-enable it regularly.[3]
This is correct. And in the end it means more or less that setting the socket option is more of a way of sending an explicit ACK from userspace than a real setting.
It's not great for common use-cases, because making userspace care about ACKs will obviously degrade efficiency (more syscalls).
However it can make sense for some use-cases. E.g. I saw the s2n TLS library using QUICKACK to avoid the TLS handshake being stuck [1]. Maybe also worthwhile to be set in some specific RPC scenarios where the server might not immediately send a response on receiving the request, and where the client could send additional frames (e.g. gRPC client side streaming, or in pipelined HTTP requests if the server would really process those in parallel and not just let them sit in socket buffers).
[1] https://github.com/aws/s2n-tls/blob/46c47a71e637cabc312ce843...
Any kernel engineer reading that can explain why TCP_QUICKACK isn't enabled by default? Maybe it's time to turn it on by default, if it was just a workaround for old terminals.
Enabling it will lead to more ACK packets being sent, which leads to lower efficiency of TCP (the stack spends time in processing ACK packets) and lower link utilization (these packets also need space somewhere).
My thought is that the behavior is probably correct by default, since a receiver without knowledge of the application protocol is not able to know whether follow-up data will immediately, and therefore not able to decide whether it should send an ACK or wait for more data. It could wait for a signal from userspace to send that ACK - which is exactly what QUICKACK is doing - but that comes with the drawback of now needing an extra syscall per read.
On the sender side the problem seems solvable more efficiently. If one aggregates data in the application, and just sends as everything at once using an explicit flush signal (either using CORKing APIs or enabling TCP_NODELAY), no extra syscall is required while minimal latency can be maintained.
However I think it might be a good question on whether the delayed ACK periods are still the best choices for the modern internet, or whether much smaller delays (e.g. 5ms, or something along a fraction of the RTT) could be helpful.
Thanks for this reply. What I find specially annoying is that the TCP client and the servers starts by a synchronization round-trip which is supposed to be used to synchronise options and this isn't the case here! Why can't the client and the servers agree on a sensible set of options (no delayed ack if the client is using the Nagle algorithm)??
Is this referring to Nagle on the server, and delayed ACK on the client?
TCP_QUICKACK is mostly used to send initial data along with the first ACK upon establishing a connection, or to make sure to merge the FIN with the last segment.
How it's possible that delayed acks and nagle's algorithms are both defaults, anywhere? Isn't this a matter of choosing one, or another?
Did the move from line oriented input to character input also occur around then?
I remember as a student, vi was installed and we all went from using ed to vi.
There was much gnashing and wailing from the admins of the VAX.
1984 would have been largely character if desired -- you already had desktop PCs with joystick and mouse too. The problem was the original party-line ethernet with large numbers of telnet clients or some other [nonstop, nonburst] byte-oriented protocol or serial hardware concentrator, which was a universal situation at educational institutions of the mid-to-late eighties. The Berkeley hack referred to above likely boosted the number of clients you could run on one ethernet sub with acceptable responsiveness.