← Back to context

Comment by musicale

7 hours ago

> how to create a reliable stream of data on top of an unreliable datagram layer, then the solution that comes out will look virtually identical to TCP. It just is the right solution for the job

A stream of bytes made sense in the 1970s for remote terminal emulation. It still sort of makes sense for email, where a partial message is useful (though downloading headers in bulk followed by full message on demand probably makes more sense.)

But in 2025 much of communication involves messages that aren't useful if you only get part of them. It's also a pain to have to serialize messages into a byte stream and then deserialize the byte stream into messages (see: gRPC etc.) and the byte stream ordering is costly, doesn't work well with multipathing, and doesn't provide much benefit if you are only delivering complete messages.

TCP without congestion control isn't particularly useful. As you note traditional TCP congestion control doesn't respond well to reordering. Also TCP's congestion control traditionally doesn't distinguish between intentional packet drops (e.g. due to buffer overflow) and packet loss (e.g. due to corruption.) This means, for example that it can't be used directly over networks with wireless links (which is why wi-fi has its own link layer retransmission).

TCP's traditional congestion control is designed to fill buffers up until packets are dropped, leading to undesirable buffer bloat issues.

TCP's traditional congestion control algorithms (additive increase/multiplicative decrease on drop) also have the poor property that your data rate tends to drop as RTT increases.

TCP wasn't designed for hardware offload, which can lead to software bottlenecks and/or increased complexity when you do try to offload it to hardware.

TCP's three-way handshake is costly for one-shot RPCs, and slow start means that short flows may never make it out of slow start, neutralizing benefits from high-speed networks.

TCP is also poor for mobility. A connection breaks when your IP address changes, and there is no easy way to migrate it. Most TCP APIs expose IP addresses at the application layer, which causes additional brittleness.

Additionally, TCP is poorly suited for optical/WDM networks, which support dedicated bandwidth (signal/channel bandwidth as well as data rate), and are becoming more important in datacenters and as interconnects for GPU clusters.

etc.