Comment by vardump
10 years ago
> A normal Pause sends this packet periodically and once it has buffers to receive will send a pause with a quanta of 0 meaning cancel previous timer... but if it's malfunctioning who knows if it ever will...
As with most protocols, it doesn't work if it's not implemented properly.
Many hardware implementations have no knowledge of when the buffer can be emptied, so it's understandable they treat it as a on/off switch. Screaming, my buffer is full, don't send me anything for 65535*512 bit-times! Which is perfectly good, because otherwise all those incoming frames would need to be dropped anyways.
Remember, small embedded devices can't often guarantee ethernet DMA slot time to DRAM, and definitely can't afford to have a dedicated DRAM channel, so those buffers are on a 2-8 kB on-chip SRAM block or equivalent.
When the buffer is "full" (above high water mark), an interrupt gets generated and device firmware will set up an appropriate DMA transfer to empty it. Once that is done, the device should of course send a PAUSE 0, and all is good.
> The sad part is that I don't even know what to recommend for a good consumer level switch that has good defaults or configurable defaults and sane buffer config...
Like you must know, you can turn it off entirely in most managed switches, see what happens to data transfer speed.
Most consumer level gigabit switches seem to have maybe 16 kB buffer. So they don't really have much buffers (or anything) to configure.
Like you said the host might have small buffers and without Pause it would drop, but who's supposed to buffer the packets, the cheap switch with 16kB of buffers and super idiotic buffer configuration such that everyone else on that switch gets paused?
You seem to think that it's bad to drop packets in the nic and while some nic might have buffers that are too small but in general you should drop. If you use TCP the window will adjust to whatever your bad nic and embedded system can handle. At least you won't affect the others by spreading pause like a cancer ( can you tell I am cynical on pause )
Usually on a switch you can usually drop packets based on the number of packets destined to a port and packets buffered per input port. This is how you can avoid head of line blocking but again if you are right with 16kB that's barely enough for a jumbo packet (~9200B)... geez that's depressing.
> If you use TCP the window will adjust to whatever your bad nic and embedded system can handle.
TCP window, sigh... It can't deal with the situation where, say, every second frame is lost, because someone thought 2 kB is enough buffer. TCP congestion control mechanisms are great for actual congestion, but when packet loss is due to other causes, it's actually pretty bad.
Again, TCP is no substitute for flow control in this case.
Doesn't matter how nice NIC you have. The problem usually happens before the packets reach your nice NIC.
Lack of buffer space is pretty much the definition of congestion.