Comment by ambrop7

It's possible to abstract IOCP and non-blocking sockets into a single interface with reasonable efficiency. For example:

- To send data, user calls Socket::send(const char *data, size_t length). This starts sending the given data; the buffer must remain available while data is being sent. IOCP implementation will initiate I/O using WSASend() or similar.

- When send operation is complete, the socket implementation calls a Done() callback of the user, and from that point on, the user is allowed to call Socket::send() again. With proper implementation, send() can directly be called from the callback.

This interface is very easy to implement on Linux. But on Windows there's a complication, because it's non-trivial to just stop an IOCP I/O operation that is in progress, in case you decide you don't need the socket any more and want to red rid of it NOW. If you just forget about it, there could be a crash when you release your buffer but Windows is still using it. A simple solution is to CancelIo() the socket and do blocking GetQueuedCompletionStatus()s until you get an event inidicating the completion of the pending I/O operation - taking care to queue up any unrelated results you may have gotten, for later processing. However, a more efficient solution is to use reference counting or similar with your buffers, so they only get released when Windows is done with them.

I didn't say it's easy, but it's possible.