Comment by formerly_proven
19 hours ago
> You can switch a file descriptor into non-blocking mode so the call won’t block while data you requested is not available. But system calls are still expensive, incurring context switches and cache misses. In fact, networks and disks have become so fast that these costs can start to approach the cost of doing the I/O itself. For the duration of time a file descriptor is unable to read or write, you don’t want to waste time continuously retrying read or write system calls.
O_NONBLOCK basically doesn't do anything for file-based file-descriptions - a file is always considered "ready" for I/O.
Is that true for all file abstractions? What happens with NFS?
Think about it, what does it means for a file to be ready? Socket and pipes are a stream abstraction: To be ready it means that there is data to read or space to write.
But for files data is always available to read (unless the file is empty) or write (unless the disk is full). Even if you somehow interpret readiness as the backing pages being loaded in the page cache, files are random access so which pages (ie which specific offset and length) you are interested in can't be expressed via a simple fd based poll-like API (Linux tried to make splice work for this use case, but it didn't work out).
Don’t block devices have a scheduler with a queue under the hood? Couldn’t that queue become full when writing?
(This is a genuine question)
4 replies →
I think you’re correct. Your file descriptor may represent an end of a pipe, which in turn is backed by a buffer of limited size. Ruby’s I/O API specifically warns that reading lop-sidedly from e.g. stdout and stderr without `select`ing is dangerous [0].
I’ve experienced deadlocks in well-known programs, because developers who were unaware of this issue did a synchronous round-robin loop over stdout and stderr. [1]
[0]: https://docs.ruby-lang.org/en/master/Open3.html#method-c-pop...
[1]: https://github.com/Homebrew/homebrew-cask/pull/21665