Comment by Pannoniae
17 hours ago
Great article :D The rabbithole actually goes even deeper because buffering on the read side is almost always good, but reads are often way more latency-sensitive than writes. Which leads to the paradox that increasing the buffer size / prefetching more with mmapping can lead to worse performance due to the added latency. And decreasing the buffer size can have the same effect due to issuing more calls, so in order to squeeze in the optimal throughput/latency ratio on reads you have to tune it per workload.... or just try a few different buffer sizes and see which one works better.
Writes are the opposite, they're usually not RAW (read-after-write), they're "fire and forget", so you don't really care about latency, but you do care a whole a lot about your writes actually arriving... again two competing parameters because the more you batch the writes, the faster they are but the bigger chance of a torn write or data loss. And unlike reads, writes are not idempotent on the application level so ideally you'd want some logic to atomically write a meaningful amount of data so it won't get torn, even if it crashes.
Now in practice just about no one actually does any of this but it's a fun thought :D
I have committed a sin of starting with the comments without reading an article, assumed it was about reading and writing text by humans and was increasingly confused by your first paragraph.
I tried to translate it as metaphor at first, but it didn't end up being coherent.
As did I. My happy impression that I was to be commended for reading slowly was turned upside down by the same realization.
Makes sense. If you are reading something you need it to make progress, if you are writing you are finished.
Most of that is just due to the fundamentally poor I/O model of synchronous operations used on Linux and other Unix derived systems followed by hardware manufacturers cheating to make that fundamentally poor model look okay.
Essentially all modern high speed hardware is batched and asynchronous. The way to get high speed performance is to batch and asynchronously submit operations then wait for batchs of asynchronous completion. In addition to being more efficient to process, batching also enables "smart" hardware to more efficiently schedule the operations if needed since it has more context.
Batching many of the reads you want upfront reduces the need for as much speculative read buffering because you are more capable of telling it "all/most" of the information you actually want instead of it having to guess that you are going to do linear reads. The reads being asynchronous, which is what the hardware does, allows you to issue speculative reads explicitly. If you intend to read a 1 GB file then the userspace I/O library can fire off 1,000 1MB reads and then you can wait until the "next" desired portion of the data comes back before continuing your processing. This is in contrast to issuing a 1 GB synchronous read followed by your processing which delays your processing by the time of the entire read even though the hardware I/O can happen in parallel.
Essentially, the only remaining case where you then want meaningful speculative read buffering (beyond the default efficient block size of the hardware) is that you either do not understand your access patterns enough to batch them, or you have a data-dependent load pattern which happens to regularly resolve to something within the speculative read buffer. For instance, you read 8-bytes which tells you the offset into the file where the desired record is and that record happens to be nearby almost all of the time.
For writes, what you want is a data-or-time flush strategy which is actually just the flush strategy you should use in basically all cases where you do buffering. Data-based flushing exists to achieve efficient batching. Issuing large amounts of small operations kills you with overhead, so you buffer the operations to attempt to coalesce them into larger operations where the overhead is amortized away. However, if inadequate data is generated to reach the data-based flush condition, then you get large or potentially unbounded latency. You resolve this by also targeting a worst-case acceptable latency in low throughput conditions. In that case your overhead has been "amortized" away as a percentage of your processing time since the processing time was long enough to reach the time-based flush condition. What you are effectively doing is allocating N% of your time to completion to overhead which either fires if you generate enough data to amortize it away or if you generate enough time to amortize it away.
Writes being asynchronous also allows you to resolve the data loss issue (if the hardware cooperates). You fire it off and wait for the hardware to signal persistence of that write. Until you see a asynchronous completion for that asynchronous write, you can not assume it is persisted. You achieve ordering by issuing a write barrier which tells the hardware to signal completion of the write barrier when all writes earlier in the asynchronous queue have signaled completion. This can also be further extended to other layers if desired to allow the detection of various levels of "persistence".
Unfortunately, Linux and other Unix derived systems use the awesomely terrible fsync() model which is antithetical to efficient I/O. Instead of you issuing a write barrier and then getting notified when it completes, allowing you to continue to asynchronously issue subsequent writes, you synchronously wait for all earlier I/O to complete. This serializes execution as you wait ensuring that any attempt at ordering or persisting data is monstrously expensive. In fact, this is so horrible that hardware manufacturers realized they could massively improve their throughput benchmarks by lying about when data is persisted so the fsync() would complete faster ensuring that their throughput benchmarks would not get shot due to data persistence latency which is a different property. With a asynchronous model you can just rapid-fire the operations and the write barriers (which is the way the hardware is designed to operate) allowing you to evaluate both read/write throughput and data persistence latency without confounding them. This is both dramatically more useful and more efficient.