Comment by adrian_b

I agree that the maximum SSD performance can be achieved with carefully planned asynchronous reads, using either the modern liburing (io_uring) or the older libaio (Linux asynchronous I/O).

Nevertheless, I do not think that "this is not really possible with mmap, even with madvise".

If the kernel is not eager enough to prefetch pages from the SSD when you use madvise with MADV_WILLNEED, then you can use madvise with MADV_POPULATE_READ, which should force the reading of the pages that you request.

Using madvise with MADV_POPULATE_READ for appropriate page ranges at the right moments should be able to provide a performance not much lower than when using explicit asynchronous reads.