Comment by pknopf
7 years ago
Imagine a juggler, juggling 5 balls at a time. At a set period of time, he will drop one ball and accept another ball thrown at him. He handles this very well because there is order/cadence.
Now imagine asking him to support being thrown balls randomly, at any interval. He may make it work, but I'd imagine he will stutter a bit.
In my experience, anytime you interrupt the page cache's normal routine, it stutters everything. I've seen the "sync" command freeze my Ubuntu machine (music player, GUI, etc).
I work on embedded devices, and my employer wanted to reduce the window at which data-loss would happen for 30mb+ files (video capture). It wasn't a supported use case, "But why not! It makes our product theoretically better!" I put my foot down. We aren't touching page-cache until there is a clear benefit to the user. It almost got me fired, but good riddance if so.
The annoying thing is that the Postgres people don't even really need "sync this file right now" for the main data files.
AIUI what they'd ideally like is "sync this file reasonably soon, and let me know when it's finished".
io_uring supports issuing async sync_file_range requests, which can be used to do most of the heavy lifting.
You will still need a final fsync to also get the metadata written, but that should faster at that point.
correction: I took a closer look at liburing and it also supports straight fsync[0], not just sync_file_range, this makes it even simpler.
[0] http://git.kernel.dk/cgit/liburing/tree/src/io_uring.h?id=90...
It should be noted that io_uring landed (in 5.1) a fair while after the whole Postgres back-and-forth started in 2018.
2 replies →
fsync != syncfs/sync
The former is for a single file, the others are for one/all filesystems
I'm aware, but the analogy still applies, which was about page-cache in particular.