Comment by thombles
4 days ago
Be cautious if you're using large databases on iOS. At least until fairly recently, iOS doesn't page dirty mmaped pages back to disk and after enough churn the app will OOM.
4 days ago
Be cautious if you're using large databases on iOS. At least until fairly recently, iOS doesn't page dirty mmaped pages back to disk and after enough churn the app will OOM.
Isn’t that why the mapping is read-only by default?
Irrelevant, since LMDB doesn't dirty the pages in the mmap. The mmap is read-only.
Wow, really?
Then what’s the point of memory mapping in the fist place? Or do they suggest manual flush/sync actions for persistence.
IIRC: it is to leverage the OS page cache rather than having a separate buffer pool in user land. By default lmdb uses normal pwrite/fsync for the write path, but can optionally use a writable mapping and (presumably) msync.
However, some people think there are problems with this usage: (pdf warning) https://www.cidrdb.org/cidr2022/papers/p13-crotty.pdf
How is pwrite/fsync any better than mmap/msync? Both go through the page cache and combine asynchronous writeback with forced flush. One theoretical advantage of pwrite might be that you can handle I/O errors, but I’d like to see a case where recovering from an I/O error makes sense (rather than just crashing the database, which SIGBUS would do anyway by default).
1 reply →