Comment by leni536
2 years ago
> The performance difference arises (we believe) because when working from an SQLite database, the open() and close() system calls are invoked only once, whereas open() and close() are invoked once for each blob when using blobs stored in individual files. It appears that the overhead of calling open() and close() is greater than the overhead of using the database
I wonder how io_uring compares.
RecordIO would be a good choice for this use case
https://mesos.apache.org/documentation/latest/recordio/
Yeah but imagine a Beowulf cluster^H^H io_uring of SQLites
It seems like nobody has suggested putting SQLite databases inside SQLite blobs yet... you could have SQLite all the way down.
https://news.ycombinator.com/item?id=41085856
Imagine the peformance we could reach. With enough SQLite layers anything would be possible.