Comment by tzs

5 years ago

> Please no. This is an incredibly complicated file format that is not particularly well suited for file saving. It's also very slow (~100 transactions per second) if you aim for integrity, or unsafe if you don't.

The kind of application files they are talking about (things like word processor documents, spreadsheets, drawings, source code control system data) would only be writing sporadically. During one of those sporadic writes they might need to update thousands of rows but those could all be done in one transaction.

16 comments

tzs

Reply
ori_b  5 years ago

I made a mistake -- it's an order of magnitude slower. That means a single transaction is within the threshold for human perception. If you block the UI thread on a transaction, you're now dropping 6 frames at 60fps. If you block other operations on the transaction completing, you end up with lag.

Reference: https://www.sqlite.org/faq.html, question 19. (I've seen similar when testing on SSDs locally).

  • lostmyoldone  5 years ago

    If you have an FPS or interactivity target that needs to be guaranteed to be better than about 0.1-1 second - depending on platform - you can't have any disk IO at all on the UI thread, or any thread that's supposed to react at interactive rates at all times.

    It has generally become much better in the last decade or two, but one should still expect most OS's to sometimes pause for excessive amounts of time on disk IO unless the API is specifically guaranteed to never pause. Even then one would be wise to measure/log deviations if it's critical for the application. OS guarantees might also be contingent on driver/subsystem guarantees, and bad drivers might sometimes affect what seems completely unrelated upstream systems.

    • ori_b  5 years ago

      > If you have an FPS or interactivity target that needs to be guaranteed to be better than about 0.1-1 second - depending on platform - you can't have any disk IO at all on the UI thread, or any thread that's supposed to react at interactive rates at all times.

      Yes, and other file formats encourage doing things in memory, so you don't have any disk i/o in the common path.

      Using sqlite as a file format strongly discourages the simple, jank-free until you press save workflow of slurping your content, operating on it in memory, and then outputting it all in one operation as a response to an explicit user action. Instead, your whole application gets small but perceptible delays across all operations and interactions.

      8 replies →