Comment by raggi

14 days ago

We have very good reasons for our checkpointing model, related to our backup + disaster recover strategy, along with resource cost. It might be worth writing about one day, so I'll not give away all the details, but in very short form, we organize a backup strategy that has minimal pause time, avoids doubling the page cache cost of the database, and enables extremely fast byte-copy restores in disaster recovery.

3 comments

raggi

Reply
inigyou  14 days ago

okay but explain why you are using sqlite and copying the file to S3 instead of using any client/server DB and its online backup feature?

  • raggi  14 days ago

    SQLite has an online backup API as well, but it is slower and requires a significant additional page cache cost.

    The team chose SQLite early on (there are some blog posts about this) and then we vertically scaled against the SQLite architecture. There are subtle ways you come to depend on the proximity/latency when you scale with local storage that mean switching requires a lot of non-obvious work - it’s probably the largest hazard for embracing SQLite in a growing saas - but at the same time you can push the vertical scale pretty far, which has great margins.

    Had we scaled a different architecture of database there’s little reason to believe it would have been plain sailing as seems to be implied here.