Comment by IshKebab

1 day ago

SQLite is really fast. There's no way that's the bottleneck.

What’s your experience with SQLite? It’s a bit hard to talk about performance without sharing code.

  • I've used it a fair bit. My biggest use was for a computer processing system that recorded gigabytes of data. If it was limited to 60 inserts per second it would have taken months to run!

    I do recall having to change some settings to make it really fast, but it wasn't 60/second slow.

    See the "update" in this answer.

    https://www.sqlite.org/faq.html#q19

    • Appreciate the answer! You’ve probably worked with raw SQLite drivers. I’m using a framework, which likely runs more transactions by default. I’m fairly confident that with a bit of digging, I can improve the ingestion speed. Good to know and thanks for sharing your experience!