{"id":45867883,"user":"IshKebab","time":1762713384,"time_ago":"2 days ago","type":"comment","content":"

That seems like insanely low throughput. What takes it so long?","comments":[{"id":45868341,"user":"kyrylo","time":1762716272,"time_ago":"a day ago","type":"comment","content":"

It uses SQLite as its database.","comments":[{"id":45870172,"user":"IshKebab","time":1762729729,"time_ago":"a day ago","type":"comment","content":"

SQLite is really fast. There's no way that's the bottleneck.","comments":[{"id":45873047,"user":"kyrylo","time":1762756392,"time_ago":"a day ago","type":"comment","content":"

What’s your experience with SQLite? It’s a bit hard to talk about performance without sharing code.","comments":[{"id":45873368,"user":"IshKebab","time":1762759722,"time_ago":"a day ago","type":"comment","content":"

I've used it a fair bit. My biggest use was for a computer processing system that recorded gigabytes of data. If it was limited to 60 inserts per second it would have taken months to run!

I do recall having to change some settings to make it really fast, but it wasn't 60/second slow.

See the "update" in this answer.

https://www.sqlite.org/faq.html#q19","comments":[{"id":45873749,"user":"kyrylo","time":1762763204,"time_ago":"a day ago","type":"comment","content":"

Appreciate the answer! You’ve probably worked with raw SQLite drivers. I’m using a framework, which likely runs more transactions by default. I’m fairly confident that with a bit of digging, I can improve the ingestion speed. Good to know and thanks for sharing your experience!","comments":[],"comments_count":0,"level":4,"url":"item?id=45873749"}],"comments_count":1,"level":3,"url":"item?id=45873368"}],"comments_count":2,"level":2,"url":"item?id=45873047"}],"comments_count":3,"level":1,"url":"item?id=45870172"}],"comments_count":4,"level":0,"url":"item?id=45868341"}],"comments_count":5,"url":"item?id=45867883"}

Comment by IshKebab