Comment by arn3n
17 days ago
Always a fun time to bring up my favorite writeups in database benchmarking: https://questdb.com/blog/lies-damn-lies-and-database-benchma...
There’s also an excellent paper called “Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing”.
Benchmarks are genuinely hard. Modern systems and hardware are notoriously complex and have different behaviors across runs and with minor tweaks to performance settings.
Yep, absolutely... and in the end benchmarks aren't going to sell a database anyways. Reliability and provable correctness and ease of integration are probably going to win in the end. It's a dubious marketing edge to lead with.
PostgreSQL itself is remarkably slow in many configurations and leaves a lot of performance on the table due to its architecture and assumptions. But it works and people trust it. So they use it.
As a SpacetimeDB dev, I absolutely agree. Reliability, provable correctness, and ease of integration are higher priority than performance. That doesn't mean we need to accept garbage performance though.
You may be interested in our technical blog post on benchmarking methodology for this benchmark: https://spacetimedb.com/blog/benchmarking
Or if you're in enterprise, whichever company's sales rep some executive has locked you into while 10 drinks in at a fancy dinner.
I am a creator of SpacetimeDB. Several assertions in this article are simply wrong. You can read our response here: https://spacetimedb.com/blog/benchmarking