Comment by kiitos
1 month ago
i mean the details are far beyond what can be effectively communicated in a HN comment but if your loadgen tool is doing anything like sleep(1000ms) it is definitely not making any kind of sound request-per-second load against its target
and, furthermore, if the application and DB are co-located on the same machine, you're co-mingling service loads, and definitely not measuring or capturing any kind of useful load numbers, in the end
tl;dr is that these benchmarks/results are ultimately unsound, it's not about optimization, it's about validity
if you want to benchmark the application, then either you (a) mock the DB at as close to 0 cost as you can, or (b) point all application endpoints to the same shared (separate-machine) DB instance, and make sure each benchmark run executes exactly the same set of queries against against a DB instance that is 100% equivalent to the other runs, resetting in-between each run
The point of the test was to test a SYSTEM on the same machine, not just the app - db and app are on the same machine by design, not mistake.
Tests on the other hand were executed on multiple different machines - it's all described in the article. Sleep works properly, because there's an unbounded thread pool that makes http request - each request has its own virtual thread.
there is no real-world deployment where both (a) db+app are colocated on the same machine, and (b) performance benchmarking is relevant. these are mutually exclusive properties. (sqlite != db)