Comment by sarchertech

Those numbers can be misleading though.

If your average per request in the application layer is say 20ms. Your p50 is probably 10-15ms and your p99 is probably something like 80ms. p99.9 is probably a good bit higher than that.

If your site is highly interactive, everyone using it is going to regularly run into actions that take long enough that they notice them, and it won’t feel snappy.

Let’s say you went all out and wrote the application layer in C with purely static allocation. You could easily see a p50 around 1-2ms and a p99 of 3-5ms. (This is obviously dependent on the app).

Even with an average time spent waiting in the database of 80ms, you’ve already made a noticeable difference.

That’s not even mentioning the biggest impact, which is that you can likely get by with far fewer application servers.

Now does it make even more sense to optimize your queries? Yes. Definitely. But with faster application stack you can start moving more things into the application layer and out of the database which is generally harder to scale.

And if you have or can build endpoints that don’t hit your database at all, those will actually be 10x faster than before every time.