Comment by jonhohle
7 hours ago
In high volume systems, 10ms is kind of crazy. I’ve run systems with operation metrics in the ms scale and the server side latency was lower than 1ms (computing business logic, or heavily cached data). Client side was closer to 3-5ms. As you mentioned, this was Java.
There also needs to be care taken in how these measurements are aggregated. Averages will almost always tell you nothing. High percentiles (95%, 99%, 99.9%) under load may show you something completely different than the average or even median case.
You can still benchmark say 100 requests and time that. For JVM for example you'll have to consider the JIT effects and GC and stuff that only happens later. So 100x 1ms requests is still a very small benchmark that's probably unreliable