Comment by sakex
1 day ago
> They won’t notice those extra 10 milliseconds you saved
Depends what you're doing. In my case I'm saving microseconds on the step time of an LLM used by hundreds of millions of people.
1 day ago
> They won’t notice those extra 10 milliseconds you saved
Depends what you're doing. In my case I'm saving microseconds on the step time of an LLM used by hundreds of millions of people.
Beauty of scale. Saving ten milliseconds a hundred times is just a second. But do it a billion times and you've shaved off ~4 months.
If you work at Google or whatever else is popular or monopolistic this week.
In most real jobs those ten milliseconds will add up to what, 5 seconds to a minute?
There is probably a non-linear function of how slow your software is to how many users will put-up with it.
Those 10 ms may quite well mean the difference between success and failure... or they may be completely irrelevant. I don't know if this is knowable.
2 replies →