Comment by londons_explore

Things needing the most compute (llm's, image and video generation) tend not to be latency sensitive.

100ms of latency is nothing when added to 10 seconds of generation time.