Comment by kolinko

You have speculative decoding that easily increases speed 2-4 times with no loss of quality, and of course MoA architectures that speed up inference 10 times or more, although with some quality loss.

Better hardware, and other techniques on top of that and you speed up even further.