Comment by philipkiely
7 hours ago
I think the biggest net new recent technique is P/D disaggregation. And that spec dec is very different now especially post DSpark/DFlash.
But overall yes the fundamentals of LLM performance optimization have been remarkably stable over the last few years.
perhaps its unfair to say this in hindsight, but it's a fairly straightforward application of little's law that's been around for some time
https://arxiv.org/html/2401.09670v2