Comment by menaerus

Well the thing is that the trajectory of people utilizing the models is only increasing so getting the most out of your HW becomes a particularly interesting optimization point for companies doing the inference at massive scale.