Comment by argee 12 hours ago Can't you just look at memory bandwidth for generation? 2 comments argee Reply josephg 12 hours ago You also need to look at memory size (to store the model). And processing can still be the bottleneck. stingraycharles 12 hours ago It’s more nuanced than that, but yes that’s the biggest factor.
josephg 12 hours ago You also need to look at memory size (to store the model). And processing can still be the bottleneck.
You also need to look at memory size (to store the model). And processing can still be the bottleneck.
It’s more nuanced than that, but yes that’s the biggest factor.