← Back to context Comment by argee 11 hours ago Can't you just look at memory bandwidth for generation? 2 comments argee Reply josephg 11 hours ago You also need to look at memory size (to store the model). And processing can still be the bottleneck. stingraycharles 11 hours ago It’s more nuanced than that, but yes that’s the biggest factor.
josephg 11 hours ago You also need to look at memory size (to store the model). And processing can still be the bottleneck.
You also need to look at memory size (to store the model). And processing can still be the bottleneck.
It’s more nuanced than that, but yes that’s the biggest factor.