Comment by Arn_Thor
2 months ago
Except I can run R1 1.5b on a GPU-less and NPU-less Intel NUC from four-five years ago using half its cores and the reply speed is…functional.
As the models have gotten more efficient and distillation better the minimum viable hardware for really cooking with LLMs has gone from a 4090 to suddenly something a lot of people already probably own.
I definitely think a Digits box would be nice, but honestly I’m not sure I’ll need one.
R1 1.5b won’t do what most people want at all.
No, it won't. But that's not the point I was making
Yeah but what was R1 trained with? 50k GPUs as far as I've heard as well as distillation from OpenAI's models (basically leaning on their GPUs/GPU time).
Besides the fact that consumers will still always want GPUs for gaming, rendering, science compute etc.
No, I don't have any Nvidia stocks.