Comment by knollimar
4 hours ago
Glm 5.3 won't fit on a mac mini. The barrier to entry to host something scary is what, like $10k? 20k?
4 hours ago
Glm 5.3 won't fit on a mac mini. The barrier to entry to host something scary is what, like $10k? 20k?
Moore's law still holds I reckon.
Even it's $10,000 to run today (FWIW, the featured article cites the M5 Mac Studio with 256GB unified memory going for $9,500 as "good enough to host something scary"), in a couple years it'll be like $2k to run, and in another couple after that, you'll have used $200 dollar smartphones capable of running a model powerful enough to do serious damage.
Haven't ram prices been a linear drop since 2010 before chatgpt rampocalypse? I don't think it's "still" anymore but perhaos some progress will be made in the model front