Comment by K0balt

Idk, I’m running 180b on $2500 of hardware at 120 T/S (single user of course) but I bought it before the clown show. Now it’s probably more like $10k lol

I expect I’ll be able to run 1/2 terabyte models in 5 years for a couple $K in hardware. Inference is going to be a lot like Bitcoin mining. Only the peak of the power/performance curve is going to be profitable. Which means a glut of surplus hardware that can’t make money but makes tons of tokens.