Comment by girvo

My GB10 Spark-alike is absolutely amazingly fun… but it is not cost effective. Step 3.7 Flash is shockingly capable (IQ4_XS and used for web dev mainly), but it cost me $6800 AUD. They’re even more expensive now. The numbers just don’t make sense: with proper triple head MTP I can get it up to ~40tk/s decode and it runs at around 1000+ tk/s prefill.

$6800 is a lot of API credits for GLM, for example, on any provider you want to use.

Now being able to run models uncensored and with privacy has value! But the cost for these is rough today.

I still am going to buy a second one haha