Comment by zer00eyz
12 hours ago
> Consumer can eat all the GPUs they have and more if we stop trying to force B2B
You should really crunch the numbers on buying and then running enough compute to run a leading edge model. The economics of buying it (never mind running it) just dont add up.
You still haven't factored in "training", the major problem right now that every one remains head in sand about.
I dont need a model to know who Tom Cruise is or how to write SQL if I am asking it "set up my amazon refund" or "cancel xyz service". The moment someone figures out how to build targeted and small it will take off.
And as for training, well having to make ongoing investment into re-training is what killed expert systems, it's what killed all past AI efforts. Just because it's much more "automated" doesn't mean it isnt the same "problem". Till a model learns (and can become a useful digital twin) the consumer market is going to remain "out of reach".
That doesn't mean we dont have an amazing tool at hand, because we do. But the way it's being sold is only going to lead to confusion and disappointment.
Consumer, as in B2C, not consumers buying directly. B2C companies will happily buy (or rent from people who are buying today) GPUs, because a huge part of the game is managing margins to a degree B2B typically doesn't need to concern itself with.
> I dont need a model to know who Tom Cruise is or how to write SQL if I am asking it "set up my amazon refund" or "cancel xyz service". The moment someone figures out how to build targeted and small it will take off.
I think people got a lot of ideas when dense models were in vogue that don't hold up today. Kimi K2.5 maybe be a "1T parameter model" but it only has 32B active parameters and still easily trounces any prior dense model, including Llama 405B...
Small models need to make sense in terms of actual UX since beating these higher sparsity MoEs on raw efficiency is harder than people realize.