Comment by BoorishBears

Consumer, as in B2C, not consumers buying directly. B2C companies will happily buy (or rent from people who are buying today) GPUs, because a huge part of the game is managing margins to a degree B2B typically doesn't need to concern itself with.

> I dont need a model to know who Tom Cruise is or how to write SQL if I am asking it "set up my amazon refund" or "cancel xyz service". The moment someone figures out how to build targeted and small it will take off.

I think people got a lot of ideas when dense models were in vogue that don't hold up today. Kimi K2.5 maybe be a "1T parameter model" but it only has 32B active parameters and still easily trounces any prior dense model, including Llama 405B...

Small models need to make sense in terms of actual UX since beating these higher sparsity MoEs on raw efficiency is harder than people realize.