Comment by measurablefunc
5 hours ago
Model segmentation & distillation like this that asks the consumers to pick exactly which version of the algorithm will solve their problem is evidence for lack of intelligence instead of its presence.
5 hours ago
Model segmentation & distillation like this that asks the consumers to pick exactly which version of the algorithm will solve their problem is evidence for lack of intelligence instead of its presence.
You really really don’t need to pick. Just use Sol on high. That’s my daily driver and I don’t touch the model picker at all.
Now, if cost is your concern, then that’s a problem in all of computing. Hence why I’m sending you short plain text messages using an iPhone with a many-core CPU and gigabytes of RAM.
it is really hard to know upfront if you have fuzzy task. sometimes i would choose a cheaper model and it will spin and spin with bad outputs ending up costing more had i chosen a more capable model.
there is mixture of experts which is also another routing. So, simple change in prompt can be a big difference.