Comment by Philpax
4 days ago
Fair enough. If you remember what you were testing with, I'd love to try it again to see if things are better now.
4 days ago
Fair enough. If you remember what you were testing with, I'd love to try it again to see if things are better now.
You have a fair point. Some LLMs are better at some tasks, and prompts can make a difference no doubt.
Perhaps at some point there will be a triage LLM to slurp up the problem and then decide which secondary LLM is most optimal for that query, and some tertiary LLMs that execute and evaluate it in a virtual machine, etc.
Maybe someday
Oh I talked to some guys who started a company that does that. This was at an AI meetup in SF last year. They were mainly focused on making $/token cheaper by directing easy/dumb queries to smaller dumber models, but it also increases output quality because some models are just better at certain things. I'm sure all the big companies already have implementations of this by now even if they don't use it everywhere
Yes they are called routers. One is https://withmartian.com/
I was suggesting optimizing for answer quality, but optimizing for cost might be useful too I suppose for "business innovation" purposes.