Comment by verelo
11 hours ago
This is the feeling i get too. Cant produce quality, but can produce something that is super fast...so take the wins where they are.
11 hours ago
This is the feeling i get too. Cant produce quality, but can produce something that is super fast...so take the wins where they are.
We don't have enough fast models, so I see this as a positive. I just test drove Gemini Flash Lite and it's crazy fast.
we need fast + cheap AI model, those gemini 2.0 flash is superb
like it literally pennies
For a coding LLM specifically, when is fast a good tradeoff for quality?
> ...when is fast a good tradeoff for quality?
When it is cheaper, and the "lower quality" model is adequate for the task at hand.
Plenty of problems have a low(er) skill/intelligence floor, anyone who uses the dual-mode agent paradigm (plan, then act) figures out the second phase can be completed by a less capable model. Even when disregarding costs - speed is important here because the agent can rapidly iterate without human supervision, based on compiler errors, lint and test failures
A coding agent driven by a large LLM can delegate smaller tasks to a faster model. For example searching through the codebase for references, examples, or established patterns. They are treated as tools and don't pollute the main agent's context.
I wouldnt say it is, but there are circumstances when speed is helpful. I wouldn't argue that coding is one of them.