Comment by jstummbillig
9 hours ago
Defining what "coding" means now, and how quickly we fall off the capability cliff seems increasingly important.
Today my "coding" sessions often enough begin with real life problems, where I discuss domain or inter-domain things, ranging from business, economics, psychology, etc. Being able to do all of that with one model is something I am willing to pay a premium for.
Of course not having to pay the premium, because the routing is smart or whatever, would be great. I just don't want to have to think about it.
> Today my "coding" sessions often enough begin with real life problems
intuition is that your sessions consists of 10% of domain related reasoning, and 90% of code plumbing. Those 90% could be moved to cheap and efficient specialized and focused model.
But that 10% is the most important part! Getting the plumbing wrong means you might have bugs or your code is brittle. Getting the domain-specific business logic wrong means your product doesn't fundamentally solve the correct problem.
Possibly! It's just hard to reason about from the outside. When does the model benefit from all the ambient knowledge? Idk.
Regardless, it's fairly obvious to me that none of what I do now will require "frontier models" for much longer. Models are getting better more quickly than my problems are getting harder.
but that is not model problem
most agentic coding app can use powerful model for planning/reasoning then use "budget" model to do ground work
> most agentic coding app can use powerful model for planning/reasoning then use "budget" model to do ground work
I've had terrible success using budget models to do ground work. The justifications that the budget models will use and document, polluting the rest of the session, are sometimes just insane. Like making code compatible with a bug that was implemented within the same session, not handling errors due to precedence in the code it just implemented, etc. I DO have success using the heavy models with lower effort, and using budget models on relatively changes post ground work. But major planning and initial ground work, I just get absolutely slop if I use a budget model.
If you're doing web stuffs, or GUI, then the budget models seem fine.