← Back to context

Comment by lunar_mycroft

18 hours ago

> For coding you always want to go with the best model in the category

This is transparently false, because the best "model" is still competent human developers. They're just more expensive. If you're willing to use current LLMs at all, it means you're willing to sacrifice quality for a better price, and your disagreement with the comment you were replying to is entirely about what the optimum tradeoff is.

17 comments

lunar_mycroft

Reply
aspenmartin  17 hours ago

Well it may be false that you always want the best model, but the point is performance of you+<agent> is far more cost effective than you+someone else

  • lunar_mycroft  4 hours ago

    Maybe, but that's a different claim than the one I was responding to. And also raises the question of "if the lower quality but cheaper output of frontier models is more cost effective than humans, is the even lower quality but even cheaper output of OSS models is more cost effective still?" With an absolute rule like GP suggested ("no, you always want the best code generator") the answer is clear, but it get much murkier if you reject such rules (as you have to to be an LLM coding proponent)

noname120  9 hours ago

It was true 6 months ago, not anymore. Frontier models now outperform developers on many tasks, be it on quality/readability/maintainability, and let’s not talk about speed…

  • lunar_mycroft  4 hours ago

    I've seen the code they produce without extensive help from human developers, this is clearly false.

    Good to see the classic "yeah the models weren't good enough six months ago, but this time they actually are, promise! Please forget you were hearing the exact same thing six months ago!" is alive and well though.

    • aspenmartin  2 hours ago

      Are you aware of performance trends though? You’re painting a picture that seems to ignore how things have consistently trended for many years now, even pre ChatGPT. It is absolutely data driven to say “an inflection point has happened within the last 6 months”. And that was also true 6 months ago (where people started using coding agents fairly consistently since sonnet 4). And it was true 6 months before that. It’s not like people are like “we’ve fixed all the bugs!” And then nothing has changed. I don’t necessarily agree with the parent poster that agents are better than humans but they are certainly much better at many tasks.

      3 replies →