Comment by rf15

> you could do the same back then with multiple models and more targeted finetuning

Definitely not, lol.

I have no idea how you could try and hold this argument without being facetious.

This is obviously not true to all of us here…

> you could do the same back then with multiple models and more targeted finetuning

I mean, come on, this is just not true. You could not achieve anything like what you can with modern agentic coding with Fable / 5.6 Sol from any combination or configuration of GPT 3.5 era models.

It's like saying that a teenager isn't an intellectually meaningful improvement over a toddler.

Sure they're both still fundamentally flawed humans prone to cognitive error, but one is clearly more likely to hit the mark than the other when assigned a task.

The only thing impressive is how wrong you are. LLMs have improved by an absolutely incredible amount in the last 4 years.

Utter nonsense.

There’s no way you could get models as smart by fine tuning. I couldn’t throw a problem like “build a pokemon database with UI to teach my son sql” and get a working system, nice ui, tests (which it iterated on) examples and explanations in one shot.

There weren’t thinking tokens. Maths is now dramatically better, making actual contributions when before they were mostly mocked for making extremely basic errors. Smearing is also something say is very rare in frontier models.

If you think they have barely changed you’ve either forgotten what they were like or not used them more recently, or you’re just being obtuse.

> you could do the same back then with multiple models and more targeted finetuning

Are you one of those anonymous billionaires as if you did this a few years ago, you would've been famous and rich.

If they use Python to fill the gap, and the end user doesn’t have to know or care, is it unfair to assess this as progress and attribute the progress to the _system_?

OK, the core technology that is the language model still can’t math as well as you’d hope, but how about the end result users see from the system when they interface with it?

“Did you know humans are better at flying today than they were a thousand years ago?” ‘No they’re not, they need planes.’ Technically correct in a way but isn’t it kind of annoying to be so stubbornly pedantic when the context is speed of reaching Point B from Point A?

Maybe more fair then would be: “I've worked with these systems for four years now and while they _have_ meaningfully improved in that time frame, they’re not perfect and remain fundamentally flawed in various ways.”

You prompt less. You need not inject search results into the context window yourself, a window much larger than years ago. You get code that’s already been run successfully once instead of finding an obvious show stopping bug yourself.

The technology is not a brand new one that fixed everything wrong with the old one, no, but not sure I would’ve noticed your comment if it had been such a bland observation. I genuinely assume good faith here… will say am tempted to assume the standards of someone posting such a thing might be impossibly high. Glad to be having a fun conversation instead of getting your grades on my work product or something :)