Comment by mcbuilder

6 days ago

LRMs are plateauing for sure, not that there won't be gains to be had in the future, but it's not like the era of rapid progress that was the past year any more.

4 comments

mcbuilder

Reply
gdhkgdhkvff  6 days ago

I agree that the rapid improvement from like 2023-24 era is over (from a perspective of going from a 3/10 to a 7/10, you can’t then go to a 11/10). There was just so much more space to grow back then.

But isn’t Fable supposed to be another step change? I never used it, myself.

Tbh, at this point I think top tier models are smart “enough” (I’m sure this will look antiquated in a year), and the way to give me MORE noticeable improvement is to make them much faster rather than much smarter. Or even a way to automatically and accurately pick faster models when it makes sense. I know that IDE’s have Auto modes, but it’s not something that I trust right now to pick smart+fast instead of picking “maybe smart enough”+”cheaper for harness owner”

  • ZeroCool2u  6 days ago

    Having used it quite a bit when it was out, it's not. It's certainly better, but in some ways it's worse. It's trained to be more "agentic" and even in cases where I wanted to talk things through first and I would explicitly tell it not to do something, it would take action on my behalf without checking first.

    It's also still just prone to the kind of "stupid" mistakes we see from all LLM's. Like it can write great code, but it doesn't really have common sense without enormous guidance.

roughly  6 days ago

A great many people were predicting this would be the case a year ago and being told they were wrong and to get on the boat.