Run the same prompt on old models and the current "SOTA" and you'll get pretty much the same answer word for word.
People think models have improved because tooling around the models (Claude Code, Cline, or your other favorite LLM wrapper) has improved, not because the models themselves have made any kind of leap.
They haven't improved at all since then in my experience. I poke at them every now and then and they still can't refrain from feeding me false info (and likely never will be able to, because they are stochastic parrots without any actual understanding). They are useless to me because I will take more time checking their output than I will just doing the task myself.
Yes?
Run the same prompt on old models and the current "SOTA" and you'll get pretty much the same answer word for word.
People think models have improved because tooling around the models (Claude Code, Cline, or your other favorite LLM wrapper) has improved, not because the models themselves have made any kind of leap.
Big time.
I'm not a true believer, but even I wouldn't say they have hit a wall.
What makes you say that?
They haven't improved at all since then in my experience. I poke at them every now and then and they still can't refrain from feeding me false info (and likely never will be able to, because they are stochastic parrots without any actual understanding). They are useless to me because I will take more time checking their output than I will just doing the task myself.
1 reply →