Comment by slopinthebag
8 hours ago
People said this for Opus 4.6 too. Every release the models get RLHF'ed into accomplishing a new task and the people who need to do this task think there was a step change.
8 hours ago
People said this for Opus 4.6 too. Every release the models get RLHF'ed into accomplishing a new task and the people who need to do this task think there was a step change.
To be fair Opus 4.6 was genuinely a really good model when it came out, in fact I'm not sure Opus 5 is even any better.
It's definitely a lot slower, though
Yeah that's kinda my point. I'm not sure if the models have gotten that much smarter, but they're certainly getting more capable. That's not the same thing though.
There are things GPT 6.0 can accomplish for me that 5.3 was not able to. But there are also things it still fails at, and it doesn't seem to be much better at the big picture. It does spam about 100x more tests though and I wonder if just RLHFing it to test everything constantly is carrying it more. 6.0 writes so many tests and spends so much time verifying it's work in python sandboxes. Slow as hell but it tends to get things right the first time more which is good, I guess. I don't love the thought of a 500loc feature adding +4000loc due to tests though.
Yep same with latest Claude models - code isn't really any better than Opus 4.5/4.6, but use 5x as many tokens doing random stuff that's mostly unnecessary.
And yeah still for some reason they often can't understand how to set up any project locally without handholding, which is something you'd think an LLM would actually be good at