Comment by glouwbug
15 days ago
We have this same point made every release. I wonder if models actually are getting better or if these companies are just secretly hamstringing the previous models before releasing a similarly performing model
15 days ago
We have this same point made every release. I wonder if models actually are getting better or if these companies are just secretly hamstringing the previous models before releasing a similarly performing model
I think it takes a while to start finding the weak points in a model. It seems fantastic until you use it enough to find out that it hallucinates APIs, or can’t count letters, or uses too many em-dashes, etc. you don’t really know someone (or something) until you’ve lived with them (or it).
I think there's definitely some quantization or nerfing happening post-release -- I definitely noticed that with Gemini Flash 3.7, it was super fast and accurate at the start and a few weeks later seemed to run in circles and overthink even a "what are you doing" message, all while doing almost identical work. I find it weird that we're so far into the LLM era and the post-launch degradation is still something of a conspiracy theory.
Models are actually getting better.
While IME 4.7 and 4.8 were not appreciably better than 4.6, Opus 5 was noticeably better, and Opus 5.5 is significantly better than Opus 5.
Try using a model from a year ago.
What would that prove if we are testing whether they "are just secretly hamstringing the previous models" as GP claimed?
This would require a conspiracy since Claude models are hosted by Google, Anthropic and Bedrock. All these companies would have to be working together to secretly hamstring the previous models.
2 replies →
Welcome to the hedonic treadmill! Just keep walking!