Comment by troupo
4 hours ago
Anthropic broke their models in spring, denied it, gaslighted everyone who said so, and then all but admitted it: https://www.anthropic.com/engineering/april-23-postmortem (basically doing Anthropic things).
> I honestly feel like basically nobody knows anything about these models, it's all just vibes
This, too. Since only providers know what they actually serve, what they change and what limits they impose.
There are some visible degradations though. E.g. Claude-ish.
As for a personal anecdote: around February I created a rather complex quiz web app for myself and friends with multiple question types, sync between screens, multiple media upload types, multiple scoring and timing types, MC inetrface etc. etc. etc. It took me a week or so in the evenings with rather vague prompts to make it.
Now Claude (and Codex) cannot reliably build a much simpler web app even with precise instructions while also maintaining the visual consistency.
But I will agree with you, it's a feeling, not a precise measurement.
I am guessing (guessing) that "great coding model" and "great model for the widest use" (agentic, long-running, unsupervised, knowledge-worker, computer operator) are competing, conflicting endgames.-
Add to that:
- Of course, "labs" (quotes) are incentivized to throw coders under the bus and aim for the biggest possible market.-
- "Sharp, focused, brief, elegant, precise" editing, as would benefit the coding use-case, is actually token-saving, ergo, undesirable.-
The only thing that can stop this, would be the quality and functionality of the codeslop generated by these models to became so low that it actually interferes with the (alledged) recursive self-improvement of models (ie. models start to perform worse/degrade).-
Until such a time, we serfs, will eat what's on our plate, pay for it, and continue to kneel before the machine god overlords. And be glad for it.-