Comment by skippyboxedhero
2 days ago
You can tell with some of the more recent models that they have been fine-tuned in such a way to produce a lot of descriptions that sound impressive but have no real meaning. In terms of production of semantic text, the recent models are significantly worse than a year ago.
It is not completely useless as a tool but there is still a huge gap in most real-world applications (I work in an org with multiple thousands of devs and I am top 5 AI user consistently, I have used Claude Code since pre-release). If you are building something like an SQL database which has decades of logic tests built by humans then it can work. If you do not have this (i.e. 99.99% of the time) then you still need to be able to issue prompts that give context. And if you start without that context, it will produce shit code in extremely basic scenarios. In particular, humans are far better at producing code that is easy to change (in practice, you need to create unintuitive, non-human rules to govern this i.e. creating many artificial boundaries which would be completely unnecessary for humans).
I am not sure whether the models just capped out in terms of coding or whether they are just trained by people who have never worked on enterprise software but the gap isn't closing. It has just been filled by people who lack the knowledge to identify when they are being bullshitted (the prevalence of consultants suggests that this markets is large).
I have worked on legacy stuff before. In most cases, there is a kernel of reason and logic that can be recovered. I have seen pure AI projects and they are the worst grade of TCS-authored abandonware. They can often do one specific thing but are basically unmaintainable outside of that.
> You can tell with some of the more recent models that they have been fine-tuned in such a way to produce a lot of descriptions that sound impressive but have no real meaning. In terms of production of semantic text, the recent models are significantly worse than a year ago.
I frequently find this in LLM generated comments, both when using an LLM myself or when reviewing code that someone else used an LLM for. I find it is often the most obvious indication that nobody really read through what was generated.
Yes, it has become quite common that Claude’s text feedback, e.g. Claude found an issue in the existing code that requires a decision and Claude tries to explain it, it is a paragraph of bullshit words stringed together.
Then you investigate it and it is often something relatively simple that could have been explained with short sentence.
Have you found a way to stop this?
I used that engineering language skill which is good when it uses it...but it doesn't like to use it.
...there are clearly some areas of alignment that they are not looking at.
We had one that hallucinated benchmarks, with graphs and everything! (The code didn't actually work.)