Comment by HNisCIS

I'm in a company going through this. Everyone outsources their thinking to LLMs and the results are painfully mediocre. The smart ones will use it to get their bearings on the topic then go to primary sources, the not so bright just ctrl-c ctrl-v.

Have you ever been in an HN thread where you're an SME on the thread topic and just been horrified by the confidently incorrect nonsense 90% of the thread is throwing around? Welcome to the training set motherfuckers.

LLMs do the same thing for what should be obvious reasons. If you search things that have some depth and you know the answer you'll be flooded by how often the models will just vomit confident half truths and misrepresented facts. They're better than they used to be, not just lying whole cloth most of the time, but truth is an asymptotic thing, not an exponential one.