Comment by SchemaLoad
13 hours ago
Even with the latest models today, the hallucination rate is absurdly high on anything deeper than surface level knowledge or something that can be directly scraped from reddit.
And you notice when it's a topic you know well or something like software where you can immediately tell the options it's giving you don't exist on the page. Leading to the amusing statement "LLMs are bad at what I do but great at everything else".
I've (elsewhere) written about this diminishing return effect on LLM utility in relation to increasing expertise.
The question: what's the net positive gain of turning people who know nothing in a given field into sub-novices, while weighing actual experts down with work slop and marginal returns?
And I wonder what the true cost is of arming so many novices with that level of dangerous knowledge.
The fact that some people get genuine value from LLMs when learning doesn’t contradict the fact that they’re Dunning-Krueger “expertise” generators. The fact that the person learning from them is in charge of ensuring they aren’t full of shit, which they frequently are, is an inescapable flaw in this process. I honestly think that reduces the value of these things to just above what you can find out with a search engine with most topics. Hey, great. An improvement is an improvement right? Is it an improvement worth trillions of dollars and screwing over writers and artists worldwide? Fuck no.
Sounds about right.
Tangentially, but related: I'm old enough to remember when the spirit of your comment was pervasive on HN.
2 replies →
I think the true cost will be some catastrophic failures.
Just hoping folks don’t get hurt due to people not understanding what they’re doing with these things but believing they’re competent.
If that is the case, I seriously doubt the blame would ever touch the people that oversold this shit.