Comment by Frieren
8 hours ago
This assessment fits with my anecdotal evidence. LLMs just cannot reason in any basic way.
LLMs have a large knowledge base that can be spit out at a moment notice. But they have zero insight on its contents, even when the information has just been asked a few lines before.
Most of the "intelligence" that LLMs show is just the ability to ask in the correct way the correct questions mirrored back to the user. That is why there is so many advice on how to do "proper prompting".
That and the fact that most questions have already been asked before as anyone that spend some time in StackOverflow back in the day realized. And memory and not reasoning is what is needed to answer them.
Please don't tell me you were one of those marking every SO question as duplicate, more often than not missing the entire nuance in the question that made it not a duplicate at all, and the answers to the so called previously asked question utterly unusable?
This was one of those infuriating things that drove so many away from SO and jump ship the second there was an alternative.
I'm not sure why duplicates were ever considered an issue. For certain subjects (like JS) things evolved so quickly during the height of SO that even a year old answer was outdated.
That and search engines seemed to promote more recent content.. so an old answer sank under the ocean of blog spam
SO wanted to avoid being a raw Q&A site in favor of something more like a wiki.
If a year-old answer on a canonical question is now incorrect, you edit it.
5 replies →
I was "playing" the gamification part of StackOverflow. I wanted to ask a good question for points. But it was very difficult because any meaningful question had already been asked. It was way easier to find questions to answer.
Every time I ask people for an example of this, and get one, I agree with the duplicate determination. Sometimes it requires a little skimming of the canonical answers past just the #1 accepted one; sometimes there's a heavily upvoted clarification in a top comment, but it's usually pretty reasonable.