Comment by mapontosevenths

> If they emulate reasoning well enough that it gets the same or better results what is the difference? Semantics?

No, of course not. The difference is that the ways in which we fail tend to be pretty ordered. You'd be hard pressed to find someone who's solved an Erdos problem but can't explain the difference between driving your car or walking to the carwash or can't count the Rs in their fruit names. Because if you can't count, you can't do math.

LLMs fail somewhat randomly because they do not have actual reasoning capabilities. It is hard to name that which they lack, because if we all knew, we would probably invent it.

Effectively, all problems are just search problems as" Newell and Simon argued as early as the 1950s. "LLM reasoning" today relies heavily on a side verifier. The coding loop that runs tests to see how it works, and so on. Which incidentally is what makes it so good at coding—that domain has a very quick and tight loop that can provide instant feedback about very targeted steps in their search.

But the corollary is LLM capability decays exactly along the gradient of verifier legibility. When you move to abstract problems that can't be easily verified, LLMs are pushovers with no real way to build nuanced abstract thought and literally think it through, find contradictions, decide on its own how to improve it and so on. They also have no spontaneous thinking, like you and I do in the shower sometimes. Because they have no agency, and those two things go hand in hand. Current transformer based models running on GPUs will never be efficient or fast enough to achieve that level of thinking. They're off by multiple orders of magnitude.

So the difference then is that their "approximate reasoning" is very useful, but is very flawed, and treating it as equivalent to human reasoning helps nobody. Believing in it is buying into hype, copium, and hopium. And, ironically, it likely delays the advent of proper AGI