Comment by circuit10
3 years ago
It’s true that their training goal sometimes rewards them hallucinations, encouraging them do that, but that doesn’t mean it’s not reasoning. If anything hallucinations involve more reasoning because they have to make up something new that wasn’t there
It's doing what LLMs do. Next word prediction. It didn't "make it up", it incorrectly predicted the relationship of words.
It isn't reasoning about the solution to a problem either, it's running as expected in relating words to each other, that doesn't mean it has any form of understanding of the words or even what it's rendering as an output.
It does just predict the next word, yes, but it turns out that predicting the next word is a complex problem that involves solving many types of subproblems that involve relatively complex reasoning, or at least something that looks very, very much like reasoning
Is it any more reasoning than Bayes' theorem is reasoning? It's statistics and derived probability from said statistics.
9 replies →