Comment by dgellow
11 hours ago
One thing about the reasoning is that models are trained to generate a chain of thoughts, but it doesn’t have to be correct, accurate, or reflect the underlying logic of the LLM. It’s the same problem we have with the output, it is something plausible, but not that reliable
I do the same thing in my head. There is no underlying logic to an llm. Logic is an external construct alien to human like forms of reasoning.