← Back to context

Comment by satvikpendem

2 months ago

Why do people keep insisting that LLMs don't follow a chain of reasoning process? Using the latest LLMs you can see exactly what they "think" and see the resultant output. Plausible code does not mean random code as you seem to imply, it means...code that could work for this particular situation.

11 comments

satvikpendem

Reply
tovej  2 months ago

Because they don't. The chain-of-reasoning feature is really just a way to get the LLM to prompt more.

The fact that it generates these "thinking" steps does not mean it is using them for reasoning. It's most useful effect is making it seem to a human that there is a reasoning process.

  • seba_dos1  2 months ago

    I love how generating strings like "let me check my notes" is effective at ending up with somewhat better end results - it pushes the weights towards outputting text that appears to be written by someone who did check their notes :D

    • andai  2 months ago

      I can't remember which lecture it was, but a guy said "they don't think, they only seem to think, and they won't replace a substantial portion of human labor, they will only seem to do so" ;)

      1 reply →

  • andai  2 months ago

    Is this position axiomatic or falsifiable? What would it take to change your mind?

    • tovej  2 months ago

      It doesn't have to be either because the burden of proof is not on me. It's on whoever claims that chaining multiple prompts together produces thinking, even though a single prompt is just predicting n-grams.

      The chain does not change the token generation process, it just artificially lengthens it.

  • satvikpendem  2 months ago

    How would you determine humans have reasoning then, in a way that LLMs do not?

    • tovej  2 months ago

      Easy, humans can synthesize new facts using logic and context. And the conclusions can be checked against the real world to check that the reasoning was correct.

      Or another way: reasoning is a socially constructed concept, developed by humans. Humans therefore have defined reasoning, and must therefore know how to reason.

      Or a third way: I experience reasoning, you experience reasoning. I am currently reasoning. You are currently reasoning. I am human, as are you. Therefore humans reason.