Comment by Peteragain

2 months ago

The article is actually about the way we humans are extremely charitable when it comes to ascribing a ToM (theory of mind) and goes on to the Gym model of value. Nice. The comments drop back into the debate I originally saw Hinton describe on The Newyorker: do LLMs construct models (of the world) - that is do they think the way we think we think - or are they "glorified auto complete". I am going for the GAF view. But glorified auto complete is far more useful than the name suggests.

10 comments

Peteragain

Reply
ptidhomme  2 months ago

Those billion parameters, they are a model of the world. Autocomplete is such a shortsighted understanding of LLMs.

  • Peteragain  2 months ago

    Sorry for the late response. Yes that is Hinton's argument, and the claim made by the believers. On the other hand, if the GAC explanation is correct, an explanation might be that what we humans write down (that is, the training corpus) is a model of the world, and LLMs reconstruct (descriptions of) human understanding.

    • ptidhomme  2 months ago

      Now of course, the only input LLMs have is human text (for text only LLMs anyway). So their model is entirely dependent on how we see the world. I wouldn't restrict LLMs to description of human understanding. They can articulate concepts in a rather sensible way, that wouldn't exist as is in the training corpus. Which exactly means that they have a model, however limited or imperfect.

      1 reply →