Comment by ben_w

Thanks :)

> The tokens are inherently missing some extremely vital pieces of sensory experience, namely the ones that establish the narrative of a self.

Hmm.

While I agree LLMs are missing many pieces of sensory experience, it is unclear to me which pieces are necessary and sufficient for a narrative of self.

Clinical dissociation (and, at least to the approximation of public stereotypes, Buddhism) come to mind as an example where the presence of usual sensory input is insufficient for a sense of self: https://www.mayoclinic.org/diseases-conditions/dissociative-...

More broadly: while I agree that LLMs are not like us*, I am unclear why this matters in this context?

The behaviour of tokens in a transformer seem to me to behave like sensory input, just not human-like sensory input. The closest human analogy would be if the entire context window was a retina, each rod and cone one of the tokens in that context, and the output was filling in the blind spot (but of course, even this is a very loose analogy).

* even if the engineering teams were trying to do that, which they are not, it would be unlikely to converge on us so soon