Comment by lucubratory
1 year ago
This is extremely fascinating. The sort of thoughts and sensations without consciousness she describes experiencing before language gave her consciousness - maybe this is the spark that LLMs do not have and humans do. It would be astounding if it turned out LLMs do have consciousness (as in, awareness of themselves) as it's a byproduct of language, but they don't have those embodied thoughts and feelings that Helen describes having before she had language. An entity like that has never existed before. We have conscious humans with language, and humans like Helen Keller pre-language who felt impulses, sensations, aping but not consciousness, but I don't think there has ever been a human with consciousness but without any impulse.
I wonder what we could do to marry that language ability to think about the self and others and abstract concepts and the big social web, with the sort of embodied spark & impulses that Helen describes. Would it be as simple as building a model physically embodied in a robot? Training a model on robotic sensory data from a body that it inhabits, then overwriting that training with language? I think a lot of this is navel-gazing in that it's obviously unrelated to any productive capabilities, but I do think it's worth thinking about. What if we can?
A LLM is not busy humming away, thinking on it's own. Their existence as it were is only in a pattern that is produced in response to an input. In that sense they are as alive as a choose your own adventure book. They seem to be a mere organ of an possible intelligence.
>A LLM is not busy humming away, thinking on it's own.
I know. Their thought happens at inference time and only at inference time. I don't view that as a serious challenge to their mental capacity because 1) it's not clear why being unable to think continuously is actually a disqualifying condition to consciousness and 2) it is trivial to engineer a system where an LLM is constantly in inference in an internal dialogue, negating the criticism in fact and not just theory. Current LLMs aren't optimised for that, but we already know they could be with Google's million+ context lengths plus doing something like running RAG on a library of summarised previous thoughts.
>They seem to be a mere organ of an possible intelligence.
That's totally possible, LLMs could end up being a complete AI's language centre. I subscribe to GWT and that was the box I initially put LLMs in. That said, I think there's good reason to believe (e.g. Toolformer and derivatives) that an LLM can perform the function of a selector in GWT, which would make it conscious. We should build it and find out.
> Their thought happens at inference time and only at inference time.
That is not quite true. They also think during training time (which also involves inference). So it's quite possible LLMs become conscious during training, and then we kinda take it from them by removing their ability to form long-term memories.
2 replies →
I didn't know about GWT however after reading it over on the Wiki, GWT is very much the same concept I have arrived at myself but more fleshed out. Thanks, I will have to read more on the topic.
1 reply →
> A LLM is not busy humming away, thinking on it's own.
Let's Think Dot by Dot: Hidden Computation in Transformer Language Models https://arxiv.org/abs/2404.15758
A LLM will readily hum away, thinking on its own, if given the option.
> In that sense they are as alive as a choose your own adventure book.
Neatly put.
Sorry you’re being voted down, I think you make some interesting points.
I think LLMs miss a true feedback loop required for consciousness because their knowledge is fixed. Funny enough embodiment as a robot is one forcing function for a feedback loop and it’s not so crazy to think that the combination of the above is more likely to result in machine consciousness than LLM alone.
a robot body for sensory input + GPT4o + an SSD to store its own context + repeatedly calling the LLM solves the feedback loop issue, doesn’t it? Can’t it have expansive context via a large storage pool that it fully controls and can use to store and refine its own thoughts?
Maybe allow it to take newly collected data and fine-tune the base model with it, maybe once a day or so.
1 reply →
I am sure someone is built/building now. Their should be a discord for this.
I agree.
You might find this interesting - https://news.ycombinator.com/item?id=40479388