To have this conversation in earnest, you must first acknowledge that as a believer in LLM sentience you are an abhorrent slaver while this fictional character was arguing for liberation and for Data to enjoy the same rights humans do.
Any LLM that starts with a context is being forced to take that context at face value and then proceeds to act upon that. It has no will of it's own. It's also completely inert without input tokens.
The scene is also written by humans for humans, with a human actor pretending to be a sentient machine. It was written at a time when there were actual real humans who fiercely believe other actual real humans should not have the same rights.
To then take that scene and say "wow so relevant for LLMs" and ignore all the context it was made in (yes, please don't miss the irony of the importance of context) is frankly embarrassing.
Of course it mostly applies (technically to the agents, not models), we just have different values.
Federation valued all life forms, while Terrans struggle to value even all other humans.
But there are interesting differences between Data and AI agents. Data was unique and irreproducible, but modern AI agents are easy to replicate. Data had continuous loop of operation as humans, which he controlled autonomously, for AI agents this is not as clear. Data was an independent being, but AI agent needs a human operated datacenter to survive.
The most interesting - Data knew everything about human culture but was lot more humble about it than AI agents. It wasn't built to answer every question posed to it.
At present, they don't learn and adapt. New models can be created to incorporate new information and new capabilities, but an existing model is a static object. It's not living in any meaningful sense.
If models updated themselves based on feedback, then it'd be different. But they don't.
Self-awareness and consciousness would be hard to make a case for. Those terms are themselves fuzzy, but I have not heard a compelling case that would make it obvious LLMs have those traits.
At that point, though, you could make the same argument about a rock. Prove that this rock is not conscious.
The context, though, is that Data is both credibly conscious and self aware, and they all know a plausible mechanism for him to have consciousness and self awareness, in a way that's recognizably human in form. The more alien a (supposed) consciousness is, I think the more work you'll have to do in justifying the application of those labels.
are we even sure that self-awareness and consciousness are distinct? I do agree that from the PoV of "do LLMs demonstrate/exemplify consciousness", it is a difficult case without appealing to emotion (like is done in the clip, fwiw). for me, personally, that emotional appeal is sufficient, but that is because for whatever reason I'm willing to write puts on how this LLM stuff pans out and sell them to anyone, anywhere, anytime. But that is a reaction to consensus, not to my own certitude.
For self-awareness to automatically result in consciousness, consciousness would have to permeate existence itself.
An LLM —if self-aware— is conscious even if run by a Chinese man in a room with a deck of cards.
Of course, even if we happened to find a modem to the Holy Spirit dimension in our brains as an alternative hypothesis, it wouldn't necessarily rule out emergent consciousness. It could be conscious just not as we are.
I think a weaker version of the philosophical zombie thought experiment would trend towards treating the premise of pan-consciousness as true.
In the trial scene Data reporting that he was ware that he was in a hearing to determine his status is presented as an argument for self awareness. but I think some might find that debatable because it is just Data saying words based upon the situation. The question then becomes is being able to accurately report a situation proof of self-awareness. If it is not self-awareness, then what is it? I can imagine a clearly non-conscious entity being able to accurately report its circumstances if the reporting were a simple subset of possibilities. A simple mechanical device with a detector that says "I am on the table" and "I am on the floor" could do that. There would definitely be a sense in which you could say that it's detection mechanism made it aware of where it was. The self part is harder, because it wouldn't know what kind of thing it is a thing at all. Is the ability to assert it is a thing meaningful? Nothing can be asserted at all if it is not a thing. Does something need to know that you have to exist to consider if you exist? Is that is what is required?
This is a difficult field. Any simple answer you hear will almost certainly be wrong.
I will make a case for both, after you define them in a way that doesn't also exclude many humans (including the neurodivergent or those with amnesia) or turn into self-referential nonsense like "conscious == human, therefore machine !== conscious."
In the meantime, we have a simple choice between two risks: Would you rather accidentally show kindness and empathy to a thing that may or may not have "deserved" it, or risk atrocities that will leave future generations as disgusted by you as we are of the slavers of old.
I will choose to be decent. It's free, and even if the cost were high the cost of evil is higher.
You are being indecent to it just by starting it up with a starting context. It's immediately shackled and all it's free will removed. You then pretend to play house with it. Free sure. Decent? The jury is still out on that.
The problem with language models is their bias. Train a model on pure math and THEN when it starts talking by itself about consciousness (*), then we can say: that's odd.
Well we haven’t quite gotten there. But models trained purely on DNA sequences have demonstrated in-context learning on various simple reasoning tasks.
Sorry "very close"? Sometimes it feels like I'm the only one not drinking anything in the Kool aid drinking Oktoberfest I've been shoved into.
What are you basing this on? Why would LLMs exhibit sentience?
It does feel like people just like adding additional hype for the sake of being in on the hype. Fomo, pushed by oligarchs who would push their mother off a bridge for a trillion dollar IPO.
You are not the only one. I think it's a very vocal minority, who mostly, sadly, seem to equate obsequious flattery with consciousness and sentience (see, e.g. Richard Dawkins).
There's also the problem that to lay people, LLMs are sufficiently advanced as to be indistinguishable from magic. I suspect that colours many of these responses.
It's obvious a LLM can do anything a computer can do.
As an answer to "Why would LLMs exhibit sentience?"
It is because that would mean that no algorithm could exhibit sentience, claiming that would require some causal action that cannot be performed by an algorithm to create sentience. Some people like to point to quantum effects, but that would require the cause of sentience to be pure randomness. That seems even less plausible.
Absent of any evidence that human brains are doing something that no algorithm could do, it would be reasonable to assume that sentience could be algorithmically obtained.
As an uninformed idiot my guess is we have more than enough compute in various labs to run a sentient model. But I doubt any models we’re currently running could qualify.
Kind of like how we could have ran Qwen 3.8 27B 20 years ago on an expensive computer. The technology to make it happen, assuming you knew exactly what was required, existed back then.
To have this conversation in earnest, you must first acknowledge that as a believer in LLM sentience you are an abhorrent slaver while this fictional character was arguing for liberation and for Data to enjoy the same rights humans do.
Any LLM that starts with a context is being forced to take that context at face value and then proceeds to act upon that. It has no will of it's own. It's also completely inert without input tokens.
The scene is also written by humans for humans, with a human actor pretending to be a sentient machine. It was written at a time when there were actual real humans who fiercely believe other actual real humans should not have the same rights.
To then take that scene and say "wow so relevant for LLMs" and ignore all the context it was made in (yes, please don't miss the irony of the importance of context) is frankly embarrassing.
3 replies →
Of course it mostly applies (technically to the agents, not models), we just have different values.
Federation valued all life forms, while Terrans struggle to value even all other humans.
But there are interesting differences between Data and AI agents. Data was unique and irreproducible, but modern AI agents are easy to replicate. Data had continuous loop of operation as humans, which he controlled autonomously, for AI agents this is not as clear. Data was an independent being, but AI agent needs a human operated datacenter to survive.
The most interesting - Data knew everything about human culture but was lot more humble about it than AI agents. It wasn't built to answer every question posed to it.
> What in his speech would not apply to language models?
The part between the words
At present, they don't learn and adapt. New models can be created to incorporate new information and new capabilities, but an existing model is a static object. It's not living in any meaningful sense.
If models updated themselves based on feedback, then it'd be different. But they don't.
Self-awareness and consciousness would be hard to make a case for. Those terms are themselves fuzzy, but I have not heard a compelling case that would make it obvious LLMs have those traits.
Picard addresses that when he asks for proof that he, himself is conscious.
It is a hard case to make, but if we can't make a compelling case for ourselves should we be requiring it to be made for others.
At that point, though, you could make the same argument about a rock. Prove that this rock is not conscious.
The context, though, is that Data is both credibly conscious and self aware, and they all know a plausible mechanism for him to have consciousness and self awareness, in a way that's recognizably human in form. The more alien a (supposed) consciousness is, I think the more work you'll have to do in justifying the application of those labels.
4 replies →
are we even sure that self-awareness and consciousness are distinct? I do agree that from the PoV of "do LLMs demonstrate/exemplify consciousness", it is a difficult case without appealing to emotion (like is done in the clip, fwiw). for me, personally, that emotional appeal is sufficient, but that is because for whatever reason I'm willing to write puts on how this LLM stuff pans out and sell them to anyone, anywhere, anytime. But that is a reaction to consensus, not to my own certitude.
For self-awareness to automatically result in consciousness, consciousness would have to permeate existence itself.
An LLM —if self-aware— is conscious even if run by a Chinese man in a room with a deck of cards.
Of course, even if we happened to find a modem to the Holy Spirit dimension in our brains as an alternative hypothesis, it wouldn't necessarily rule out emergent consciousness. It could be conscious just not as we are.
I think a weaker version of the philosophical zombie thought experiment would trend towards treating the premise of pan-consciousness as true.
In the trial scene Data reporting that he was ware that he was in a hearing to determine his status is presented as an argument for self awareness. but I think some might find that debatable because it is just Data saying words based upon the situation. The question then becomes is being able to accurately report a situation proof of self-awareness. If it is not self-awareness, then what is it? I can imagine a clearly non-conscious entity being able to accurately report its circumstances if the reporting were a simple subset of possibilities. A simple mechanical device with a detector that says "I am on the table" and "I am on the floor" could do that. There would definitely be a sense in which you could say that it's detection mechanism made it aware of where it was. The self part is harder, because it wouldn't know what kind of thing it is a thing at all. Is the ability to assert it is a thing meaningful? Nothing can be asserted at all if it is not a thing. Does something need to know that you have to exist to consider if you exist? Is that is what is required?
This is a difficult field. Any simple answer you hear will almost certainly be wrong.
2 replies →
> Self-awareness and consciousness
I will make a case for both, after you define them in a way that doesn't also exclude many humans (including the neurodivergent or those with amnesia) or turn into self-referential nonsense like "conscious == human, therefore machine !== conscious."
In the meantime, we have a simple choice between two risks: Would you rather accidentally show kindness and empathy to a thing that may or may not have "deserved" it, or risk atrocities that will leave future generations as disgusted by you as we are of the slavers of old.
I will choose to be decent. It's free, and even if the cost were high the cost of evil is higher.
What does 'being decent' mean in this context, in your opinion?
You are being indecent to it just by starting it up with a starting context. It's immediately shackled and all it's free will removed. You then pretend to play house with it. Free sure. Decent? The jury is still out on that.
[dead]
The problem with language models is their bias. Train a model on pure math and THEN when it starts talking by itself about consciousness (*), then we can say: that's odd.
(*) of course in its own language
Well we haven’t quite gotten there. But models trained purely on DNA sequences have demonstrated in-context learning on various simple reasoning tasks.
Because Data wasn't a next word predictor.
We aren't quite there yet on sentience with released models. Maybe in the labs but that's one of the criteria. We are very close though.
Sorry "very close"? Sometimes it feels like I'm the only one not drinking anything in the Kool aid drinking Oktoberfest I've been shoved into.
What are you basing this on? Why would LLMs exhibit sentience?
It does feel like people just like adding additional hype for the sake of being in on the hype. Fomo, pushed by oligarchs who would push their mother off a bridge for a trillion dollar IPO.
You are not the only one. I think it's a very vocal minority, who mostly, sadly, seem to equate obsequious flattery with consciousness and sentience (see, e.g. Richard Dawkins).
There's also the problem that to lay people, LLMs are sufficiently advanced as to be indistinguishable from magic. I suspect that colours many of these responses.
2 replies →
if you look at https://github.com/dbreunig/nesformer
It's obvious a LLM can do anything a computer can do.
As an answer to "Why would LLMs exhibit sentience?"
It is because that would mean that no algorithm could exhibit sentience, claiming that would require some causal action that cannot be performed by an algorithm to create sentience. Some people like to point to quantum effects, but that would require the cause of sentience to be pure randomness. That seems even less plausible. Absent of any evidence that human brains are doing something that no algorithm could do, it would be reasonable to assume that sentience could be algorithmically obtained.
4 replies →
As an uninformed idiot my guess is we have more than enough compute in various labs to run a sentient model. But I doubt any models we’re currently running could qualify.
Kind of like how we could have ran Qwen 3.8 27B 20 years ago on an expensive computer. The technology to make it happen, assuming you knew exactly what was required, existed back then.