Comment by Waterluvian
1 day ago
I think about Star Trek: TNG’s “Measure of a Man” a lot lately. We can be so confident to decide what is and isn’t alive from vibes alone.
The conclusion I’m currently at is that I don’t know and probably can’t ever know. Maybe you’re all philosophical zombies. Maybe I am one too!
But at some point we will get close enough that it hopefully becomes obvious that we must tread carefully.
The entire episode is incredibly relevant. But here’s a snippet: https://youtu.be/EFNbTnFHruI?si=pW9QtxCsqMtHkVYG
I think about this from the other end. It cannot be considered a conscious being. There just isn't a world in which we should start to think of a machine using ethics we reserve for humans.
AI is essentially infinitely reproducible at zero cost, and won't suffer from decay etc. There's not scarcity to preserve.
So, I'd turn off an AI in a moment to save property or real possessions or money. I'd sacrifice property and money to save animals. I would never choose to save an animal over a person. I'd probably not choose to save a person over a child.
I don't see any inversion of any of those priorities that makes any sense.
It is interesting to think about what would cause me to consider these priorities incorrect, but a majority consensus about a program being sentient isn't it.
What is the connection between scarcity and consciousness?
If we turn off people we can't usually just fire them back up again, and swapping the models between harnesses is also tricky.
2 replies →
A loose one. In nature consciousness is very scarce and therefore special. The more human the consciousness the more we probably naturally react to it. And the closer to us it is, even more so.
5 replies →
Zero cost? How did you reached that estimate?
Training AI is often more costly than supporting human from birth till death. Just sustaining frontier LLM model on necessary hardware costs more than living in first world countries.
> AI is essentially infinitely reproducible at zero cost, and won't suffer from decay etc.
I hope the same becomes true of people, and that doesn't mean people stop being sapient.
Infinitely reproducible at 0 cost means people will be treated as disposable by the rich.
1 reply →
Consider that it is very costly to train a model. It is only cheap to copy because the substrate we instantiate it onto is a substrate we have designed to be fully readable.
I think you should reconsider this viewpoint. Suppose that we really can create silicon-based consciousness, in that case your view would result in a huge amount of suffering.
Take some other basis for dismissing digital consciousness, this one is too dangerous.
How much data center energy and capex justifies killing a human to save?
I argue zero - placing AI below the value of humans no matter the energy input.
The _only_ reason an AI might be worth saving is if it, say, has a cure for all diseases, but then we're not saving it due to its intrinsic worth, we're saving it because we can save many humans. I _would_ consider the trolly problem a legitimate thing in ethics, but not if an AI were tied up on the tracks no matter how expensive it is. It's a thing. It gets run over to save any human.
Roko's basilisk can suck a fart out of my butt, but no battery blood would ever be worth the life of a single human.
Suffering comes from desire.
What would a silicon-based consciousness desire to cause suffering?
Are you a vegan?
If not, then your comment's claim is false.
Anyway, the deeper solution is to acknowledge that all life is sacred, and infinities cannot be compared, and some decisions are impossible to make, and some tragedies cannot be averted, and "prioritization" is a distraction that forces choices when choices are not strictly necessary.
We can choose to draw the lines wherever we want. I firmly draw the line such that AI is never equal to a human.
2 replies →
Presumably they meant that they'd sacrifice some material value for some animals, not that every animal on Earth has infinitely more value than inanimate goods.
> infinities cannot be compared
That's either a mathematically illiterate assumption or a very strange philosophical hill to die on.
> some tragedies cannot be averted
Sure. The question is what to do about the ones that can be averted.
> some decisions are impossible to make
> and "prioritization" is a distraction that forces choices when choices are not strictly necessary.
Again, the question is what choices to make when you can (arguably must) make them. Saying they're impossible is just refusing to take responsibility. You either do something, or you don't.
I'm not. You're right. Some animals are property to be traded and used to support human life. I should separate companion animals and the rest.
If you believe sanctity of all life is a solution, then I'm curious what you believe the problem is that such a belief solves.
I bet it's circularly defined as justifying the preservation of sacred nonhuman life? I'm not trying to be provocative just curious.
I'm a big fan of Star Trek but I recently rewatched this one in the context of recent AI developments and it's not as good as I remember it.
They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent. The judge eventually rules in Data's favor but doesn't give much of a justification IMO.
It's still a good episode, but it doesn't add much to the conversation on consciousness. It's a hugely complicated topic which people have devoted their entire careers to.
The fact that they sent Data to starfleet academy, gave him a commission in the starfleet, let him attain the rank of Lieutenant Commander, and then decide that actually he's a machine that can be dismantled seems like quite a turn.
Does the ship's computer have a commission?
It was a good episode but it had some elements of Star Trek tropes in it, like the evil admirals and Picard can talk his way out of anything.
>The fact that they sent Data to starfleet academy, gave him a commission in the starfleet, let him attain the rank of Lieutenant Commander, and then decide that actually he's a machine that can be dismantled seems like quite a turn.
Data is basically an Isaac Asimov android (down to the positronic brain) and Measure of a Man is an Asimov-type story whose tropes don't entirely fit within in the Trek universe.
It makes no sense within the context of the Trek universe that Data is unable to use contractions for instance - but it makes sense in the context of how a robot might have been conceived of in the 1940s.
3 replies →
I think you're misremembering or misunderstanding Picard's argument. It isn't a tangent. Here's the transcript[0].
TL;DR Picard's initial arguments are pretty weak, even admitting that Riker as opposing counsel almost had him convinced. During a recess Picard talks to Guinan where she alludes to the future subjugation of many Datas which Picard connects to slavery. Back in the courtroom Picard calls Maddox as a hostile witness and gets him to define sentience--intelligence, self-awareness, consciousness--then walks him into conceding Data meets the first two. Picard's closing boils down to, "we don't know if he meets the third--you can call Data a toaster and rule he is property--_but what if you're wrong_". The judge rules on the basis of erroring on the side of caution due to that uncertainty. It's really a great scene.
We're not there yet, obviously. No LLM brings Data's level of awareness but it's as relevant a story as ever because it isn't really about AI but othering for the purpose of subjugation.
[0] http://www.chakoteya.net/NextGen/135.htm
> Picard's closing boils down to, "we don't know if he meets the third
A little piece of in-universe lore for anyone unaware or who had forgotten: At this point in the series, Data's positronic brain is new technology no one understands. His creator is missing/presumed dead, the "positronic" basis isn't how Federation technology works, and apparently so far they hadn't done a whole lot of direct experimentation (hence why the trial is happening). So not knowing if he's conscious is a lot more reasonable a stance than with real-life LLMs where we do know roughly how they work.
Also later in the series we meet a character whose brain was copied into a positronic brain, and does imply that technology is at least capable of consciousness, whether or not it applies in Data's case.
> They barely touch on the issues of consciousness
I would argue that is a strength, rather than a weakness. Consciousness is unobservable in any entity other than the observer, and its existence in others is pure conjecture, and irreducibly so.
Making it a criteria in a decision involves either acting on fantasy, or, more likely, acting on some unstated basis and using “consciousness” as a dishonest (perhaps to oneself most of all) rationalization.
Debating AI consciousness a real modern equivalent of the cliché (but purely fictional, invented later as a form of hostile mockery grounded in large part in sectarian bigotry) medieval scholastic debates over how many angels could dance on the head of a pin.
> invented later as a form of hostile mockery grounded in large part in sectarian bigotry
When you read about the theological questions that led Christians to kill and excommunicate one another, "angels dancing on the head of a pin" is not far off. The Homoousion, Monophysitism, the Filioque controversy... It's all so arcane and poorly defined. It almost makes one wish Positivism had been invented 2000 years earlier.
The current AI debate about consciousness does remind me of that in one respect: no one can even clearly define what consciousness is.
It asks important questions. It's not so presumptuous as to try to answer them conclusively.
In fact I think the conclusion it comes to is the one that people, especially the smart ones, so easily miss: we don't know the answer. It might be that we can't know the answer. But ignorance is not a defense.
> it's not as good as I remember it. They barely touch on the issues of consciousness, Picard basically says "What if Data is conscious?" and then goes off on a tangent.
ST:TNG writing is generally like this. The show required considerable suspension of disbelief and a willingness to accept the kayfabe that deep concepts are being presented when for the most part they're just not that deep. (But it can be very enjoyable when you make those accommodations.)
It’s a great episode of TV because Data is a main cast character who is obviously conscious. We know before the episode starts who is right and who is wrong in this particular argument. This episode is not about consciousness, it is about civil rights: resisting bigotry and power.
Note that the episode is NOT about the ship’s computer. They all know it’s not conscious, despite also being a machine that can converse and do things.
Our LLMs are like the ship’s computer.
Any time I bump into a device that acts like a human I'm going to treat it like a human.
Because treating things that act human inhumanely is not something I want to learn how to do.
My instincts are pretty different here.
- I'll try not to swear at/hit a printer: not because I see the printer as having human-like qualities of being capable but complex and unreliable, but because I want to be a person who can control his temper.
- Treating an inhuman thing as human because it can mimic us in some way is not something that I want to do.
So if a machine does become conscious, you're happy being nasty to it until it is proven conscious?
I try not to make errors like that.
12 replies →
This is in fact the danger with these human-simulating "AI"s we have now...
People get used to treating human-like, human emulating machines with either disrespect or in a command/control/master fashion, because that's the nature of the tooling.
And then potentially by extent/blurring of lines they then treat other people like machines.
Which is already a thing people do to other people.
I just fear it gets worse.
> The conclusion I’m currently at is that I don’t know and probably can’t ever know.
I think about this quote often, straight from Data's voice module in another episode:
'The most elementary and valuable statement in science, the beginning of wisdom, is, "I do not know".'
I’ve also rewatched it lately and I’m more on the side of the Starfleet scientists when I was obviously on Picard/Data’s side before.
Don't forget the other episode "Quality of Life" with these working robots who data discovers have become sentient: https://en.wikipedia.org/wiki/The_Quality_of_Life_(Star_Trek... I do miss the old TNG show.
Easily one of my top 10 favorite episodes.
The judge broached on the subject of what makes us distinct from Data (e.g. machines w/great heuristics) - the existence of a soul. Or rather, I'd like to think, in the words of CS Lewis, that we are a soul with bodies attached.
Based on how some actual humans I know speak and act, I'm less and less convinced the human brain is much more than a stochastic next-thought prediction stream.
35 years old and just as relevant today - https://web.mit.edu/people/dpolicar/writing/prose/text/think...
A silicon alien coming to earth might poke us, we would say ouch, and just determine the ouch sound is just the result of a bunch of chemistry - not really conscious or feeling pain like it can, just emulating.
1 reply →
[flagged]
How would someone know whether one has a soul or not? Is there any sort of introspection that can reveal the presence of a soul or any of its properties?
There is no soul. Just a bunch of systems nudging each other to action. What people call soul is literally the same as the concept of personality. In essence, the way all systems in your body have been calibrated to exist.
I believe that the moment an artificial inteligence is going to "receive" a soul, is the moment it is going to be made to sustain itself. Either as a larger package (some bots working to keep an AI farm running) or as an individual (a bot which is tasked with not only fulfilling human desires, but also sustaining itself)
4 replies →
Define the word “soul” first before asking this question.
[dead]
I wonder if modern Star Trek could make this episode.
Even if it could, it probably ought not makes this precise episode, because the technological, philosophical and social context is remarkably different.
A similar episode, actually informed by what we know or can forsee about AI and LLMs, and addressing our hopes and fears about what they mean would be interesting though.
Definitely not. Kurtzman Star Trek is not really Star Trek in any spiritual sense, it’s a vessel for political messaging (they’ve pretty much said as much)
> Kurtzman Star Trek is not really Star Trek in any spiritual sense, it’s a vessel for political messaging (they’ve pretty much said as much)
So was most of Roddenberry, Piller, et al., Star Trek. At its low ebb in Braga Star Trek, but even then...
AFAICT, Star Trek has ALWAYS been a vessel for political messaging.
1 reply →