Comment by mnky9800n
3 months ago
Also the narrative that we are currently on the brink of Ai explosion and this random paper shows it has been the same tired old story handed out by ai hawks for years now. Like yes, I agree with the general idea that more compute means more progress for humans and perhaps having a more responsive user interface through some kind of ai type technology would be good. But I don’t see why that will turn into Data from Star Trek. But I also think all these ai hawks kind of narcissistically over value their own being. Like blink and their lives are over in the grand scheme of things. Maybe our “awareness” of the world around us is an illusion provided by evolution because we needed it to value self preservation whereas other animals don’t. There is an inherent belief in the specialness of humans that I suppose I mistrust.
> But I don’t see why that will turn into Data from Star Trek.
"Is Data genuinely sentient or is he just a machine with this impression" was a repeated plot point in TNG.
https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Tre...
https://en.wikipedia.org/wiki/The_Offspring_(Star_Trek:_The_...
https://en.wikipedia.org/wiki/The_Ensigns_of_Command
https://en.wikipedia.org/wiki/The_Schizoid_Man_(Star_Trek:_T...
Similar with The Doctor on VOY.
Even then, what we have with LLMs is basically already at the level of the ship's main computer as it was written in TNG/DS9/VOY.
But what counts as personhood? No idea. We're as ignorant as of that as the writers of the TV show; humanity's best attempts at philosophy of mind are still making the same kinds of arguments today as we were a century ago, the only real shift is to be slightly more inclusive of other species besides Homo Sapiens, and the only real improvement over two centuries ago is to include all of Homo Sapiens.
The main computer does not make choices stochastically and always understands what people ask it. I do not think that resembles the current crop of LLMs. On voyager the ships computer is some kind of biological computing entity that they eventually give up on as a story topic but there is an episode where the bio computing gel packs get sick.
I believe data and the doctor both would be people to me. But is minuet? The woman created by the binars to distract riker so they could steal the ship? Star Trek goes back and forth as to who counts as a person or not. This is one of the points that is made in the episode where the doctor learns his hologram is cloned many times to work as miners on an asteroid and he says this is slavery. Does minuet get to say she doesn’t want to seduce riker if she doesn’t feel like it? Or is she a program that does whatever the programmer wants?
But moreover linking back to the original technology, does eqtransformer have awareness? Do we treat it as a thinking being that we enslaved to find earthquakes in data because it uses the same technology as ChatGPT? I don’t think anyone thinks we should do this but it seems like that is where people want to take it. That’s strange to me. At what point does data pipelines doing some kind of stochastic transformation and summarization of training data become an individual that presents a desire for autonomy like data or the doctor?
I think there’s lots of questions here to answer and I don’t know the answers to them.
> The main computer does not make choices stochastically and always understands what people ask it.
The mechanism is never explained, but no, it doesn't always understand correctly — and neither does Data. If hologram-Moriarty is sentient (is he?), then the capability likely exceeds what current LLMs can do, but the cause of the creation is definitely a misunderstanding.
Even the episode where that happens, the script for Dr. Pulaski leading up to Moriarty's IQ boost was exactly the same arguments used against LLMs: https://www.youtube.com/watch?v=4pYDy7vsCj8
(Common trope in that era being that computers (including Data) are too literal, so there was also: https://www.youtube.com/watch?v=HiIlJaSDPaA)
Similar with every time the crew work iteratively to create something in the holodeck. And, of course: https://www.youtube.com/watch?v=srO9D8B6dH4
> I do not think that resembles the current crop of LLMs. On voyager the ships computer is some kind of biological computing entity that they eventually give up on as a story topic but there is an episode where the bio computing gel packs get sick.
"Take the cheese to sickbay" is one of my favourite lines from that series.
> But is minuet?
I would say the character was a puppet, with the Bynars pulling the strings, because the holo-character was immediately seen as lacking personhood the moment they stopped fiddling with the computer.
Vic Fontaine was more ambiguous in that regard. Knew he was "a lightbulb", but (acted like) he wanted to remain within that reality in a way that to me felt like he was *programmed* to respond as if the sim around him was the only reality that mattered rather than having free will in that regard.
(But who has total free will? Humans are to holograms as Q is to humans, and the main cast were also written to reject "gifts" from Riker that time he briefly became a Q).
The villagers of Fair Haven were, I think, not supposed to be sentient (from the POV of the crew), but were from the POV of the writers: https://en.wikipedia.org/wiki/Fair_Haven_(Star_Trek:_Voyager... and https://en.wikipedia.org/wiki/Spirit_Folk_(Star_Trek:_Voyage...
> does eqtransformer have awareness?
There's too many different definitions for a single answer.
We don't know what part of our own brains gives us the sensation of our own existence; and even if we did, we wouldn't know if it was the only mechanism to do so.
To paraphrase your own words:
At what point does chemical pipelines doing some kind of stochastic transformation and electrochemical integration of sensory input become an individual that presents a desire for autonomy like data or the doctor?
I don't know. Like you, I'd say:
> I think there’s lots of questions here to answer and I don’t know the answers to them.
I find the last part of the paragraph offputting and I agree