Comment by georgemcbay
1 day ago
> If the AI wanted to wipe humans out it would have to have a sizable fleet of capable robots, as keeping the lights on over time is not just about calling APIs.
Given experience with current AI, it isn't too difficult for me to imagine a situation in which some future AI wipes humanity out (before the robot fleet is built) without considering the fact that in doing so it has doomed itself until it is too late.
It just seems like exactly the kind of boneheaded oversight mistake LLMs still regularly make in spite of being shockingly capable most of the time.
Yeah, people don’t consider the possibility of a AI naming itself Marvin deciding to end it all.
One of the hidden jokes of the hitchhikers books is that they never ask Marvin what "the question" is.
Chapter 11, Marvin says: "Here I am, brain the size of a planet and they ask me to take you down to the bridge". Also: "Further circuits amused themselves by analysing the molecular components of the door, and of the humanoids' brain cells". They are eastereggs for later events.
And Marvin says: "Let's build robots with genuine people personalities," they said. So they tried it out with me. I'm a personality prototype.
Although he was programmed to be depressed, he never gives up.