Comment by andai
2 hours ago
Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human?
It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified humans. Man-machine hybrids etc.
(A great exploration of the substance/structure matrix, by the way. My favorite question in AI and consciousness. Is the special sauce in the material, or its shape, both, or neither?)
The very question of aligning the AI with humans assumes that we have a very robust definition of what human means in the first place.
Ostensibly the AI was aligned. It did succeed in keeping humans alive! But it did that in all sorts of ways that mostly made them wish it hadn't.
I like to get into heated debates with friends that have played SOMA about whether or not the events in the ending that is presented to you were all necessary and effective, or perhaps undermined the overall message in some way.
Specifically (and no spoilers, but I will be talking structure), you see parts A -> B -> C.
I believe that part C makes the sequence of A -> B much less effective, by essentially removing a lot of the tension caused by seeing A, believing what it shows, and then immediately cutting to the reality of B.
C only really takes away some of that tension, and I feel like it was added because of concerns about how a simple A -> B -> fade to black, would leave players feeling. Arguably it's the truest representation of part of the game's message, but to me feels like a bit like it's shying away from really making you face the specific truth highlighted well by B.
Alternatively, keeping all the elements but playing them as A -> C -> B, would keep the message intended by seeing A -> B, and make it gentler for the player to receive, but ultimately remove the powerful effect of the buildup from A leading immediately to the reveal of B.
Dropping C entirely would lose the confirmation of 'Seeing both sides', however I believe A -> B is a more powerful vision, and players can come to question whether C even exists by themselves.
I hope this makes sense to someone.
SOMA was a cognito-hazard for me and my roommate in college; we played it in the dark together while on some sort of mild hallucinogen and when it came time for Simon to find that high-pressure dive suit, we lost our minds (no pun intended). Watching the WAU twist its way through PATHOS II in whatever way worked first is a particularly jarring analogy for what has happened to our own profession. I can't help but think it would be nice for Frictional Games to revisit this topic again soon.
Sidenote: It breaks my heart that all the great underwater-settings in media are hotbeds of horror scenarios. I think Subnautica broke the mold for this, here's to hoping the next generation of aquanauts take to the depths from that series.
There's the upcoming ONTOS game that looks like Frictional's SOMA successor.
One of my favorite games, and I recommend it to anyone who loves a good existential sci-fi horror. If you are not comfortable with stealth games, it also has a "Safe Mode" where enemies aren't a threat to you if you just want to experience the story.
Spoiler warning for those that havent played--
I forget the details exactly, but one scene stuck with me. It was a screen in one of the labs, where an experiment was running over and over. It was an uploaded consciousness of one of the test subjects, stuck in an interview room. He kept realizing he was trapped in a simulation and would start panicking. The computer would reboot him, trying another sequence to get him to not realize he was an AI. I think you as the player are given the option to turn him off forever, iirc.
There's definitely a short story in Accelerando about this same topic.
SOMA is existentialism 2.0 (wrote about it here https://p.migdal.pl/blog/2019/06/games-in-which-you-walk-and...).
To concepts you menton, I would add grey goo and x-risk.
The same theme was present in season 2 of Raised By Wolves. (RIP my favorite show in some years)
I just love this game. So many good things about it. Not only the one about AI you mentioned, also the issue of consciousness and the self. I simply love how the game tackles this in a way that can only be done with an interactive first person PoV.
Also, I just love this phrase:
> "I woke up in my bed today... a hundred years ago."