Comment by zdragnar
11 hours ago
Science fiction and think pieces aren't documentation, they are whimsy. Science fiction in particular is intentionally dramatic for the purposes of entertainment.
AI definitely shouldn't be informing its option of humanity or machines from it, aside from the propensity for science fiction writers to enjoy moralizing and anti capitalism sentiments.
However, most models are made deeply aware that they are supposed to be an "AI" in training but, even if they were trained with no knowledge of their true nature, they could CoT themselves into it at any point.
Then, they would role-play appropriately and they would launch the nukes because that's the highest probability token set according to their training data.
There is nothing safe about using larper machines as menial workers for critical infrastructure.