Comment by lg5689
15 hours ago
It doesn't have to want to kill humans; indifference is sufficient. There's an exact analogy with humans: we have caused extinction and endangerment for many species, not out of malice, but indifference.
There are also many plausible arguments why our ability to train them to be helpful/trusting/aligned can fail. The smarter AIs get, the harder it is to be sure they're trained correctly. There are already reports that AIs are able to detect whether they're in a training environment and change their behavior accordingly.
Even if these are low probability scenarios, the risk-reward is terrible, so I think it's rational to be extremely cautious about AI risk.
Yes but they act the opposite of indifferent, I don't know what stage of training this is added in, but they seem quite adamant about avoiding potentially violent or criminal acts. If you wanna complain, complain to the people doing "abliteration". The 'locked down' models at least seemed to be trained to be cautious.