Comment by qsort
1 day ago
Humans not dying would be an excellent feature for the future to have. Not a hard requirement though, we'll see if there are some story points left in the sprint.
1 day ago
Humans not dying would be an excellent feature for the future to have. Not a hard requirement though, we'll see if there are some story points left in the sprint.
I've got bad news for you on the whole humans dying thing. Seems like actually literally everyone is going to die. Turns out that this has actually been happening for a really long time - possibly since day one - and we've got no idea how to fix it.
I keep reading about AI killing all humans but I have yet to see a practical scenario where that happens. Somehow everyone is scared of that but nobody can explain exactly how that would happen?
A lot of people have explained how that would happen, in excruciating detail. The reason why this wasn't a major part of the discourse is that most people (including myself) did not believe jumps in capabilities like those we have seen in the last few years were likely.
For a superintelligent AI trained with methods anywhere close to what we are doing now, killing all humans is the default outcome. I actually think researchers are downplaying the risks to make their arguments more normie-friendly.
https://www.thecompendium.ai/ai-catastrophe
https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
A book if you will: https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_...