Comment by ben_w

> Even this would require monumental advances in physical compute and HMI, but I wouldn’t rule this out within 20 years.

Would it require so much? Cochlear implants exist if you care about HMI, and compute is already doing "reasoning"[0] faster than humans do snap judgements. Actually winning in an adversarial environment against the best human political minds (let alone economic or military) will be harder, but I would be (pleasantly!) surprised if it takes 10 years, let alone 20.

> I could see Elon Musk having this kind of idea. But even in this case, I don’t think “nuke everybody” would be the highest-ROI strategy advised by the AI.

I hope a sensible AI would not. However, Musk's LLM called itself "MechaHitler", which suggests his training process does not avoid such things. If his in particular becomes dominantly competent, he has also spoken of wanting more Optimus robots than there are humans[1], has called it a "robot army" (and his justification for even more Tesla stock was to keep influence over said army[2]), and he has spoken of factories on the moon churning out space-based data centres by the petawatt which is enough that if placed in LEO it could have about the same effect on Earth's climate as all anthropomorphic greenhouse gasses currently in our atmosphere (and that's even if not weaponised, that's just simply by having that much stuff that close).

On the plus side, Musk's got a huge gap between what he says his AI will do and what they end up doing on those timescales, so hopefully we'll be safe from him: https://en.wikipedia.org/wiki/List_of_predictions_for_autono...

> This is probably closer to the imagined doomsday scenario, but again I find it hard to imagine that such an AI would advise nuking everybody.

I don't. First, because AI being "better" doesn't make it infallible; and second, an argument shaped like this has historic echoes with actual nukes and the cold war: if the AI is competent to take over conventionally, and everyone knows this, then every other nation with nukes will use them, therefore the AI should use a first strike. (Note: the argument does not need to be perfect or flawless in reality, it only needs to be the kind of thing which humans have found convincing, and far too many humans have argued their side would win a nuclear exchange).

> If AI takes over the world, I imagine it will be because of the production efficiencies it enables. Which is a different kind of takeover - one that would eventually have broad benefits for many people - but would likely go through a period of major instability due to labor displacement.

This is my best case scenario. Some people also object to this, they like their careers and fear what retirement will do to their sense of self-worth, but I suspect this can become an opportunity for flourishing in other ways if, and only if, it is handled correctly.

[0] something something swimming submarines, I don't care about using anthropomorphic language myself, but there's always someone ready to complain if anthropomorphic language doesn't get scare-quotes

[1] "And, and then there'll be obviously robots in industry making stuff. And so I mean, I think the ratio of humanoid robots to humans will probably be at least two to one, something like that. One to one for sure. So which means like somewhere on the order of 10 billion humanoid robots, maybe, maybe 20 or 30. And so then it's like, okay, well let's say, you know, you kind of make, let's say the build rate is, I think the build rate will be probably something ultimately like a billion a year." - https://www.elonmuskarchive.org/video/tesla-shareholder-meet...

[2] “If we build this robot army, do I have at least a strong influence over this robot army?” he told investors. “Not control, but a strong influence… I don’t feel comfortable building that robot army unless I have a strong influence.” - https://futurism.com/robots-and-machines/elon-musk-remarks-r...