Comment by coldtea

>By "hostile ASI," are you speaking in terms of an AI that decides to go rogue independent of human instructions? Or of AI being used by bad actors? The latter is a more defensible concern. The former is a scifi premise that had more credibility before the current era than since, IMO.

You seem to be moving the goalposts a little, but I don't see why "rogue AI" is less credible after the LLMs. It's obviously more. And we're just at the start of constantly running agent style harnesses.

>To the extent that you mean to say the latter is also part of it, it's undercut by the defense being AI-augmented

And yet, in many cases, those on the offense have a big advantage that any defense lacks. Especially if there's no concern for one's own life, which whether it's an AI or a human AI user removed from the field via his use of AI to delegate doesn't need to have (or they're just mad/fanatics and don't have in general).

>Anything that could cause actual existential risk would involve more than disruption, but long-term orchestration of systems that are air-gapped, time-gapped, and meat-gapped.

As if Stuxnet didn't exist?