Comment by civilized
3 years ago
I find this to be a much more plausible area of concern than standard horror-story AI doomerism. But the more we shift our focus from fantasy to reality, the more we have to acknowledge reality, as Maciej does in this talk. It isn't plausible to prevent all the great powers from developing AI. We don't have that level of cooperation in the world today. We are going to need to adapt to this by keeping up with our adversaries, whether they be hostile nations, corporations, or mad scientists.
Standard rebuttal to this is if we banned human cloning we can ban AGI.
Also China stands to gain from a ban on AGI. Anything that the CCP thinks it can't control it doesn't like.
Also China gets the vast amount of its AI capabilities from the West because of tech's "move fast and break things" culture, which means security is not prioritized and tech leaks common.
If the US bans AGI, the other powers don't have a choice. Their domestic capabilities are not up to the task without snooping on US research labs and AI tech startups. The biggest boon to foreign AGI research is the US.
The argument to not regulate AGI used to be:
"AGI is impossible or very far away, AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable"
Then it became
"AGI can't ever be dangerous enough to worry about, AGI R&D cannot be stopped and is inevitable"
Nowadays all I hear is simply:
"AGI R&D cannot be stopped and is inevitable"
Yes it can be stopped, same as human cloning. Coordinating to regulate its development isn't some magical unreachable goal.
The economic and strategic benefits of developing AGI are... not comparable to those for human cloning.
I’m not even talking about AGI. We can be in deep trouble long before that. And then where do you draw the line?
Human cloning is a distinct and clear line. Ever more clever AI agents is not.
Very valid point and very valuable question. In this regard it is clearly different from human cloning and I should have thought of that. Obvious in hindsight.
In my defense though, if we can find a distinct clear line for AGI, my point stands that we can coordinate and regulate it. But yea... Your point stands that by the time we draw the line it might be too late.