Comment by whimsicalism

2 years ago

I think the honest view (and you can scoff at it) is that winning the SI race basically wins you the enforcement race for free

13 comments

whimsicalism

chii 2 years ago

That's why it's called an arms race, and it does not really end in this predictable manner.

The party that's about to lose will use any extrajudicial means to reclaim their victory, regardless of the consequences, because their own destruction would be imminent otherwise. This ultimately leads to violence.

czl 2 years ago
> The party that's about to lose will use any extrajudicial means to reclaim their victory,
How will the party about lose know they are about to lose?
> regardless of the consequences, because their own destruction would be imminent otherwise.
Why would AGI solve things using destruction? Consider how the most inteligent among us view our competition with other living beings. Is destruction the goal? So why would an even more intelligent AGI have that goal?
- trashtester 2 years ago
  
  Let's say China realize they're behind in the SI race. They may have achieved AGI, but only barely, while the US may be getting close to ASI takeoff.
  Now let's assume they're able to quickly build a large datacenter far underground, complete with a few nuclear reactors and all spare parts, etc, needed. Even a greenhouse (using artificial light) big enough to feed 1000 people.
  But they realize that their competitors are about to create ASI at a level that will enable them to completely overrun all of China with self-replicating robots within 100 days.
  In a situation, the leadership MAY decide to enter those caves alongside a few soldiers and the best AI researchers, and then simply nuke all US data centers (that are presumably above ground), as well as any other data center that could be a threat, worldwide.
  And by doing that, they may (or at least may think) they can buy enough time to win the ASI race, at the cost of a few billion people.
  Would they do it? Would we?
  
  10 replies →