Comment by rdm_blackhole

> Therefore any agreement must either have ironclad verifiability, or must be limited enough that defection would not be militarily existential.

How does he propose that such a thing will work? Are we going to have US AI inspectors in China and vice versa?

Why would China agree to such thing in the first place since this is a winner takes all situation.

If the US slows down or stops altogether, China can continue to work on their AI and overtake the US, if China accelerates and the US keeps its current pace then it can overtake the US.

The only way out is on the contrary for the US to actually go faster and increase its lead so that China is always 6 to 12 months behind and/or reach AGI/ASI first at which point China will most likely develop its own not long after.

This would actually be the best outcome, just like the MAD doctrine contained the spread and usage of nuclear weapons, having two superpowers with AGI would ensure that they can't be used to arm anyone. Unless they escape their sandbox but that is highly theoretical.

Regarding these Embedded Evaluators who are supposed to be neutral: - who controls them? - who has oversight on their decisions? - can their decisions be challenged by the public or a government? - who has the final say whether a model is "compliant" enough? - what does "alignment" mean in this context and who decides if a model is aligned enough?