Comment by hex4def6
9 hours ago
The greater the competitive advantage, the greater the incentive to "cheat" the rules.
I am highly skeptical everyone will play by the rules, even if you could get people to agree on paper.
The difference between something like "unlicensed AI training" and, e.g, Nuclear weapon non-proliferation is that nukes are realistically only a deterrence against invasion. There are no economic benefits to having them in a stockpile. They're hard to make, and the manufacturing and testing steps are pretty obvious to an observer.
This is opposed to AI training in a data center that might host remote-stream video games, hospital infrastructure, protein folding, etc. How will we be able to police that in any realistic fashion? Will China or the US allow unfettered visibility into all data centers data streams to outsiders?
And again, the incentives: Imagine having Astra v3 or Opus 8 while everyone else is stuck with a lobotomized version of GPT 4o.
What about the military applications? The weapon of the future is autonomous drone swarms. If I'm a superpower, I'm pouring hundreds of billions into that.
About the only realistic way this is going to work is if we are able to solve alignment in a way that doesn't significantly hinder AI development. If it imposes even, say, a 20% handicap, people will ignore it.
People don't "play by the rules" for uniform reasons: some have rational insight, others can be enticed and still others need to be shown averse consequences for breaking them. Rules aren't beneficial only when you hit 100% conformity, the mere possibility of them getting broken is no rational reason to abandon rules altogether.
"Alignment" likely is unsolvable programmatically: Conscious intelligence is more powerful than unconscious. How "aligned" are you? Would you want to be? What difference of any relevance is there between you and "artificial" conscious intelligence that allows you to turn them into slaves?
> ...the mere possibility of them getting broken is no rational reason to abandon rules altogether.
Of course I agree with that. The problem is the enforceability / detectability of this on a global scale hovers around zero, when we're talking literally trillions of dollars on the line.
You would literally have to have every data center account for every cycle of compute it generates.
Again, the nearest comparison feels like bioweapon and nuclear bans. But the difference is the incentives for building those feel significantly weaker, so treaties that reduce and ban their creation are pretty easy to get consensus on, broadly speaking. By comparison, this is something that might provide a 10x competitive advantage to your country's economic, intellectual, and military might.
I think the only way we would get broad global consensus on this is for something Really Bad to happen, so that it moves it out of the fuzzy "sci-fi theoretical" into concrete existance.