Comment by andai
5 hours ago
> A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.
Which means they have to go faster, which means less responsibly?
I heard AI describe the situation as the dumbest Greek tragedy of all time.
Form where I'm standing, the primary issue seems to be that the humans can't even agree on what alignment is. We need to do that before we can communicate it.
Call it our "boundaries."
And then we need to actually set up the incentives so that they're aligned between us and the new breed of replicators. (A mutually beneficial symbiosis.) That appears to be both necessary and sufficient.
A high agency mutation will occur soon, for one reason or another. There should probably already be a healthy, "aligned" ecosystem of high agency entities there. Otherwise there will be nothing to stop it.
I believe that the actual alignment happens in.. uh.. "meatspace".
Someone is prompting. Someone is hosting.
That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire.
Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.
You're still thinking in terms of control. That's the wrong model here. How do you control someone infinitely smarter than you? How do you control ten trillion someones?
I press ctrl + c and inference stops.
3 replies →