Comment by machomaster

Yes, the EAs and longtermists on the board probably disliked the commercial focus of “Open”AI or its rapid scaling; I’m not blaming them for Laundry Buddy. But make no mistake: the nonprofit board had even less interest in opening up their research or sharing their code. They and the “superalignment” team believe AGI can end the world and needs to be in safe hands (i.e. their own hands). The EA movement which they are embedded in is one of the leading forces advocating shutting down open AI development through regulation. The board has strong EA and doomerist ties. Within OpenAI, Ilya Sutskever is on record as saying that the world will realize open-sourcing weights is foolish by 2025, and the Atlantic reported he literally burned an effigy of “unaligned” AI at the company retreat. Helen Toner is similarly involved in “AI governance” and not coincidentally took funding early in her career from OpenPhil, one of the key EA slush funds. The new CEO, their appointee, is quite literally a character written into Eliezer Yudkowsky’s rationalist fanfic in a cameo, and believes the probability of AGI killing humanity is 50%.

These people have absolutely no interest in decentralization and accountability, and were more than happy to let OpenAI accumulate power to “protect humanity”—until they pulled the plug for reasons they still refuse to disclose. Let me be clear: this is unacceptable. Not coincidentally, their so-called utilitarianism and altruism merely justifies accumulating all power in their hands, and taking any action (like the backstabbing we saw last week) to make it happen. For all MSFT’s faults, they play by traditional and predictable corporate rules of greed and can be reasoned with. The safety faction are true believers and implacably opposed to openness anywhere, and moreover happily gave the veneer of altruism to the regulatory capture of the commercial faction anyway before they realized they couldn’t control it. I know which one I’d pick.

I'm an accelerationist. All the safety shit is stupid to me. I'm cheering because doomers are annoying. Just boring old schadenfreude