Comment by jquery
2 hours ago
If you read between the lines, that's not exactly a pause. That's just them lamenting that unsupervised training has temporarily hit a limit, and using that as yet another reason the industry needs to be regulated so that OpenAI, Claude, and MechaHitler are the only approved AIs.
Oh my god, we have these "it's regulatory capture!" populists even on HN now? I thought we were better than this.
They want coordinated action to prevent a multipolar trap and potential catastrophe by future superintelligent AI (which may emerge quite rapidly if RSI ever hits an inflection point). It's not rocket science. The rules would apply to everyone. The rules they're advocating for are not "only OpenAI and Anthropic can make AI".
Give me a break. It's possible to actually think Eliezer Yudkowsky is a complete lunatic and his Roko's Basilisk essay should've been the end of him as a "thought leader" in the AI space. He has close ties to Altman, Musk, Thiel, and others, so as far as I'm concerned it's just an End Times cult.
You were misinformed about his response to Roko's basilisk. He never thought it was plausible. He freaked out for an unrelated meta reason, basically, and later admitted he overreacted. (Basically he did not want forum members putting attempted mindviruses out into the ether as a general principle.)
Critics of Yudkowsky have been trying to misportray that for years to make him seem delusional. You can see an example of that here: https://www.tracingwoodgrains.com/p/reliable-sources-how-wik...
(Roko himself is genuinely crazy, but largely because he became a neo-Nazi.)
If you think ASI is plausible this century, you would have no reason to not fear the end times yourself. If you don't think it's plausible it'll be created this century or perhaps ever, fair enough, but I'd bet against that.