Comment by davesque

Yes, it would be correct not to pursue a technology that has that chance of wiping out the world. But where are the people who are suggesting these probabilities getting their numbers? I personally can't imagine where, and I'm an engineer with a specific technical interest in LLMs. And I haven't heard one clear description of the methodologies used to calculate these chances.

The much more likely explanation to me is that people are just spitballing, either because they've watched too much sci-fi, or they have some weird counterintuitive agenda (e.g. Anthropic and OpenAI trying to position themselves as the amazing, trustworthy keepers of this dangerous technology before their IPOs).