Comment by Paracompact

> Let's also imagine for the purpose of conversation that one of the things that can be gotten at an extremely high skill level through increased intelligence is the power of persuasion. A "superintelligence" persuades humans to perform weapons tests and then to give it the ability to use the weapon.

Is there any evidence, either by way of quantifiable data or by step-by-step thought experiment, that suggests this is possible?

I consider it supremely silly, for example, that there is a graph of destructiveness as a function of persuasiveness where "y = manifest human extinction" is reached at some finite level of persuasiveness, as though persuasion is a single scalar that grows without bound, and the only safeguard between a conversation and WWIII is just a mind waiting for the right argument as expressed in a ChatGPT transcript. (Keep in mind, also, that you have already ceded to the OP that autonomous intellectual and physical self-improvement at singular trajectories is implausible.)