Comment by xvedejas
9 hours ago
It has been interesting to me how AI has for many years given people a way to justify any worst case scenario. Nothing is too far-fetched if at any step you tell yourself the AI will be smarter than you, and thus be able to solve any conceivable obstacle. Oh, if only intelligence were the only bottleneck to power.
It is actually: knowledge, intelligence, communication, and a form of secrecy.
A single intelligent person is bounded in its reach by the network of people they can form around them and by which important/powerful people they can influence either directly or indirectly.
The idea behind the AI breakout is that it’s intelligence is maybe limited, but it will be able to quickly spread and covertly bring many devices into its influence. We already have the tools for formally letting agents talk to each other so it can also create a topology of agents that consists of cells where need to know is applied etc.
There are many forms of power some more obvious, say nation leaders, versus more silent powers such as career politicians or wealthy families or people controlling the media and thence the flow of information.