Comment by HAL3000

Why would you allow a handful of private companies to control all the resources? Society allows this today because it's the most efficient economic model. But once we reach recursive self improvement and true superintelligence, no human will be able to compete with machines at any task, including managing resources and running companies.

So why would you allow that kind of power to be concentrated in the hands of a few companies? If the state can use this technology, which keeps improving on its own, to manage an entire nation's resources better than any human or group of humans, why would we still need private companies to do it? Especially when the ones with the most resources/capital would be able to just win with everyone else and accumulate all the existing resources.

But the bigger issue lies elsewhere. When you replace human labor, you also replace soldiers with robots. Until now, a leader with autocratic tendencies couldn't easily order the military to turn against domestic "terrorists" who are actually their political opponents. But once armies are replaced by robots, that could all change, and we could see democracy go into decline.