Comment by remexre
11 days ago
If I interpret "a machine superintelligence" as "a classroom of 300IQ humans," I'm not really sure how this is true? You still have material and energy constraints, you can't think your way out of those.
11 days ago
If I interpret "a machine superintelligence" as "a classroom of 300IQ humans," I'm not really sure how this is true? You still have material and energy constraints, you can't think your way out of those.
For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale.
Bostrom's Superintelligence (2014) is a bit of a dreary read, and I didn't finish it, but it pulls no punches about the leverage that a superintelligence might have in our highly-connected world.
> For the concrete problem we're discussing, you can hack your competitors out of existence, replace all of your knowledge workers to shed costs, hyperoptimise your logistics, etc. It's not just intelligence, it's speed and scale.
For the concrete problem we're discussing, that hypothetical belongs in a Marvel movie, not reality. In the real world, you can't 'hack your competitors out of existence', and you'll be going to prison very quickly for trying this sort of thing.
I did say
> especially if you're willing to break the law / normal operating decorum
in my original post. If you have a superintelligence, you have something that can find and take advantage of every exploitation vector in parallel - technical, social, bureaucratic - and use that to destroy a company from the inside. A superintelligence that is subservient to its operator is an informational superweapon.
I agree that this sounds fanciful, but you can see what existing cyberattacks can do to organisations; it does not take that much imagination to gauge how much worse it could be when the process can be automated and scaled.
2 replies →