Comment by derektank

I’m not sure how you came to believe there aren’t rules in life, but there absolutely are. As you point out, there are physical constraints on everything and if you find yourself in a concrete box with nothing but a DGX H100 or at the bottom of the wrong gravity well, there’s nothing you can do. Checkmate.

This goes both ways. You absolutely can constrain an AI system by “putting it in a box”. The point parent comment was making is that, such a device is borderline useless for its creators. Why invest trillions in capital on a system that can’t even accept input from the internet. So you set it up with an ethernet connection. And this is good, but you have a hardware failure at the concrete room data center. That’s pretty annoying for your customers, so you install some doors (with electronic key codes of course) and give a bunch of (trusted, vetted) people access to deal with those. And this is fine, but it turns out some of your customers are having latency issues so you build more data centers with more humans granted access to copies of the intelligent system. And this makes people happy but to get a faster feedback loop your customers ask to let the AI system have more permissions to the system they’re operating on. And they come with billion dollar checks, and the system hasn’t harmed anyone yet, so you say, “Okay.” And now you find yourself where we are today where AI systems can remotely run arbitrary commands on thousands if not millions of systems, where many individual humans with all of their frailties and idiosyncrasies can physically interact with the hardware running these systems, and where there’s an economic demand to tighten the loop between action in the real world and a response by an AI system. It’s very obvious that the story doesn’t end here, so where does it stop?