Comment by smaudet

For all its talk of inoculation, this is a terribly written essay. They do not make a point, nor even arguments, instead, opting to ramble in hopes that you forget whatever it was you were thinking.

The issue is simple. Just like us (who are arguably complex, look at what we're building over here, this AI computer stuff!), entities have simple core needs (like food, water, power, etc.).

An infinitely smart AGI has the potential, nay, likely cause, to require infinite resources. We're already seeing the effect in the computing sector on e.g. chips, there's no reason to think this trend won't continue...

Lets circle back to the hydrogen argument, will we blow ourselves up. Real concern, abated by hard numbers. Different atmosphere, different concentrations, different pressure, different possible outcomes.

Today, we don't have those numbers. We don't have those calculations. I don't disagree with the point at the end "about how people can exploit other people, or through carelessness introduce immoral behavior into automated systems". These are issues, too. But saying there are other issues, don't worry about this big issue over here, is the absolute worse argument possible.

That's hand waving.