Comment by usrbinbash
3 years ago
> Einstein could be convinced to tell you that 7 + 4 = 5
I think Einstein would have laughed at me if I tried to convince him to do that.
Because other than an LLM, Einstein knew what these symbols denote, what their relation to reality is, and how math works. Einstein didn't mimick math by completing sequences of tokens, and relying on humans antropomorphizing the sequence completion engines output to an actual understanding of the topic.
> Ants can't define nuclear weapons, but nuclear weapons exist.
Ants also cannot build nuclear weapons, nor create anything that would make the emergence of nukes any more likely, among other things because they don't have the ability to define them. So if we accept this premise, then the discussion is moot: We can be fairly certain that we are the most technologically capable entities on this world, so unless we can understand a technological creation to the extend that we can bring it about, nothing else will.
In short: If we are ants to the superintelligence, then we have nothing to worry about, because we likely lack the understanding and ability to create it, or even something that could act as its precursor. If we are not ants, then we should be able to predict when this can happen.
> Since we can't accurately predict the future
We can accurately predict a lot of things. Global warming is an example. And the things that we can predict, and determine how likely they are, we can and should prepare for.
AI doomsday preparation demands the exact opposite: That we prepare for something that we cannot predict, and cannot demonstrate if it is possible, or how likely it is. That's like asking to prepare for an ice age. Theoretically an ice age is possible on this planet, however nothing we can see, measure and demonstrate right now, supports the prediction that an ice age is about to destroy us.
Einstein was not hobbled by having to tell the truth. He was capable of joking, playing a prank, doing it as a favour, doing it as a challenge, as an experiment, exploring the scenario, etc.
> "unless we can understand a technological creation to the extend that we can bring it about, nothing else will."
Where did human intelligence come from? Are you a Creationist? Self-improving AI brings itself about. With the right feedback loops and the right software, the fear is that an AI will grow itself - and no humans and no aliens are needed up front to design it. People are trying to make machines behave like people, like pets, like the world, and emerging out of this is machines which behave more and more like people with every passing year.
> "we should be able to predict when this can happen."
Who says we can't? Ray Kurzweil has been predicting it will happen by around 2030 for years and years.
> Where did human intelligence come from?
From ~290-300 million years of mammal, and ~7 million years of hominid evolution, give or take. Which is a natural process and not something an intelligent creator started, is observing, powering or influencing in any way shape or form. Which makes the next statement...
> Self-improving AI brings itself about.
...a bit interesting, because, all the parameters in a comparison with natural systems are different: The system is designed by an intelligent creator, we are observing it, it's development is entirely powered by us, and we completely control it's development.
And so far, the sample size for self-improving AI, in the sense that would be required for the doomsday scenarios to happen, is zero.
> and no humans and no aliens are needed up front to design it.
Last time I checked, matrix multiplication wasn't one of the things observed in the Miller-Urey experiment.
> Who says we can't?
Since so far no one could demonstrate how to even measure the distance, in whatever unit, of AI systems to AGI, I'm not holding my breath.