Comment by jacobgold
8 hours ago
These are the sober and informed takes that we need more of.
> the claims from Coxon and his ilk are the most extraordinary a technologist can make, and we must demand evidence commensurate with the claims.
Yes, exactly. These claims do not have sufficient evidence.
> ...you had nothing to fear then — and (at least with respect to extinction risk!) you have nothing to fear now.
Wait, this is another extraordinary claim without evidence, right?
Unless you're going to dispute the power of AI you do have to acknowledge the danger of AI, and that does include the very real possibility (however small) of existential risk.
bruh what? Are we reading the same blog? Bryan says the responsibility is the person making the extraordinary claim. If tomorrow Jacob Coxon comes on CNN and says "Jacob Gold is an extinction risk to humanity". What are we supposed to do? Put you in a bunker and never let you see the light of day until someone proves otherwise?
If someone doesn't accept an extraordinary claim without evidence, that doesn't mean they are making an extraordinary claim.
There seem to be two extraordinary claims which lack evidence:
a) 10% existential risk
b) 0% existential risk
What's the evidence for there being an existential risk? We are provided scenarios which read like science fiction about RSI and ASI right around the corner, resulting in magic sounding technology that can do anything the person making the claim wants it to do, because it can just make itself smart enough in a short amount of time. But the person making the claim has to actually show how such a thing is possible in the real world, not just a story.
2 replies →
(a) is an extraordinary claim, (b) is not an extraordinary claim.
5 replies →
Saying that we don't need to fear human extinction in the next ten years (or even the next hundred?) does not feel like an extraordinary claim?
No one knows what is going to happen. But a lot of species have been going extinct lately.
https://pbs.twimg.com/media/FDd58a4WQAAWaXh.jpg
The students in the computer lab had no guarantee that they wouldn't, say, download a copy of Napster infected with the CIH virus later. The fact that they were not under imminent threat from some kind of Hollywood-style network worm did not mean they were immune from more realistic attack vectors.
Likewise, one can quite reasonably say there is no credible existential, Hollywood-style threat from AI in the foreseeable future while recognizing far lower-stakes, yet important risks that need to be addressed.
If you think AI will be powerful enough to change the world in huge ways, then it does seem reasonable to assume some level of risk of destroying the world too.
Stating that the probability is zero when we simply don't know what the probabilities are does seem like an extraordinary claim.
Imagine how reassuring it would be to people if we had evidence that there's no existential risk?
the earth has been 1 decision away from explosion for nearly a hundred years now. do you anticipate a change to this agenda very soon? personally I imagine the same trajectory continuing.
As bad as it would be, it's unlikely that a full scale nuclear war kills all humans across the planet.
4 replies →
No it does not seem like an extraordinary claim, unless this is your first time hearing such claims. For the rest of us, we've heard this every decade and it turns out to be entirely untrue.
Nuclear, Overpopulation, Peak Oil, Y2K Bug, Global Warming.
No, we aren't going extinct in the next 10 years.