Comment by janalsncm

5 hours ago

I don’t need to know the value of their limit, I just need to know their bounds. Just like a prison doesn’t need to know the strength of each inmate, just that they can’t bend or bite through steel bars.

Cryptography is real, physics is real, networking requires a substrate, CPU clock cycles are real, magic is not real. I think those are pretty reasonable premises.

4 comments

janalsncm

ben_w 26 minutes ago

Cryptography is real, but nobody in that field seems to be hubristic enough to think their methods are flawless, and there's a degree of suspicion than the best models may have secret weaknesses engineered into them by the governments who sponsored them.

Physics is real and networking requires a substrate. But there are already known exploits which can misuse the compute hardware as an antenna, e.g. my first search result: https://github.com/fulldecent/system-bus-radio

(Older nerds may remember https://en.wikipedia.org/wiki/Van_Eck_phreaking)

> Magic is not real

  Turning lead into gold isn't the magic of alchemy, it's just nucleosynthesis.

  Taking a living human's heart out without killing them, and replacing it with one you got out a corpse, that isn't the magic of necromancy, neither is it a prayer or ritual to Sekhmet, it's just transplant surgery.

  ...

  Reading someone’s thoughts isn't magic telepathy, it's just fMRI decoding.

  ...

  Seeing someone's bones without flaying the flesh from them isn't magic, it's just an x-ray.

  Curing congenital deafness, letting the blind see, letting the lame walk, none of that is magic or miracle, they're just cochlear implants, cataract removal/retinal implants, and surgery or prosthetic exoskeletons respectively.

- me, https://www.lesswrong.com/posts/hAwvJDRKWFibjxh4e/it-isn-t-m...

bottlepalm 5 hours ago

Imagine 200 years ago saying the same thing. As if you have any idea the limits/bounds of anything. Especially in the face of a super intelligence, it’s absurd.

janalsncm 4 hours ago
It doesn’t matter how smart it is. 200 years of technology were not accomplished by thinking harder. It required empirical observation, new materials and tools, and supply chains.
We could send a cracked team of scientists and engineers that knew everything there is to know about how to make a CPU. But you can’t build a photolithography machine when you barely have electricity or any way to sufficiently purify silicon.
Magic can just wish things into existence. Technology requires a supply chain. When it works, the latter looks like the former but they are not the same.
- bottlepalm 38 minutes ago
  
  I think some people are just immune to understanding the implications of super intelligence. Like a severe lack of imagination, they only believe something once they see it and afterward claim it was, ‘obvious all along’.
  I don’t really want a disaster to happen to convince you that it is possible. Is there any other way?