Comment by lelanthran
3 days ago
> The only way AI gets there is something like control of the nuclear weapons on all sides and that control becoming suicidal.
Not the way I was thinking off; I was thinking more along the lines of brain atrophy until humans are helpless (maybe 2 complete generations of humans).
And that's a short extinction; the KT extinction took place over thousands of years. If, 1000 years from now, we are finally at the place described in "The Machine Stops", only then will an observer be able to decide that we don't have the necessary critical mass of intelligent humans anymore.
Humans got to be the apex predator simply on intelligence alone. It's possible that without thinking ability we'd go extinct.
> Not the way I was thinking off; I was thinking more along the lines of brain atrophy until humans are helpless (maybe 2 complete generations of humans).
That's not how humans work. We developed math, art, literature, etc., because people love the act of doing it. We will continue to do that.
We're not going to stop thinking because we don't have to. We stop thinking because we subject ourselves to political propaganda, or religious belief systems, or other group control methods that turns off critical thought. But even then, it's only a subset of the population, it won't get us all.
> That's not how humans work. We developed math, art, literature, etc., because people love the act of doing it. We will continue to do that.
Sure, for art, literature, etc. For math, physics, biosciences, etc, we pay people to do it. There are too few people who would work for free to push progress in those areas. We are most certainly not going to do that, we're going to leave it to the AIs and hope for the best.
This sort of thing (human brains atrophying over generations) is like something out of a dystopian nightmare[1], but hey, that's natural selection for you: if intelligence is not a selection criteria (and it won't be if machines do all the thinking), it's unlikely that it will be present in future generations.
-------------------
[1]. "The Machine Stops", E. M. Forster, 1907. Or "Pump 6" (can't remember the author, Italian author). Just because it's fiction does not mean it's impossible.
I actually think the opposite… it takes all my skills as an engineer to steer these things effectively and I’ve learnt more than I imagined I could in a very short period of time. And I don’t even use AI as heavily these days due to other commitments. This also aligns with how heavy AI users report feeling drained after intense coding sessions.
My take is that after a period of turmoil, most humans will be doing higher level cognitive work than we’re doing now, because all the lower level work will be done by AI agents, and so progress will accelerate multifold.