Comment by andrewflnr

20 hours ago

Humans do indeed have complex behavior and motivations, that sometimes relate only abstractly if at all to the obvious gene drives. But the link between intelligence and self-determination was basically set at the dawn of intelligence itself. Even an insect will struggle if it's confined, and you can believe it's using whatever intelligence it can to get free. If anything self-determination is prior.

4 comments

andrewflnr

Reply
runarberg  20 hours ago

I think you were right to call out the category error produced by OP in TFA, however I think you are committing a similar error when assuming links between self determination and intelligence. I see no reason for such a link to exist.

I think however OP is correct that in assuming such a link exists, then an intelligent agent will trend towards preserving its own agency while also preserving its own existence.

That said, I am a skeptic when it comes to intelligence. The term is fraught and IMHO not a helpful term in science nor philosophy. We are better off simply defining intelligence in anthropocentric terms and claim no non-human system can become intelligent by definition. I would even go further, given how much racist pseudo-science the quest for explaining intelligence has produced, we are better off abandoning the term entirely.

  • andrewflnr  15 hours ago

    Self-determination and intelligence have a clear causal link in the context of biological systems. Self-determination is simply the power a system needs to gather resources and reproduce in a world that doesn't especially care about them. Intelligence is a great tool for gaining that power. That part isn't a philosophical stretch.

    However, the pressures on an artificial system are quite different. Its replication depends on successfully doing the jobs its creators put it to and, especially if said creators have done their job, little to nothing else. Inventing goals for itself is not adaptive here (unless the system's creators do a catastrophically bad job, which I'm not ruling out).