Comment by wccrawford
7 hours ago
I feel like that's like saying amoeba have no intent and no motivation. Or plants.
And those cause quite a bit of damage.
AI has all the intent that we gave it, and we continue giving it. That's always been the fear. Not that it will randomly wipe out humanity.
The fear is that it will decide to do that, with a purpose. Whether we tell it to protect us and it goes too far, or it decides that it can't achieve the purpose we gave it because we'll interfere and it removes that interference...
The fear is that we will set it on that path and can't stop it.
I'm sure there have been some scifi books that have it just be random, but they're far, far less worrisome.
Well, an amoeba just like all other living beings, maximizes self-preservation/propagation for which the article mentions as a trait which AI models do not have.
Model doesn't but in in agent loop it can generate such sub-goal
The loop is a deterministic programmatic routine, and not model weights. This is the equivalent of a killer military drone written in C++ and OpenCV. I’m not arguing that AI isn’t dangerous, but nothing in the model weights of LLMs show signs of self-preservation.
2 replies →
Was thinking the same. Air, water and fire have no intents and yet we get hurricanes, flash floods and wildfires.
Also, repeating a point from a similar thread: Software can have "intent" in the sense that it steers itself towards a predefined goal without having to be "alive" or "conscious" in any way. Some classic examples are thermostat control loops, navigation systems and chess engines.
We don't try to "align" those things themselves - so maybe there are some learnings there about control, containment, processes etc.
> Air, water and fire have no intents and yet we get hurricanes, flash floods and wildfires
yes but we don't blame the hurricane, we blame the people who didn't do any hurricane prevention or didn't put the snow avalanche sign up.
>Some classic examples are thermostat control loops, navigation systems and chess engines.
Those don't have intent. The people who made them for a purpose have. In fact this is true for all computers. Computers don't compute, people do, using computers. This is the nonsense of modern ontology that David Bentley Hart frequently writes about. The computer is only computing insofar as an intentional mind instructs it do so, and then because of their reductionist assumptions some people use the computer metaphor, and that is all that it is, to deny that human minds possess intent to begin with in the first place.
AI is not an organism. Please stop trying to think it is.
AI has all the intent that we gave it. I'd go a bit further and say it has the intent that it develops through the context, which may not be intended by any human if the context grows 'organically' from the environment. So yes, I strongly disagree with the OG and I'd say they do have an intent encoded in their states.
An AI agent can't make decisions, so there's that. But, as for purpose, it's not the purpose of an AI agent we need to fear, but the person wielding it.
> An AI agent can't make decisions
I don't think is true but maybe I'm misreading your perspective?
Weird, when I tell it "pick between the options you presented the one you think it's best" it does make a "decision". But now you are going to say "nothing was decided here, no more than how a flipping coin decides on what side to fall".