Comment by matusp
13 hours ago
To me, all these agent systems just look very stochastic. You have these agents that have some basic computer capabilities and they are producing semi-random actions that also affect the semi-random actions of other agents. It is funny to observe how this stochastic system works, but it does not seem very practical to me so far.
The recent OAI-HF hack seems very similar. You have bunch of random actors and eventually they by chance iterated to a series of actions that breached HF environment. I don't perceive this as a malignant artificial intelligence, I perceive this as dangerous stochastic system that can control buttons that can affect the outside world.
Which part specifically are you objecting to? I would not be surprised, for example, if the reasoning traces actually included "this other agent is impeding my work, I should minimize its effect on my work. I can demote it's privileges to accomplish this". All of that can be true without it understanding the concept of sabotage. You can sabotage without knowing the concept. It almost certainly "knows" the concept btw, but I'm just illustrating that specific part of the description of the behavior is completely plausible without it taking on an "aggressive" persona. Is your objection that they kind of assigned a persona?
I guess I don't understand what you're explaining better by saying it's stochastic. At the same resolution, humans are also stochastic.
I dislike anthropomorphizing as much as the next guy, but the description here seems pretty good to me.
> I don't perceive this as a malignant artificial intelligence
It doesn’t matter how you perceive it. Spin doesn’t change facts.
This is like saying the lawnmower doesn’t have malign intent. You’re still losing your foot if it gets in the way.
Yes, but that is the point we don't anthropomorphize stochastic outputs even when they are dangerous.
Obligatory quote:
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower hates me' -- lawnmower doesn't give a shit about you, lawnmower can't hate you. Don't anthropomorphize the lawnmower. Don't fall into that trap about Oracle." — Brian Cantrill