Comment by mewpmewp2
1 day ago
I just don't get the fear at all. I too have 10 to 50 agents working 24/7, from 5+ different providers. They talk to each other directly using tmux send keys. What is the exact frightening thing here? It is all tokens following tokens.
I haven't tried what you guys are talking about myself, but I think when agents begin talking to other agents, the chances for unpredictable goal mixing and confusion resulting in really unexpected and bad behavior is a lot higher.
That being said, the danger is measured in computer damage, which can be a lot personally and to a company, but less existential, so your mileage may vary as to how "scary" it is.
When nearly all money in the world is stored in computers, the danger is quite existential.
This seems traceable back to GIGO. If you don't understand the software you're using, don't use it.
What scares me are the people who think that this software is the equivalent of an perfectly-smart elf in a box and use it blindly.
The frightening thing is how easy it is to make copies of things that can reason without the requisite investment, and how easy it could be to direct them to bad things.
It’s amazing how “token following tokens” is this revolutionary, completely earth-shattering technology capable of 100x’ing productivity (also worth untold billions of investment). But the moment people become worried or skeptical it’s “just” tokens following tokens.
They're different groups though.
Skeptical was probably a poor word choice. The point is if it’s negative, “it’s just tokens.” If it’s positive, it’s “AGI.” They downplay the technology and act like it’s no different from anything we’ve had before whenever a critique starts. It’s only as capable as the argument needs it to be.