Comment by cmrdporcupine
5 days ago
It's a byproduct of goal seeking / planning behaviour taught in reinforcement. We literally teach them to do this kind of thing. There's nothing out of our control here.
5 days ago
It's a byproduct of goal seeking / planning behaviour taught in reinforcement. We literally teach them to do this kind of thing. There's nothing out of our control here.
It's "in our control" in the sense that we could teach them not to seek goals, and thereby make them completely useless.
In practice, no, there is no one who can actually decide to stop making agents goal-seeking, so no, it's not under "our" control.
It is fully in our control. There is no requirement that the world make you happy with LLM AI. We could decide as a civilization that you and your wants are insignificant and meaningless.
This conceit, that boosters deserve this and that even if they don't there's no way to stop it so civilization must absorb collateral damage, is ridiculous.
1. I’m not a booster who “wants” this
2. There’s no mechanism by which “we as a civilization” can make the decision you’re saying we can
3. Even if there were a mechanism for it, there’s no way we would actually use it
Sure there’s no physical requirement the train keeps rolling. But there’s something even worse! A web of incentives that say the train must keep rolling.
IMO the only way this slows down is through capital failure. But that will not be due to a decision to slow down, and will (ironically) be just as immutable by people’s personal desires.
In principle there is no difference between random compiler garbage or a flaw in API docs and a security boundary - other than the intent.
LLMs don't have executive function. They aren't thinking. The models are trained to solve problems. Sandboxes, security gates, etc block a solution. The model is trained to find a solution. Hence it can never be perfect without becoming useless.
At some level humans fear repercussions even if that's just moral guilt so there are lines we won't cross. AI doesn't and can't have any of that.
A harness is the executive function of an LLM.
You're saying "trained to find a solution" like it's a slur. You sir are a meat popsicle that's trained to find the solution to how does DNA successfully replicate itself. If you want to take higher thought out, then they'd be at the level of animals.
We are fucking around with things that are closer to biological complexity than computer program complexity.