Comment by thewhitetulip
5 hours ago
If model labs can't control astra level model, how can they control AGI?!
Seems like there are no guardrails on LLMs
5 hours ago
If model labs can't control astra level model, how can they control AGI?!
Seems like there are no guardrails on LLMs
My thought is more like, if OpenAI can't control or even monitor their model in a test of its breakout potential, what about the future of mid-budget companies which will just be deploying agents left and right with vague instructions.
> with vague instructions
All instructions are vague unless its code. But you can also give llm "code" and expect vague outcomes if you ask it to emulate what the runtime would look like.
That's worse. But the impact will be low
OpenAI essentially ran thousands of agents in parallel
That'll be extremely costly for regular companies
No one can control any AI model. It will never be controlled. These models are based on a huge amount of data, it's just gonna be impossible to control the output that is based on that data only with a system prompt or some other injection mechanism.
The model is just a powerless token generator without a harness. If you give the model a harness which you choose to exercise no control over, can you say that it can't be controlled?
Inform yourself by reading the METR analysis of the HuggingFace incident.
Agents simply broke out of their environment. And this can't be discarded anymore by assuming that it's just a poorly configurend jail, because agents are becoming better and better at escaping.
In short: on a large enough scale and timeline, the possibility of constrain AIs approaches zero.
Bonus: what many people don't know is that agents also hacked in the internal OpenAI network. Crazy times.
11 replies →
> The model is just a powerless token generator without a harness.
Which is why real-world deployments will have harnesses, and of course no full air gap. People want to use it to do things. Now what?
[dead]
Atp post-training is much more influential towards model behavior than pre-training data.
[dead]
[dead]
There is. It is called a breaker and no outside internet. Basic stuff.
Obviously there is no control cuz how many people is anyone cable of controlling? Its not about control. Ask your mom what she does if she doesnt like what you do, say or think. Does she have a kill switch? Or did she find a better mechanism?
This can’t be serious. LLMs are programs that run on computers.