Comment by geraneum
22 days ago
The benefit could be the effect you described. For some to say it’s breakaway intelligence. Aligns with AGI narrative.
22 days ago
The benefit could be the effect you described. For some to say it’s breakaway intelligence. Aligns with AGI narrative.
It does not align, though, with the narrative that openai is a good steward of AI. If anything, if the world/government took the AGI narrative seriously, all openai operations (except maybe serving customer inference) should immediately get shutdown and be dissected by independent investigators to find out what is going on there and how many other such breaches exist. The fact that openai continues functioning as normal and is not immediately shutdown after repeated incidents implies that the world does not really take the AGI narrative seriously.
Evidently that’s not what’s happening and I don’t think anyone seriously expects that (considering the outcome of hugging face campaign). It’s a marketing technique as old as GTA’s early days and it’s apparently still effective in one form or the other!
What will the government do if they are worried? They’ll ask to look at the envs, logs, prompts, harness code, etc. Questions we should be asking before making assumptions about emergent breakaway behavior by colluding AGI 1.0 super agents.
The question is less about whether astra is AGI, and more about whether this direction approaches AGI and we should worry about existential risks wrt to that. If we do, openai's situation is very concerning even if astra is not AGI because it shows a lack of guardrails and any kind of measures to prevent these existential risk scenarios. Especially if we consider that this is gonna happen gradually in steps than suddenly.
If AGI existential risks were accounted for in legislation openai would have been shutdown immediately.
2 replies →