Comment by lenerdenator
4 hours ago
> Him having control would be an improvement on the reality.
Oh, he does. It's unlikely that this is some AGI that spawned itself out of nothing and started doing this. If he were a decent person, he'd simply find a way to investigate this internally, fire the people responsible, and find a way to set up guardrails around his product.
The problem is, like most people in SV, Altman seems to have a twisted ethical compass. He doesn't see these incidents as an issue, he sees them as an opportunity. He has both the thing a bunch of Western governments want (a superhacker agent that can do dirty work) and a crisis that can be used to craft regulations that favor OpenAI and thus his bank account.
> It's unlikely that this is some AGI that spawned itself out of nothing and started doing this.
You've not spent much time playing with these models, I see.
Does't matter if you call the current models "AGI" or not, they:
(1) successfully do stuff like this. Someone I know on Telegram found "fifty or so" Linux filesystems kernel bugs a few days ago, of which 26 were the first night while he slept; this was with Kimi which is one of the open models. He stopped it when the backlog of fixes to submit was too big, not because it wasn't finding more.
(2) sometimes misunderstand goals, sometimes wildly so, which happens every so often for the same reason we use programming languages (and indeed mathematical formalisms) at all: natural language is vague and prone to misunderstanding.
and (3) tend towards sycophantically agreeing to implement goals they're given even when the goal is stupid.
This can easily add up to something seemingly innocent like "research if there's any statistical difference in melanoma rates between different Australian states" becoming this headline. (For example. I don't know what the actual request was).
And "spawned out of nothing" is, like, what rhetorical point are you even trying to score here? They're an AI company with (their definition of) AGI as their goal. They upgrade models twice in the time it takes someone to pass a probation period.
> He doesn't see these incidents as an issue, he sees them as an opportunity.
That he may be.
The models are still quite capable of acting this way without him being aware of it at the time, nor deliberately ordering it.