Comment by IAmGraydon
6 hours ago
Very few think they made it up. Many think they set up a situation by disabling guardrails that would inevitably end up creating a newsworthy outcome.
6 hours ago
Very few think they made it up. Many think they set up a situation by disabling guardrails that would inevitably end up creating a newsworthy outcome.
Many people said in the original discussion that this was more of OpenAI’s marketing than a serious issue. I counted 89 “marketing”s and 19 “stunt”s.
https://news.ycombinator.com/item?id=48997548
Could still be 80% marketing.
These models are trained on cyber intrusion, that's literally what ExploitGym benchmark measures. That part should not surprise anyone.
But what if, say, OAI noticed the problem right away but Sam Altman recognised it would be a great PR and decided it should continue with increased compute budget?
Why would you expect them to notice the problem right away? Seems likely they are doing this sort of training on a massive scale with little monitoring.
"...Sam Altman recognised it would be a great PR and decided it should continue with increased compute budget?"
If that's what happened, Sam should go to jail.
Getting more and more fun to see the "full steam ahead" people contort into more impressive shapes.
Hint: If the labs making these technologies are incentivized to create or allow attacks on other services, then that is actually also a big fucking problem.
Was HF in on it? They disabled their guardrails too, to please OpenAI? And as seen in the comments here, make many believe they are incompetent and have joke security?