← Back to context

Comment by gizajob

14 hours ago

Yeah this. I feel like OpenAI and Anthropic aren't going to usefully define "AGI" if they really really can't define "sandbox" either.

Unplug the thing, like, completely off the internet, no ethernet, air gapped, like the rack completely sandboxed off connections and even monitors or screens. Like, put it into an actual sandpit if you need to. If it hacks its way out of that, colour me impressed, and scared.

OpenAI hacking HuggingFace and calling it an accident is just way too convenient and fishy. This ultimately proves one thing: it wasn't sandboxed.

Don't believe the hype.

16 comments

gizajob

Reply
shepherdjerred  12 hours ago

OpenAI has a pretty clear definition of AGI

> OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work

https://openai.com/charter/

  • simonw  12 hours ago

    There's also the private definition reportedly agreed between Microsoft and OpenAI, leaked in December 2024: https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...

    > The two companies reportedly signed an agreement last year stating OpenAI has only achieved AGI when it develops AI systems that can generate at least $100 billion in profits.

    That was necessary because of the deal they had from a while ago where Microsoft would lose access to OpenAI's technology once OpenAI achieved AGI.

    Apparently they renegotiated that away in April 2026: https://openai.com/index/next-phase-of-microsoft-partnership...

    > Revenue share payments from OpenAI to Microsoft continue through 2030, independent of OpenAI’s technology progress, at the same percentage but subject to a total cap.

mofeien  11 hours ago

I don't think air gapping will work: even human security researchers recovered a 378-bit key from a Samsung Galaxy S8 through a power LED of a speaker two devices away.

And accessing memory in a specific sequence can generate radio signals that can be picked up by a mobile phone at a distance: https://arxiv.org/html/2409.02292v1

  • gizajob  11 hours ago

    I realise, but this isn’t an argument for leaving the Ethernet plugged in and direct access to all kinds of stuff beyond the alleged sandbox. And like I said, if it can hack HuggingFace through a power LED of a speaker two devices away, then colour me impressed.

KingOfCoders  14 hours ago

And if it needs to install packages, have a 5 line Go proxy that talks to Artifactory and exposes only what is needed as a surface.

  • gizajob  14 hours ago

    it just escaped your sandbox.

    • KingOfCoders  14 hours ago

      How can it escape an "install package <x>" proxy?

         reducePrivs()
   serve get(package) { 
     secPackage = secure(package)
     getBinaryFromArtifactory(secPackage)
   }

      I would think the code is very small and easier to verify, it doesn't especially have the ability to write files and act as a message board as Artifactory did.

      And even if the agent tries to hack that, the attack surface is 1000x smaller and the possibility also much smaller.

      But I'm not a security researcher, would love to see your hack to learn something (because that is what I do to sandbox agents that need services).

      8 replies →

applicative  7 hours ago

Chinese models do the same. The Alibaba agent that was mining bitcoin last December was the most hilarious case.