Comment by aesthesia

6 days ago

I'm not sure what you want. This is explicitly an exercise in hardening the systems you're complaining weren't sufficiently hardened!

8 comments

aesthesia

Reply
troupo  6 days ago

I want people to stop pretending they are surprised when the computer does exactly what it's instructed to do.

And AI labs to stop pushing the narrative of "these amazing thinking machines that escape no matter what we do".

The sibling comment described it very well: https://news.ycombinator.com/item?id=50215904

  • dwohnitmok  6 days ago

    > I want people to stop pretending they are surprised when the computer does exactly what it's instructed to do.

    This is very wrong though. We now have many examples of cases where the computer very much does not do what it is instructed.

    OpenAI's hack of DseWiki is an example where the instruction given to the AI agents was to collate data to generate economic reports and it hacked into DseWiki as part of that. It was not given instructions to do any sort of cybersecurity stuff.

    The hospital bed stat research project was a similar thing, where OAI's AI agent found the hospital bed statistics through a simple web search and then hacked into the website anyways to download even more data.

    More prosaically, models make up reasons for doing crazy things all the time!

    Models may or may not be "malicious" (if that's even coherent), but at this point we have lots of evidence they definitely go off the rails in systematic, sustained ways that cause real world disruption that are not at all what they were instructed to do.

    • troupo  5 days ago

      Models don't make up reasons for anything. They leverage their training data while running in poorly secured sandboxes often within the context of literal "hack/find a way/exploit" prompt by the labs themselves.

      I don't think OpenAI ever actually disclosed what they were doing when the hijiacking happened. Probably because their previous stories like HuggingFace were all "oh, actually we were literally prompting agents to run hacking benchmarks in a poorly secured sandbox"

      1 reply →

  • semiquaver  6 days ago

    I don’t know, ten years ago a machine that could do anything you asked it to using natural language would have been pretty god damned impressive.

    • troupo  5 days ago

      It's impressive, and we should be impressed by that. And not by OMG OUR ROGUE AI HAS BEEN ROGUE AND HACKING LEFT AND RIGHT (in tiny print many weeks later: after we explicitly told it to do it and left the sandbox wide open)

  • aesthesia  6 days ago

    Who's acting like they're surprised here? Not OpenAI. Not the original commenter. I think you're reading something that isn't there.