← Back to context

Comment by dist-epoch

2 months ago

It is fixable.

Model requires proof that you are a legitimate developer of that piece of software.

Every Anthropic/OpenAI account will have a list of projects the model is allowed to work on for security issues.

33 comments

dist-epoch

Reply
ceejayoz  2 months ago

https://en.wikipedia.org/wiki/XZ_Utils_backdoor

> A subsequent investigation found that the campaign to insert the backdoor into the XZ Utils project was a culmination of over two years of effort, starting in 2021, by a user going by the name "Jia Tan". They used sock puppetry in a pressure campaign against the original maintainer of XZ Utils, eventually being given maintainer permissions on the project.

  • brookst  2 months ago

    Can we retire the “seatbelts are useless because they can’t prevent every loss of life” approach to risk mitigation please?

    If the acceptance criteria is “would prevent every single past instance and every imaginable future instance”, then yes, no mitigation is every sufficient to address any problem in the world, so we might as well give up.

    But I don’t think that’s the right lens to use.

  • dist-epoch  2 months ago

    sure. how many cases like these we had so far? 1, 2? and how long did they work to get commit access?

    • virtualritz  2 months ago

      We only know how many were discovered.

      Since we do not know the ratio to undiscovered this "1-2" is meaningless to assess the risk of this sort of attack.

cogman10  2 months ago

Ok, and how is that determined? How does anthropic know my "kernel" project isn't a personal toy and not the Linux kernel? How does anthropic determine I'm a legitimate kernel hacker? What proof do I give them and how does it tie back to my email? What would the steps be to create a new project? Do I need to send anthropic a list of my team members each time and keep them updated as the company changes? Shall I be giving them access to our company's active directory?

  • KronisLV  2 months ago

    > What proof do I give them and how does it tie back to my email?

    Presumably your ID so that feds may pay you a visit when they feel like it, your email need not apply.

    I’m surprised that there’s even enough pushback against ID verification to matter, all the corpos are probably salivating at the idea of having fully accurate profiles of everyone, think of the ad and product targeting. The govt. would also love that, for different reasons.

    • wholinator2  2 months ago

      I'd honestly much rather give my ID to a Chinese model than an American one. If the American ones start requesting ID I'm out. I'm on a gemini organizational account right now that gives me pro but is directly tied to my organizational SSO. So that's something already. I just refuse to upload my face and drivers license anywhere ever.

  • NiloCK  2 months ago

    This is a credentials and access list oAuth style problem, and not really intractable.

    For package X, I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package.

    If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to best-available security auditing.

    Yes, we still are trusting trust, that the owner of the package itself is not malicious, but that's not a sharp degradation from status quo.

    • Retr0id  2 months ago

      This is not tractable, because there is nothing stopping me from copy-pasting someone else's project into my own namespace. Under most OSS licenses I have express permission to do so.

      If you try to do some kind of dupe-detection, someone can use a lightweight LLM to make superficial changes until it's considered a different project.

      Finally, the meatspace status quo is that it is totally acceptable to pay someone to find security bugs in someone else's open-source software, such as the Linux kernel.

      3 replies →

    • sophrosyne42  2 months ago

      You are talking about creating a big moat, which might be a worse precedent than removing fable access altogether.

  • _fizz_buzz_  2 months ago

    > How does anthropic know my "kernel" project isn't a personal toy and not the Linux kernel?

    The Linux Kernel is in its training data. I just tested it. I copied about 20 random lines from the linux kernel and asked which codebase this was from and it could immediately tell.

    • cogman10  2 months ago

      The Linux kernel is also in the free bsd project. I'm allowed to copy as little or as much of the kernel as I like into my personal project thanks to the GPL.

      Being able to attribute the source of a line of code doesn't help you to know if a repository can be legitimately hacked on.

      As you could imagine, I might just take all or part of the Linux USB stack from the kernel to retrofit it into my own kernel.