Comment by CamperBob2

21 hours ago

I don't ship AI-written code without at least having an unrelated model of equal capability review it. That should cover a multitude of sins like intentionally-embedded backdoors and zero-days...

... as long as the reviewer model hasn't been deliberately lobotomized, that is.

(I don't disagree with the [dead] comment, either -- fortunately I tend to make small changes to code I'm very familiar with, so it's fairly easy to vet in person. The more code gets generated by the clanker, the more nervous I get, just because it's impossible to keep some complacency from creeping in.)

5 comments

CamperBob2

Reply
figassis  20 hours ago

I don’t think parent was talking about the shipped code, but about your one development environment while using the models.

  • scott_weber  19 hours ago

    OP could be read either way, but the dev environment threat is not rocket science to manage: LLMs can only do what the harnesses their responses are embedded into explicitly enable them to do [1]. And once the setup is secured, it tends to stay secured without additional effort, unless one explicitly wants to alter its granted capabilities.

    Auditing output by contrast is a continuous cost. On the third hand, it's one that most will want to pay anyway.

    The "embed into shipped code so subtly humans can't spot it" vector is sci fi ish. Less plausible than old fashioned trusting-trust issues, and most folks have managed to sleep at night for the last 40ish years that spectre has haunted.

    [1] I guess the model could return output that buffer overflows the harness and exceeds its intended ability, but that seems far-fetched too

jdjdjdjjdbznnx  21 hours ago

insane. I can't imagine doing this and I hate that I'm being forced by upstream communities to run code nobody has read

just read the damn code yourself!

  • hattmall  20 hours ago

    >just read the damn code yourself!

    It's basically impossible to do this. Claude produces so much excessive / verbose code and comments that reading it would take far longer than simply doing the implementation "by hand". It's kind of like, Claude can write the code in 1/2 a day. It can be reviewed by another LLM, unit tested by another for 1 day. Or I could write the code in probably 3-4 days. Reading it and validating / testing it myself though would easily take 6+ days.

    Now if I were fully in control I wouldn't run code that wasn't truly reviewed at all, but in the situations where Claude is being used that's not my call. TBH a lot gets caught, and I have some automated scripts that find a lot of the more problematic stuff automatically, flag it, send it back and make note for the reviewer model as well.

    I will also spot check areas where I think an issue could arise.