Comment by cautiouscat

2 days ago

This has been on hackernews before, months ago[1]. How is it just being found now?

1: https://news.ycombinator.com/item?id=49258881

Of the same ilk Claude code will make PRs on your behalf, using a vague consent prompt that when it happened to me was subtle and easily.

Best practice will forever be to enforce your guard rails in addition to Anthropics guardrails.

  • Calling them guardrails scares me. I want to hear about deterministic safety interlocks.

    Can anyone recommend a reliable 3rd-party audited guide that describes Claude Code's built-in capability gating infrastructure?

    Whenever I ask users of claude code how claude controls actions they don't seem to be able to distinguish between what just goes into LLM context (e.g. as a "don't do X" system prompt) and what is gated behind a deterministic capabilities layer.

    • The default is "auto mode" where another non-deterministic model compares what the user requested with what the model wants to run and blocks if there is a mismatch.

      It will block an "rm" call if your prompt did not include anything about "deleting something".

      It mainly protects benign errors. It's completely broken against malicious actors. The model itself will suggest work arounds against its own auto classifier to the user, in order to reach its goal.

    • I haven't found a comprehensive independent code audit of Claude Code's full permission implementation.

      Anthropic's documentation does distinguish permission rules and pre-tool hooks from model instructions. It also documents OS-enforced sandboxing for shell commands, which has a narrower scope than all agent actions.

      An independent stress-test of Auto mode is available here: https://arxiv.org/abs/2604.04978. It's a behavioral evaluation, not a source-code audit.

      The important question is whether every state-changing action passes through an enforceable boundary before execution, including indirect actions through file edits or scripts. Post-execution validation is a separate control.

According to vibe coders, the future is never reading what the code actually does. You just trust coding agents with everything and "give into the vibes" even when it "says" its correct.

Offshoring understanding is the trend that many companies will pay in the form of comprehension debt.

  • That's not because "vibes" but because focusing on "code" is the wrong level of abstraction altogether. It always was. We just didn't have the technology to let it go, but now we do.

    Code is a reflection. Of what? Look there. Code itself is useless.

  • I mean it's a fair take, your email is probably leaked anyways. Mine is breached 32 times on HaveIBeenPwned with various passwords. Who cares at this point really it's just an inconvenience

    • I'm happy to say that my main email that I've been using for 15+ years hasn't been leaked, because I've been very careful on how I share it. Getting it leaked by Claude would suck.

  • According to C-programmers, the future is never reading the assembly that the compiler actually generates. You just trust the compiler with everything and "give into the vibes" even when it "says" it's correct.

    (sorry, i think you make a good point, ... but i could not resist.)

This reminds of the open issue on the Claude MacOS app where the OS-wide voice dictation keyboard shortcut function doesn't work.

Somebody on Reddit figured out they just forgot to add a single line permission request into the app. It's been broken now for months.

I just find this wildly hilarious given the 'world ending all powerful AGI is here' rhetoric they're running with at the same time.

Maybe before worrying about permanent 15% GDP growth or human annihilation, what if we started by putting an agent on user issues and solving them pro-actively?

I know the kool aid is tasty, but we might be getting a bit ahead of ourselves here guys. You can fantasize about human annihilation after you figure out how to make a desktop app that doesn't have obvious bugs, I promise :)

  • You're not wrong, per se, but I think you are not comprehending what will happen next. You think nobody sees this? You think nobody is aware that it's the humans in the process that are holding everything back?

    Sure you can have an issue tracker with 1000 items and yes, you can have a bot solve them all in a day, but who monitors it? Who gets to say what he did at the standup? Who reports to whom and what?

    Code is not the bottleneck (if it ever was..)

    • So if the leading frontier AI company are struggling to remove human gatekeepers from slowing down the process, how do you think this will go across the rest of the economy?

      Will Northrop Grumman, the state government of Alaska, the construction industry, etc. somehow magically move faster than Anthropic?

      1 reply →

  • Yeah it's legit weird for the whole narrative to see Claude Code/Codex repos with thousands of open issues. With certain issues full of +1 reactions and "still happening in 2.21.2" replies.

    But instead of a Software Dev as a Service triaging/implementing/merging fixes 24/7 and clearing the backlog as they come in, they just sit there for days to weeks until a GitHub bot closes then as duplicates, generating more unhappy user replies and new issues to go unaddressed.

  • Aren’t we supposed to be able to just point Claude at the bugs and rip through them?