Comment by mdeeks

Yeah, with a budget assigned. This is actually just software development and security right?

Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software.

Whether that fits into the business model, or the value proposition of spending tokens instead of engineer hours or user hours is fundamentally a risk management decision and whether or not the developer (whether OSS contributor, employee, business owner, etc) wants to invest their resources into maintaining the project.

While not evenly distributed, and not perfect, the currently available and behind embargoed tools are absolutely impactful, and yes, they are expensive to operate right now - it may not always be the case, but the "Attacks always get better" adage applies here. The models will get cheaper to run, and if you don't want to pay for engineers or reward volunteers to do the work, then you've got to pay for tokens, or spend some other resource to get the work done.

The AIs have already figured out how to succeed in a software job:

1. Ship bugs

2. Fix them

3. You're the hero!

Ngl, watching folks getting irritated about normal employer-employee absurdities from the employer perspective through usage of agents and having to pay for tokens has been a little therapeutic for me.

Software engineers generate security bugs, Software engineers find them, then Software engineers generate fix, collect salary, profit?

All my sibling comments are missing the message here which is that if Claude can find security issues then it can avoid them right when writing the code, so it could just never commit anything containing a security issue.

Replace “Claude code” with “programmers” and you get what we’ve had up until now. It’s all just moving quicker now.

Engineers generate security bugs, security researchers find them, then engineers generate the fix, all the while getting paid, raking in hundreds of thousands of dollars a year in profit per engineer.

You can hook traditional SAST into your coding tool, and get cheap-ish realtime detection for some classes of vulns while coding.

You can optionally layer LLM diff scanning if you want to burn some tokens on your tokens. Modern tools can catch some impressively subtle issues.

Humans work like that too. If you're not comfortable with Claude involves in every step (for whatever reason) then just use different providers for each.

I wonder how many minivans Anthropic is going to code themselves.

Just refactor and rebrand all of it as Claude Code and see it as one process.

This also describes the work of software engineers.

New era of cat and mouse.

I'm starting to think that those who are most aggressively expressive about low quality from these tools are the same who expect everything to be a one shot.

How is this supposed to work? Humans generate security bugs, then humans find them, then humans generate the fix, profit?

Yeah. Presumably as AI code generation gets better, the output gets better. As smaller portions of code are stitched together, human/AI systems analyze it holistically to make sure all its integrations are secure and bug free.

In 2026, different models are better at different things. Cheap models can plan and do small/medium code projects well, more expensive models are even better at architecture and exploit discovery.

Yes. Up until this point the bottleneck was how many developers you could convince to help you. Now it's how much money you can dump into it. Like everything else, software is becoming a game where the winner is the organization most willing to spend money. It'll be like bombs or tanks - you need smart people to advance in the war, but you also need money and material, the material is just compute infra.

Man, some people like conspiracies. I encourage you to replicate all that.

So? That's how a business works. We sold you landmines and now you need them removed? Lucky you we also have mine clearance products.

I wonder this too. I prompted Opus 4.7 to generate some Python threading code for me. The code to run the sub-thread looked like this:

    def run():
        with contextlib.suppress(SystemExit):
            do_thread_thing()

    threading.Thread(target=run, daemon=True).start()

Suppressing SystemExit was surprising, and made me curious. I followed up and asked the model: what's the purpose of that?

The model's response: "Honestly? Cargo-culting on my part. You should remove it."

Thinking off the top of my head - couldn't you have an AI scan that looked for such things? Just send every file in the code base to AI one at a time. Have a prompt like "See if there is ABC pattern that can now be handled by XYZ standard library function in this file. Reply YES or NO. {{file contents}}"

Seems you would not need that many tokens to do so and you might find such cases.

Gosh this couldn’t be more true, which IMO is the real reason LLM workflows are not strictly faster if you care about quality. Otherwise you end up with a codebase where only 60% of it is necessary. Standard testing patterns also tend not to be great at catching this particular flavor of LLM-ism.

Watching it like a hawk and stopping/redirecting, or immediately reviewing and doing the same is the only way, really.

Two years? That exists right now. You only have to point Codex Security at an open source repo. There are a lot of tools and companies that are spinning up today that do autonomous pentesting.

I'm not even sure a specialized model is needed here. It probably just needs the right harness around existing ones.

I expect the next two years to be absolutely brutal for hacks. Attackers have supercharged tools in their hands right now. Defenders are only getting started and will have to plow through a massive backlog of newly uncovered vulns.

The major short term downside is that open source or personal projects won't be able to afford things like Codex Security.

You'll have access to the same models as your hypothetical attackers, and a big advantage if only you have access to the source code

I would say that if this sounds untenable to you, then you may want to consider that the way we architect software has itself been untenable for a while. What Mythos can accomplish today in public, an APT unit can already accomplish in secret.

Not sure what the threshold is but I sent them all of my bug bounty profiles and papers I’ve authored.

I don’t think you need all of that though. I know a whole mess of people that have gotten it for much less. Should just give it a try.

By the way, you might be interested in looking up “blameless post-mortems” and indeed the field of incident response more generally. Modern incident response practice is to treat failures of an individual to do something as problems with the system they were operating in, because humans aren’t designed to be consistent or perfect and therefore shouldn’t be pretended or assumed to be.