← Back to context

Comment by WithinReason

8 days ago

> [...] beating Claude Code (32%) at roughly $0.17 per vulnerability found

Claude Code is an agent harness, not an LLM.

Claude is a brand (or group of LLMs), not an LLM.

11 comments

WithinReason

Reply
raincole  8 days ago

Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.

  • mkagenius  8 days ago

    It looks like the author is specifically avoiding model's name, because results are really weird.

      Opus 4.8/4.7 scored 28%

  Opus 4.6 score 37%

    So the author thought as let's not get into that just write Claude.

    • happycube  8 days ago

      Not weird at all, given the variance in Opus' quality over the last few months.

      wild guess - I wouldn't be surprised if Opus 4.6 was run quantized for a while, and 4.7/4.8 have QAT for that nerfed size.

    • insiderphd  8 days ago

      Hello! Author here (Katie) Ty for your comments, 4.6 and 4.7 both scored 28% on our benchmark, I just wanted to have 10 things in the list because I wanted a round number.

Onavo  8 days ago

Claude code it's the only way to get access to the actual amortized cost of running a Claude-scale model. The consumer non-enterprise API is extremely expensive (with increasing marginal costs for the user and fat profit margins for Anthropic). If you want to approximate a State level attacker's cost where they can have the model on their own hardware, Claude Code is probably the best guess at the amortized cost.

croemer  8 days ago

The dollar amount is meaningless without comparison - and no other model has a price tag. Sloppy article.