Comment by SwellJoe
3 hours ago
I've found in my current work on a security auditing harness and benchmarks, both Fable and Opus are useless. I recently switched to using GPT for Nelson and the security benchmarks I've been doing because Opus started refusing to do the work. I guess I probably could also use GLM or DeepSeek or MiMo, and I'll probably do some experiments to see the shape of all of their guardrails in this area soon, now that I see it's more than one model that behaves this way (Gemini in Antigravity also refuses any security auditing task, even as simple as "find security bugs").
I blogged about it: https://swelljoe.com/post/why-i-had-to-switch-to-gpt/
This underscores a huge risk of broad agentic adoption in an enterprise. Your engineers atrophy and if the agent provider decides to squeeze you, you’re SOL.
Yeah, I guess I could also write the code myself, if all the models refuse. But, it seems worrying to have a handful of the largest corporations and a few governments having access to the best models while the rest of us are using hobbled ones.
So far, we're not in that boat. I've had two models refuse to participate, but most just do what they're told.