Not in my experience (yes, sample ~= 1), where they are subpar in terms of cybersecurity (e.g. source code review). This is one of my reports using GPT 5.6 (High), GLM 5.3 High, Fable 5.1 [1]:
> On the full benchmark of 31 audited revisions, GPT-5.6 Sol High reached a weighted recall of 38.27% and GLM-5.3 High reached 30.86%. On a common subset of 12 projects, Claude Fable 5.1 High reached 65.70%, against 42.98% for GPT and 35.54% for GLM.
Not in my experience (yes, sample ~= 1), where they are subpar in terms of cybersecurity (e.g. source code review). This is one of my reports using GPT 5.6 (High), GLM 5.3 High, Fable 5.1 [1]:
> On the full benchmark of 31 audited revisions, GPT-5.6 Sol High reached a weighted recall of 38.27% and GLM-5.3 High reached 30.86%. On a common subset of 12 projects, Claude Fable 5.1 High reached 65.70%, against 42.98% for GPT and 35.54% for GLM.
[1] https://github.com/CoinFabrik/soroban-audit-harness/blob/mai...
How about Kimi?
That assumes you have the hardware capable of running inference on them at a reasonable speed
Or $10 on open router.
Or it assumes you have $5 to spend on the tokens for inference. Not everyone has the money.
it takes a lot more than that....
3 replies →