Comment by howunfortunate
9 hours ago
I think this is the main thing preserving the (miserable) jobs of people at call centers.
But I'm not convinced. With today's models, is it really that much easier to defraud an AI agent than a human?
I do think a lot of people are much more WILLING to try and defraud an AI agent than a person, so maybe that's the difference. There's a parallel there to people being much more willing to mess with Waymos. I think both are a huge shame.
Largely this can be solved the same way it is at a lot of call centers: first line support isn't actually given permission to do much more than the customer can do themselves.
I wouldn't exactly call that "solved", this still sucks for the customer.
I think the ideal solution would be to track and punish fraudsters, but there seems to be little political will to do anything about petty crime these days.
The difference is AIs are identical and don't learn. If you figure out how to defraud a human once it may not work on the next guy. If you figure out how to defraud an AI once you can repeat it indefinitely until the AI is shut off.
Fortunately they can escalate to a generally intelligent model, opus deception benchmarks are approaching 0%.
Intelligence has little to do with it, Opus relies on prompt injections done by a pretty zealous classifier or a cascade of classifiers. It has a ton of semi-black box machinery around it, which can be taken apart with enough understanding of what you're doing. Despite Anthropic's benchmarks and claims, Claude was always the easiest to deceive or JB compared to e.g. GPT, which the RP community knows pretty well. They definitely improved it in 5.5 (due to distillation fears, ironically), but I'm not trusting their or anyone else's numbers this time either. These benchmarks always look good, and then there's someone routinely jailbreaking it layer through layer just for fun.
Still, all instances of Opus are identical and do not learn. As long as the rate of mistakes is nonzero, my point stands.
2 replies →