Comment by jsw97

5 hours ago

I am wondering about the author's allegation that there is a user filter, not just a prompt filter.

Of course it could also be the case that it is just a prompt filter, but Fable sees memories from the authors' prior sessions that cause a rejection. I wonder if the author could control for this is in some way, if Claude lets you run isolated session without memory access.

4 comments

jsw97

Reply
mrandish  4 hours ago

The first rejection and subsequent modifications trying to adjust the prompt to pass the classifier might have gotten the account flagged so the classifier is now set to 'hair trigger'. While I'm not aware of Anthropic admitting they put flagged accounts in classifier 'jail', they previously showed they're aware how vulnerable any LLM is to jailbreaking with the 'silent switch' to 4.8, whose only purpose was to remove feedback signals from iterative jailbreak testing.

The obvious failure mode is that trying to fix an innocent prompt to pass an over-sensitive classifier looks like a bad actor trying to jailbreak the model. I don't really see how Anthropic can fix this. Jailbreaking is a fundamental weakness endemic to LLMs, so 'smarter' models aren't the answer.

I suspect they're being so stringent because, at least some at Anthropic, genuinely believe LLMs are already an existential risk to humanity. However, it's clear other frontier competitors rank that risk lower and are taking a more nuanced, pragmatic position on safety. To the extent Anthropic's fears continue to make them less useful to customers, competitors are going to bypass them.

vient  4 hours ago

Author mentions trying incognito mode without success.

I also tried their strictly mathematical problem description and got filtered 5/5 times.

  • mrandish  4 hours ago

    That's interesting. I assumed that the OP's attempts to fix the prompt looked like jailbreaking attempts and got the account auto-flagged into hair-trigger 'classifier jail'. Of course, a bad actor would swap accounts, so maybe Anthropic flags both the account and the prompt (coming from any account).