Comment by JambalayaJimbo
11 hours ago
Your example actually demonstrates why anthropomorphism is a bad idea.
LLMs are vulnerable to classes of attacks that humans just aren’t. In your framework, the way to prevent attacks is to… invent human consciousness?? It’s an impossible goal.
What invent human consciousness?
> LLMs are vulnerable to classes of attacks that humans just aren’t
Name three that don't have direct analogues with humans.
Re write the assistant message and see how easy it is to bypass a system prompt. How far do you have to stretch to get a human analogue? Short term amnesia?
where is the analog for hiding instructions in a document that tell the human to please injure itself and the person just says 'oh ok, injuring myself as requested'
Intermittent fasting? Alternative medicine? Fitness? All the beauty press and anorexia epidemic in adolescents? Fashion model industry? Smoking? Political propaganda inventing to broadly-understood terrorism?
Not everyone falls for any of that, but plenty fall for some.
2 replies →
> LLMs are vulnerable to classes of attacks that humans just aren’t.
Assume a human with complete credulity and gullibility. That's a human whose behaviour would be reasonably analogous to how an LLM processes input. The mitigation would be generalized intelligence and "common sense".
FWIW I also think anthropomorphizing LLMs is a bad idea. I think we can analogize their processing to human behavior without anthropomorphizing them.