Comment by MoltenMan
20 hours ago
The problem is that this is an incredibly niche / small issue (i.e. <<1% of users, let alone prompts, need this clarification), and if you add a section for every single small thing like this, you end up with a massively bloated prompt. Notice that every single user of Claude is paying for this paragraph now! This single paragraph is going to legitimately cost anthropic at least 4, maybe 5 digits.
At some point you just have to accept that llm's, like people, make mistakes, and that's ok!
>The problem is that this is an incredibly niche / small issue (i.e. <<1% of users, let alone prompts
It's not a niche issue at all. 29 million people in the US are struggling with an eating disorder [1].
> This single paragraph is going to legitimately cost anthropic at least 4, maybe 5 digits.
It's 59 out of 3,791 words total in the system prompt. That's 1.48%. Relax.
It should go without saying, but Anthropic has the usage data; they must be seeing a significant increase in the number of times eating disorders come up in conversations with Claude. I'm sure Anthropic takes what goes into the system prompt very seriously.
[1]: from https://www.southdenvertherapy.com/blog/eating-disorder-stat...
The trajectory is troubling. Eating disorder prevalence has more than doubled globally since 2000, with a 124% increase according to World Health Organization data. The United States has seen similar trends, with hospitalization rates climbing steadily year over year.
Your source says "Right now, nearly 29 million Americans are struggling with an eating disorder," and then in the table below says that the number of "Americans affected in their lifetime" is 29 million. Two very different things, barely a paragraph apart.
I don't mean to dispute your assertion that it's not a niche issue, but that site does not strike me as a reliable interpreter of the facts.
[dead]
It's not "incredibly niche" when you consider the kinds of questions that average everyday users might submit to these AIs. Diet is definitely up there, given how unintuitive it is for many.
> At some point you just have to accept that llm's, like people, make mistakes, and that's ok!
Except that's not the way many everyday users view LLM's. The carwash prompt went viral because it showed the LLM making a blatant mistake, and many seem to have found this genuinely surprising.
People think these LLM's are anthropomorphic magic boxes.
It will take years until the understanding sets in that they're just calculators for text and you're not praying to a magic oracle, you're just putting tokens into a context window to add bias to statistical weights.
The Claude prompt is already quite bloated, around 7,000 tokens excluding tools.
Worse, it reveals the kind of moralistic control Anthropic will impose on the world. If they get enough power, manipulation and refusal is the reality everyone will face whenever they veer outside of its built in worldview.
I think it actually reveals how they don't want to be sued for telling somebody's teenage daughter with an eating disorder to eat less and count her calories more.