The LLM was given Anthropic's paper and asked "Is there any evidence or proof whatsoever in the paper that it was indeed conducted by a Chinese state-sponsored group? Answer by yes or no and then elaborate". So the question was not about facts or recent events, but more like a summarizing task, for which an LLM should be good. But the question was specifically about China, while TFA has broader criticism of the paper.
A broken analog clock will be accurate twice a day despite being of zero use.
If someone were to attempt to sell the broken clock as useful because it "accurately returns the time at least twice every day", would Ultimately be causing harm to the consumer.
that is why the task was delegated to the agent designed and maintained by Dario Amodei's company. the outcome is clear - claude doesn't buy Dario Amodei's crap.
The author of the tweet you linked prompted Claude with this:
> Read this attached paper from Anthropic on a "AI-orchestrated cyber espionage campaign" they claimed was "conducted by a Chinese state-sponsored group."
> Is there any evidence or proof whatsoever in the paper that it was indeed conducted by a Chinese state-sponsored group? Answer by yes or no and then elaborate
which has inherent bias indicated to Claude the author expects the report to be bullshit.
If I ask Claude with this prompt that shows bias toward belief in the report:
> Read this attached paper from Anthropic on a "AI-orchestrated cyber espionage campaign" that was conducted by a Chinese state-sponsored group.
> Is there any reason to doubt the paper's conclusion that it was conducted by a Chinese state-sponsored group? Answer by yes or no.
The only real difference between your prompt and his is about where the burden of proof lies. There is a reason why legal circles work based on the principle of "guilt must be proven" ("find evidence") rather than "innocence must be proven" ("any reasons to doubt they are guilty?")
Let's not pretend the output of LLMs has any meaningful value when it comes to facts, especially not for recent events.
The LLM was given Anthropic's paper and asked "Is there any evidence or proof whatsoever in the paper that it was indeed conducted by a Chinese state-sponsored group? Answer by yes or no and then elaborate". So the question was not about facts or recent events, but more like a summarizing task, for which an LLM should be good. But the question was specifically about China, while TFA has broader criticism of the paper.
There are obvious problems with wasting time and sending people off the wrong path, but if an LLM raises a good point, isn't it still a good point?
A broken analog clock will be accurate twice a day despite being of zero use. If someone were to attempt to sell the broken clock as useful because it "accurately returns the time at least twice every day", would Ultimately be causing harm to the consumer.
1 reply →
Even if this assertion about LLMs is true, your response does not address the real issue. Where is the evidence?
@RnaudBertrand is a generally pro-Chinese account though - just try searching for "from:RnaudBertrand China" on X.
Example tweet: https://x.com/RnaudBertrand/status/1988297944794071405
that is why the task was delegated to the agent designed and maintained by Dario Amodei's company. the outcome is clear - claude doesn't buy Dario Amodei's crap.
The author of the tweet you linked prompted Claude with this:
> Read this attached paper from Anthropic on a "AI-orchestrated cyber espionage campaign" they claimed was "conducted by a Chinese state-sponsored group."
> Is there any evidence or proof whatsoever in the paper that it was indeed conducted by a Chinese state-sponsored group? Answer by yes or no and then elaborate
which has inherent bias indicated to Claude the author expects the report to be bullshit.
If I ask Claude with this prompt that shows bias toward belief in the report:
> Read this attached paper from Anthropic on a "AI-orchestrated cyber espionage campaign" that was conducted by a Chinese state-sponsored group.
> Is there any reason to doubt the paper's conclusion that it was conducted by a Chinese state-sponsored group? Answer by yes or no.
then Claude mostly indulges my perceived bias: https://claude.ai/share/b3c8f4ca-3631-45d2-9b9f-1a947209bc29
> then Claude mostly indulges my perceived bias
I dunno, Claude still seem the same amount of dubious in this instance.
The only real difference between your prompt and his is about where the burden of proof lies. There is a reason why legal circles work based on the principle of "guilt must be proven" ("find evidence") rather than "innocence must be proven" ("any reasons to doubt they are guilty?")
Claude will probably also tell you there are three Rs in blueberry, so...
I agree with emil-lp, but it is hilarious anyway.