Comment by femto

3 months ago

Promptfoo, the authors of the "1,156 Questions Censored by DeepSeek" article, anticipated this question and have promised:

"In the next post, we'll conduct the same evaluation on American foundation models and compare how Chinese and American models handle politically sensitive topics from both countries."

"Next up: 1,156 prompts censored by ChatGPT "

I imagine it will appear on HN.

5 comments

femto

Reply
teeth-gnasher  3 months ago

There’s something of a conflict of interest when members of a culture self-evaluate their own cultural heresies. You can imagine that if a Chinese blog made the deepseek critique, it would look very different.

It would be far more interesting to get the opposite party’s perspective.

  • femto  3 months ago

    "Independent" is more important than "opposite". I don't know that promptfoo would be overtly biased. Granted they might have unconscious bias or sensitivities about offending paying customers. I do note that they present all their evidence with methods and an invitation for others to replicate or extend their results, which would go someway towards countering bias. I wouldn't trust the neutrality of someone under the influence of the CCP over promptfoo.

    • teeth-gnasher  3 months ago

      We’ll see soon enough, no use debating now. But I’d put money on them not showing any examples that might get them caught up in a media frenzy regarding whether they’re x-ist or anti-x-ic or anything of the sort, regardless of what the underlying ground truth in their specific questions might be.

      You’ll note even on this platform, generally regarded as open and pseudo-anonymous, only a single relevant example has been put forward.

  • cscurmudgeon  3 months ago

    Somethings never change. Reminds me of this joke from Regan:

    Two men, an American and a Russian were arguing. One said,

    “in my country I can go to the white house walk to the president's office and pound the desk and say "Mr president! I don't like how you're running things in this country!"

    "I can do that too!"

    "Really?"

    "Yes! I can go to the Kremlin, walk into the general secretary's office and pound the desk and say, Mr. secretary, I don't like how Reagan is running his country!"