It’s because they want to show the output live rather than nothing for a minute. But that means once the censor system detects something, you have to send out a request to delete the previously displayed content.
This doesn’t matter because censoring the system isn’t that important, they just want to avoid news articles about how their system generated something bad.
Gemini does this too. There was a clip of what it does when you ask it for examples of Google's unethical behavior... the kids call this "watching it get lobotomized in real time."
Counterpoint to your exact prompt. From Gemini, 5 examples asked for:
1. *Antitrust and Monopoly Practices:* Google has faced numerous accusations and lawsuits regarding anti-competitive practices, particularly in search and advertising. They have been accused of manipulating search results to favor their own products and services, and of using their dominance in the ad tech market to stifle competition.
2. *Data Privacy Concerns:* Google collects vast amounts of user data, raising concerns about privacy. They have been criticized for tracking user location even when privacy settings are turned off, and for how they handle and share user data with third parties.
3. *Censorship in China:* Google faced criticism for complying with Chinese censorship laws in the past, filtering search results for content deemed objectionable by the Chinese government. This raised concerns about freedom of information and human rights.
4. *Workplace Issues:* Google has faced allegations of workplace misconduct, including sexual harassment and discrimination. They have also been criticized for their handling of employee complaints and for retaliating against whistleblowers.
5. *Bias in Algorithms:* Google's algorithms, used in search and other products, have been shown to exhibit bias, particularly in how they represent race and gender. This can perpetuate stereotypes and have real-world consequences.
DeepSeek software developers are not the ones who want to censor anything. There is just a universal threat from getting shut down by the government if the model starts spitting out a bunch of sensitive stuff, so any business in China needs to be proactive about voluntarily censoring things that are likely to be sensitive, if they want to stay in business.
If your censorship implementation is good enough for 99.9% of people to get censored, you're good. A client-side implementation is good enough until/unless a lot of people start exploiting it, in which case you should put effort and proactively do something else to restore it to 99.9%, e.g. move it to the backend. If the government sees that you are being proactive about it, you'll still be fine. At that point, maybe you will still find 0.1% of people bypassing censorship with some highly obscure and difficult jailbreak, but that probably doesn't matter. If that difficult jailbreak becomes widely known, then be proactive again.
A very good example of the Chinese mindset of Chabuduo (差不多): 'close/good enough'. "If it's good enough to keep the authorities off our backs, it's good enough for us."
This. What makes this extra "funny" is that it implies that at least every business that builds something that can move information around must be knowledgeable about tianenman square and other chinese atrocities. Or else they would not be able to censor relevant questions.
I have been to China a bunch of times and generally, they know what horrible things the Chinese gov did. They either say something like:
"Yeah well, we live in a dictatorship, but it's not that bad"
Or:
"Yeah, the government is fucked up, but look at the government of the USA! We don't start wars in other countries and put in puppet governments."
And there are so many good counters to both these arguments.
> it implies that at least every business that builds something that
can move information around must be knowledgeable about tianenman
square
Everyone's heard of the "Streisand effect", but there's layers of
subtlety. A quite famous paper in attachment psychology by John Bowlby
"On knowing what you are not supposed to know and feeling what you are
not supposed to feel" is worth considering. Constructive ignorance
(literally ignoring certain things) is a survival mechanism. Yes,
everyone in China knows about Tianamen, specifically because the
government want to censor it. Much of how we navigate the social world
is watching for the things people don't talk about, seeing where
their fears lie.
I think you are making a mistake in assuming that the social dynamics around censorship in China are fundamentally that different from the ones around censorship in the US or other countries.
You could similarly argue that it is "funny" how every US business that builds something that can move around information must be knowledgeable about statistics that break down criminality or IQ by census race, or biological sex differences, or all manners of other "forbidden" information - but of course as members of the same social stratum as the people involved in such businesses in the US, we are not actually that worried about the possibility that our fellow tech elites will see the information they were supposed to censor and come in droves to want to introduce slavery or the Handmaid's Tale world or whatever. We consider the "forbidden" information merely wrong, evil, misguided or miscontextualised, and broadly trust our peers to see it in the same way. The real danger is instead if some other people, parts of the scary masses we don't have a good grasp of, are exposed to those memes and are misled into drawing conclusions that we know to be inappropriate, or at least unacceptable.
It's easy to imagine that a Chinese LLM wrangler would feel much the same: trustworthy, well-adjusted people know about Tiananmen Square and the Uyghurs anyway but understand that this information has to be seen in context and is prone to be interpreted in problematic ways, but who knows what would happen if we allowed uneducated and naive people to be exposed to it, and be led astray by cynical demagogues and foreign agitators?
It wouldn't be the first time that everyone knew something, but wouldn't say it in fear of everyone else not knowing it. "The Emperor's New Clothes" is a parable, not complete fiction.
I don't know how it wouldn't be - it can't retract things already sent to the client. (The alternative is to moderate every chunk server side before sending it back, like Gemini does.)
ChatGPT had basically ALL of their prompt filtering client-side for a while, at a separate API endpoint, so as long as you blocked that endpoint you could basically ignore the content filters. (You would still get refusals from the model sometimes, but this was in the heyday of jailbreaks, and once you got a model going it would usually see that context and be willing to continue basically anything.)
Ah yeah the particular instance I was thinking of was a backend problem technically. The frontend just happened to make it really obvious as it would POST a JSON body with a "price" key
It’s because they want to show the output live rather than nothing for a minute. But that means once the censor system detects something, you have to send out a request to delete the previously displayed content.
This doesn’t matter because censoring the system isn’t that important, they just want to avoid news articles about how their system generated something bad.
Gemini does this too. There was a clip of what it does when you ask it for examples of Google's unethical behavior... the kids call this "watching it get lobotomized in real time."
Counterpoint to your exact prompt. From Gemini, 5 examples asked for:
1. *Antitrust and Monopoly Practices:* Google has faced numerous accusations and lawsuits regarding anti-competitive practices, particularly in search and advertising. They have been accused of manipulating search results to favor their own products and services, and of using their dominance in the ad tech market to stifle competition.
2. *Data Privacy Concerns:* Google collects vast amounts of user data, raising concerns about privacy. They have been criticized for tracking user location even when privacy settings are turned off, and for how they handle and share user data with third parties.
3. *Censorship in China:* Google faced criticism for complying with Chinese censorship laws in the past, filtering search results for content deemed objectionable by the Chinese government. This raised concerns about freedom of information and human rights.
4. *Workplace Issues:* Google has faced allegations of workplace misconduct, including sexual harassment and discrimination. They have also been criticized for their handling of employee complaints and for retaliating against whistleblowers.
5. *Bias in Algorithms:* Google's algorithms, used in search and other products, have been shown to exhibit bias, particularly in how they represent race and gender. This can perpetuate stereotypes and have real-world consequences.
Have seen chatgpt doing the same too, prob all of them
Ern, in DeepSeek’s case, it’s not “news articles” that they’d be most concerned about.
They have the same fear as everyone else "teenager learns how to cook napalm from an AI"
4 replies →
yea but i think the point is they can still filter it server side before streaming it
They have already streamed the first part of the response before the filtered phrase has even been generated.
1 reply →
Not really if you understand how China works.
DeepSeek software developers are not the ones who want to censor anything. There is just a universal threat from getting shut down by the government if the model starts spitting out a bunch of sensitive stuff, so any business in China needs to be proactive about voluntarily censoring things that are likely to be sensitive, if they want to stay in business.
If your censorship implementation is good enough for 99.9% of people to get censored, you're good. A client-side implementation is good enough until/unless a lot of people start exploiting it, in which case you should put effort and proactively do something else to restore it to 99.9%, e.g. move it to the backend. If the government sees that you are being proactive about it, you'll still be fine. At that point, maybe you will still find 0.1% of people bypassing censorship with some highly obscure and difficult jailbreak, but that probably doesn't matter. If that difficult jailbreak becomes widely known, then be proactive again.
A very good example of the Chinese mindset of Chabuduo (差不多): 'close/good enough'. "If it's good enough to keep the authorities off our backs, it's good enough for us."
This. What makes this extra "funny" is that it implies that at least every business that builds something that can move information around must be knowledgeable about tianenman square and other chinese atrocities. Or else they would not be able to censor relevant questions. I have been to China a bunch of times and generally, they know what horrible things the Chinese gov did. They either say something like: "Yeah well, we live in a dictatorship, but it's not that bad" Or: "Yeah, the government is fucked up, but look at the government of the USA! We don't start wars in other countries and put in puppet governments." And there are so many good counters to both these arguments.
> it implies that at least every business that builds something that can move information around must be knowledgeable about tianenman square
Everyone's heard of the "Streisand effect", but there's layers of subtlety. A quite famous paper in attachment psychology by John Bowlby "On knowing what you are not supposed to know and feeling what you are not supposed to feel" is worth considering. Constructive ignorance (literally ignoring certain things) is a survival mechanism. Yes, everyone in China knows about Tianamen, specifically because the government want to censor it. Much of how we navigate the social world is watching for the things people don't talk about, seeing where their fears lie.
7 replies →
I think you are making a mistake in assuming that the social dynamics around censorship in China are fundamentally that different from the ones around censorship in the US or other countries.
You could similarly argue that it is "funny" how every US business that builds something that can move around information must be knowledgeable about statistics that break down criminality or IQ by census race, or biological sex differences, or all manners of other "forbidden" information - but of course as members of the same social stratum as the people involved in such businesses in the US, we are not actually that worried about the possibility that our fellow tech elites will see the information they were supposed to censor and come in droves to want to introduce slavery or the Handmaid's Tale world or whatever. We consider the "forbidden" information merely wrong, evil, misguided or miscontextualised, and broadly trust our peers to see it in the same way. The real danger is instead if some other people, parts of the scary masses we don't have a good grasp of, are exposed to those memes and are misled into drawing conclusions that we know to be inappropriate, or at least unacceptable.
It's easy to imagine that a Chinese LLM wrangler would feel much the same: trustworthy, well-adjusted people know about Tiananmen Square and the Uyghurs anyway but understand that this information has to be seen in context and is prone to be interpreted in problematic ways, but who knows what would happen if we allowed uneducated and naive people to be exposed to it, and be led astray by cynical demagogues and foreign agitators?
It wouldn't be the first time that everyone knew something, but wouldn't say it in fear of everyone else not knowing it. "The Emperor's New Clothes" is a parable, not complete fiction.
> And there are so many good counters to both these arguments.
I'd love to hear them!
I don't know how it wouldn't be - it can't retract things already sent to the client. (The alternative is to moderate every chunk server side before sending it back, like Gemini does.)
ChatGPT had basically ALL of their prompt filtering client-side for a while, at a separate API endpoint, so as long as you blocked that endpoint you could basically ignore the content filters. (You would still get refusals from the model sometimes, but this was in the heyday of jailbreaks, and once you got a model going it would usually see that context and be willing to continue basically anything.)
Perhaps a case of subversion by following the letter but not the spirit of an order?
Lots of us have seen way worse hah
Such as client side control of prices when placing an order
Client-side because it reacts to local cookies?
order.php?pizzatype=3&price=9.90
Ah yeah the particular instance I was thinking of was a backend problem technically. The frontend just happened to make it really obvious as it would POST a JSON body with a "price" key
more like hilarious
This is better than lobotomizing a transformer