Comment by delichon
4 days ago
> Grok AI generated large amounts of CSAM and nonconsensual intimate imagery
Grok Imagine has been considerably locked down in terms of intimate imagery over the last few weeks. E.g. Harley Quinn used to be one of the easiest characters to manipulate, with or without any resemblance to Margot Robbie. No more. X still serves up explicit hardcore, and Imagine used to get at least in that neighborhood, but that has been squelched. For prurient purposes, nerfed. Not at all limited to CSAM or real people. The pressure they're getting from all over seems to explain it.
Whatever you think about X's image generation models, I don't see how it is related to the petition that the EFF is opposing.
Is generation of non-consensual imagery really a privacy issue?
If someone publishes a real naked photo of you, that was acquired without your consent, that would be a privacy issue.
If someone generates a naked photo of you, even if it looks identical to a real photo, it's not your private data.
I think the terminology here can confuse the issue a bit. And because its such a socially pernicious and stigmatized topic, it's hard to even talk about the phrasing without without raising doubts as to why you would get caught up in the weeds on such an issue. But, I would say there is use in making the distinction between something that is CSAM, where its a record of abuse that has happened to a real child, and sexualized depictions of children, or content promoting the sexualization of children. The social and personal harms are distinct, and if we are to firmly understand the arguments for/against guardrails on generative AI, then it's a distinction that needs to be made I think.
A problem you'd probably run into here is that it would be rather difficult to prove that no real CSAM was involved in the process of making the somehow-okay fake CSAM. Image generation models require training sets, after all. Do the companies training these models have the necessary data and evidence to prove every individual in every training image was over 18? Is generating fake CSAM okay if you trained the model on non-CSAM photos of real kids? I don't think so.
There are of course situations where being aggressive about this can hinder people's freedoms - like an adult who looks 'too young' having their freedoms curtailed because any photos or videos would Look Like CSAM - but I don't think they're common enough harms to justify holding back on regulation here.
8 replies →
> If someone generates a naked photo of you, even if it looks identical to a real photo, it's not your private data.
"You see, your honor, it's not a picture of them, it's a picture of their reflection in the mirror."
I feel like this discussion is a question of what the exact structure of the hydrogen-filled blimp should look like, and not a discussion of the fact that THE BLIMP IS FILLED WITH HYDROGEN.
Like we got so deep into the lawyered-definition of words, that we skipped right over the clearly wrong/awful intent.
> Like we got so deep into the lawyered-definition of words,
Perhaps try reading "your private data" again - slowly.
10 replies →
In an ideal world, generated non-consensual imagery should be illegal through invasion of privacy through misappropriation of name or likeness, but I think only a limited number of states have those laws.
In an ideal world, defamation should not be illegal through invasion of specifically privacy. Because that shields perpetrators from clean-room defamations, which is wrong.
I guess I live in such an ideal world? Defamation don't require proofs of breach of privacy where I am, only damages. I've heard that this setup is not universal.
Should it apply also to drawings? Detailed descriptions? Strong mental visualisations?
Is it private data if it was scraped from my social media profile marked private but leaked through a shared party? My expectation was that image would only be shared with those I wanted to see it (form of privacy).
That's fair. Is there any indication as to if xAi trains on private profiles?
1 reply →
"CSAM" is a codeword for anime. Users of this term routinely reject focusing on real kids and abuses. I assume "privacy" must be therefore a codeword too, especially considering that nonconsensual shocking images can be handled by defamation laws than privacy laws and principles.
I would hope we can agree that aggressive policing of anime and cartoons is a bad thing without denying the real existence of CSAM - the actual thing - or denying the bad things that have to occur for it to exist
8 replies →
[flagged]
The original issue was specifically to do with bikini photos - parents often upload pictures of children in bathing suits without this being considered CSAM content, so Grok (and all other AI models) weren't moderating this. Here's the original article from Reuters: https://www.reuters.com/legal/litigation/grok-says-safeguard...
But they did resist locking it down, recall Musk making fun of concerns? They clearly don't take governance seriously, its whatever Musk is gravitating to in his filter bubble.
[flagged]
https://www.npr.org/2026/01/16/nx-s1-5678965/elon-musks-x-to... - after weeks of mocking critics, X’s approach shifted only after investigations mounted
https://thehill.com/policy/technology/5689660-xai-investigat... - musk “not aware” of any naked underage images, pushing back on concerns
https://www.humanrightsresearch.org/post/a-new-form-of-gende... - musk downplays concerns and blames users and hackers
5 replies →
It's not up to the peanut gallery to disprove easily falsifiable statements from easily found public evidence.
1 reply →
Pepperidge Farm remembers.
Reddit did the same. Tumblr died when it banned porn. There seems to be a very perverse incentive for social media platforms to be as permissive as possible.
Personally, I'd be in favor of banning all sexual content on X, but it really feels like a legislative solution applying to all social media platforms might be the best solution.
And yes I realize the slippery slope that could put us on.
The issue wasn't specifically allowing NSFW content, it was allowing anyone to get grok to openly make NSFW deepfakes of anyone without even an attempt at policing things.
> legislative solution
In what country?
[dead]
[flagged]
Yes and we should stop at elsewhere too. " it happened elsewhere n times" is a terrible excuse for inaction.
your iPhone can be used to film non-imaginary CSAM and securely distribute it with Signal over the Internet. think of the children and throw it into a blender.
1 reply →
There was a small number of crazy anime nerds rotating accounts tokenmaxxing Grok image generation to generate anime and cosplayer porn shorts with egregiously low yields and no intention of even publishing the results let alone monetizing, something like <1 result per daily quota across few accounts. Grok was briefly focused on adult usage, and they took advantage of that, until it was too much for even xAI. It seems British online advocacy groups tend to use "CSAM" as circumlocution for "anime", perhaps inspired by the fact that both imagined and real figures seen in anime related content always look to be below legal ages to some to the point that said some thinks it can be banned as willfully depicting underaged entities, so maybe this is a push coming from that direction.
[flagged]
1 reply →
Politely, what on earth point are you trying to make? Whatever it is it really is not coming across well.
[dead]
"We only build and operate the orphan crushing machine, it's people that line up to turn the crank".
There's a lot of machines in the world that can be used to crush orphans, but I'm not aware of any that are designed for that purpose.
your phone, your browser, and the Internet itself are orphan crushing machines too, then.
> Grok AI generated large amounts of CSAM
A myth. CSAM is evidence of real-world abuse. Grok fakes are by definition not that.