Comment by ben_w
2 days ago
(Based on https://www.euronews.com/my-europe/2026/07/07/eu-to-extend-t... and https://www.euractiv.com/news/how-the-epp-pushed-the-chat-sc... as well as the stuff in the link).
Here's a quote from the article itself, which works for both pro and con arguments:
"What is still NOT being scanned: End-to-end encrypted chats, such as those on WhatsApp, have always been exempt from these scans. Additionally, European providers of messaging and email services have never implemented chat control measures."
As I'm not trained in law, I have no strong opinions on if this proposal is a net positive or negative, almost any big name LLM will do a better job than I can manage by looking at the legal text, stroking my goatee and saying "I recon…". But what I can say that I've just seen a headline about a class action lawsuit in the USA due to grok making CSAM and the company failing to assist the police in their investigations, and another about Meta facing a lawsuit in India for delivering advertising for CSAM on Instagram.
My steelman in favour of the legislation:
The regulation closes a legal gap that would otherwise force platforms to stop using existing CSAM detection systems; it's a temporary framework that doesn't require universal mandatory scanning or ban E2EE, just keeps the legal basis for companies which choose to use detection/scanners while lawmakers continue negotiating a more comprehensive longterm solution.
My steelman against the legislation:
Scanning private communications, even allowing companies to "voluntary" do this, sets the precedent that the confidentiality of private correspondence is conditional rather than fundamental. Also, automated scanning inevitably has false positives. Also, has chilling effect on free speech, undermines trust in encrypted messaging.
Also, situationally, that it's "voluntary" means offenders can migrate to platforms which don't "voluntarily" do this.
>CSAM detection systems
Blackboxes which scan your messages and photos for anything 3rd party want with undisclosed criteria.
Yes, indeed.
In principle "for anything 3rd party want" would be illegal in the EU. However, Big Tech clearly doesn't care what's illegal in the EU.
Pertinent to this case: https://stateofsurveillance.org/news/big-tech-defies-eu-law-...
Previously: https://www.edpb.europa.eu/news/french-sa-cookies-and-advert...
Even earlier, when they cared about the law: https://www.trtworld.com/article/13092354
> Blackboxes which scan your messages and photos for anything 3rd party want with undisclosed criteria.
Well the criteria is CSAM?
Do you have any evidence that these systems have been used to imprison someone that wasn't CSAM related?
>Well the criteria is CSAM
Could you confirm that in any way?
2 replies →
Real time notifications here would solve a lot of issues...
Imagine Alice, an 18, 19yo girl, having a boyfriend, Bob, and since Bob is on a student exchange, she decides to send him a boob photo. Since alice is skinny, her boobs are on the smaller side.
Now imagine Alice hitting 'Send', and getting an automated message from whatever CSAM AI bot:
"Your message has not been sent, the system detected the breasts in the photo to be probably underage, the photo was forwarded to <your local police station> for manual review"
And half an hour later
"Detectives Rob Johnson, John Robson and Bob Bobson from police department XY, have done an extensive manual review of the photo of the breasts and have 2:1 decided that they're probably not underage, so the photo was sent to the intended destination. Than you, your friendly CSAM AI bot!"
I think you're probably wildly overoptimistic about the ratio of police officers to private nudes.
No government really wants to be fully enforcing all their own laws, just because it's way too expensive to hire that many cops. I think the closest anyone got was the Stasi, and they had a lot of "volunteers": https://en.wikipedia.org/wiki/Unofficial_collaborator#Other_...
I think a more realistic system would be hashing images and comparing them to known CSAM in some database.
I think Apple was going to implement something like this a few years ago before scrapping it.
Such systems are unfortunately trivially defeated by, for example, adding rings of colourful blocks around the edges of the image. If the scanning systems are then updated to notice rings around the image, bad actors will start cutting images in half and adding the colours there, etc. It's a never-ending arms race that's bound to leave regular people worse off.
2 replies →
That assumes no new material is being created.
"But the call was coming from inside the house": https://www.robertkinglawfirm.com/mass-torts/grok-lawsuit/
I don't really want governments, Apple or Google to build databases of CSAM, thank you :/
2 replies →