Comment by Gatsky
4 years ago
What do you think is the ratio of innocent to nefarious pictures of naked children Google encounters in aggregate?
This is relevant to how outraged one should be by this story. I think it is probably > 1:100000. As such, probably not much outrage is warranted, although it’s obviously not great for this one guy.
Wow, I'd guess the opposite when we consider the base rate. Seems like a classic Bayesian problem.
It's fairly normal for parents to take pictures of their children naked in the bathtub/at the beach/ camping/etc.
Conversely, I'd expect actual pedophiles and CSAM producers to be really quite rare.
So even a relatively low base-rate of normal parents with normal nude photos would likely dwarf CSAM upon detection.
So, if we say 1/100 are pedophiles, and 30/100 are parents, and of the parents 10% have such photos, the ratio I'd expect without getting into detection rates is like 3:1 in favor of normal parents.
> It's fairly normal for parents to take pictures of their children naked in the bathtub/at the beach/ camping/etc.
And what I understand is that each of these private photos of their children that parents take on an Android phone with Google cloud enabled gets specifically flagged to be shown to a stranger working for Google.
That sounds pretty insane to me.
What's more a "professional" CP distributor would at the least encrypt or compress their photos, and probably not even use Gmail, maybe use a hijacked cracked account.* So that would lower the "true" positive rate even further.
*In fact that seems like it could be an effective form of extortion or swatting. Someone with access to your credentials could (threaten to) send CP over your account and ruin your life. Or destroy a small business.
This quite the insane realization, and it needs to be shouted from the rooftops every time the encryption "debate" comes up. Authoritarians browbeat us by overstating the diffuse possibility of harm, while whitewashing the effects of their own actions. Meanwhile their actions create centralized, institutionalized, inescapable harm - specifically here, creating a ripe target for pedophiles to remotely creep on everyday families just trying to go about their lives.
I don't think it is even legal for google to watch my photos, even if i accepted everything they said, and permitted them to do so.
1 reply →
If the types of photos you mention were getting flagged as per the case in the article, we would be hearing about a lot more cases. The case itself involved a medical photo, not smiling children at the beach. As such the denominator you use isn’t relevant here.
There is a lot of child pornography in image sharing platforms. Facebook, which reports most reliably, had 20 million CSAM reports in 2021, and that’s photos posted to a social network or sent via messenger, not even in private albums. As I understand it CSAM is more focused on reporting re-uploads or transmission of known abuse material, rather than new material. So I still maintain that if we take the type of image referenced in the article as the denominator, vastly more of such photos would be child abuse material. Granted, 1:100000 is too high, I would revise that down to 1:1000.
Some fair points, but I have a couple of counter points...
One, We really need to know the base rates and false positive rates to reason well here.
Two, I think you'd also need to consider the number of people and number of photos. So like, a photo can be considered a trial, OR flagging a person can. I would imagine a given pedophile has a large number of CSAM photos, but there are few pedophiles. There are however lots of normal parents with fewer photos that might trigger a false positive, but because of that base-rate issue, even if the probability that a given photo is falsely identified as CSAM is quite low, the probability that a given person is falsely flagged/reported may NOT be low.
Also, according to the article, Google trained the AI on nude bathtub photos so that they wouldn't (or at least shouldn't) get flagged.
If it wasn't clear, people aren't objecting to the use of automated tools to prevent crime. It's that there is absolutely no avenue of appeal or review against it even if the law enforcement exonerates them and a big news media reports it.
Google has inserted itself into almost all spheres of digital life by hook or crook. It's practically difficult to avoid them in many services - especially email. And now they play judge, jury and executioner. I don't understand any of these are acceptable, much less justifiable. The old argument 'think of the kids' used to justify digital authoritarianism is such a cliché by now.
> big news media reports it
The guy in the article works in the tech industry and lives in a city where a lot of journalists live, so he could get his story into the media.
What about others?
I’d put pretty good money on almost every family having pictures of their naked kids doing some shenanigans. I know I have those pictures. My parents have those pictures of me and my sister. Quite a few, as I seemed to enjoy trying to run about naked…
Bill Waterson somehow managed to sneak watercolor paintings of a naked little boy into every major newspaper under the guise of being a “comic strip”— the perv.
A lot of people don't, specifically because they are aware of the tremendous risk of an insane government or corporation ruining their life over innocent behavior.
Dare I say that it is a peek into how Black people feel when they go out in public to do...anything.
Somehow I think there more parents who sometimes need to take a photo of their naked children than there are paedophiles.
Or at least the ratio is clearly not 1:100000, maybe more like closer to 1:10.
You would need statistics how many times google have reported police and how many times it have turned out to be a false alarm. Does google even keep record of false alarms? Most likely they don't to avoid responsibility.
I'd expect it to be a lot lower than 1:10. All parents have nude children at some point, they're sort of made that way. The number of pedophiles is higher than anyone wants it to be but it's not 1:10 the number of parents or they'd be a voting bloc. Probably 1:10000
Well as a data point I have pictures of my children naked. As another data point my parents have photos of me as a child naked, and as a third data point my grandparents have photos of themselves as children naked. Whereas I don't knowingly know any paedophiles.