Comment by areoform

This is one of the most legible, well-detailed, and well-written article I've seen on perceptual hashing. It must have taken months of effort to pull off, and I'd love to see the author write about other things.

But the article fails to take its statements to their logical conclusion, in one section, he writes,

    > Every false positive means an innocent person's content was flagged — a family photo, a medical image, a piece of art. It means unnecessary investigation, potential harm to reputation, and erosion of trust in the system. At scale, even a 0.01% false positive rate means thousands of wrongful flags per day.

and,

    > In practice, the industry errs heavily toward minimizing false negatives — catching every possible match — and then uses human review to resolve false positives. This means the system flags aggressively but confirms carefully. The cost of a false positive is an investigation. The cost of a false negative is a child.
    > 
    > This is also why the hybrid approach from Chapter VI matters. Perceptual hashing against a verified database has a low false positive rate — but not zero. Certain images (blank, solid-color, simple gradients) produce hashes that collide with database entries by coincidence, not because they depict abuse. Production systems include collision detection to filter these out before matching. Classifiers for unknown material have a higher false positive rate still (the model is making a judgment, not a comparison). By layering them — hashing first, then classifiers, then human review — the system can be both aggressive and precise. But no layer is perfect, and the threshold remains a human decision.

If there is a way to "include collision detection to filter these out before matching" then why do they "then human review?" The author starts the next section with, "Three Steps. No One Sees the Image."

But they do human review to eliminate false positives? Both statements can't be simultaneously true - "no human ever sees it," or "by layering them — hashing first, then classifiers, then human review — the system can be both aggressive and precise."

Secondly, although I'm not a researcher, I think I and a lot of researchers would love to see this "aggressive, but precise algorithm" that eliminates collisions (an imprecise term - while here it means an image of a background or a setting that ticks off the similarity system; it's still not exactly a collision in the classical sense as the algorithm is a type of clustering with hashes) without making the algorithm useless? As far as I'm aware, no such algorithm exists without either becoming useless or having significant false positives. But I might be wrong.

At one point in the article, the author says, "The cost of a false negative is a child." This "aggressive and precise" system diverts resources from actual investigations and prosecution. A few examples,

A very famous case from 2022, https://www.nytimes.com/2022/08/21/technology/google-surveil...

A more precise example, as the author mentions PhotoDNA,

    > LinkedIn found 75 accounts that were reported to EU authorities in the second half of 2021, due to files that it matched with known CSAM. But upon manual review, only 31 of those cases involved confirmed CSAM. (LinkedIn uses PhotoDNA, the software product specifically recommended by the U.S. sponsors of the EARN IT Bill.) 

PhotoDNA's "aggressive and precise" have a 58.6% false positive rate when tested. That means nearly 60% of the cases it generates for investigations wasted investigators time, leading to fewer investigations overall.

from, https://www.eff.org/deeplinks/2022/08/googles-scans-private-...

These systems are also flagging photos of adults,

    > In the process of reporting images, the occurrence of false positives—instances where non-CSAM images are mistakenly reported as CSAM—is inevitable. *One officer told us that there are “a lot” of CyberTipline reports that are images of adults.124* More false positives will mean fewer cases going unreported, and platforms must decide what balance they are comfortable with. False positives and false negatives can be minimized with better detection technology. One respondent criticized platforms for relying on their in-house technology. They perceived those as inferior to solutions offered by start-ups, suggesting that this choice might be driven by profit motives.125 Platforms, however, might have reservations about using third-party services for screening potential CSAM due to legal and ethical considerations. An NGO employee highlighted platform concerns, asking, “Can we trust these organizations? What ethical due diligence have they done?”

via https://purl.stanford.edu/pr592kc5483

The uncomfortable truth is that people are trying to use technology to fix a structural problem. Usually, most victims of CSA (including me) know the abuser. In my case and others, at least one adult knew (or suspected) and did nothing. More maddeningly, even when reported and the CSA is discovered and the perpetrator is punished, the victims are reabused within the foster care system. https://ballardbrief.byu.edu/issue-briefs/sexual-abuse-of-ch... 40% of children in foster care experience some type of abuse. Most never get the help they need.

I think the impulse to create systems to monitor everyone's phones for CSAM comes from a good place. But it's energy misdirected; better investigations into exploitation networks, investment in foster care and care for abused children and teens, heck even child AI companions capable of reporting abuse for children suspected of being abused would lead to better outcomes than scanning everyone's phone.