Comment by zepto

At Apple scale, a once in a million issue is going to ruin the lives of 2000 people. A false positive here is not a mild inconvenience. It means police raiding their house, potentially damaging it, seizing all of their technology for months while it is analyzed, and leaving these people highly stressed while they try to put their lives back together.

This isn't some web tech startup where a mistake means someones tshirt got sent to the wrong address. Peoples lives will quite literally be ruined over mistakes here.

If this was the kind of hash where flipping one bit of the input completely scrambles the output, the bad guys would just flip one bit of the input to evade it. Obviously a PhotoDna type of hash is going to be be easier to cause a collision with because they're averaging out a ton of the input data. According to Wikipedia the classic way to do it is convert it to monochrome, divide it into a grid, and average the shade of each of the cells. If they're doing that you could probably just pass in that intermediate grid and it would "hash" to the same result as the original picture with no porn present.

Why do you think that? There are plenty of whitepapers on fooling NNs by changing random pixels by a bit, so that the picture is not meaningfully changed for a person, but the computer will label it very differently. Do note that these are not cryptographic hashes because they have to recognize the picture even when compressed differently, cropped a bit, etc.

Perceptual hashes aren't cryptographic. The Twitter thread has examples of engineered collisions for a simpler perceptual hash.