Comment by Hackbraten
1 month ago
> The chance of a hash colliding is near 0%
Until someone finds a successful collision attack.
> Even so a human is in the loop to review what got a hit.
Until shareholder/growth pressure causes them to replace that human with an AI.
> Until someone finds a successful collision attack.
Indeed, and within hours of the hash function being made available to me I developed a second preimage attack (strictly stronger than a collision attack)... allowing me to modify images in a visually non-objectionable way to match an arbitrary hash value.
> Until shareholder/growth pressure causes them to replace that human with an AI.
Indeed, but more than that:
The "human" is still a dire privacy loss. Perhaps Apple's review might have protected you from some erroneous reports to law enforcement, but does it protect you from an apple-employed stalker ex? does it protect you from paparazzi? Does it protect you from the double thinking ("do I photograph my kids playing in the sprinklers? do I take a nude photo of my spouse?") due knowing that your private activity is being watched?
One could easily argue that some AI second level review is an "improvement", which is another reason why your assumption that even that backstop would eventually be removed is a reasonable one.
> Until someone finds a successful collision attack.
The only way to get a successful collision attack is to have a picture/video that is stored in that hash database.
To put it in perspective. You randomly picking the same atom twice in the observable universe has a higher chance than getting a mismatch.
> Until shareholder/growth pressure causes them to replace that human with an AI.
How do you think it will impact shareholders to know that the company is not actively scanning for illegal content.
Also it's Interpol/FBI that get involved.