← Back to context

Comment by ablob

18 hours ago

The mentioned accuracy in the comment you are replying to already encapsulates the relation of true positives to false positives.

5 comments

ablob

Reply
cortesoft  3 hours ago

It really doesn’t, and it is easy to demonstrate by using an extreme example.

Suppose I invent a device that can detect whether there is a giant invisible dragon living in your house, and it has an accuracy of 99.999%

Now, I use it in your house and it tells me there is an invisible dragon… so what are the chances that there is a dragon in your house?

Based on your statement, it would be 99.999% likely that there is an invisible dragon in your house. However, we actually know that there is a 0% chance there is an invisible dragon, so even with the positive test result we still know there is a 0% chance a dragon is there.

fc417fc802  15 hours ago

No I don't believe it does. I interpret 99.9% accurate to mean 1 in 1000 false positives. If 0.1% of your population are terrorists that means each alert has a 50% chance of being correct. That's nowhere near good enough to fully automate things but it is quite reasonable assuming this is merely information provided to a human agent.

Whereas if only 0.001% of your population are terrorists then 99 out of 100 alerts are false positives at which point the system is well on its way to being useless.

There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.

  • ablob  10 hours ago

    You're right it doesn't. At least not completely. I was thinking about precision (i.e.: if the test is positive, what are the odds that its prediction is true). It turns out, that accuracy is not defined as "true positive / (true pos. + true neg.)", but "correct predictions / all predictions". The whole point of OP's statement: "It's kind o remarkable how even a 99.9% accurate heuristic is insufficient at scale.", which you actually support with your example.

    > There is an important difference between scenarios where we care about the relative versus absolute frequency of errors.

    The context is chat control without probable cause over the whole population of Europe with a low prevalence. My point, and presumably that of OP, is that even a small relative frequency of errors will yield an unsustainably high absolute frequncy of errors.

    > This is merely information provided to a human agent.

    It will be in theory. In practice the human agent will just forward the decision. A human agent is not sufficient; you need to test only with probable cause for the kind of scenario we're talking about. The exact opposite of "Chat Control 1.0 and 2.0".

    P.S.: The comment I originally replied to choose a very convoluted way of saying that the false discovery rate of the test matters for a proper evaluation. Both you and they explain this by throwing numbers without context in combination with slightly inaccurate definitions. I got the definitions mixed up differently, which led to this follow-up.

    • fc417fc802  8 hours ago

      I think we largely agree about being opposed to chat control however we seem to disagree somewhat about the underlying reasoning leading us to that conclusion.

      > even a small relative frequency of errors will yield an unsustainably high absolute frequncy of errors.

      That depends entirely on the rate of true positives in the general population and the rate at which the test successfully catches them. If the success rate is reasonably high and the rate of true positives is within one base ten order of magnitude of the rate of false positives then regardless of volume the stream of reports would be expected to prove quite useful.

      To put this in concrete terms, if 1 billion messages are scanned, there are 100 violations, 99 of those violations are successfully detected, and there are an additional 1000 false positives reported, then you've got about a 10% hit rate when examining reports. That would provide a genuinely useful starting point.

      But it's not at all clear that we can expect numbers like that. Both because the scanners are likely much worse but also because criminals can't reasonably be expected to stick around on conforming platforms in the event that such measures are enacted.

      Even if the reports were 100% accurate I'd still be opposed to it on ideological grounds. I don't think pervasive surveillance of that nature is compatible in the long term with a free and democratic system of government.

      > Both you and they explain this by throwing numbers without context in combination with slightly inaccurate definitions.

      It was my intent to provide reasoning for all the numbers I put forward. They were meant as examples.

      As to definitions I wasn't going by anything formal. I tried to spell out exactly what I meant by each term. Apologies if I wasn't entirely clear about that. Regardless, the precise definitions of the terms aren't what matters here. It's the practical end result - what percentage of the alerts are false?