Comment by klik99
9 hours ago
> You might even convince yourself that these questions are “privacy preserving,” since no human police officer would ever rummage through your papers, and law enforcement would only learn the answer if you were (probably) doing something illegal.
Something I've started to see happen but never mentioned is the effect automated detection has on systems: As detection becomes more automated (previously authored algorithms, now with large AI models), there's less cash available for individual case workers, and more trust at the managerial level on automatic detection. This leads to false positives turning into major frustrations since it's hard to get in touch with a person to resolve the issue. When dealing with businesses it's frustrating, but as these get more used in law enforcement, this could be life ruining.
For instance - I got flagged as illegal reviews on Amazon years ago and spent months trying to make my case to a human. Every year or so I try to raise the issue again to leave reviews, but it gets nowhere. Imagine this happening for a serious criminal issue, with the years long back log on some courts, this could ruin someones life.
More automatic detection can work (and honestly, it's inevitable) but it's got to acknowledge that false positives will happen and allocate enough people to resolve those issues. As it stands right now, these detection systems get built and immediately human case workers get laid off, there's this assumption that detection systems REPLACE humans, but it should be that they augment and focus human case workers so you can do more with less - the human aspect needs to be included in the budgeting.
But the incentives aren't there, and the people making the decisions aren't the ones working the actual cases so they aren't confronted with the problem. For them, the question is why save $1m when you could save $2m? With large AI models making it easier and more effective to build automated detection I expect this problem to get significantly worse over the next years.
>Imagine this happening for a serious criminal issue, with the years long back log on some courts, this could ruin someones life.
It can be much scarier.
There was a case in Russia when a scientist was accused in a murder that happened 20 years ago based on 70% face recognition match and fake identification as an accomplice by a criminal. [0] He spent 10 months in jail during "investigation" despite being incredibly lucky to have an alibi -- archival records of the institute where he worked, proving he was in an expedition far away from Moscow at that time. He was eventually freed but I'm afraid that police investigators that used very weak face recognition match as a way to improve their work performance stats are still working in the police.
[0] https://lenta.ru/articles/2024/04/03/scientist/
The UK Post Office scandal is bone-chilling.
Update this to a world where every corner of your life is controlled by a platform monopoly that doesn't even provide the most bare-bones customer service and yeah, this is going to get a lot worse before it gets better.
We need strong and comprehensive regulations. Some places have enacted partial solutions but none anywhere near as complete as needed. EU has GDPR and some early AI laws, India has the IT Act that requires companies to provide direct end-user support.
And that's the early game.
Imagine when AI will be monitoring all internet traffic and arresting people for thoughtcrime.
What wasn't feasible to do before is now quite in reach and the consequences are dire.
Though of course it won't happen overnight. First they will let AI encroach every available space (backed by enthusiastic techbros). THEN, once it's established, boom. Authoritarian police state dystopia times 1000.
And it's not like they need evidence to bin you. They just need inference. People who share your psychological profile will act and speak and behave in a similar way to you, so you can be put in the same category. When enough people in that category are tagged as criminals, you will be too.
All because you couldn't be arsed to write some boilerplate
That's why there are transparency laws that indirectly forbid the use of black box decision systems like these for anything government-related.
Also AI for accountability laundering. It gives plausible deniability. It's a sociopathic manager's dream.
[dead]