Comment by nrmitchi
1 day ago
Flock isn’t an “AI” though? It doesn’t put together a case and say “yes, this is the criminal”. It’s not a crime-solving AI.
It’s a camera system, and data can be queried by police. They have to search for something. So the cops searched for a vehicle matching the description in the general area, seemed to take the first one, and called it a day.
That’s not an AI-uncanny-valley. That’s the equivalent of (pre-flock/cameras) looking at vehicle registration list and assuming the one at the top of the list must be it.
The automation-trust is a legitimate concern but it’s not applicable here.
I don't know the details on how Flock is internally designed, but I suppose AI is at least involved in the image recognition and correlation, at least to this scale. I also don't know exactly why Flock misidentifies Issacs' car in this case, because I don't find it in the article or the interview, but I assume it is due to a mislabeling of the plate number or the car damage imagery, which is related to AI. Correct me if I'm wrong.
Flock didn't misidentify Isaacs' car in this case. It correctly identified that her car was vaguely near the accident, and the police incorrectly concluded that it was the one involved in the accident.
Correct, but what's more, there is a lack of accountability issue that's part of it. When people are not held accountable for the screw ups, then processes don't or are very slow to change. That's how you can just call it a day, regardless of the harm done to innocent citizens.
Ahem, reality would like a word about flock not being AI.
https://cybernews.com/privacy/flock-ai-os-investigate/
Flock is not an AI. That they have AI features on certain products does not mean "Flock" is an AI. That's the fallacy of composition.
Flock does have AI components (this one seems LLM based). Basically every ANPR system will also use some level of AI.
Also, on "reality would like a word". Why not just speak normally instead of using what seems like a high-school grade comeback?
The main Flock related issue here is that officers treat results that come out of the systems as gospel. The same thing has happened with face recognition systems. There is also the broader issue of access controls to Flock, 5A, and whether places should be building up a technology so powerful (surveillance infrastructure can easily be used for evil).