Comment by danso
6 years ago
This story is really alarming because as described, the police ran a face recognition tool based on a frame of grainy security footage and got a positive hit. Does this tool give any indication of a confidence value? Does it return a list (sorted by confidence) of possible suspects, or any other kind of feedback that would indicate even to a layperson how much uncertainty there is?
The issue of face recognition algorithms performing worse on dark faces is a major problem. But the other side of it is: would police be more hesitant to act on such fuzzy evidence if the top match appeared to be a middle-class Caucasian (i.e. someone who is more likely to take legal recourse)?
I think the NYT article has a little more detail: https://www.nytimes.com/2020/06/24/technology/facial-recogni...
Essentially, an employee of the facial recognition provider forwarded an "investigative lead" for the match they generated (which does have a score associated with it on the provider's side, but it's not clear if the score is clearly communicated to detectives as well), and the detectives then put the photo of this man into a "6 pack" photo line-up, from which a store employee then identified that man as being the suspect.
Everyone involved will probably point fingers at each other, because the provider for example put large heading on their communication that, "this is not probable cause for an arrest, this is only an investigative lead, etc.", while the detectives will say well we got a hit from a line-up, blame the witness, and the witness would probably say well the detectives showed me a line-up and he seemed like the right guy (or maybe as is often the case with line-ups, the detectives can exert a huge amount of bias/influence over witnesses).
EDIT: Just to be clear, none of this is to say that the process worked well or that I condone this. I think the data, the technology, the processes, and the level of understanding on the side of the police are all insufficient, and I do not support how this played out, but I think it is easy enough to provide at least some pseudo-justification at each step along the way.
That's interesting. In many ways, it's similar to the "traditional" process I went through when reporting a robbery to the NYPD 5+ years ago: they had software where they could search for mugshots of all previously convicted felons living in a x-mile radius of the crime scene, filtered by the physical characteristics I described. Whether the actual suspect's face was found by the software, it was ultimately too slow and clunky to paginate through hundreds of results.
Presumably, the facial recognition software would provide an additional filter/sort. But at least in my situation, I could actually see how big the total pool of potential matches and thus have a sense of uncertainty about false positives, even if I were completely ignorant about the impact of false negatives (i.e. what if my suspect didn't live within x-miles of the scene, or wasn't a known/convicted felon?)
So the caution re: face recognition software is how it may non-transparently add confidence to this already very imperfect filtering process.
(in my case, the suspect was eventually found because he had committed a number of robberies, including being clearly caught on camera, and in an area/pattern that was easy to narrow down where he operated)
> and the detectives then put the photo of this man into a "6 pack" photo line-up, from which a store employee then identified that man as being the suspect.
This is absurdly dangerously. The AI will find people who look like the suspect, that’s how the technology works. A lineup as evidence will almost guarantee a bad outcome, because of course the man looks like the suspect!
The worse part is that the employee wasn't a witness to anything. He was making the "ID" from the same video the police had.
I'm also half guessing if the "lineup" was 5 White people and the a photo of the victim.
> Essentially, an employee of the facial recognition provider forwarded an "investigative lead" for the match they generated (which does have a score associated with it on the provider's side, but it's not clear if the score is clearly communicated to detectives as well)
This is the lead provided:
https://wfdd-live.s3.amazonaws.com/styles/story-full/s3/imag...
Note that it says in red and bold emphasis:
THIS DOCUMENT IS NOT A POSITIVE IDENTIFICATION. IT IS AN INVESTIGATIVE LEAD ONLY AND IS NOT PROBABLE CAUSE TO ARRREST. FURTHER INVESTIGATION IS NEEDED TO DEVELOP PROBABLE CAUSE TO ARREST.
Dear god the input image they used to generate that is TERRIBLE! It could be damn near any black male.
The real negligence here is whoever tuned the software to spit out a result for that quality of image rather than a "not enough data, too many matches, please submit a better image" error.
4 replies →
This is why you should be scared of this tech. Computer assisted patsy finder. No need to find the right guy when the ai will happily cough up 20 people nearby who kinda sorta look like the perp enough to stuff them into a lineup in front of a confused and highly fallible witness.
Yep, the potential for abuse here is insane.
I'm becoming increasingly frustrated with the difficulty in accessing primary source material. Why don't any of these outlets post the surveillance video and let us decide for ourselves how much of a resemblance there is.
Even if the guy was an exact facial match, that doesn't justify the complete lack of basic police work to establish it was him.
1 reply →
Do they have it? Police haven't always been forthcoming in publishing their evidence.
9 replies →
Because they're not in the business of providing information, transparency or journalism.
They are in the business of exposing you to as many paid ads as possible. And they believe providing outgoing links reduces their ability to do that.
1 reply →
I can see why you'd only get 6 guys together for a physical "6 pack" line-up.
But for a photo lineup I can't imagine why you don't have least 25 photos to pick from.
Excellent point. In fact, the entire process of showing the witness the photos should be recorded, and double blind. I.e the officer showing the person should not know anything about the lineup.
1 reply →
> the detectives then put the photo of this man into a "6 pack" photo line-up, from which a store employee then identified that man
This is not correct. The "6-pack" was shown to a security firm's employee, who had viewed the store camera's tape.
"In this case, however, according to the Detroit police report, investigators simply included Mr. Williams’s picture in a “6-pack photo lineup” they created and showed to Ms. Johnston, Shinola’s loss-prevention contractor, and she identified him." [1]
[1] ibid.
Just a tip in case it happens to anyone - Never, ever agree to be in a lineup.
It wasn't just that the employee picked the man out of 6 pack; the employee they interviewed wasn't even a witness to the crime in the first place.
>into a "6 pack" photo line-up
How did the people in the 6 pack photo line-up match up against the facial recognition? Were they likely matches?
No clue about the likelihood of police using similar facial recognition matches for the rest, but normally the alternates need to be around the same height, build, and complexion as the subject. I would think including multiple potential matches would be a huge no-no simply because your alternates need to be people who you know are not a match. If you just grab the 6 most similar faces and ask the victim to choose, what do you do when they pick the third closest match?
1 reply →
Even worse, the employee who was asked to pick him out of a line up hadn't even witnessed the crime in the first place.
> Does this tool give any indication of a confidence value?
Yes.
> Does it return a list (sorted by confidence) of possible suspects,
Yes.
> ... or any other kind of feedback that would indicate even to a layperson how much uncertainty there is?
Yes it does. It also states in large print heading “THIS DOCUMENT IS NOT A POSITIVE IDENTIFICATION IT IS AN INVESTIGATIVE LEAD AND IS NOT PROBABLE CAUSE TO ARREST”.
You can see a picture of this in the ACLU article.
The police bungled this badly by setting up a fake photo lineup with the loss prevention clerk who submitted the report (who had only ever seen the same footage they had).
However, tools that are rife for misuse do not get a pass because they include a bold disclaimer. If the tool/process can not prevent misuse, the tool/process is broken and possibly dangerous.
That said, we have little data on how often the tool results in catching dangerous criminals versus how often it misidentifies innocent people. We have little data on if those innocent people tend to skew toward a particular demographic.
But I have a fair suspicion that dragnet techniques like this unfortunately can be both effective and also problematic.
I think the software would be potentially less problematic if the victim/witness were given access, and (ostensibly) could see the pool of matches and how much/little the top likely match differed from the less confident matches.
> The police bungled this badly by setting up a fake photo lineup...*
FWIW, this process is similar to traditional police lineups. The witness is shown 4-6 people – one who is the actual suspect, and several that vaguely match a description of the suspect. When I was asked to identify a suspect in my robbery, the lineup included an assistant attorney who would later end up prosecuting the case. The police had to go out and find tall slight-skinned men to round out the lineup.
> ... with the loss prevention clerk who submitted the report (who had only ever seen the same footage they had).
Yeah, I would hope that this is not standard process. The lineup process is already imperfect and flawed as it is even with a witness who at least saw the crime first-hand.
Intresting and related, a team made a neat "face depixelizer" that takes a pixelated image and uses machine learning to generate a face that should match the pixelated image.
What's hilarious is that it makes faces that look nothing like the original high-resolution images.
https://twitter.com/Chicken3gg/status/1274314622447820801
Interesting... Neat... Hilarious... In light of the submission and the comment you're responding to, these are not the words I would choose.
I think there's genuine cause for concern here, especially if technologies like these are candidates for inclusion in any real law enforcement decision-making.
That should be called a face generator, not a depixelizer.
Basically. The faces look plausible but less useful than the original blurred image.
What's sad is that a tech entrepreneur will definitely add that feature and sell it to law enforcement agencies that believe in CSI magic: https://www.youtube.com/watch?v=Vxq9yj2pVWk
And another entrepreneur can add a feature to generate 10 different faces which match the same pixelation, and sell it to the defence.
2 replies →
Ironically, if the police had used and followed the face depixelizer then we may not have had the false arrest of a black man - not because of accuracy but because it doesn't produce many black faces
I wonder if this is trained on the same, or similar, datasets.
One of the underlying models, PULSE, was trained on CelebAHQ, which is likely what the results are mostly white-looking. StyleGAN, which was trained on the much more diverse (but sparse) FFHQ dataset does come up with a much more diverse set of faces[1]...but PULSE couldn't get them to converge very closely on the pixelated subjects...so they went with CelebA [2].
[1] https://github.com/NVlabs/stylegan [2] https://arxiv.org/pdf/2003.03808.pdf (ctrl+f ffhq)
People are not good at understanding uncertainty and its implications, even if you put it front and center. I used to work in renewable energy consulting and I was shocked by how aggressively uncertainty estimates are ignored by those whose goals they threaten.
In this case, it's incumbent on the software vendors to ensure that less-than-certain results aren't even shown to the user. American police can't generally be trusted to understand nuance and/or do the right thing.
I blame TV shows like CSI and all the other crap out there that make pixelated images look like something you could "Zoom" into or something the computer can still understand even if the eye does not. Because of this, non tech people do not really understand that pixelated images have LOST information. Add that to the racial situation in the U.S. and the the inaccuracy of the tool for black people. Wow, this can lead to some really troublesome results
I lose hours every day just yelling "enhance" at my computer screen. Hasn't worked yet, but any day now...
Bladerunner
> But the other side of it is: would police be more hesitant to act on such fuzzy evidence if the top match appeared to be a middle-class Caucasian (i.e. someone who is more likely to take legal recourse)?
Honest question: does race predict legal recourse when decoupled from socioeconomic status, or is this an assumption?
Race and socioeconomic status are deeply intertwined. Or to be more blunt - US society has kept black people poorer. To treat them as independent variables is to ignore the whole history of race in the US.
> To treat them as independent variables is to ignore the whole history of race in the US.
Presumably the coupling of the variables is not binary (dependent or independent) but variable (degrees of coupling). Presumably these variables were more tightly coupled in the past than in the present. Presumably it's useful to understand precisely how coupled these variables are today because it would drive our approach to addressing these disparities. E.g., if the variables are loosely coupled then bias-reducing programs would have a marginal impact on the disparities and the better investment would be social welfare programs (and the inverse is true if the variables are tightly coupled).
>Honest question: does race predict legal recourse when decoupled from socioeconomic status, or is this an assumption?
I think the issue is that regardless of the answer, it isn't decoupled in real world scenarios.
I think the solution isn't dependent upon race either. It is to ensure everyone have access to legal recourse regardless of socioeconomic status. This would have the side effect of benefiting races correlated with lower socioeconomic status more.
> I think the issue is that regardless of the answer, it isn't decoupled in real world scenarios.
Did you think I was asking about non-real-world scenarios? And how do we know that it's coupled (or rather, the degree to which it's coupled) in real world scenarios?
> I think the solution isn't dependent upon race either. It is to ensure everyone have access to legal recourse regardless of socioeconomic status. This would have the side effect of benefiting races correlated with lower socioeconomic status more.
This makes sense to me, although I don't know what this looks like in practice.
1 reply →
Middle class black people often get harassed by police, and there is a long history of far steeper sentences for convictions for drugs used more by the black population (crack) than that used more by the white population (cocaine).
So unequal treatment based on race has quite literally been a feature of the US justice system, independent of socioeconomic status.
I’m aware, but that doesn’t answer my question about access to legal recourse.
1 reply →
> The issue of face recognition algorithms performing worse on dark faces is a major problem.
This needs to be coupled with the truth that people (police) without diverse racial exposure are terrible at identifying people outside of their ethnicity. In the photo/text article they show the top of the "Investigative Lead Report" as an image. You mean to say that every cop who saw the two images side by side did not stop and say "hey, these are not the same person!" They did not, and that's because their own brains' could not see the difference.
This is a major reason police forces need to be ethnically diverse. Just that enables those members of the force who never grew up or spent time outside their ethnicity can learn to tell a diverse range of similar but different people outside their ethnicity apart.
It wouldn't make it into the newspapers, so it doesn't matter.