Comment by PaulDavisThe1st
2 months ago
There seems to be implication of confusion of what a qualified applicant means in your example above.
If there's a test used as the basis of consideration, and some process has decided that any score over X makes the candidate qualified, but then you are later going to claim that actually, given that there were candidates with a score of X+Y, a score of just X does not really constitute "qualified" and the higher scoring candidates should have been chosen, then the whole nature of the test and the ranking becomes rather suspect.
So either everyone who is judged to be qualified really is qualified, and it makes no difference that they were not necessarily the highest scoring candidates ... or ... the test for "qualified" is not suitable for purpose.
Suppose you have a test which is a decent proxy for how well someone will do a job. The median person currently doing the job scored 85 and their range is 70-99. If you put someone who scored a 4 in the job, people will die almost immediately. If you put someone who scored a 50 there, people will be at a higher risk of death and you'd be better off passing on that candidate and waiting for a better one. From this we might come up with a threshold of 70 for the minimum score and call this "qualified". Then if you have to fill 5 slots and you got candidates scoring 50, 75 and 95, you should hire the latter two and keep the other slots unfilled until you get better candidates.
But if you have to fill 5 slots and you have 10 candidates who all scored above 70, you now have to choose between them somehow. And the candidates who scored 95 are legitimately expected to perform the job better than the ones who scored 75, even though the ones who scored 75 would have been better than an unfilled position.
Assumptions:
1. there is a test that is a decent proxy for job performance
2. the relationship between job performance and test score above some passing score is linear
These both sound "common sense", but I suspect fail for a huge number of real world scenarios.
According to the article they actually tested the first assumption and it was true.
The second assumption is not required. If people who score a 95 are only 5% better at the job than people who score a 70, all else equal you'd still pick the person who scored a 95 given the choice.
3 replies →