Comment by PaulHoule

The whole problem of text understanding is a problem of reasoning under uncertainty, that is, you can't really be sure which witch people are talking about all the time. A person you might hire might be successful or unsuccessful at the role, no matter what hiring process you use. Two people might look at the same resume and come to the same conclusions. Two patients with the same symptoms and clinical presentation might have different diseases, etc.

I don't buy the story that the old AI died primarily due to the cost of knowledge base maintenance [1], but rather the lack of a universal system of reasoning over uncertainty.

For me it's a running gag that Spock was always saying things like "Captain, we have a 21% probability of surviving this mission" when Bayes teaches us your probability distribution has a probability distribution, "we have a β(5,1) chance of surviving this mission" is more like it.

To that end it wouldn't be too crazy to run a resume through that machine 100 times and look at the probability distribution of the score.

[1] then again I am the kind of maniac who will sort images on a tablet lying in bed until my visual system malfunctions