Comment by cgriswald
6 years ago
It’s because tests aren’t just testing knowledge. They are testing the ability to express the entirety of that knowledge under completely arbitrary conditions (time limits, schedules, no references, etc.) with outsized consequences for mistakes.
Also good tests are hard to write. I’ve seen T/F questions that could go either way. Multiple choice questions with more than one correct answer. (Professors will tell you to choose the “best” answer. But that’s a matter of opinion in many, if not all, cases.)
I think what someone can DO with their knowledge is more important than what individual bits of knowledge they maintained. I’d rather hire or work with someone who can get things done and knows certain algorithms exist than hire or work with someone who can’t get anything done but can recite the same algorithms from memory. Tests favor the second person. (I was that person in HS calculus. Aced the class without understanding a thing just because I have a gift for remembering and applying rules. I had no idea what I was doing.)
> Also good tests are hard to write. I’ve seen T/F questions that could go either way. Multiple choice questions with more than one correct answer. (Professors will tell you to choose the “best” answer. But that’s a matter of opinion in many, if not all, cases.)
This is why I am so glad that all of my university exams (and the _vast_ majority of exams before that, at least post-Y9/age 13) were open-ended questions[1], then marked by someone who will (likely) know the subject better than you even will. Even if you couldn't get the the answer, but could understand and articulate the starting points, or made a compelling argument but misread or misunderstood part of the question, you will at least get partial credit. The physics exams would also have a standardised formulae reference sheet.
[1] A typical paper would be three hours, answering 5-6 questions, and a typical subquestion can be as open-ended as "Write brief notes about a tree representation of functional arrays, subscripted by positive integers according to their representation in binary notation. How efficient are the lookup and update operations?"
It seems like you'd be surprised to learn that the ability to recall and utilize something under constraints is highly indicative of a person's understanding of that material. The kids who "memorize formulas" aren't the ones getting straight A's. The ones who understand and grasp the material are the ones who will have no trouble performing under pressure, because if they forget something, they can recall it using their knowledge structures.
E.g. Say I forgot the formula for Simpson's Rule on a test, but remember that it had to do with approximating integrals with trapezoids. Someone who thoroughly understands the course material could re-derive this formula in under 5 minutes if they had forgotten.
Good test questions are hard to write. That's why standardized tests usually have lots of them so deficiencies with individual questions become statistically irrelevant. At the end of the day, we aren't trying to probe exactly what the students know. We aren't trying to read minds here. We just want an accurate distribution curve. We don't need great test questions for that. We just need some correlation between the correction answers and academic abilities.
> They are testing the ability to express the entirety of that knowledge under completely arbitrary conditions (time limits, schedules, no references, etc.) with outsized consequences for mistakes.
Right. And if you know the material, this isn't a problem.