Comment by dragonwriter

6 years ago

> She scored 95% in one subject. But the qualified cut-off rate is 98%!

> My eyes just rolled looking at that number. That means they only pick 98 and 99 percentiles.

Either one of these is wrong, or they somehow managed to craft a test where the score percentage matches the score percentile, which while possible to engineer is somewhat improbable and also contradicts the next sentence:

> For a first grader to score that high, she needs to answer the exam perfectly.

Irrespective of one’s current grade level, it doesn't require answering an exam perfectly to get a 98%. It might to get a 98 percentile score (depending on what other people taking the score get).

> If you have some statistics training, you'll see this score is like shooting yourself in the foot.

If you have some statistics training you'll recognize the difference between percentages and percentiles and which of the former corresponds to a perfect score.

The test-retest validity of IQ tests (which tests for gifted programs either expressly are or are equivalent to) is such that it's not unreasonable at all to think that people will consistently score at about the same place based on ability over a short time period (over a longer span there'll be some variation) and that being a prepared test taker isn't a particularly significant factor, though anxiety about testing or the particular test could depress scores and soothing that is the main benefit of test prep.

Yes, the levels most districts use a gifted cutoff are very high (usually 97th-99th percentile). Yes, that means very few (1-3%) will make the cut. No, this doesn't mean the people that make the cut are just test-taking prodigies.

10 comments

dragonwriter

Reply
kabouseng  6 years ago

It's a form of Goodheart's law [1]. If you use the test scores to select for gifted students and the requirement is this high you will only select for highly test prepared students (the set of prepared and gifted or prepared and capable students). You're final student selection will be higher correlated to preparation and less to how naturally gifted a student is. This is not necessarily a bad thing though.

[1] - https://en.wikipedia.org/wiki/Goodhart's_law

  • soared  6 years ago

    You'd expect a first grader who is highly prepared for a test to be likely to be gifted, correct?

    • singingboyo  6 years ago

      Up to maybe 90-95%, I'd agree. Past that... well, that last 5% is often weird bits that you have to specifically have come across before.

      For something at a first grade level, consider, say, spelling bees. I read a lot as a kid, and correspondingly did very well on things like that, but across the hundreds or thousands of words these things go through... well, the people at Scripps aren't just reading lots of books.

      It's also been my experience that the people who implicitly understand something tend not to want to bother with recommended prep. That was me for CS courses, some calculus, and english/writing, where I got high grades but often not top of the class. I was on the opposite side for history and statistics (and outscored at least some of the people who are far better at those subjects than I am).

      Basically, if you're gifted and already have knowledge of a subject, you probably don't really want to spend lots of time studying it. This leaves you vulnerable to the weird 5% of edge cases you haven't seen before. If you don't know the subject, and have to do a lot of prep, you're going to come across those cases during prep.

usgroup  6 years ago

If the optimisation problem is “pick a set of kids that can finish the curriculum quicker as measured by standardised tests” then any effective screening test is likely to select for “test taking prodigies” since that is the leading measure.

RE percentage vs percentile: I think you may be wrong here. Take for example, uni exams. 40% is the pass mark here in old Blighty. The exams are standardised and the 40% threshold is not a percentile. In fact, it makes almost no sense at all to stackrank every cohort of test takers. It wouldn’t be fair at all not comparable over time. I think the parent poster is correct. Doubtless the empirical distribution of real scores are used to decide cutoffs for grades, by 99% in the parents posts very likely refers to a percentage.

  • fjsolwmv  6 years ago

    Gifted classrooms are funded by a budget and have a certain number of seats available. In must be a percentile selection (mixed with subjective judgments, diversity, etc)

    • usgroup  6 years ago

      Anyone with the same percentage score will obviously be in the same percentile ... so if you have a limited number of places and everyone scores the same mark, then the fact that you’re filtering by a percentile doesn’t help you in the slightest: it’s tantamount to picking at random.

      If none scores 99% then you just don’t take anyone ... in your method if everyone scored 0% you’d still take a bunch. I don’t think that makes any sense.

      2 replies →