Comment by pminimax
2 years ago
It is common practice in psychometrics to use two levels in a forced choice and model responses as a logistic regression, which is what's done here. Adding an N/A option turns the thing into an ordered logistic regression with unknown levels, which is tricky to fit, but it's possible. Having done a lot of psychophysics, having more options generally doesn't make the task easier.
Sounds like psychometrics is unsuitable for modeling this problem, according to what you're saying. When you have a hammer everything looks like a nail.
The way that XKCD did it is the best, you ask people to give a name to each color then the responses are entirely natural and unprompted.
I don’t think that forced choice can give accurate results if a substantial number of people perceive green and blue as being non-adjacent - i.e. there exists a color between green and blue (turquoise/cyan/teal).
Otherwise it’s like asking people whether a color is red or yellow, when it’s clearly a shade of orange.
Some shades of orange are closer to red and some are closer to yellow.
Yes but saying that a shade of orange is closer to yellow is different from saying that it is yellow.
Orange is closer to green than blue but I wouldn’t say that it’s a shade of green. It’s just orange.
> Otherwise it’s like asking people whether a color is red or yellow, when it’s clearly a shade of orange.
No it's like asking people whether a color is red or green, when it's clearly a shade of yellow.
https://en.wikipedia.org/wiki/Color_wheel#/media/File%3ALine...
That’s why I took the test 5 times, and my scores varied between 63% and 69% “green” so I took the average at 66.4
Are you sure that it is common practice for a problem that has three valid answers A, B and C, to only allow people to answer A or C?
Your website is not talking about "levels" of colour.
It's asking "is this blue or green", not "is this closer to blue or closer to green".
The question (1) "is this blue or green" has three valid answers: blue, green or neither.
The question (2) "is this closer to blue or green" only has two valid answers.
I would assume that with these types of surveys, the first thing to do is to qualify the proper categorization of the question.
Sorry to say, but to me it seems that almost all of the confusion in the discussion here is because you're asking question (1) (which has three valid answers) but expecting an answer from (2) (which indeed has two valid answers).