It’s easy enough to get more than 1,500 people to answer your survey. But every pollster is plagued by the fact that people who voluntarily answer surveys are usually not representative of the broader population on the relevant issue.
Basically any method for sampling the population introduces bias. The sample of people who pick the phone, who respond to texts or emails, who come to the door for a stranger, are all subpopulations with their own biases. A pollster’s ability to correct for these biases is what separates the good from the bad.
This is a counter intuitive result around surveying. Best way I’ve found to explain it is if you want to test if a soup is too salty, it’s about the size of the spoon, not the size of the pot.
For questions with only a handful of answers, it is indeed definitely enough, as long as it's actually a random sample without too much bias. Going to larger numbers but with the same bias in sampling of people won't give an better answer!
So the actual question is how they checked that their sample is representative and not too heavy on any demographic.
1000 people is what they usually do for political polls. I was taught that a sample size of 1000 is sufficient for pretty much any purpose. At some point what becomes more significant is the differences between the poll-answering population and the general population. For eg if you call people at random on the phone, you're biased towards people who are likely to pick up the phone and talk to a pollster (eg retirees are home more often and not busy). Of course nobody is that naive nowadays, but that doesn't mean systemic error is completely eliminated.
https://archive.is/K3y9y
For some reason the economist's page was not loading on my browser.
I could only read the blurb, but this stood out to me: "To survey 1500 Americans".
Is that really enough to get a representative sample across 340 million people? Also, with tech as it is, why is that the best they can do?
It’s easy enough to get more than 1,500 people to answer your survey. But every pollster is plagued by the fact that people who voluntarily answer surveys are usually not representative of the broader population on the relevant issue.
Basically any method for sampling the population introduces bias. The sample of people who pick the phone, who respond to texts or emails, who come to the door for a stranger, are all subpopulations with their own biases. A pollster’s ability to correct for these biases is what separates the good from the bad.
This is a counter intuitive result around surveying. Best way I’ve found to explain it is if you want to test if a soup is too salty, it’s about the size of the spoon, not the size of the pot.
For questions with only a handful of answers, it is indeed definitely enough, as long as it's actually a random sample without too much bias. Going to larger numbers but with the same bias in sampling of people won't give an better answer!
So the actual question is how they checked that their sample is representative and not too heavy on any demographic.
> as long as it's actually a random sample without too much bias
This is the "spherical cow" of statistics.
1000 people is what they usually do for political polls. I was taught that a sample size of 1000 is sufficient for pretty much any purpose. At some point what becomes more significant is the differences between the poll-answering population and the general population. For eg if you call people at random on the phone, you're biased towards people who are likely to pick up the phone and talk to a pollster (eg retirees are home more often and not busy). Of course nobody is that naive nowadays, but that doesn't mean systemic error is completely eliminated.
Yes, it is, if its sufficiently random.
In fact, 385 people is enough for 350M people, if sufficiently random.
n = (ZZp(1-p))/(EE)
n = sample size
Z = confidence (studies are usually 95%)
p(1-p) = variance of binormal proportion, usually set p=.5
E = margin of error (is 1.00-.95 or .05)