Comment by jmyeet

Polling, particularly in US elections, is hard. A lot of people, particularly tech people, got very excited about Nate Silver and 538 after the 2012 presidential elections but they shouldn't have. When you look at any polling or election predictions, the US has voluntary voting so it's not just a question of how people will vote but who will vote. So, if your predictions just guessed an outcome without explaining why (accurately) then it's just astrology, basically.

In polling circles, the voters tend to be segmented into high and low propensity voters. High propensity voters will always vote. Low propensity voters won't. But the differences are often so small the results can flip on unexpected turnout in just one segment of the voters.

As an example, the 2024 election turned on 3 big factors:

1. Millions of Biden voters in 2020 stayed home. Affordability was the biggest factor but there are were other huge factors too, most notably Palestine;

2. Trump retained the white vote while increasing his share of the Hispanic vote; and

3. Trump activated younger, low-propensity voters. You'll often osee this described as the "podcast sphere". We're talking the people who end up in alt-right pipeline on Youtube and in podcasts (eg Andrew Tate).

Most recent presidential elections come down the results in about 7-8 states. The other 42-43 are known before you go in with some rare exceptions. The most recent exception was Obama in 2008 who won Iowa, for example. Other big sweeps were Reagan in 1984 and Nixon in 1972.

So, with a modern election you can just flip a coin 7 times and you have a 1 in 128 chance of just being correct, 50 out of 50. This is why you need to show your work with any prediction modeling and polling. You need to show how you reached your prediction in terms of turnout as well as how major demographics will vote.

Every election cycle complicates this with external factors and per-state issues. Covid loomed large over 2020 but it also made voting easier than ever, with easy access to early voting and mail-in voting. This changes the high and low propensity voter math significantly. Also, the Arizona legislature went on a mission to punish Native Americans for flipping Arizona blue in 2020 by disenfranchising Native Americans in subsequent elections in many, many ways.

So what tends to happen is that results are close enough that it become s abit of a guess. No pollster wants to be an outlier AND wrong so there's a convergence to mean thing that happens where they all tend to make the same prediction because everybody being wrong is way better, optically, than you being wrong and everyone else being right.

Add to all this, population sampling used to be done on landlines decades ago. Now we just don't have an equivalent and if your sampling algorithm is off, your results are off. Garbage in, garbage out.

I guess my point is that Nate Silver got kinda lucky in 2012 and came back to Earth in 2016 so I've never been that impressed and honestly I jus tdon't care if 538 exists or not.