It's a bit more loaded than that. 538 post-Nate Silver had a model setup that was apparently kind of a mess. 538 was apparently sending strange messages to Republican leaning polling agencies, demanding they gave far more detailed audit information than usual (with the implication obviously being that they were fraudulent pollsters), and the guy running the site had fairly openly tuned his model on the assumption people cared about certain talking points. 538 was predicting Biden victories even when the polls were so overwhelmingly against him that not even the most Democratic leaning polling agencies had trust in him; even if you aren't running difficult math, that means something has gone wrong with the model.
(Something which got worse after Harris was picked, although every polling aggregator went barmy - there's suspicions that a lot of polling agencies aggressively normalized their data to avoid being seen as biased, leading to an almost 50/50 split.)
So, I would actually agree with you that 538 was a mess in that period. I would actually trace the first problems back to Enten's departure, but it did get worse when Silver left.
It's just that this particular criticism (that they got it "wrong") doesn't seem very well founded.
Biden got literally 0% of the vote despite 538 predicting him to win.
They chose to pretend Biden's mental decline wasn't happening because that was the Democrat party line at the time. How can you trust predictions from someone who is willing to manipulate his results to prop up his political party?
What would you have them do in that period? Give Trump a 100% chance of winning? Make up a candidate who wasn't actually running? Arguably the most realistic thing to do would be to give X% Trump, Y% Some Democrat -- but had they done that, they'd have been rightly criticized for making an obvious vibe-driven decision. You can't change a quantitative prediction based on qualitative observations; all you can do is try to find more and better data sources.
The issue the models had in that period was pretty well documented: the models rated fundamentals (like economic indicators) more highly relative to polls when the election was farther in the future. Those were the same economic indicators that famously do not capture the pain average Americans have felt since COVID, and that politicians across the spectrum have tried to use to deny that pain when they are in power.
You can argue that this issue was well known at the time, and that a better group of analysts might have taken action to mitigate it. I'd argue that myself. It turns out that it's kind of hard, but that's a terrible excuse not to try. That is different than deliberately manipulating results for political ends, though. It's unclear what political ends those would even be -- who exactly benefits from an "unclear winner" forecast in that situation?
Because of broad statistical illiteracy regarding early forecasts?
538: "We have no idea who will win. It seems like basically a coin toss."
Reality: someone wins.
Internet: "How did 538 mess this up so badly?"
It's a bit more loaded than that. 538 post-Nate Silver had a model setup that was apparently kind of a mess. 538 was apparently sending strange messages to Republican leaning polling agencies, demanding they gave far more detailed audit information than usual (with the implication obviously being that they were fraudulent pollsters), and the guy running the site had fairly openly tuned his model on the assumption people cared about certain talking points. 538 was predicting Biden victories even when the polls were so overwhelmingly against him that not even the most Democratic leaning polling agencies had trust in him; even if you aren't running difficult math, that means something has gone wrong with the model.
(Something which got worse after Harris was picked, although every polling aggregator went barmy - there's suspicions that a lot of polling agencies aggressively normalized their data to avoid being seen as biased, leading to an almost 50/50 split.)
So, I would actually agree with you that 538 was a mess in that period. I would actually trace the first problems back to Enten's departure, but it did get worse when Silver left.
It's just that this particular criticism (that they got it "wrong") doesn't seem very well founded.
Biden got literally 0% of the vote despite 538 predicting him to win.
They chose to pretend Biden's mental decline wasn't happening because that was the Democrat party line at the time. How can you trust predictions from someone who is willing to manipulate his results to prop up his political party?
What would you have them do in that period? Give Trump a 100% chance of winning? Make up a candidate who wasn't actually running? Arguably the most realistic thing to do would be to give X% Trump, Y% Some Democrat -- but had they done that, they'd have been rightly criticized for making an obvious vibe-driven decision. You can't change a quantitative prediction based on qualitative observations; all you can do is try to find more and better data sources.
The issue the models had in that period was pretty well documented: the models rated fundamentals (like economic indicators) more highly relative to polls when the election was farther in the future. Those were the same economic indicators that famously do not capture the pain average Americans have felt since COVID, and that politicians across the spectrum have tried to use to deny that pain when they are in power.
You can argue that this issue was well known at the time, and that a better group of analysts might have taken action to mitigate it. I'd argue that myself. It turns out that it's kind of hard, but that's a terrible excuse not to try. That is different than deliberately manipulating results for political ends, though. It's unclear what political ends those would even be -- who exactly benefits from an "unclear winner" forecast in that situation?
... so an event that had a (simulated) 47% percent chance of happening happens, and an event that had a 53% chance of happening didn't.
You must _hate_ it when you need <= 4 and you roll a 5...
They had Trump 1/3 and Clinton 2/3 IIRC. That was a bigger spread. I remember Nate Silver saying that is what 1/3 probability looks like later.