Comment by epihelix
3 hours ago
> When you have the mind of "the models are wrong", you tend to be brought there by select counter examples, and in that space you only feed on counter examples.
Box's (possibly apocryphal) aphorism, "all models are wrong, but some are useful", is the better mindset.
Obviously we can never perfectly model nature, but we can often get close enough to form useful predictions. A minor predictive failure does not necessarily mean you throw the model, and all of its predictions, out entirely.
> "The experts are wrong (and you can be right in 15 minutes if you consume this media!)" is abundant.
Social media, sadly, preys on the feeble-minded or willingly-deceived :(
the unspoken permanent wrench in the gears lies in chaos theory. models always have some level of consolidation of reality that gets treated in blocks. While this has intuitive inaccuracies like rounding errors, edge cases, and such, we also know reality often shows emergent behavior from chaotic interactions in a way only understandable after it's actually happened. these could be erorrs that could completely break the models relevance
After observing the emergent behavior, we can often then incorporate it into the model by breaking up prior consolidations into more parts, but the fundamental problem still remains. This makes models for things that arent part of a relatively rapid feedback loop for model improvement (like climate change) to be very vulnerable unknowns. thats not to say they are completely useless, but it can very much take away the weight behind any specificity of the models results, and should open up a second conversation about the direction in which the model might be expected to fail.
Additionally, when creating models that involve human behavior you can appeal to game theory, psychology, sociology, statistics, etc but ultimately chaos will be in full force. now with AI it is going to become a factor amongst automation as well in a way it previously was not. there is no practical way to model how the weights of neural networks might act unexpectedly in various contexts.
all this to say models are not often that great at giving useful predictions as much as they are great at building foundations of understanding of relationships in complex/complicated and what "clean" situations might look like, which can then be taken into consideration of what reality is likely to look like and happen. this distinction is important because currently, automation tools and AI lack that final chaotic adjustment that astute humans are able to apply. Ai has gotten very good at making complicated models but it ultimately is, by design imo whether intentional or not, limited in the same way models themselves are.
that final adjustment to make real world decisions and have personal accountability is more rooted in beliefs/feelings than it is in model outputs, albeit the model outputs help to refine it.