← Back to context

Comment by alexpotato

10 hours ago

I was reading Nate Silver's book "On The Edge" and there is an interesting part where he takes predictions on the usage of nuclear weapons taken from just after World War 2 and compares them to what the Bayesian prediction would be given what actually happened.

Post World War 2, some people had the odds per year at 10%. Some of that is probably a mix of recency bias + not understanding how to use new weapons etc etc but as Silver points out, the odds were much lower.

I mention this only b/c the "could something trained on LLMs of the time predict the future" always makes me think of it.

8 comments

alexpotato

Reply
defrost  10 hours ago

Predicting the future is problematic, agreed.

Re: the Nate Silver nuclear weapons example, that's pretty weak - eg: given (say) I've just seen three heads in a row (exactly once) .. does that alter anything about "the odds".

Having seen nuclear weapons not used post WWII ... does that inform us about "the odds" or the several times their use was almost certain (eg: Cuban missile crisis) save for out of band behaviour by individuals that averted use and escalation?

  • energy123  7 hours ago

    Historical base rates are the starting point unless you have an unusually good causal theory of the thing you're modelling. In the case of a coin flip you do. But the large majority of the time when it's a complex system you don't.

    Most people's first instinct when faced with a complex system is to try to model it with words and use those words to predict. It's a beginner's error.

  • nl  10 hours ago

    > Having seen nuclear weapons not used post WWII ... does that inform us about "the odds"

    This is what Bayesian prediction does

    > save for out of band behaviour by individuals that averted use and escalation?

    This is kind of the point being made.

    • defrost  9 hours ago

      > This is what Bayesian prediction does

      Repeatedly, in a reproducible way, for events in the arrow of time? We can test this by going back to 1945 and running forward again?

      > This is kind of the point being made.

      Was it?

      ( assume I did a little math some decades past and have some poor grasp of Bayesian statistics )

      3 replies →