Comment by rodonn

8 hours ago

You can't easily evaluate the success of any single forecast, but as soon as you have a collection of forecasts there are a lot of ways to evaluate which forecasters are more accurate and which are well calibrated in their predictions.

For example you can look at the calibration of the forecasts: Take all the outcomes where you estimated ~10% probability. How many of them actually happened? If you are an well-calibrated forecaster, the actual number should be fairly close to 10%.