← Back to context

Comment by ezwoodland

4 days ago

Not if you use a proper scoring rule.

See: https://en.wikipedia.org/wiki/Scoring_rule

I'd say selecting a scoring rule that depends on the probability distribution (the Brier score, for example) changes the task from predicting the results of coin flips to predicting the distribution. Maybe it will work with a corresponding prompt.