Comment by yaoke259

Whilst general self-reporting of confidence will be hard to achieve (we can't even do it as humans), I'm sceptical of the notion that we can't train models to give some confidence scores accurately. I think over the past year that with the correct RL, models can showcase some pretty insane capabilities. If reporting the correct confidence was a RL training target say for predicting weather events or a game outcome that is supported by real RL environment or simulated data, then I suspect LLMs will develop a strong intuition for confidence similar to how the recent JEV and decision models are trained.