Comment by bravura
3 months ago
I appreciate your argument, but add the following nuance:
Latent space reasoning can represent and manipulate UNCERTAINTY more concisely and elegantly than token space reasoning.
3 months ago
I appreciate your argument, but add the following nuance:
Latent space reasoning can represent and manipulate UNCERTAINTY more concisely and elegantly than token space reasoning.
If uncertainty is an important signal then a model RL conditioned to perform good COT should be expected to learn how to encode an uncertainty sidechannel in its COT.
If we're fortunate it'll do so using language choice that would also convey uncertainty to humans. Before you complain that English uncertainty has poor precision, consider that nothing prevents the LLM from overloading it with a more precise meaning. Like how "MAY" in an RFC means something much more concrete than in general English. Though unless somehow conditioned for it the uncertainty signal could be something else entirely (including, perhaps, sounding more certain).
This also goes for pretty much any other side information you might hope could be conveyed.