Comment by causal

I’m not convinced removing RLHF would really make the probabilities generator give us distributions that can diverge from the mean while remaining useful.

In other words, this might not a problem that can be overcome in LLMs alone.