Comment by LoganDark

I have a hunch that language models have a hard time controlling their voice depending on context. Like there are very particular voices used in certain locations in a user interface, and a model doesn't necessarily have an intuitive sense of that. You can instruct it to use particular voices, but it doesn't usually know to apply that intuitively, you have to know for it and tell it. To me, that example you quoted sounds like a potentially desirable quality (for the model trainers...) for communicating with the prompter during coding tasks, but I would not ever in a million years transfer that verbatim to a user interface. The way you speak to users is fundamentally different than the way you speak efficiently to prompters.

The reason why I consider it may be potentially desirable for communicating with prompters is that prompters usually need a way to verify the model has done what they asked, but without necessarily needing to review all the code or very long runs of text. They only need a bare minimum to know that their requirement has been met and not a full explanation of everything, so the super terse and efficient way of communicating how the constraints are implemented can be helpful. I understand that a lot of prompters don't need this style, or that some people just hate it unconditionally and need it to be different, that's just my guess for why it might've been reinforced during training.

Experienced prompters also typically know what they're doing, so they don't need the full explanation, only the bare minimum details that apply to this particular case. Again, users are different here, but the model doesn't know the difference in order to write it, so that's another way this can end up happening.