Comment by antonvs

What's weird though is how consistent it is, even across models to some extent. Were the RLHF people given a really specific style guide?

While I agree that no-one used to write like that as a whole before, all the elements can be found in different places. Short sentences to avoid discouraging poor readers. Maximally impactful statements are commonly used in marketing or other business communication that's focused on selling what it's saying. A bullet-pointy style is used in many kinds of business communication. Etc.

It makes me wonder if part of what happened was a kind of melding of common styles from several different kinds of writing.