Comment by zbentley

My unprovable pet theory is that, especially for writing about technical topics, the RL process has an open-ended way to weight things for quality: textbooks and first-party docs preferred to old stackoverflow answers and obscure blog/forum posts, and so on. The open-endedness of that quality gradient results in something in the RL process (maybe HF, maybe not) massively over-weighting some very small corpus of “quality = near infinite” content. The distribution of quality scores that inform the degree to which RL affects output has some extremely influential outliers, in other words.

Whatever that small corpus is, it contains some very specific grammatical tics, and that’s how we get Claudish.

Anyone who thinks a company/project as big as Anthropic/Claude wouldn’t make such a big mistake should take a look at how Azure cross-account federated login used to work.