Comment by numpad0

Isn't it just it getting increasingly incoherent as non-English data fraction increases?

Last I checked, none of open weight LLMs has languages other than English as its sole dominant language represented in the dataset.