← Back to context

Comment by nik736

3 hours ago

The weird thing about this model is that it spits out Chinese characters just like the chinese models when chatting with it in German. Happened to me quite often during the past few days.

15 comments

nik736

Reply
speedyStuff_  3 hours ago

Opus 4.8 and Fable 5 did the same; they would output random Chinese characters and then say “I made a typo, let me fix it” and it’s just them removing that stay Chinese characters. Haven’t seen this since 5.1 family of models.

bigwheels  3 hours ago

Does this suggest Mistral is distilling Deepseek et. al.? Can distillation keep passing through with high fidelity, or is it lossy like copying a VHS tape?

  • orbital-decay  3 hours ago

    It doesn't, even remotely. It's a typical failure mode in all models, some do this more often. It's not related to samplers either. Not necessarily Chinese characters either, they likely just occupy a large portion of vocabularies. For example Gemini 2.0 used to glitch in Telugu and Ukrainian, Claude 2.x and 3.0 used to switch between random languages for a few words when writing in non-English, once in a blue moon.

aftbit  2 hours ago

I've seen this quite a bit on bad quants as well. I ran an early and very lossy quant of one of the Gemma3 models, and it loved to just stick Chinese and Korean characters into the text, sometimes without seeming to realize it had mixed languages.

bmk44  3 hours ago

Ha, ha. Just like when Tilly Norwood switched to Chinese while interviewing with Piers Morgan [1].

On a serious note, I remember getting spooked when I saw a bunch of Chinese characters dumped out into some of my server logs (pre-LLM era). When I pasted the Chinese content from my log into google translate, it was just complete nonsense. Later I found a bug that was dumping raw binary data into my log file which interpreted as Chinese characters when viewed in a text editor. Perhaps similar bugs existed in the models.

[1] https://www.youtube.com/watch?v=8VOfGcTqqq8

whiplash451  3 hours ago

Can you please share: - what specific input you see this issue with? - which interface you used (API, playground, specific sdk version or a wrapper app)? - whether the chinese chars appear in the thinking section or the output?

Thanks.

  • nik736  2 hours ago

    Sorry it was not one time but it happened easily 5-10x during the past few days on different topics and prompts and I didn't store them. It felt like when pushing the model on some specifics in some niche topics in German (I have been prompting in German and the model responded in German). It was the acutal output and not the thinking traces and it was through the API. I jokingly asked it what the character meant and it didn't even make sense in the context so it seemed like random chinese characters.