← Back to context Comment by luckystarr 7 days ago Tried it on Android and got "!!!!!!!!!!!!!" for answers. 5 comments luckystarr Reply gunalx 7 days ago The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage. aesthesia 7 days ago Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior. amelius 7 days ago Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error. jldugger 7 days ago After the third !, the probability of a fourth probably skyrockets =) verdverm 7 days ago That's what happens when you quant too hard. I'm working on quant strats and evals for the same underlying qwen 27b models.When I saw 27b on a phone, I thought not fitting, big phone, or aggressive quant. NVFP4 still takes 27G before KV cache.
gunalx 7 days ago The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage. aesthesia 7 days ago Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior. amelius 7 days ago Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error. jldugger 7 days ago After the third !, the probability of a fourth probably skyrockets =)
aesthesia 7 days ago Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior.
amelius 7 days ago Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error. jldugger 7 days ago After the third !, the probability of a fourth probably skyrockets =)
verdverm 7 days ago That's what happens when you quant too hard. I'm working on quant strats and evals for the same underlying qwen 27b models.When I saw 27b on a phone, I thought not fitting, big phone, or aggressive quant. NVFP4 still takes 27G before KV cache.
The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.
Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior.
Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?
Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.
After the third !, the probability of a fourth probably skyrockets =)
That's what happens when you quant too hard. I'm working on quant strats and evals for the same underlying qwen 27b models.
When I saw 27b on a phone, I thought not fitting, big phone, or aggressive quant. NVFP4 still takes 27G before KV cache.