Comment by xienze

I think it's extremely quantization and engine specific. I run Gemma4-31B at FP8 on vLLM and it's fantastic, no issues anymore[*].

* I will say that early on there were a LOT of issues with the chat template, across all engines. I dunno who decided using crappy Jinja templates was a good idea, but clearly it has its limitations. In the latest version of vLLM (0.25) they've ditched the Jinja templates for an in-engine parser and I've seen no issues.