Comment by moffkalast
6 days ago
> slightly worse on knowledge and tool calling
Worse than Gemma at tool calling? Gemma's already bottom tier at that (at least when there's Qwen to compare to), that would just be unable to do tool calling at all.
6 days ago
> slightly worse on knowledge and tool calling
Worse than Gemma at tool calling? Gemma's already bottom tier at that (at least when there's Qwen to compare to), that would just be unable to do tool calling at all.
I think it's extremely quantization and engine specific. I run Gemma4-31B at FP8 on vLLM and it's fantastic, no issues anymore[*].
* I will say that early on there were a LOT of issues with the chat template, across all engines. I dunno who decided using crappy Jinja templates was a good idea, but clearly it has its limitations. In the latest version of vLLM (0.25) they've ditched the Jinja templates for an in-engine parser and I've seen no issues.
I think that depends on how you run it. Llama.cpp has several fixes for the somewhat unusual tool call semantics in Gemma 4. I don't think I have noticed any issues.