Comment by msp26
21 hours ago
Google is singlehandedly carrying western open source models. Gemma 4 31B is fantastic.
However, it is a little painful to try to fit the best possible version into 24GB vram with vision + this drafter soon. My build doesn't support any more GPUs and I believe I would want another 4090 (overpriced) for best performance or otherwise just replace it altogether.
You could keep multimodal projector (understanding of audio, images & PDFs) in system RAM with `--no-mmproj-offload` in llama.cpp. Of course, then it is not accelerated with GPU, but you save its VRAM.
Interesting, I might try that, thanks!
Qwen is still better that Gemma though. Also you can tune it more for different tasks, which means that you can prioritize thinking and accuracy versus inference speed.
Qwen is better at some things (code, in particular), but Gemma has better prose and better vision. At least, it feels that way to me.
gemma is also just way faster. i dont wanna wait 10min to get a 5-10% better answer (and sometimes, actually worse answer).
best is to use your own model router atm, depending on the task
4 replies →
Genuine question: how do you tune it?
I thought "fine-tuning" meant training it on additional data to add additional facts / knowledge? I might be mistaking your use of the word "tune", though :)
You can fine-tune relatively easily in Unsloth Studio.
It’s a heck of a lot faster too.
Yes I would just go with qwen.