Comment by tyfon

2 days ago

To add for me (and claude assisted but tuned myself), here is for 3060 12gb + 5950x 128 gb ram without image support: https://pastebin.com/HsAUBGSY

Here is the chat template I used (and renamed to qwen.jinja): https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/r...

For image support, drop to NGL 24 and add:

  --mmproj-url https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/mmproj-F16.gguf \

  --no-mmproj-offload \

  --image-min-tokens 1024 \

And drop the --no-mmproj

2 comments

tyfon

Reply
tommica  2 days ago

Thanks for this! I got the same card, but not the same CPU nor RAM, but this is a great starting point to start tweaking

acidburnNSA  2 days ago

How many tokens/second you getting? I have the same CPU but a 5070 and only 64 GB of ram. I just got llama.cpp built and am now hitting a whopping 5-6 tokens/s.