Comment by hank2000 19 hours ago how was it? I'm doing this today 1 comment hank2000 Reply robertkarl 18 hours ago I will report back... but I have to recommend this comment on a post about Qwen 3.6 https://news.ycombinator.com/item?id=47843466 by daemonologistit goes into detail about llama-server args; quants to try; and layer/kv cache splits. I plan to try the techniques there.
robertkarl 18 hours ago I will report back... but I have to recommend this comment on a post about Qwen 3.6 https://news.ycombinator.com/item?id=47843466 by daemonologistit goes into detail about llama-server args; quants to try; and layer/kv cache splits. I plan to try the techniques there.
I will report back... but I have to recommend this comment on a post about Qwen 3.6 https://news.ycombinator.com/item?id=47843466 by daemonologist
it goes into detail about llama-server args; quants to try; and layer/kv cache splits. I plan to try the techniques there.