Comment by diseasedyak

9 hours ago

I've been running Qwen3.6-27B-IQ4 (4-bit quantized) locally and it's been great. I can't run the non-quantized version as I only have a 4090 w/24 GB of VRAM and it won't fit and leave any context room, but the quantized version only uses 18GB.

0 comments