Comment by teaearlgraycold
12 hours ago
I don’t see how you could run Qwen3.8 27B on 16GB of memory that’s shared with Linux. Are people running models at 2bit quants? Are they even worth bothering with? I had assumed you go down to 4bit and if you need to go smaller you have to lose parameters.
You can, it's kind of cool to have this capability on something gaming at such a low price tag. Its not optimal for your time but beats nothing by a LOT. And that model is pretty reliable.
https://unsloth.ai/docs/basics/dynamic-3.0-ggufs