Comment by MatthiasPortzel
1 day ago
I posted this elsewhere, but Unsloth says the 27B model should run in 18GB. That leaves little RAM for other tasks, but it depends on your tolerance for slowness I suppose. I haven’t tried it in 24GB so report back if you do.
No comments yet
Contribute on Hacker News ↗