Comment by jboss10 1 day ago They can be ran on 32GB with 8GB VRAM. I don't think these will be on 16GB for a while. (35B MoE) 3 comments jboss10 Reply TheCycoONE 1 day ago I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that? slim 1 day ago use llama.cpp with cuda TheCycoONE 21 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
TheCycoONE 1 day ago I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that? slim 1 day ago use llama.cpp with cuda TheCycoONE 21 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
slim 1 day ago use llama.cpp with cuda TheCycoONE 21 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
TheCycoONE 21 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that?
use llama.cpp with cuda
The problem may be that it's a 7800XT which handles memory contention by freezing.