← Back to context Comment by jboss10 1 day ago They can be ran on 32GB with 8GB VRAM. I don't think these will be on 16GB for a while. (35B MoE) 3 comments jboss10 Reply TheCycoONE 1 day ago I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that? slim 21 hours ago use llama.cpp with cuda TheCycoONE 20 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
TheCycoONE 1 day ago I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that? slim 21 hours ago use llama.cpp with cuda TheCycoONE 20 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
slim 21 hours ago use llama.cpp with cuda TheCycoONE 20 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
TheCycoONE 20 hours ago The problem may be that it's a 7800XT which handles memory contention by freezing.
I have 32GB of RAM with 16GB VRAM and I haven't had a lot of luck running larger models like this. Are you able to expand on that?
use llama.cpp with cuda
The problem may be that it's a 7800XT which handles memory contention by freezing.