Comment by tom_0
5 days ago
GGML still runs on llama.cpp, and that still requires CUDA to be installed, unfortunately. I saw a PR for DirectML, but I'm not really holding my breath.
5 days ago
GGML still runs on llama.cpp, and that still requires CUDA to be installed, unfortunately. I saw a PR for DirectML, but I'm not really holding my breath.
You don't have to install the whole CUDA. They have a redistributable.
Oh, I can't believe I missed that! That makes whisper.cpp and llama.cpp valid options if the user has Nvidia, thanks.
Whisper.cpp and llama.cpp also work with Vulkan.
1 reply →