Comment by paradite
6 months ago
Could it be that supporting multiple platforms open up more support tickets and adds more work to keep the software working on those new platforms?
As someone who built apps for Windows, Linux, macOS, iOS and Android, it is not trivial to ensure your new features or updates work on all platforms, and you have to deal with deprecations.
They already support ROCm, which probably introduces 10x more support requests than Vulkan would!
ollama is not doing anything. llama cpp does all that work. ollama is just a small wrapper on top.
This is not quite correct. Ollama must assess the state of Vulkan support and amount of available memory, then pick the fraction of the model to be hosted on GPU. This is not totally foolproof and will likely always need manual adjustment in some cases.
the work involved is tiny compared to the work llama.cpp did to get vulkan up and running.
this is not rocket science.
3 replies →
Ok assuming what you said is correct, why wouldn't Ollama then be able to support Vulkan by default out of the box?
Sorry I'm not sure what's the relationship exactly between the two projects. This is a genuine questions, not a troll question.
check the PR, it's a very short one. It's not more complicated than setting a compile time flag.
I have no idea why they have been ignoring it.
Ollama is just a friendly front end for llama.cpp. It doesn't have to do any of those things you mentioned. Llama.cpp does all that.
4 replies →