Comment by rpdillon
8 days ago
What? The obvious move is to never have switched to Ollama and just use Llama.cpp directly, which I've been doing for years. Llama.cpp was created first, is the foundation for this product, and is actually open source.
8 days ago
What? The obvious move is to never have switched to Ollama and just use Llama.cpp directly, which I've been doing for years. Llama.cpp was created first, is the foundation for this product, and is actually open source.
But there's much less that works with that. OpenWebUI for example.
Open WebUI works perfectly fine with llama.cpp though.
They have very detailed quick start docs on it: https://docs.openwebui.com/getting-started/quick-start/start...
Oh thanks I didn't know that :O
I do also need an API server though. The one built into OpenWebUI is no good because it always reloads the model if you use it first from the web console and then run an API call using the same model (like literally the same model from the workspace). Very weird but I avoid it for that reason.
1 reply →