← Back to context Comment by satvikpendem 5 months ago Just use llama.cpp or Unsloth Studio which wraps it, I don't know why anyone use Ollama anymore. 1 comment satvikpendem Reply verdverm 5 months ago I switched from llama.cpp to vLLM because of prompt cache bugs in qwen/gemma modelsThis is a good starting issue with a bunch of linked/relatedhttps://github.com/ggml-org/llama.cpp/issues/22746
verdverm 5 months ago I switched from llama.cpp to vLLM because of prompt cache bugs in qwen/gemma modelsThis is a good starting issue with a bunch of linked/relatedhttps://github.com/ggml-org/llama.cpp/issues/22746
I switched from llama.cpp to vLLM because of prompt cache bugs in qwen/gemma models
This is a good starting issue with a bunch of linked/related
https://github.com/ggml-org/llama.cpp/issues/22746