Comment by bilekas
2 days ago
But you're assuming I'm using the Olama studio. This model as far as I see doesn't have a gguf download.. Unless I'm missing something on the page.
If I want to download the model myself, it's not clear. I thought it was supposed to behave like a package manager. But even in nuGet I can download a zip of the package.
What on earth are you talking about? llama.cpp != Ollama. You can (and should) just use llama.cpp directly. Upstream llama.cpp can take the shorthand huggingface path and automagically download it into a cache folder as part of the launch. Have you read any of the docs?