Comment by MrDrMcCoy
2 days ago
Official llama.cpp releases ship with huggingface support. If you don't want to download it yourself, you can just use the `repo/model:quant` convention and it will handle downloading locally for you.
2 days ago
Official llama.cpp releases ship with huggingface support. If you don't want to download it yourself, you can just use the `repo/model:quant` convention and it will handle downloading locally for you.
But you're assuming I'm using the Olama studio. This model as far as I see doesn't have a gguf download.. Unless I'm missing something on the page.
If I want to download the model myself, it's not clear. I thought it was supposed to behave like a package manager. But even in nuGet I can download a zip of the package.
What on earth are you talking about? llama.cpp != Ollama. You can (and should) just use llama.cpp directly. Upstream llama.cpp can take the shorthand huggingface path and automagically download it into a cache folder as part of the launch. Have you read any of the docs?
Plain question for you, where can I find the gguf model of this to direct download ?
Here: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main
Excuse me, but thats a direct link you've just sent. I asked where I can find the links. I like to believe in the source of truth.
They shared a lot of links, I'm struggling to find yours. Where did yours come from ?
Who is Unsloth AI? Have they modified the model ? Is this really the source of truth ?
Do you see how steep the barrier for entry is to do anything right ?
unslothai is not a name qwen has ever used. So you're sharing a link to a model that isn't from the owner, while saying it's the owner's. I'm not comfortable with that, and I want AI to be a better tool.
6 replies →