Seems I'm missing something. Does this model support other inputs?
Image outputs are supported, videos I'm not sure but I don't think that's an output, just a preview of the equirectangular example, so, same question here, what does this model outputs that isn't supported?
Not all architectures are supported by llama.cpp . The GGUF format encodes the NN in a standardized way, but then you need code that can use that NN structure.
I understand that llama.cpp could only output text, last time I checked (I do not know how to find a good source for that though).
I haven't used llama.cpp for image generation either but I recall an issue about it. Unfortunately I can't pinpoint it now and there is the older closed issue https://github.com/ggml-org/llama.cpp/issues/4408 so unless mtmd supports also multimodal outputs out of the box safe to assume output is still limited to text.
Seems I'm missing something. Does this model support other inputs?
Image outputs are supported, videos I'm not sure but I don't think that's an output, just a preview of the equirectangular example, so, same question here, what does this model outputs that isn't supported?
Not all architectures are supported by llama.cpp . The GGUF format encodes the NN in a standardized way, but then you need code that can use that NN structure.
I understand that llama.cpp could only output text, last time I checked (I do not know how to find a good source for that though).
See https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... , the
...
I haven't used llama.cpp for image generation either but I recall an issue about it. Unfortunately I can't pinpoint it now and there is the older closed issue https://github.com/ggml-org/llama.cpp/issues/4408 so unless mtmd supports also multimodal outputs out of the box safe to assume output is still limited to text.
It's a diffusion model, completely different from autoregressive attention models.
like https://github.com/ggml-org/llama.cpp/tree/master/examples/d... ?
At the risk of stating the obvious llama.cpp isn't just about LLaMa as https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... someone else pointed out.
2 replies →