Comment by sullivanmatt
2 hours ago
https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF
I have no idea how you could shrink this down to fit on any type of prosumer hardware though. I doubt it's possible.
2 hours ago
https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF
I have no idea how you could shrink this down to fit on any type of prosumer hardware though. I doubt it's possible.
Yes, you are right, and I was too optimistic. I'd have to level up.
This is the stuff I've put together and it is clearly too small for this:
https://jacquesmattheij.com/local-inference-with-large-langu...