Comment by Archit3ch
2 days ago
Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.
I will be delighted to try. I have a 128gb macbookpro m4 waiting for this.
ds4 is pretty optimized for this btw - https://github.com/antirez/ds4
This is the way.
Link it already!
Followed you on GitHub to get notified when you are!