Comment by SwellJoe

I got their previous model working in their custom fork of llama.cpp (https://github.com/PrismML-Eng/llama.cpp). I haven't tried this one yet, but will find some time to benchmark it sometime this week.

Though this says mainline llama.cpp has their patches for Metal and CPU backends, so maybe it's simply "use current llama.cpp" if you have a Mac or fast enough CPU/memor to use the CPU backend.