Comment by colingauvin
2 hours ago
DS4 Flash Q2/Q4 mixed quant fits on a DGX Spark (a $4000 device which is not particularly unheard of expense for Apple customers), and is indistinguishable for me from Opus for my personal daily use/assistant benchmarks[0].
[0]https://humanparadox.org/local-vs-frontier-benchmarks-for-my... - note here I tested Q8 but have found no difference at lower quant.
Indeed. I like using Macs mostly, and the bargain M1 Max MBP I am using for local LLMs is a fabulous experimentation platform and does loads of other stuff well, so I am in no rush, but if I reached the point of buying dedicated hardware for an LLM, I'd be looking at the DGX Spark machines.