Comment by peri-cl
3 days ago
I think they're sandbagging Qwen pretty heavily by benchmarking Meta's model at 4-bit, against Qwen only at native BF16 (and nothing else shown). That's not a quant most users would choose.
I get that this is cross-brand marketing for Apple's partner, Meta; but really Muse Spark isn't even in the top *ten* models I'd consider if I had a 512 GB M5 in my paws. Or a cluster of them.
They clearly list out the quants in Table 1.
Most of the following tables show relative speedups; not tok/s numbers.
I agree no home AI users should really use bf16^; but I see two defensible justifications: (1) showing that their new hardware's speedups scale to full precision too [crucially; NVIDIA is focusing more and more on lower quants], and (2) trying to sell to the crowd running IQ3XXS or something on 16GB.
^q8 ~= bf16 is a good general rule of thumb. And friends don't let friends quantize KV cache.
> cross-brand marketing for Apple's partner, Meta
Partner!? I don't see where you get that from the article, and Apple & Facebook have traditionally not been best of friends (see everything iOS implemented to limit Facebook's overreach).
I'm guessing they chose Muse because 1. It's not Chinese so it'll look good for Trump's AI cronies (Apple is in the administration's spotlight) and 2. It's fairly recent and thus on people's mind.
Eh, Apple spent years suing Samsung while also buying parts from them for their products. You can be a partner and an enemy at the same time at the high levels of the game.
I mean that they're doing joint marketing on this particular thing. Muse Spark marketing features Mac Studios quite prominently, and Apple's material spotlights Muse Spark.
For what it’s worth, Muse Glimmer is by far the most usable model on my 32GB M1 Max. It gets stuff done very competently while Qwen never gets done reasoning.
[dead]