Comment by beastman82
7 hours ago
can confirm.
I dont' know why people spend huge money on these and Spark. The 5090 is running qwen 3.8 at 200+ tps!! That's 1-2 orders of magnitude faster.
7 hours ago
can confirm.
I dont' know why people spend huge money on these and Spark. The 5090 is running qwen 3.8 at 200+ tps!! That's 1-2 orders of magnitude faster.
Have a 5090, and yes it's very fast. But it's like the worst ADHD team member and requires constant supervision and review from larger models. It's context size on-card is good for super, suuuuuper shallow precision work. The gb10/spark on top of it, that thing can refactor enormous monorepo architecture. The time it takes the 5090 to compact, reiterate and execute a plan is often the same time as the gb10.
[flagged]
How are you deciding which work to send to the 5090 vs a frontier model, or making the two work together nicely?
Correct is much more important than fast for me, but if I could get correct and fast, that would obviously be amazing.
People don't buy Sparks and M5 Ultras to run a 27B model - you buy it to run an MoE model like Qwen Next which this M5 excelled at.
Exactly; when I first got my RTX 5070 Ti (16gb, to game with!!!, upgrading from VEGA56), I loaded then-latest Qwen3.6 (~30B, cannot remember exactly). My only prior LLM experience was with models <8gb, primarily llama3.1.
My technical-expert twin played around with these LLMs, for about an hour, and then correctly reasoned "it's able to be WRONG, faster."
This seems apt. My next LLM machine will be closer to 96gb+ vRAM.
Once I get some kind of settlement after getting beaten up by a cop my first purchase will be some RTX Pro 6000s.
Is a 5090 still cost efficent when it is (currently) unobtainable? Or when obtainable only at current prices (min. $6500 USD)?
Personally I think the price is way too high right now. It’s a power hungry gaming GPU. The efficient single card equivalent would be a 4500 Blackwell which launched at about $3500. Or you could get a 9700 32GB or an Arc B70 for well under $2k, today. You only buy a 5090 if you want absolute speed.
32GB is still not that much. I would rather get a Spark and have the RAM to experiment with larger LLMs, even if it was slow.
Because you can run Qwen 3.8 Flash Next, Laguna S 2.1 and other medium-sized models that simply don't fit on a 5090?
A) the macos value add is enormous if you have any investment in the ecosystem, B) for me at least a GPU is completely useless for anything but being a token generator.
> for me at least a GPU is completely useless for anything but being a token generator.
No thanks to the "macos value add" that forces you to use Metal while Valve customers frolick in Protonland.
> No thanks to the "macos value add" that forces you to use Metal while Valve customers frolick in Protonland.
Crossover works on macos, too. So does moltenvk, so does vanilla wine, etc etc. You can run most games without a hitch these days (allegedly, according to /r/macgaming). But I don't play video games so a GPU would probably be better off in some kid's computer.
1 reply →
A 5090 has a 1.79TB/s memory bandwidth. Qwen 3.8 27B NVFP4 is 22GB. You cannot generate tokens faster than the weights can traverse the GPU memory, so that makes max generation speed without MTP to be 81T/s. Say MTP is giving you 0.5 acceptance rate (very good), that is 1.5 * 81 is 121T/s. Even with a perfect acceptance rate you would only get 162T/s.
It really does get it, because MTP is usually run at "3 token" depth. It's pretty shocking to watch
Off the top of my head, I'm guessing we're missing sparse attention. But I'll run your challenge through and see where the gaps are. I promise I'm telling the truth :)
I think you’re missing that MTP can predict more than 1 token in advance.
Qwen3.8-Flash-Next is pretty damn worth the extra ram you need.
same reason they spend huge amounts of money on rolexes when seikos work better (the tech crowd isn't immune from vanity).
If you seriously think apple products are nothing but a status item, you're deluding yourself and probably have been for decades.
If you seriously think apple cares about anything other than cell phones, you're deluding yourself and probably have been for decades.
7 replies →
This 1000%. Data centres don't equate to medium sized labs and businesses. A stack of Macs is up and running without digging trenches, an electrician on staff and a department of PhDs to justify the spend.
3 replies →