Comment by ismailmaj
21 hours ago
1.5 TB/s 1.3 PFLOPS with no fast multi-GPU, it's a consumer card that pretends to be a server card.
It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while.
That's the point I believe fast local 2-4 user service for cheap on very large models.
You have to pretend it's for servers to keep investors happy, it would be banks and similar with in house servers that have to run local large models buying these for experiments and cybersecurity. Likely.
Reminds me of the early 2000s era where we'd fight in my household if too many people used the internet at the same time, can see that happening here if it's not 2 to 4 users but 10.
I mean in companies you can just buy more, and I think AI compute is going to get a lot cheaper in the next 4-5 years, photonics are just around the corner, assuming AI is even 1/10th as good as AGI they should help prove substantially useful for just pushing photonics based computing over a couple decades sooner.
1 reply →