Comment by draxen
16 hours ago
One of the fascinating dynamics around Chinese open-weights models is the competition among American (and some European) inference providers to deliver the best inference performance on them. They've increasingly beaten the Chinese model makers themselves on inference throughput and latency.
Together.ai, Baseten, and others have already achieved higher throughput, lower latency, and lower E2E latency for DeepSeek V4.1 Flash 1017 than DeepSeek's own inference service.
That’s just likely the NVDA chip ban at play. I wouldn’t bet on that being an enduring truth
Why? Next gen chips are coming very soon (VR) and VR Ultra will follow, with Fyenman afterwards. Huawei will find it very difficult to compete with those products.
Well presumably the load on the deepseek inference service is a lot higher, probably most users of deepseek use it through them (presumably more in SEA, less in US/EU). And for the users from the EU/US, if they are going to use deepseek, their usage is distributed among all those providers (because of the need for zdr)
these labs have access to far better NVIDIA chips than deepseek