Comment by tibbar
2 days ago
I wonder how the Chinese labs are training a 3 trillion parameter model on what has to be vastly smaller compute resources. If the U.S. compute advantage is persistent, it's hard to imagine that Chinese labs will be able to keep pace forever, as a matter of physics, but... so far they seem to be doing just fine.
Or they just don't actually have any compute access restrictions of significance? Chinese companies can just go use those GPUs in neighboring countries that aren't export-restricted, like Malaysia. Like ByteDance openly did: https://www.tomshardware.com/pc-components/gpus/chinas-byted...
and Tencent is rumored to have done via Japan: https://wccftech.com/china-tencent-gains-access-to-nvidia-bl...
And that's not even considering just smuggling the GPUs in by eg buying them in Singapore.
AI-specific chips also seem to be on the easier side to design & create relative to high performance CPUs & GPUs, so there's no particular reason to expect Chinese domestic designs to continuously lag behind. They have access to the same fabs, after all
Even ignoring chip export ban, Chinese companies have way less funding than American counterparts, maybe 1 or 2 orders of magnitude less depending on which company you look at. Deepseek’s recent big funding round being “only” a couple billion $ at $50B valuation, for example. Bytedance and Tencent are tech giants for sure, nonetheless they’re not Google kind of giant.
> Bytedance and Tencent are tech giants for sure, nonetheless they’re not Google kind of giant.
$186 billion and $105 billion revenue in 2025 respectively vs. $402 billion? Yes, Google is larger, but they're all in that same ballpark?
ByteDance's 2025 net income isn't that different from Anthropic's Series H funding even ($50bn vs $65bn respectively).
But this is all also ignoring how much of China is state owned (25% of the GDP!), so the available resource pool is dramatically larger than it would appear depending on what the government decides is important
Chinese companies likely aren't paying millions/year for their researchers but a tenth of it.
Gamers Nexus has a good video where his team travelled to China and did some actual investigative journalism: https://www.youtube.com/watch?v=1H3xQaf7BFI
Firstly, the export-restricted GB202s (e.g. 5090, RTX 6000 Pro Blackwell) are fabled in TSMC, and then packaged/made in... China before they supposedly have to be sold out (by US law; but not by Chinese law). You can immediately see the problem there.
Secondly, despite the supposed 'crackdowns' and et al, NVIDIA and their channel partners pretty much will sell to anyone in countries like Singapore without any questions.
Third, there's human "smugglers" who just physically carry em on trips, and Chinese customs is obviously not going to care about the US's laws on Chinese soil.
It's not like same parameter count models are identical, so that doesn't appear to be an indicator for quality, or even compute requirements?
There seems to be more to producing a better model than brute forcing parameter count after all.
Training and serving large models does require increasingly more compute, though. (The Chinese labs have clearly found some massive optimizations, but my point was that you'd think at some point even those optimizations wouldn't be enough to keep up with exponentially increasing model sizes.)
The Chinese just saved the world economy by draining their absurdly enormous oil storage reserves nobody knew they had, wouldn't surprise me if they had lots of hidden compute too.
Huawei Ascend chips were used to train DeepSeek v4 over 4 months ago, and they shared their kernel with the other Chinese labs. China also has their own DDR5 fabs.