Comment by awongh
2 hours ago
In a recent Dwarkesh podcast Dylan Patel breaks down how little compute the chinese labs actually have- not even the fact that they don't have access to new Nvidia chips and they're stealing them through shell companies- just that, even if they have cheap electricity, the compute just doesn't compare. Maybe even two orders of magnitude less. They couldn't get it even if they had the money. And if you look at how much more efficient newer chips are, that cuts the effective compute in half again. The conclusion was that they are at least 2-3 years behind.
For frontier labs the current compute seems to be driving model progress (in training) at least to some degree, even without true RSI, and this seems like it'll continue to keep any chinese model from drawing even with the frontier labs, at least for the foreseeable future.
Inevitably the chinese government will drive more funding in chip fab technology and the money will come around to build chinese data centers, but who knows how far off that is. A few different things in the tech tree need to fall into place. It doesn't seem like it'll be next year.
The counterpoint to that, though, is that the Chinese companies have to figure out how to be competitive, regardless of their significant compute deficit. And, as far as I can tell, they're actually doing that. They're trailing the frontiers in model effectiveness, but not by years. It's single digit months.
If there is no upper bound how how these things scale with compute, and if China does really begin to catch up to Nvidia (and they're probably not going to feel encumbered by US patents for domestic AI hardware, given how important AI seems to be to the Chinese government), there will come a day when China leapfrogs the US on AI.
I think on the timescale of 10 years, that's a super likely scenario. But will it be any sooner?
For instance a Chinese EUV machine seems like it's very far away. Even if they have (steal/borrow) the necessary IP.