Comment by HarHarVeryFunny
4 hours ago
On a related note, I was reading yesterday that apparently the real bottleneck for Chinese production of AI accelerators is HBM production, not processors or ASML equipment.
The lack of ASML EUV machines certainly hurts, and pushing DUV so hard results in abysmal yields of good chips, but you can compensate by running more wafers or making smaller chips, and the net result is that Huawei's Ascend production volume is limited by CXMT's HBM capacity not processor dies.
The problem is that HBM manufacture requires many steps (die thinning, via drilling, plating, alignment) where the equipment used by everyone else (Samsung, SK Hynix, Micron) is also blocked by sanctions, so the Chinese are having to develop all of this themselves too, which they have, but yields are currently low, even when using shorter HBM stacks.
Tokens per second is almost entirely memory bandwidth at inference time, training obviously needs more compute but you can add more chips for that.
According to SemiAnalysis, both inference and post-training (RLVR) is mostly memory bandwidth bound. Only pre-training is compute bound, but it now only takes a small share of overall data center capacity.
https://x.com/EugeneNg/status/2099315982959616369
[dead]
China should invest in an analog inference chip. It's a hail mary but why not.
The two challenges there are firstly - that analog design has been a separate electrical engineering school for most of a century, so there are few who could design it - and secondly - that every single chip will have subtle variations in its computations, necessitating some sort of model finetuning per chip. Possibly the chip could be characterised at the factory, and ship with the characterisation data burned into a controller rom or something, but if that doesn’t pan out the whole thing is likely a non-starter.
If it could be made to work, you could run a fable-grade model in tens of watts.
They can probably afford to do both.
That would be like skipping land line phones for mobile...
Developing countries have done exactly that in many cases.
I'd assume that they have - but will keep mum 'till they have a major breakthrough or large-scale operational deployment to announce.
Huawei uses their own non-standard HBM called HiZQ probably not produced by CXMT.
Actually CXMT make both standard HBM3E, and also Huawei's custom HiBL and HiZQ non-JEDEC-standard HBM variants.
As I understand it, yields are different for all three:
25% for standard 8-Hi HBM3E
45-50% for HiZQ (used for Huawei's Ascend 950DT decode/training chip)
75-80% for HiBL (used for Huawei's Ascend 950PR prefill chip)
HiBL yield is best because it only users 4 layers, resulting in lower memory bandwidth (prefill is more dependent on compute than memory).
HiZQ uses 8 layers same as their 8-Hi HBM3E, but is higher yield due to using a Huawei specific base logic die that relaxes a lot of the specifications.
I read HBM yields are 25-30% (vs 80-90%) making them 3 to 5 times as expensive. They are 4-5 years behind, that probably means 1-2 in Chinese time.
Does that include yield from packaging?