Comment by jacquesm
19 hours ago
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
They'd be wrong. Especially about Europe. Many companies are in a bind about this: they can't upload their confidential files to a service provider but they do want AI. So right now they end up with a choice: do nothing, breach their contracts, use a service provider that pinky promises not to share their data (which we have already seen they will violate) and run local models.
The latter is the reason you can't buy powerful GPUs for reasonable prices because a local model (GLM 5.3, DS 4, etc) + some hardware is all you need to solve the problem in a way that avoids the legal issues.
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center buildout, not because some mid tier European insurance co needs to serve deepseek
Yes, well, you're wrong about that. They're buying GPUs by the pallet here.
Mid tier insurance co is working with local inference providers and some of these now have thousands of GPUs up and running. Don't drink the HN cool aid that the EU is a backwater where nothing happens too much or you'll end up surprised one day.
The one thing where Europe really does lag is in training, that would be fair. But inference providers are popping out of the ground like mushrooms and there are some billion euro+ projects underway. We -unfortunately- also have some scammers that are bilking (or attempting to bilk) investors pretending they have more than they do, this is annoying but I would expect that other countries have similar players as well.
Wow, whole pallets!?
There are roughly 70 GW of datacenters under construction in the US right now, and a GW is roughly 500k GPUs, so 35 million GPUs.
9 replies →
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center build
Its not just the US ... Where do you think all those 5090's end up? Its just crypto mining all over again. Most people do not realize that China was the biggest crypto mining operation because of its cheap electricity and people willingness to take big risks.
The same happens now also, with people buying up 5090's (and lower class cards) in bulk, to strap into AI rigs. With Nvidia discontinuing the 32GB 5090s, we already see pressure on the lower end cards that had a better price / performance for a while now.
China will do China things *lol* ... People there are just more willing to take bigger gambles (and the cheap elec helps a ton). But that is also a factor why a lot of consumer level hardware gets under pressure. Its not just the supply chain to the enterprise market that is eating a lot for AI.
That does not matter so much anymore, when Huawei starts selling those NPUs for a fraction of the NVIDIA price. It's all downhill from there on for OpenAI, Anthropic and NVIDIA.
It's all downhill for OpenAI and Anthropic investors, because they will never recoup what was invested. NVDA is safe(r) - Huawei first needs to catch it, and as AMD learned, it's a fast moving target.
1 reply →
> US frontier labs are spending nearly a trillion dollars this year
They are not, not even close. I wonder if you're aware how money is being "spent" between Nvidia and US AI corporations!
The other choice is AWS Bedrock, if it happens to have the model you're interested in. They do have GLM-5.3. They don't have DS-4 or 4.1 yet, and updates are intermittent at best, but they're already on the trusted provider list for anywhere large enough to be interesting.
That may not be acceptable for many companies outside of the USA.
Depends on the parameters of "acceptable", but they do offer inference in non-US zones. If the company is already on AWS for meaningful workloads with sensitive data (which is not uncommon), they've already been through the hard part.
That being said it's not quite as simple as would be ideal, you do need to check whether there's an additional EULA on your specific model choice.
5 replies →
Yeah, about that... My kid works for a very small company. Chatted the other day about in-house inference vs ZDR providers and all. His team is trying to get access to GLM on AWS Bedrock. It is a second week they are trying to schedule a simple meeting to discuss what they need. Random silence, and back and forth from a new person each time. 2 WEEKS to just schedule a meeting... And as far as I know, no luck so far.
You do realize there are hundreds of datacenters in Europe and the US that rent GPUs to European companies, right? You don't have to buy a GPU or even figure out how to use one. Pay for SageMaker.
Europe isn't in a bind. They can, and have been, running open weights on inference providers around the world. And they pay a premium for US models and accept its risk for the same reason they remain US allies. The cost is worth it.
Yes, I do. Not sure what in my comment prompted you to write all this.