Comment by michalsustr
2 days ago
I talked with Hochreiter about 1.5y ago they should go for it and properly scale xLSTM to LLMs, not just playing around with few B params for experiments. They felt like they are too late to the game to get investment and traction. So much for European ambitions I guess...
Yes, but how would it have been if the EU governments just piled on, subsidized whoever had reasonable qualifications who wanted to try, straight-up funded accelerator development so that the chips don't just become investment in US AI companies, straight up built the supercomputers for training, straight up offered access to groups that had previously had reasonable success in training some model.
I don't think it would even be that expensive. I think with EU chips we could do it for 20 billion EUR + 5 billion EUR per year. After all, an H200 doesn't cost 40k to manufacture, probably one-tenth, and if we are making the chips we can get the money circulating just as the Americans have.
Add in some state effort to create suitable training data-- maybe encourage academics to create some things that are AI-friendly, and let all the people trying have that. There would hardly be anyone in the game other than us if we went for this package.
We already have VSORA. Maybe Euclyd is ready soon too. If enough are ordered we have 85% of the inference need gone, so that whatever accelerators are freed can be repurposed for supercomputing and training? Maybe we could have some organized effort to make many of the VSORA and Euclyd chips so that the chips we need for training are freed up while we wait for our own training chips.
Edit: So in the end I think the thing we need to do is to order enough VSORA Jotunn8 cards and enough Euclyd cards that we flood the inference market so much that the price of H200eds etc. drops and we can buy them second hand and assemble huge supercomputers out of them. This may require some diplomatic effort from the commission, something like 20B EUR almost immediately and some coordination, but it's easy, it's doable and it requires only being a little bit decisive. Since these cards are faster less memory will be tied up in serving the models, so it'd have a positive effect on memory prices too, which in turn would have a positive effect on European supercomputer centers, who are currently of course constrained by high memory prices.
What's their track record for other state sponsored high tech work?
I remember a few high profile failures; but perhaps on average these things work out?
I am actually not quite sure.
Here in the EU Rhea 1 was delayed and thus maybe a bit of a failure, but Rhea 2 might be good and seems to be on time. OpenChip seems to be on time too. The university work, people like Hochreiter etc. is clearly good.
What are the high-profile failures?
The EU spends EUR 300B yearly on agricultural subsidies and social programs.
Mm.
Still, maybe less money could be enough.
If Euclyd's chip costs 40k per chip and we want to substitute all worldwide H200 inference capacity, so let's say 700,000 H200eds and Euclyd are right about how good their chip is, then we'd need 28,000 Euclyd cards, this is only 1.120 billion.
So maybe even 1 billion for VSORA chips this year and then two billion for VSORA and Euclyd chips next year, with the split based on how well the chips perform and some money for the lagging firm so they can keep designing things too.