Comment by elo02
8 hours ago
I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models
8 hours ago
I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models
The SemiAnalysis piece on this is long but very much worth reading:
> The company appears burdened by far too many disparate groups that are over-optimizing for certain metrics as opposed to delivering usable technology for the company as a whole.
> And because Meta has a reputation for throwing money at problems and executing at high speed, these U-turns end up becoming more costly versus other companies that take a more disciplined or conservative approach. Suppliers also lose faith when given design wins are later cancelled. This has lead to less supply chain prioritization on new designs. Some suppliers favor focusing on Amazon or Google designs due to Meta’s frequent reshuffling.
> Few inside Meta’s chip division have a full understanding of why the company bought Rivos in the first place, and those who championed the deal internally have since gone quiet.
etc etc
It goes into a lot of depth.
https://newsletter.semianalysis.com/p/metas-infrastructure-t...
he seems to be bullish on meta though and considers muse on slope than an intercept
https://newsletter.semianalysis.com/p/the-future-of-meta-sup...
The cynic in me says maybe they would be further along if they hadn't spent $80 billion on trying to build the "Metaverse" VR world. I've never met anyone who actually uses it and to the best of my knowledge it has very low mass market uptake.
https://finance.yahoo.com/sectors/technology/articles/mark-z...
Not exactly the best use of dollars and the labor hours of some of the best minds of our generation.
Particularly when you could vibe-code Metaverse VR for a lot less than $80bn.
Because lack of talent and organizational disfunction matters a lot more than you think. The reason why OAI and Ant are always at the top is because of this and I’d say compute is third on the list.
I would argue that they actually don’t lack talent, they have an insane bench of really smart people. What they lack is any sort of direction and leadership. They are a ship lost in the ocean and up until now have been lucky to find a few treasures along the their way.
> they have an insane bench of really smart people.
Filtered heavily into those who care only about money. Many don’t want to work there. Smart people have other choices.
1 reply →
because imagine starting work every week and finding that your dumb ass CEO pivoted the company again and is ruining other peoples lives, and he then reorgs the management again so you now have your 5th leader this year.
Morale and momentum are huge things in companies, Zuck has been murdering both of those in Meta since... well naming it Meta.
Turns out sitting quietly in a room and doing math is worth more than all the money and network in the world.
At this point in time what does meta get from releasing open weight models? Why devote the resources to it.
Why would there be any motivated person left in this place.
You can make the same argument for closed models. Why spend hundreds of billions training larger and larger models when you can just use Chinese models? Spend that money somewhere else further up the stack where there’s more value. Let China do the training since they’re so efficient at it.
Because without bigger players, Chinese models don't get anywhere. Playing catch-up is a radically different game.
1 reply →
as a big tech company you have the resources to make many bets and do a lot of things at the same time. it's good to have some specialists with knowledge of model training "just in case".
1 reply →