Comment by John23832

> I cannot judge his research output at Meta but he failed pretty bad at the LLM race. Since so many other organizations succeeded at creating open source models of far higher quality at much lower cost, it would be instructive to understand what exactly went wrong there.

What? Until the Chinese jumped in Llama was the premium open source model. The reason that the Chinese were successful at MOE was just that they were limited with chips and had to think outside the box. US labs are operating on the power law. They also, arguably, distilled from western models (llama).