Comment by crystal_revenge

There are plenty of people theoretically capable of doing this, I secretly believe some of the most talented people in this space are randos posting on /r/LocalLlama.

But the truth is to have experience building models at this scale requires working at a high level job at a major FAANG/LLM provider. Building what Meta needs is not something you can do in your basement.

The reality is the set of people who really understand this stuff and have experience working on it at scale is very, very small. And the people in this space are already paid very well.