Comment by earthnail
2 days ago
The analogy is a good analogy. It is used to demonstrate that a larger workforce doesn’t always automatically give you better results, and that there is a set of problems that are clear to identify a priori where that applies. For some problems, quality is more important than quantity, and you structure your org respectively. See sports teams, for example.
In this case, you want one foundation model, not 100 or 1000. You can’t afford to build 1000. That’s the one baby the company wants.
> In this case, you want one foundation model, not 100 or 1000. You can’t afford to build 1000. That’s the one baby the company wants.
I am going to repeat the footnote in my comment:
>> [1] It's even more facile in this context: you're looking to strike gold (AGI), so the analogy is trying to get one genius (160+ IQ) child. Good luck getting there by getting 1 woman pregnant at a time!
IOW, if you're looking for specifically for quality, you can't bet everything on one horse.
You're ignoring that each foundation model requires sinking enormous and finite resources (compute, time, data) into training.
At some point, even companies like Meta need to make a limited number of bets, and in cases like that it's better to have smarter than more people.