Comment by lelanthran
2 days ago
> In this case, you want one foundation model, not 100 or 1000. You can’t afford to build 1000. That’s the one baby the company wants.
I am going to repeat the footnote in my comment:
>> [1] It's even more facile in this context: you're looking to strike gold (AGI), so the analogy is trying to get one genius (160+ IQ) child. Good luck getting there by getting 1 woman pregnant at a time!
IOW, if you're looking for specifically for quality, you can't bet everything on one horse.
You're ignoring that each foundation model requires sinking enormous and finite resources (compute, time, data) into training.
At some point, even companies like Meta need to make a limited number of bets, and in cases like that it's better to have smarter than more people.