Comment by yogthos
1 day ago
This is precisely why we need more projects like this https://github.com/bigscience-workshop/petals
There's no fundamental reason why models couldn't be developed and trained using community efforts. It might not be as fast and efficient, but it's definitely possible.
See the recent development of DiLoCo at Nous Research and Prime Intellect.
oh that's very cool, exactly the kind of stuff we need