Comment by fooker
9 hours ago
This trope was valid maybe in 2022.
Model training now is not a straight forwards process of input data -> run tools -> get model.
There's a whole lot of alchemy going on. We don't quite understand what works and what doesn't. Think of it like painting with water color and having to improvise very often.
The only advantage over water color is that we can revert to a working state.
And since none of the source is provided to make the thing, it’s not open source.
Really, it’s no more open source than a free calculator. It’s free, and you can use it to no or great effect. But, you sure as hell can’t make one.
So calling it open source is without question, wrong.
There's no program you run to 'make the thing'.
It's mostly ad hoc scripts and some pretty horrible hacks being run by a hundred engineers trying to improve a thousand different things at once with a hundred thousand GPUs.
I'm sure once we understand the tech better, the training process will look like running a program.