Comment by mschuster91
3 days ago
> The weights is the source code.
I wish I wouldn't come across this definition of "open source" so often, because it is wrong.
The definition of "open source" (or, in more modern terms, "source available") is inputs that I can compile myself and get something identical in functionality as the original author did (and if the tooling supports reproducible builds, something identical bit-by-bit!).
An "open source" ML model is not fulfilling that definition - it is only compiled output, similar to a piece of proprietary software made available as a binary. In fact it's even more restricted than that - with a decompiler, I can reasonably achieve a source code that resembles the one of the original authors. With an ML model, there is no way of reversing the "training" process.
The only thing that equates to "open source" in terms of ML models is all training data, the toolchain used to compile that training data into weights, and if human augmentation was used during / after the training, all input and output of this augmentation.
But no one of the large players will ever release that. First of all, the training data is heavily contaminated. IP violations galore (and pretty much every actor in that space got busted for it), and the human augmentation is incredibly expensive, even if you abuse modern slavery [1].
[1] https://www.theguardian.com/technology/article/2024/jul/06/m...
Even if you had all that you would get completely different weights out at the end, and you also don't have the resources to "compile" an LLM because the compilation can cost $100M. If you were given the training data but not the weights, would you consider that open source?