If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
Not sure why it matters whether weights are software or not.
One can equally argue that firmware is just settings for the CPU.
If weights aren't copyrightable then doesn't this argue against the idea of them being able to have the quality of open-sourceness? Meanwhile the training code could be copyrightable.
They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.
If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
They don't even say what the ingredients are.
1 reply →
I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?
The antenna firmware is software. Model weights are essentially learned settings, and they are not software.
Note also that software is copyrightable and model weights aren’t (in the US anyway).
Not sure why it matters whether weights are software or not.
One can equally argue that firmware is just settings for the CPU.
If weights aren't copyrightable then doesn't this argue against the idea of them being able to have the quality of open-sourceness? Meanwhile the training code could be copyrightable.
1 reply →
You can hand-patch a binary, too.