Comment by HPsquared
13 hours ago
Compiling source code also takes computing resources, only the scale is different. It's a very close analogy: source code with training data, and compiled binaries with model weights.
The weights are literally a binary blob.
The weights are the modifiable representation. You modify them with gradient updates.
They are a cooked stew; the ingredients have already been chopped and mixed together. You can add things, yes, but you can't inspect the ingredients.
If we start with the same ingredients and independently make our stews they're likely still going to taste a bit different due to the non deterministic nature of the process. Isn't it similar with training LLMs at scale? Do you get identical weights if you do multiple runs?
2 replies →
I can also open broadcom's antenna firmware in any hex editor. Why are you arguing that makes it open source?
The antenna firmware is software. Model weights are essentially learned settings, and they are not software.
Note also that software is copyrightable and model weights aren’t (in the US anyway).
2 replies →
You can hand-patch a binary, too.