Comment by diggan
10 hours ago
> fork them
This requires all of the "source" to be available. For PyTorch and a bunch of other projects, this is trivial as all the source is straight up on GitHub. But for proprietary things like Llama, it's really hard to fork something when you don't even have access to what they used to build it (software-wise, not even thinking about the hardware yet).
How could you fork something like Llama when Meta don't even speak clearly about what data they used, literally none of the training code is available, and you have to agree to terms and conditions before you're "allowed" to do anything with it?
> you have to agree to terms and conditions before you're "allowed" to do anything with it
I don’t have experience with this so I’m taking it at face value; if this is true, it’s so strange that I have an idea of this being an “open” model. As in, not that they PR’ed to make people believe it but that people who were required to accept those terms seem to believe it (as users seem to repeat it). Seems a little bit of critical thinking should dispel that notion. Are there any, more reasonably open models? Is LLaMa just called open because it’s the most accessible?
> Are there any, more reasonably open models? Is LLaMa just called open because it’s the most accessible?
Indeed there are! They aren't exactly SOTA, but they're 100% open source and you could build them yourself from scratch, granted you had the compute, knowledge and time for it. OLMo 2 from Ai2 is probably the most notable one.
I think Llama is called "open source" because that's what Meta, Zuckerberg and the Llama website says it is, and people take it at face value. Then people see "Oh but it's free, who cares about the license?" not understand how we got here in the first place...