Comment by simias
4 hours ago
I absolutely want that to be available.
The only way to make sure that a model was trained without hiding weird biases and backdoors is to monitor this process. It also allows anybody (with the hardware resources) to experiment with the model and how it's trained. Will most people want to do that? No, but most people also don't want to recompile their own kernel, that doesn't negate the value of Linux being FOSS.
I hope that, in the long term, I'll be able to use competitive, completely open models in the same way that I can run Linux on my computer.
You're completely right that the licensing of the training data is a big issue though (and I would argue that it should be a much bigger issue right now, even for closed models). I hope that eventually our understanding of the tech will let us build competitive LLMs using purely public domain/Creative Commons-style training data.
I realize that we have a long way to go... but that's a very worthwhile goal IMO.
Even if they did that, you would have no way of verifying that the data they upload and the model they claim to have trained is the weights they have shared. It’s a nonsensical idea.
If you want a trusted public model, we need to start a foundation that does it. The Linux of AI.
You’re going to need some serious money though.