you dont need to reproduce it, if you can finetune it.
The alignment in these models is really narrow, it doesnt take many finetuning steps to get out of the alignment basin.
the alignment is not data centric but done after the fact using RL...
You can also modify binaries. This doesn't address the benefits of reproduction more than a smidgeon. How can you answer "why the fuck does it work like this" if you can't inspect the process that built it? There's no replacement for "what was this built from?"
Of course, corporate/national investments need some moat, so I don't expect open source models to be competitive for a few years
you dont need to reproduce it, if you can finetune it. The alignment in these models is really narrow, it doesnt take many finetuning steps to get out of the alignment basin.
the alignment is not data centric but done after the fact using RL...
You can also modify binaries. This doesn't address the benefits of reproduction more than a smidgeon. How can you answer "why the fuck does it work like this" if you can't inspect the process that built it? There's no replacement for "what was this built from?"
Of course, corporate/national investments need some moat, so I don't expect open source models to be competitive for a few years
You can write mods for Minecraft too but that doesn’t make Minecraft open source.
Yeah, but don't call it open source if it's not.