Comment by sleepyeldrazi
6 hours ago
Finetuning takes little resources, the base model training is the slow and expensive part. Architecturally 3.5 models are identical to their 3.6 counterparts, that is why there is a consensus that those are probably finetunes and not re-trained from scratch, like you will se many people publish their own on huggingface.
Understood, but look at their larger cadence over the years and the breadth of models. They are clearly not all finetunes. Meta for all its billions, doesn't have anything comparable.
In the china AI scene, there seem to be two separate types of companies.
Companies or labs like deepseek that produce less but larger and more innovative models, so seem to be more research oriented.
then there are companies like z.ai (GLM), Minimax, and Qwen which focus more on commercializing the AI and so produce far more versions, but with far less improvements between them (usually fine tunes)
Commercial providers like anthropic probably do the same thing, maybe even without labeling it like a different version if the model is similiar enough.
> Meta for all its billions, doesn't have anything comparable.
Maybe nothing released to the public. I don't know that all of their models are public. I think all they really care about is that they aren't relying on one or two cloud providers for a critical piece of their infrastructure.
competent leadership goes a long way