Comment by loveparade
5 hours ago
I give it one to two more years before open source models have fully caught up. Products are commodities and models are commodities too. GPUs cores are still hard to get for inference at scale right now. They need a platform with lock in but unsure what that would look like and why it wouldn't be based on open source models.
What does "fully caught up" mean in the context of an ever evolving technology? I think I'm in support of open weight models (though there are safety implications), but these things aren't cheap to train and run. This fact alone gives no incentive for leading labs to release cutting edge open weight models. Why spend the money then give the product for free?
Now if "fully caught up" means today's level of intelligence is available for free in two years, by then that level of intelligence means very little
It’s never free your shifting costs from paying a company for their api use vs the power costs of running it locally.
Yeah I don't understand it, it's a marathon with three companies perpetually a minute ahead, and people keep saying "I expect the stragglers to catch up".
The only thing I can see them meaning is what you said, "in a minute the stragglers will be where the leaders were a minute ago", which, yeah, sure.
By my estimation, there is a point where these models are "good enough" for the vast vast majority of all appropriate tasks, after which point further investment by the major labs will have diminishing returns. While they might stay ahead by some measure, the open models will be good enough too, and I assume significantly cheaper like they are now.
Or AGI hits and this theory collapses, but that's feeling less likely every day.
It's not a marathon, or any race. There is no a finish line. It doesn't matter that much that someone is a minute ahead.
It makes perfect sense if you think things cannot improve indefinitely
2 replies →
Why do people have such faith in "open source" models? There's nothing "open source" about them. No individuals have the ability to train such modules. They are just released by companies to commoditize the models of the competition.
If Mythos is the endgame, companies won't release open-weight equivalents, and no private individuals have the capital to train such models.
The open models cannot be taken away. Anyone with the right hardware can host these. Unlike the API/subscription services where you can be banned from, may have drastic price increases or reduction of their limits.
What is the transition state where people start using open source models that you imagine actually happening?
Play out a scenario. An open source model is released that is capable as Mythos. Presumably it requires hardware big enough that running it at home is unfeasible. You are imagining that individuals can run it in the cloud themselves for cheaper than api tokens would cost? Or even small companies? And that Anthropic and OpenAI won't be able to cut costs deeper than their competitors while staying profitable?
If it is fundamentally a commodity, that means "running it yourself" also isn't really interesting as a proposition. Many of the world's biggest companies sell commodities. It's a great business to be in if you can sell them cheaper than anyone else.
The value add here isn't the model, it is "having a bunch of compute and using it more efficiently than anyone else".