Comment by hintymad
3 hours ago
> That's why we had huge backlash w
Training a frontier model is prohibitively expensive, so it’s understandable that open weights aren’t truly open source. In fact, I worry whether open weights will even survive[1] once the current AI cycle cools down. An even bigger issue is that as long as model development remains this capital intensive, a genuinely open-source community around frontier AI is virtually impossible. That will have massive implications for our careers. The more capital concentrates at the top, the less leverage we’ll have in the job market.
[1] Arguably hardware companies like Nvidia have incentives to subsidize open models to drive GPU demand. That said, I'm not sure how sustainable and sufficient that incentive can be.
We could create an open-source SETI@HOME but for training open source models?
Covenant-72B
https://arxiv.org/abs/2603.08163
https://huggingface.co/1Covenant/Covenant-72B
Training neural networks is requires extremely high speed interconnect between the humongous GPUs
Someone’s surely doing it atm we just don’t know yet. And then we scan all books but the stuff stays local. Perhaps possible, perhaps not. But I don’t see why everyone could not run given the epoch on his content only, reporting back findings (loss).
There is a Swiss model by ETH university completely trained on fair and open data that everybody can use and is from top to bottom open source.
1 reply →
> In fact, I worry whether open weights will even survive once the current AI cycle cools down.
Surely they'll outcompete everyone on cost alone
Their existence is the only thing keeping costs down across the board. If they had gotten open-weight model importing banned in the US a year ago, I'm sure they'd have 100% increased token cost relative to the advertised 100%+ intelligence improvements.
Open weight models are kind of weird because they're usually just a bunch of weights and maybe a chat template, not even test vectors or anything. Quite odd, and probably one of the reasons why vllm, sglang, the llamas etc. so often produce buggy output. It's actually quite strange how little attention is given to verifying these setups work.
Why cant the community piece together the open-weights using open community-based like, implementation or whatever magic sauce integrates those weights
No it's not understandable, they were created with our stolen data.
They stole your data? Like, they took the text you wrote on the internet and removed it and kept it to themselves?
Ownership of data was a farcical concept to begin with. Now we have a farce squared
It's not understandable for them to lie or mislead. Calling something open weights and not being remotely open in the commonly understood sense is misleading and not ok.
I don't think "you can do whatever you want other than just turn around and sell a chat wrapper for this" makes it immediately not open weights. Could it be more open? Sure. That doesn't mean it's misleading or a lie and I would strong disagree that it's "not being remotely open."