Comment by zozbot234
4 hours ago
Open source AI is actually a lot less "powerful" than genuine frontier models, i.e. it has a much tighter inherent capability ceiling. This is "decelerationist" from a purely AGI-pilled point of view but it's actually great if you're worried about a capabilities arms race putting AI Safety at severe risk.
Kimi K3 is plausibly a lot less dangerous than a totally jailbroken ChatGPT/Gemini/Claude Sonnet (let alone Opus or Fable!) and it's quite deeply weird how no one seems to be calling for those models to be banned or restrained by further regulation. Why the double standard against the less concerning (but more efficient!) open weight models?
Do you think they are inherently less powerful? I'd imagined that closed labs have a head start / more funding so the open labs are playing catch-up.
Is there a world where open source models end up at the frontier, or do you think there are structural/first-principles reasons why this won't happen?
If you're targeting widespread local/on prem deployment which is what many open weight models are doing, that inherently limits your scale in terms of total model weights/inference-time compute compared to running in a few centralized datacenters. A centralized model will always be able to leverage a larger scale of deployment, placing it much closer to the genuine "frontier".