Comment by shafyy
2 hours ago
Yes, of course this is the real reason. This and regulatory capture. Anybody believing a single word those CEOs say must have been living under a rock for the past 20 years.
2 hours ago
Yes, of course this is the real reason. This and regulatory capture. Anybody believing a single word those CEOs say must have been living under a rock for the past 20 years.
They see the threat of open weight models, which are approaching the usability of their models.
By arguing that the AI is dangerous, they are really arguing that someone 'trusted' needs to monitor the situation, so that the outcome is aligned with existing power.
They are trying to scare the world into forbidding others the right to use and develop AI.
Open weight models are "decelerationist" (i.e. they drain funding from the training of frontier proprietary models), not fine-tuned on any of the even slightly dangerous stuff (e.g. middling scores on cyber benchmarks, with the bulk of ability focused on defense. Biology will be much the same story, except that it's orders of magnitude harder than even cyber!) and not generally deployed with any of the extreme test-time compute (10,000 parallel agents working for a week or 1,000 working for months, seriously? A multi million-dollar token cost for a single task?) that are enabling the proprietary stuff to engage in the dumb casual mischief we've been witnessing as of late. If you're worried about AI being dangerous, you should be welcoming open models.
I think the current state isn't what the frontier labs are worried about. The real issue is the dynamic that open weight models create.
The frontier labs are investing TONS of capital into training the new 'best model ever'. Today, that doesn't make economic sense, it loses money. So where's the business? The future, where token costs reflect the costs involved.
For that to be corrected, the labs need to get people using the models for things they depend on, then they turn dial up on cost. The ordering there is critical. It has to become indispensable first, then costs go up.
The trouble is, open weight models are rapidly becoming usable in what is looking like the big market: coding. Step 2 (the costs go up) doesn't work if there is a ready off-ramp that doesn't require frontier models.
However, if they can effectively make open weight model development illegal (its dangerous!), their business strategy can remain intact. A nice side effect is that they will be effectively be granted an oligopoly "for humanity's safety".
If they can convince the populace that a text generator is an existential threat to humanity, that's a hard argument to combat.
Edit: I think it is ridiculous that autocomplete on steroids is an existential threat. There's no way anything harmful happens until you actually use it as part of important systems. By that reasoning, you could make it illegal to use AI in important systems, but do you think that's the argument the frontier labs are making to governments?