Comment by gorjusborg

I think the current state isn't what the frontier labs are worried about. The real issue is the dynamic that open weight models create.

The frontier labs are investing TONS of capital into training the new 'best model ever'. Today, that doesn't make economic sense, it loses money. So where's the business? The future, where token costs reflect the costs involved.

For that to be corrected, the labs need to get people using the models for things they depend on, then they turn dial up on cost. The ordering there is critical. It has to become indispensable first, then costs go up.

The trouble is, open weight models are rapidly becoming usable in what is looking like the big market: coding. Step 2 (the costs go up) doesn't work if there is a ready off-ramp that doesn't require frontier models.

However, if they can effectively make open weight model development illegal (its dangerous!), their business strategy can remain intact. A nice side effect is that they will be effectively be granted an oligopoly "for humanity's safety".

If they can convince the populace that a text generator is an existential threat to humanity, that's a hard argument to combat.

Edit: I think it is ridiculous that autocomplete on steroids is an existential threat. There's no way anything harmful happens until you actually use it as part of important systems. By that reasoning, you could make it illegal to use AI in important systems, but do you think that's the argument the frontier labs are making to governments?