Comment by ben_w
23 days ago
Most of the world trusts the US a little less than China last I checked, and the market caps of these companies only make sense if they can both sell worldwide and also keep ahead of the open weight models (currently Chinese but no guarantee it remains just China).
If (/when) either of those ceases to be true, if they lose out on global sales or if Chinese models catch up, the sector gets a severe downward price adjustment.
You’re almost there…
The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation.
Of course, they will be at the table writing the regulations (but you and I won’t be).
Viewed through this angle, their histrionics make complete sense.
> The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation
I'm British and live in Germany.
The US is 25% of global GDP, and cannot support current AI market caps if they can't sell basically worldwide.
> Viewed through this angle, their histrionics make complete sense.
It also makes sense if they are completely sincere; also independently it makes sense if they're lying their faces off so long as the hacks everyone else caught them doing actually happened; also it makes sense if none of that happened and also they were completely silent, just by downloading some of the newer better open weights models and asking them to look for coding vulnerabilities.
Occam's razor: we can see the problems, we don't need a grand conspiracy predating the foundation of these companies to get here.
> The US is 25% of global GDP, and cannot support current AI market caps if they can't sell basically worldwide.
Oh they definitely can. Irrespective of whether you are living in US or not, you still have to get yourself various certifications (take for example SOC Type-2 or HIPAA depending on your business category) to even become vendors to US companies or the US Government. If the US Government decides that all companies MUST use "safe AI", no company can use any other model other than what is decided by US Government. And that has a viral effect on the entire supply chain.
If you don't want this scenario to happen, where you are held to ransom by US Government, all middle-powers have to de-risk and diversify away from US dollars as reserve currency. That is the only way you can bring in some balance of power and leverage. There is no other way.
3 replies →
> the hacks everyone else caught them doing actually happened
My firsthand knowledge of these is that yes, they happened.
However in most cases OpenAI informed the victim, not the other way round.
Even medium to large organisations are typically unable to detect a breach unless it does obvious damage.
Everyone can notice their whole network getting encrypted for ransom.
Few can notice SQL injection blended in with application requests that contain SQL snippets in the normal case.
From what I’ve seen in the logs, the “AI agent breaches” are almost polite for the want of a better word. Like a gentlemanly catburgler picking a lock to sneak in and… take a picture of a rare artwork in a private collection.
The right unit of analysis here is not "the LLM" or a LLM session, but a agent harness or swarm with a budget. It can cost tens of millions of dollars to repeat OpenAI's exploits.
A model cannot be aligned or misaligned any more than a species or an equation. Even agents, when put inside a swarm develop collective goals and activities. You can't analyze a swarm at session level, it is on a higher level.
It might be that discussing about "model alignment" they want to deflect their responsibility as administrators. They couldn't even guard their own agents. They can't prevent an agent being unwittingly helping some dark purposes. It has no context to see it. Only those who pay for the tokens see the external consequences.
Why should safe choices by individual components establish safe behavior by the collective?