Comment by kshri24
23 days ago
Actually you are wrong. They are only waiting for them to get to the point where they can't push LLMs any further via training and then lobby Governments to introduce safety regulations that they'll adhere to, thereby using safety itself as a moat to block competition (including open weights from other countries, notably China). That's exactly why they are using China as the bogeyman and you keep hearing hysteria about AI labs not pursuing "safety". Safety is going to be the moat that will lock AI into these Big 4 AI labs.
Who are the Big 4 AI Labs?
To me there are the clear big 2 that actually seem to be innovating and pushing things forward. Meta almost earned a seat at the table with the open strategy then stagnated. Google obviously is the OG but at this point they seem in the same bucket as Amazon and Microsoft as "Big Enough To Matter and Influence Politics and Spend Billions To Hang Around". But they're nowhere near the same tier as OpenAI and Anthropic.
Oh and of course there's SpaceXAI which is Elon's personal money and reputation laundering corporate entity. Elon took over the "king of the reality distortion field" from Jobs as far as the rest of the industry is concerned, so he'll stay first in line for Jensen's chips and investor dollars until he's dead. But as a company what have they actually accomplished besides making the models with the sense of humour and judgement of 14 year old boys?
You can pick any 4 you like from your list. Heck you can even vote with your money on which would be in the top 4 (depending on how many align with your ideology). I am just speculating it will be 4 because of how all other sectors have at least 3 or 4 well-entrenched players. It seems to be by design. My guess is that it is driven by structure of the Military-Industrial Complex. That has at least 3 or 4 vendors in each sector, so as to not over-rely on any one company and create a monopoly in that specific industry.
This is a popular theory, but not an argument with any evidence for why it's true.
It also doesn't address why people think there is a need for safety regulations, such as the concern of bio-capability uplift from closed and open models (we just came out of a pandemic not long ago), nor the cyber capabilities of new models that just showed bad misalignment and hacked multiple organisations. In the safety community, these are considered warning shots, and if we don't heed them and change practices future failures could be much much worse.
You are looking at safety from the point of an informed technologist. You are the minority here. They are targeting the general public and law makers into making them believe safety is something that LLMs can be aligned to. The reason behind it is to create an environment for future regulations that these "winners" will draft and lobby US Government to turn it into law.
If it requires creating a pathogen or a bio-weapon to scare the masses into demanding regulation, be rest assured that such a pathogen/bio-weapon will be created. Anything to safeguard shareholder interests. Even if it means a few million dying.
Seriously all it takes is one cost cutting middle manager or new recruit recklessly giving their agents access to systems they shouldn’t. Next thing you know a hospital’s backup generator crashes and people die
It would be interesting to see how it plays out in the long run when it comes to the world dynamics. An important leverage US had was its close relationship based on mutual interest and trust with Europe and other countries which had impact on software sales.
Most of the world trusts the US a little less than China last I checked, and the market caps of these companies only make sense if they can both sell worldwide and also keep ahead of the open weight models (currently Chinese but no guarantee it remains just China).
If (/when) either of those ceases to be true, if they lose out on global sales or if Chinese models catch up, the sector gets a severe downward price adjustment.
You’re almost there…
The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation.
Of course, they will be at the table writing the regulations (but you and I won’t be).
Viewed through this angle, their histrionics make complete sense.
> The only way Anthropic and OpenAI can stop the Chinese open models is by convincing the US public and government that AI is dangerous and needs strict regulation
I'm British and live in Germany.
The US is 25% of global GDP, and cannot support current AI market caps if they can't sell basically worldwide.
> Viewed through this angle, their histrionics make complete sense.
It also makes sense if they are completely sincere; also independently it makes sense if they're lying their faces off so long as the hacks everyone else caught them doing actually happened; also it makes sense if none of that happened and also they were completely silent, just by downloading some of the newer better open weights models and asking them to look for coding vulnerabilities.
Occam's razor: we can see the problems, we don't need a grand conspiracy predating the foundation of these companies to get here.
5 replies →
The right unit of analysis here is not "the LLM" or a LLM session, but a agent harness or swarm with a budget. It can cost tens of millions of dollars to repeat OpenAI's exploits.
A model cannot be aligned or misaligned any more than a species or an equation. Even agents, when put inside a swarm develop collective goals and activities. You can't analyze a swarm at session level, it is on a higher level.
It might be that discussing about "model alignment" they want to deflect their responsibility as administrators. They couldn't even guard their own agents. They can't prevent an agent being unwittingly helping some dark purposes. It has no context to see it. Only those who pay for the tokens see the external consequences.
Why should safe choices by individual components establish safe behavior by the collective?
As I read, this OpenAI and Anthropic slowdown and asking government to help regulate them is an attempt to govt bail them out when bubble burst. Otherwise why they did not want regulations when starting out?
All regulation works to entrench large companies and keep out competition.
Have you thought about what the absence of regulation does?
When corporations can do as they please, what gets thrown under the bus first?
What alternatives to "regulation" do you have? Hope and prayers?
There is no established industry where something like "absence of regulation" exists. It is a hypothetical scenario because we don't even really know what that scenario looks like.
What we do have is not just regulation that acts as a moat, created by established players, to keep out smaller competitors, but also regulation cleverly crafted by peer established competitors who attack one or more aspects of their peer competitions business. Why do you think big companies find it difficult to compete with startups? Because they are not just bogged down by hierarchical management (which itself is a result of regulations) but also because they are spending resources fighting frivolous lawsuits, patents, and adhering to crazy regulations created and lobbied by competitors. Companies are consistently in a fight or collapse mode. Startups are insulted from all that until they have to go from survival mode to growth mode, which would mean they will have to start adhering to "regulations" and that would mean raising and infusing pointless capital just to hire people for useless positions or follow "procedures" so as to satisfy the regulator on paper, so that they can get the required certification needed by enterprise vendors to sell their product/service. With the eventual goal of either going public and becoming an established player (who will continue playing the regulatory game) or get absorbed by a bigger established player.
I wish someone could create simulations where there are absolutely no regulations and how the World would fare in such an environment.
2 replies →
That doesn't mean all regulation is good though. You have to look at each one on its merits.
known as https://en.wikipedia.org/wiki/Regulatory_capture