Comment by Chance-Device
2 days ago
What disturbs me is that there likely won’t be a big enough reaction to this policy wise.
There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong hands.
I’d remind them that the stock market is a few coordinated hacks away from crashing on any given day, so maybe they should think about that.
I think all that regulation will do at this point is help the incumbents who are failing. Protectionism. I don't think they deserve that help. I also don't see any reason to think the current administration would have anything resembling competence around this. And it's worth noting that Greg Brockman is a huge MAGA donor, so it's likely the policies would be very corrupt. (Don't worry, he justified his donations as "apolitical", he just wants to buy the politicians, he doesn't believe in their causes. I hate these people.)
> all that regulation will do at this point is help the incumbents who are failing
This depends on the specific regulation. The datacentre moratoria probably give open-weight models time to catch up by tempering the extent to which the leading companies can turn their capital advantage into market share.
> datacentre moratoria
What infrastructure will these open weight models be trained on?
3 replies →
[dead]
> What disturbs me is that there likely won’t be a big enough reaction to this policy wise.
Anthropic was blocked from releasing Fable without any such level of incident. OAI was also briefly blocked from releasing 5.6. Why do you think there is no policy appetite?
Because that was just an attack on Anthropic by a hostile administration. And it worked, didn’t it? Anthropic had to turn their filters up to absurd levels, OpenAI didn’t. It’s got nothing to do with safety.
> It’s got nothing to do with safety
Doesn't change the effect. Plenty of good policy is enacted by self-interested politiicans.
45 replies →
yeah tell me about it... Fable today refused to turn on Row Level security on my internal db on in development app becuase of cyber-securty safeguard..had to switch to Codex
> Why do you think there is no policy appetite?
Because China seems pretty eager to serve the rest of the world's needs if the USA doesn't stop their idiotic "safety" nonsense.
How do you know that? How do you know that the Chinese aren’t exactly as uneasy about rapidly advancing AI capability and feel locked into the race because they think that the US will race ahead if they stop?
During the Cold War the nuclear arms race was brought under control gradually, because it was mutually beneficial, but it took time to build trust. This is no different. Nobody wins from the race.
35 replies →
They won’t when citizens run AI on their phone that contradicts Xi thought
Do the Chinese models have anything to say about Tiananmen Square? Or if they can act as a surrogate girlfriend/boyfriend?
Both countries are engaging in different flavors of censoring.
2 replies →
>Anthropic was blocked from releasing Fable without any such level of incident.
The head of the NSA said Mythos breached almost all of their classified systems, though it was in an intention red-team test.
It's such a freak incident of history that right at this critical time the dumbest, most incompetent leadership is at the helm in the US...
Follow the money, eg. investors and their connections to the Govt and media.
> There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons.
Let's be honest: it's financial and national security reasons.
China has a long and storied history of hacking attacks on American and western targets.
There are other parts of the world that make open weight models; Mistral is a European option. You don't see the worry about that because most people in the US are used to existing in a world order where European powers are considered ambivalent to the US at worst and holders of a special political relationship at best.
If Mistral had the same backing that Chinese AI companies did, there probably wouldn't be as much hemming and hawing. Sure, American companies would take a haircut, but that haircut wouldn't be seen as a move towards software hegemony built on top of manufacturing hegemony. It'd just be you calling into Paris or Frankfurt to talk to your vendor in the future.
This is marketing.
Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?
It’s marketing the same way shitting your pants in public is marketing. People notice you.
It’s more like eating your own fiber supplement in public, and then shitting your pants and telling everyone about it. Sure, it’s embarrassing. But it shows how potent your product is.
1 reply →
Apparently this is totally legit marketing strategy now. It truly is, especially if there are enough people who think that shitting your pants is cool, and the people that form the "market" nowadays may have a very different idea from yours about what is cool. Their ideas about coolness are very different from mine, that's for sure.
2 replies →
Anyone can have bad security. No one cares. But you can convince those who don’t know better that breaking bad security with an LLM is a once in a civilization investing opportunity. You just need to convince a handful of billionaires and market makers to get on board.
How much would someone have to pay you to take the fall for bad security? A million? A billion? 500b? The stake at play puts it in the realm of geopolitics.
I think all of western (or at least American) discourse of all kinds has recently devolved into who can shit their pants the loudest. I'm hardly surprised when it becomes a dominant advertising strategy
You guys have created this un-falsifiable "marketing" narrative. Why is it that Jensen is pushing back on the doomer stuff, and complaining that it is hurting AI investments?
https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...
Jensen Huang wants to sell more hardware, he doesn't care whether it's Anthropic, OpenAI, or some Chinese company that wins. So he has a vested interest in AI being perceived positively so that the construction of datacenter continues unopposed.
OpenAI and Anthropic have completely different motives, they're in a zero-sum game with each other and with cheap Chinese models. Regulatory capture that results in artificial barriers of entry is their best bet at healthy profit margins even if it comes at the cost of less overall AI buildout.
It's not like "the AI industry" is a single entity with a single purpose, these are different companies with different objectives.
Edit: Google pretty much spells out the problem that AI labs are faced with in the leaked "We have no moat" memo[1]:
> People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality. We should consider where our value add really is.
And
> All this talk of open source can feel unfair given OpenAI’s current closed policy. Why do we have to share, if they won’t? But the fact of the matter is, we are already sharing everything with them in the form of the steady flow of poached senior researchers. Until we stem that tide, secrecy is a moot point.
> And in the end, OpenAI doesn’t matter. They are making the same mistakes we are in their posture relative to open source, and their ability to maintain an edge is necessarily in question. Open source alternatives can and will eventually eclipse them unless they change their stance. In this respect, at least, we can make the first move.
[1] https://newsletter.semianalysis.com/p/google-we-have-no-moat...
1 reply →
This is marketing, totally. HF conveniently created a weak sandbox