Comment by razighter777
10 days ago
I hope this doesn't become the new norm where government becomes the bottleneck for innovation in the AI space.
It's worrying that with no formal and transparent policy framework that the government will be picking winners and losers and stifling innovation.
There's been no public policy, executive order, legislation, or otherwise on this, I wonder if anyone has filed FOIA requests for these decisions or the conversations between the Executive Branch and AI companies.
Indeed, I find quite ironic that some people in tech in the US complain about EU "regulations first" approach, but then their government seem to arbitrarily stop things from being released because, well, there is no established policy on safety guarantees or other similar aspects.
I see it too, but worth noting that this is basically unprecedented at least within the last 25 years; I think you have to go back to export controlled cryptography for another example of this kind of abrupt and targeted regulation.
We’ve seen more examples recently. TikTok, wireless routers, polestar cars…
38 replies →
A real headscratcher isn't it? And from a government that is supposedly priding itself on small government. How should companies navigate this? What's the framework they should operate within?
33 replies →
Munitions exporting. I fondly remember the PGP feasco. I spent years using PGP to encrypt my emails to several people who refused to use email without it. Good times.
release the weights! freedom of speech!
7 replies →
Competent government wouldn't do this either... ...also why I think it won't last.
Doubtful it'd hold in court; this admin would have to show that it's not corruption, because we'd all assume otherwise.
Between $5-10T of the US economy is subject to export controls. Nobody disputes that Mythos is dual-use technology, which means it has been export controlled since the day it was created.
Companies are responsible for demonstrating criteria to export (for example) a nerfed version (Fable) of an export controlled item (Mythos)
Nothing here is novel, unusual, capricious, or … fascisistic.
1 reply →
[flagged]
1 reply →
This arrangement is already dubiously legal. The government is already being sued over the Fable incident with Anthropic.
No amount of rules can stop people who are willing to break them. Only enforcement can.
Anthropic just needs to donate millions of dollars to a “MAGA Inc” like Greg Brockman did and they’ll get regulated properly from now on.
It’s a perfectly good system for government regulation.
9 replies →
But your government is constantly acting illegally. Isn't it time for Americans to... do something? It's clear that your legal framework isn't working.
8 replies →
And enforcement cannot work if you’ve captured all three estates.
8 replies →
[dead]
It would only be ironic of you assume those same people who thing the EU over-regulates also support this US government regulation.
It's N=1, but I believe both that the EU approach discourages investment and innovation in the EU, and that this US policy will do the same in the US.
It’s a bit in general, because if you actually read the EU AI legislation, most of it follows the right ideas and provides more safety, in the sense that OpenAI and Anthropic used to pretend to care about, but never really did.
The ideas are debatable but generally correct. The EU's problem is that regulation stops at the ideas, and it is intentionally designed so to be impossible to ensure compliance in advance. So the regulation is really after the fact and a subjective judgment by regulators. So there's tons of risk even if you genuinely believe you're complying with the prescribed intents.
My opinion on EU regulation would flip 180 degrees if they offered any kind of pre-clearance where you could propose a product, feature, or policy and be told in advance if it meets their subjective requirements.
IMO you can have clear, specific requirements in advance, or you can have a body that provides interpretations of spirit-of-the-rules regulations in advance. Having neither is a problem.
(yes, I'm aware of the argument that if you tell companies what's legal in advance they will just do the bare minimum or find loopholes... I don't find that to be a legit rule of law system)
2 replies →
With the way things are, having to disclose training data will basically make it impossible for an EU AI to compete.
6 replies →
Our AI czar, David Sacks, whined and moaned about the idea of regulation, even said Anthropic begging for some guidance was asking for “regulatory capture” and was gloating about how right he was they wanted it, 2 weeks ago.
I wonder if he understands why, now.
> Anthropic begging for some guidance
Anthropic was "begging" to make it harder for competing companies to be founded.
8 replies →
Is it really ironic or just yet another example of how the current administration just keep finding ways to line their pockets? Big Tech has lots of money, and they'd just like to get a little taste. Placing arbitrary restrictions is a pretty good motivator for those being restricted to find some way to make necessary contributions.
On some level though we have to be cognizant of the potential for harm these models have.
LLMs are still a little loosey goosey, and we are right on the cusp (if not there already) for an agent to hack a bank and steal money for some rando teenager with a penchant for jail breaking.
The regulations are and will be negative, but don't lose sight of what LLMs can do off the leash.
>but don't lose sight of what LLMs can do off the leash.
there is no such thing as an LLM "off the leash", it's not a dog, and even if it was a dog the person responsible is the owner. What is this bizarre attitude to a piece of software that makes people think existing laws don't apply?
If your LLM agent hacks a bank, you have hacked a bank, you will go to prison and that's entirely sufficient. People have been hacking banks for decades now, it didn't require the government to regulate C compilers and Emacs.
11 replies →
> On some level
The appropriate level would be regulation though? Like I just don't get how we can argue that arbitrarily throttling companies is ok.
5 replies →
Robbing banks is already illegal
3 replies →
Bank should be more secure, if a random person with an LLM can hack them, they should have paid 100 random blue teamers with LLMs to hack them first to get more secure. Not AI's fault.
3 replies →
Maybe this is a good opportunity for the European AI companies to jump ahead?
AI is probably old italian tech
Lol
3 replies →
> there is no established policy on safety guarantees
Which is the government’s own fault.
Elon Musk talked back in 2018 about how he went to Washington and met with Obama and Congress, but they did nothing.
In 2020 Andrew Yang’s entire run for president was centered around the risk of AI displacing job. He lost, no one did anything.
A couple years later we say the consumer facing LLMs start to roll out. Still, no one does anything.
They have time to micromanage the industry, but in all these years they haven’t found the time to establish any meaningful policy?
Deeper than that, what to do with a super smart AI has been floating around intellectual circles since the late 00's.
Google had been pussy-footing AI research due to deathly fear of awakening an ASI, because their safety teams still had no answers.
Then Altman came along...
In your opinion, what was the appropriate action you believe should have been taken in 2018?
Who specifically? Probably just different people.
I'm fine with this in principle, it's more like regulations last. They looked at the end result and decided it was too powerful to let loose. But also expect the Trump administration to unfairly use it as leverage against US corps.
Meanwhile EU prevented itself from building competitive LLMs in the first place.
its the typical US regulations for consumer but not for corporations, disgusting
Regulation is a good thing, even if HN hates it.
It's a way to clearly agree on ground rules that you can plan around, not more, not less.
The alternative is not no rules, do whatever you want. The alternative is executive capriciousness arbitrarily setting the rules based on whims and messing up your plans.
This applies to most things when it comes to the USG/citizens. Protectionism is communist unless they do it. Thinking about developing a nuke? Well bomb you first despite being the only people to ever use them. Free speech and press - unless we don’t like what you say.
Let’s be real, as an EU citizen I have zero doubts that those models would also have been blocked if developed in EU.
I like the US approach better: regulate when the need for it arises, not before when you don’t know how the situation is going to evolve.
They're not regulating though – they're arbitrarily blocking releases based on no clear criteria. The EU may be legalistic and rules-based, but I'd take that over capricious and arbitrary.
5 replies →
You can’t be serious because “When the need arises” means when your company does not lavish praise on the current administration.
Let's plan a fire fighter division only once we are actually having some buildings in the city burning down. That people who fear that ridiculous perfectly controlled fire in chemines are ridiculous.
Regulating when the need arises requires also compensating the people who get hurt in the meantime.
If they were real about risk, they would have to block a lot more models.
It sounds nice but you end up with entrenched special interests that later oppose all regulation regardless of the consequences. We have pesticides you wouldn't want anywhere near your children casually used to control weeds on kid's playgrounds, insanely huge trucks that kill hundred each year, the food is garbage...the list is long and tiresome. Trust me brother, if I could live in the EU, I would.
> regulate when the need for it arises
I agree. But that need has absolutely arisen. The US government is not exactly the best steward for this kind of thing, but some model other than "race each other as fast as we can" is desperately needed here.
Well, when the leaders of this movement go around doom-trolling for years on end this is what happens. It turns out you need to be careful what you say if you're a highly visible public figure. Amazing!
Honestly, with open source models I don't think this regulation means anything because there's no way they can really regulate what's coming out of china. I don't think this affects innovation in AI much at all (unless your definition of innovation is "pour more money into diminishing scaling"). It's mostly just bad news for the US frontier labs, and based on their behavior I don't feel sad for them AT ALL. Like, they've basically alienated the vast majority of people by outright threatening their livelihoods or even society at large, and now we're supposed to feel sorry for them because they can't just go around saying "THIS WILL REPLACE ALL JOBS IN A MONTH!" without consequences?
> because there's no way they can really regulate what's coming out of china
Do you think the Chinese will go parading around that they've created the greatest cyberweapon known to man, and the CCP will be totally cool with the Americans being first in line to buy tokens, because hey, free market?
They would sooner put all their own employees in an incinerator than allow that to happen.
People genuinely think that China is just being a bro and has no ulterior motives.
the truth is that doom is a high likelihood
sorry we don't lie about that
If they actually think that, then.. stop working on AI? Being the first person to end the world doesn't exactly gain you anything. I guess there's a bleak humor to Altman and Musk and Amodei being forced to live out the rest of their days in their bunkers I guess.
Like seriously, to the people that think this is a doomsday thing, if you're serious about that line of thought then STOP DOING IT. It's like the people that are arguing that AI is conscious. IF you truly believe that, then we've just reinvented slavery, and again, STOP!
I 100% do not accept the "it's inevitable anyway so morality is out the window". We also have nuclear weapons but we don't need to rush into World War 3. Also nothing inevitable requires trillions of dollars a year to advance because it's so deeply unprofitable.
1 reply →
It’s likely that this would slow down the rate of advancement at the Chinese labs as well
Or significantly increase their market share outside the US and give them some breathing space to catch up with the currently available closed models
I don't see how, other than that it will make it harder for chinese labs to train their models on OpenAI/Anthropics' (which honestly I can't get that worked up about plagiarism in this space considering where they got their data from..)
I remember when Republicans told us they want less regulation and smaller federal government. Now they want their dementia riddled god king to control everything from pool liners to the information you're allowed to see, which is all in books and readily available online.
It was never about principles. It was always about justifying getting the things they wanted.
What surprises me more is that any of the AI CEOs believed them.
They were in the tank for Trump because they thought Biden/Harris would stifle them… and here we are.
You're assuming that their base is rational
Always good to dehumanize your enemies
2 replies →
I don't see how you can make a case for a $700bn+ IPO when the government might not even let you sell your product. America is ceding the lead in the AI race. The winners here will be the Chinese AI companies.
If the Chinese models remain predominantly open source then it would probably be for the best. Unfortunately I'm not convinced they will, with examples like Qwen Max showing what could happen.
Z.ai and MoonShot (and StepFun and some others who are another six months behind or targeting smaller use cases) are still open, surprisingly enough.
Alibaba making Qwen close up shop for its best models isn’t that surprising, though sad.
The worry is that if the US models are locked up like this, then there’s less reason for China to commodify its complement through open weights…
It will just mean US providers will rapidly loose their moat. Their moat is already shrinking. If they can't release their best models, it'll shrink a whole lot faster...
It won’t, the government will change its mind again
This is what OpenAI/Anthropic want, it's better marketing than they can pay for -- and it creates a precedent for permanently banning the next generation of open weights models
Banning next gen open models would never happen globally, and would be a major disadvantage to any country that does.
If the USA continues to put barriers into the release of models, open and/or foreign models will start to out perform them.
If open models are competitive enough nothing will stop even US companies from running them locally.
damn, never thought about this - but yeah this is where we are heading.
open weight models - will be deemed too risky to be out in the open - since they can be abused by "bad actors" (unwashed masses)
100% this is the direction all govts will go. This isn't specific to any political party, it's just about communications control. I don't think open source models will be directly capped, necessarily, but all commercial/easy-to-setup models will be heavily regulated.
I wouldn’t be surprised if they started going after open-source as a whole and labeling it “communism” like the old Microsoft days.
3 replies →
[dead]
It’s going to be a bit trickier to do that, even banning US providers from hosting them legally might be tricky to do.
I think it's going to be like DMCA, like hard to convict you for having the files but distributing them might be illegal
The EU bans ASML for American chip use.
America bans intel and amd from exporting chips.
Whats next.
Yup, regulatory capture.
But in this model, there is no regulator, there is just the Whitehouse deciding who gets to use the AI. Nobody has “captured” Trump here.
3 replies →
I wonder what kind of scheme the administration is up to. The obvious play is a squeeze where OpenAI and Anthropic are forced to give parts of their company away, like Intel. But they could also be toying with the idea of limiting frontier AI access to companies that bend the knee, which would further cement their grasp on the tech industry.
> what kind of scheme the administration is up to
I’m sure they are wondering just as much. I assume exhorting Anthropic/OpenAI for personal bribes, favorable government contracts with no restrictions and public acts of submission.
Maybe include some election guides for poor, misguided Americans that would hurt themselves by not voting for God President Donald Trump I as well?
It's protecting people from themselves, so basically like the safeguards already included in the models.
Not a fan of the phased release but I do remember when access to gpt-3 was gated and access to gpt-4 had a staged release.
ppl are acting like limited release is unprecedented when, in fact, has been the norm until a few years ago.
Gated by capacity constraints for all users is very different that picking “trusted partners” that get preferential access. Especially when that access is based on political connections
Gated by the company that made it is not remotely the same thing as gated by the government.
Was it the norm for Trump's team to hand-select the specific customers who get access in the staged rollout, and to choose the date of wide release?
The AI companies were all asking for the government to regulate them. The government is doing what the companies asked for them to do.
You can argue that, by government, they meant some legislative process, but I'd argue that regulation via bad executive order is much better than regulation via bad legislation because the former is tractable. I say this as an EO minimalist.
20 replies →
That only happens in governments that treat regulations as a racket, not something to be used for public good.
It's worrying that with no formal and transparent policy framework that the government will be picking winners and losers and stifling innovation.
The market will demand such a framework. I suspect that's the larger idea here, in that Amodei not only wants to be in the room when that framework is written, he wants to be at the head of the table.
He apparently wants it so badly he's willing to set back his own company's IPO to make it happen, given that there can be no pure-play AI IPOs until the regulatory picture is sorted out.
With _this_ admin? No way.
Checks from the major model providers will already be on their way to Congress, hand-delivered by the highest-paid lobbyists on K Street. Look for them to wake from their ent-like slumber tout suite to pass legislation that the courts can use for guidance.
What Trump is doing at the moment is, as usual, only a distraction.
I largely blame people like Amodei for such outcome. As product owners, they could've done it the old way: telling people how great this product is, how much potential it has, and what kind of guardrails the companies are building and etc. But oh no, Amodei has to do the doom trolling 24x7, while in the meantime plays a cult leader by telling people only he knows how to the guarding angel of the AI or the humanity thereof. Ironically, the same people also push their companies to develop more powerful AI in full speed. They think ordinary Americans are so stupid that they can't see through them?
It definitely will be. They are trying to preserve their offensive cyberwarfare and cyberespionage capabilities. It’s not about “bad actors”. It’s about defensive capabilities becoming pervasive and cheap.
So OAI are you also silently dumbing down your models when you detect "inappropriate topics" like Anthropic did with Fable?
Fable didn't silently dumb it down. It printed a warning that it has detected a possible inappropriate topic and you are being switched back to Opus. I hate it, but it isn't dishonest.
Unless it thought you are trying to distill it, then it would silently sabotage you.
1 reply →
The big companies want this. It's a moat for them, a way to keep competitors (especially overseas) out of the US market.
They might try to extend it to downloadable open weight models, but honestly they might not even bother with that. The goal is to keep people from competing on lucrative contracts or the hosting market.
> I hope this doesn't become the new norm where government becomes the bottleneck for innovation in the AI space.
I hope that not too far in the future a responsible government* will become the bottleneck for innovation in the AI space.
*obviously not the current
> There's been no public policy, executive order, legislation, or otherwise on this
I mean, insofar as you could frame each new model as its own patentable invention, "patent secrecy" would be an existing policy framework that clearly justifies what's been happening here.
1. Some private company or individual invents something.
2. But the state wants a monopoly on the new thing's military use-cases (against other states/militaries.) So the state forcibly classifies the invention at patent time.
3. But the individual/company still wants to make more money than the state is giving them as compensation for their own use of the patent. They want to sell to the private market. But, of course, the state doesn't want to permit this, as arbitrary private parties could in turn resell to foreign state actors.
4. So a compromise is struck: private use is now made deny-by-default. The state permits the individual or company to sell into the private market, given: a rigorous per-customer KYB background-checking process; strong supply-chain tracing; contractual stipulations prohibiting resale; and the customer use-case being transformative or cost-prohibitive to extract the original invention from.
5. As such, big established enterprise customers who want the invention for private use in their internal industrial processes, can somewhat-easily jump all those hoops to acquire access to it; but everyone else is now basically locked out.
Sound familiar?
Caused by AI marketing teams hyping everyone up.
> conversations between the Executive Branch and AI companies
You think there is a record?
I agree that this is all ridiculously arbitrary right now, but it shouldn't be surprising either.
I can't find the exact blog post (maybe on simonwillison.net ?) but I read people predicting that know your customer laws would be coming to AI if it gets more powerful several years ago already.
Powerful technology that can do immense harm in the hand of individuals/small groups is the most obvious (and legitimate!) target for regulation. Maybe Anthropics hubbub around Mythos made all of this happen earlier than it would have, but it was going to happen (if the models are going to get as capable as valuations imply they will).
(Edit: Of course this doesn't mean it can be effectively regulated in practice)
Further more, no one actually gets hurt if we start rolling these things out more slowly.
Rolling them out selectively according to the whims of an administration that disdains fair process, tears down the institutions that could potentially provide and legitimise reasonable regulation, etc... well, that's another topic.
Shadow of export controls is very long indeed.
The Project is almost here.
But think about how terrible it would be if “foreigners” (including the ones that work on these models) got access!
We must clutch our pearls and cite National Security as a reason to pick winners and losers, just like the government did for Fable.
There would be real risks yeah.
This is not something to joke about, its real.
for america yeah. for the world the only real risks are american, chinese or corporate dominance. thats why its important to support open models wherever they come from and smaller players like mistral in france or black forest in germany.
Imagine what brown skinned people could do if they were granted the privilege of accessing the Internet!
That’s a lot of information that could fall into the boogeyman’s hands.
It’s already the new norm.
I'm going to get downvoted here, but all the E/acc people that loudly allocated for Trump, someone known for amassing power by any means necessary including strong arming industry should be publicly eating crow right now. This was something that was always in the cards when you vote for someone who only cares about himself.
All the tech CEOs had no qualms about groveling before Trump and licking his boots, so yeah I assume they must be 100% onboard with stuff like this as well
I agree. It’s crazy the backwards reasoning that is being used to blame anthropic for this!
I am still waiting for a government to try “nationalizing” AI by saying anything produced by AI belongs to “everyone” and thus hugely taxing the profits and proceeds from the product, as soon as Bernie Sanders thinks of it you can bet we’ll hear all about it.
Apparently your hearing isn't that good, since you haven't heard of the American AI Sovereign Wealth Fund Act.
https://www.sanders.senate.gov/press-releases/news-sanders-i...
ah I must have been out of audible range
How can you simultaneously be a bottleneck for innovation while being their largest customer, and pouring tons of money and resources into it to help accelerate development?
The startup-brained among us never learned the first rule of business which is to not fuck over your benefactor.
For some conspirational reasons, I am thinking that this was the part of the deal between Anthropic and US government, all buzz was for PR, but behind the closed doors Anthropic asked US to regulate the LLM space
I'm honestly surprised there isn't more political outcry. The administration has a party affiliation that, typically, insists on free market principles and is against government overreach and regulation.
You would think that this government, attempting to puppeteer the most rapidly growing industry in the world, would have more people outraged.
Where are all of the people crying "Communist"? This is one of those moments where it is less of an overreaction.
[dead]
[flagged]
You have fallen victim to what is known as “marketing”.
> We don't let passengers fly on unvetted jumbo jets either and we prevent them from flying when they have problems.
This is Mr. Fart's Favorite Colors all over again. Our "vetting" process is not any more useful than the billion-dollar metal detector you can skip with a TSA Precheck. It arguably does not deter the most dangerous attacks even slightly. What happens when a mentally-ill pilot locks their copilot out of the cockpit? Well, we write off a crowd of passengers and then "vet" the next jet as a safe vehicle.
AI will be the same way. These "safety" measures are performative and do not even slightly address the actual threat surface of the technology. Arguably, it cannot even be done.
What knowledge or skills do you have to be hating on the certification process for airplanes?
It’s just getting ridiculous at this point. There are plenty of industries regulated and certified by national or international agencies. And no they don’t get to do what they want.
2 replies →
Ah yes all regulation is actually just performative and for capture. Give me a break.
What a terrible take
First of all, who said this is a disaster?
Second of all, OP never even said anything about no regulation - they specifically said they wanted transparency which is 100% valid and better than a world where the government baby proofs everything for you
Models are already censored - and who they are or aren't uncensored for has a lot of implications which are way worse
And the jets is a terrible example - you picked one of THE highest regulated industries where NOBODY has a problem with regulation
>And the jets is a terrible example - you picked one of THE highest regulated industries where NOBODY has a problem with regulation
The potential risk from serious AI disasters far far outweighs anything the Airline industry has ever done. Like its a joke that I even have to invoke that in the first place.
In the EU that's the norm not the exception. A little taste of Europe for our American friends :-S
Except that EU has clear rules, not arbitrary lawless approvals.
Yeah it’s called diffusion of responsibility and accountability. Impersonal forces slowly close in with no face.
The precedent for this is terrible.
MAGA is bad enough. Imagine if the current batch of US progressives, who have 0 idea how any of this works, wins the presidency and gets to decide who gets to use it.
From the other side of the world, the current batch of US progressives looks quite liberal. It looks to me they'll either completely restrict AI or not touch it at all.
Do you think they'll try to dictate who gets to use it?
Just because you don't get access, doesn't mean they're not innovating.
Their innovation relies on a huge amount of investment made under the assumption that they'll continue to be able to provide frontier models to a global audience. If it turns out the US government only lets them sell gimped models to non-citizens then they'll forfeit the whole global market to China and investors will flee like rats.
Maybe, but you know who is also innovating, not gating access, and at most 6-9 months away from reaching parity with US frontier labs?
[dead]
Also applies to Chinese models. Give it 5 more months of US admin locking out US models and let's see what the market will look like for OoenAI and Anthropic IPO.
This move was obvious the moment Anthropic pleaded to the government to regulate them.
As predicted, [0] it has now been applied to OpenAI and soon anyone else releasing highly capable models.
[0] https://news.ycombinator.com/item?id=48511849
Seems like it hasn't been applied to openai. Anthropic can't even release this to partners. Openai can. I wonder why.
Wondering how long it'll take for the US to make it... more difficult to use Chinese models once they've caught up.