Comment by rao-v

The reason is money. They want regulation to make it harder for new competitors and competitors from other countries.

They invested billions into training the models but there is no competitive advantage, we see that within a couple of months everyone catches up. There is no way to profitability unless they get some policies to shields them against competitors that can't comply with the regulatory requirements.

That is also why there are things like Claude, Codex and Cursor. They are trying hard to build a customer relationship with a higher switching cost that hopefully sticks.

But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.

Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.

Does anyone know what are the proposed regulations? Controlling software is impossible, so the only option is banning hardware ownership. No more mac studio.

Dario has always wanted the AI development to slow down and be more careful. Safer AI development was a core reason that Anthropic split off from OpenAI.

What's different today is that now all the big LLM firms want to slow down AI development. When men like Musk and Altman (both known for habitually shooting their mouths off and saying whatever they need to whoever needs to hear it regardless of truth) suddenly agree with Amodei, that's when things start to smell off.

I might turn this into a blogpost if folks are interested, but my god there is so much clever info in that dashboard.

Here is one really neat bit:

A cutting edge training idea (for agents, it's been used elsewhere for ages) is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".

Basically, it's more training efficient to improve models with small tweaks to do more of the right thing they are already doing sometimes than from some perfect oracular "this is the way" answer.

(if you've ever tried to teach humans new skills, you’ve probably noticed this too!)

When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (agentic rollouts for hard problems can take hours with lots of tool calls, so you can't keep redeploying every slight improvement).

Lo and behold, the dashboard literally has:

partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)

train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how differently the two models generate tokens)

How cool is that?!

And don't get me started on the clever ideas hiding behind dynsam/avg@n ...

The best thing they did is being open about all the setbacks they had to deal with. They logged every restart with a reason, talked about dropping a cyber dataset after it degraded coding benchmarks. Also published real time training loss, benchmark scores after every checkpoint and running cost estimates.

Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.

the existence, who else has a live dashboard for the RL late-training?

Open tech is cool. Speeds up all progress...