Comment by vhantz

I'm very confident that it was staged and coordinated. This propaganda started because they cannot evolve their models further. See Fable and Sol, they are lame. They seem incredible at first but the more you use you can see the trickery. It is a matter of time for someone to prove they are marginally better only because they inject more information to the harness at server side.

I case you're wondering, HF = Hugging Face

Worth checking, elsewhere in this thread someone points out glm only assessed the damage after the fact, it didn't stop the attack, and Hf apprently never sought access to a trusted defender program with a closed model either the open model saved them farming might not hold up.

I think the fair nuance here is, an administration which used Executive Orders to force guardrails, meaning US companies must retain them, even if they might want to drop them now.

And on top of that, with low/no guardrails, people call you a child pornographer(grok), so the public is also against it. Yet mysteriously few complain about Chinese open models being child pornographers.

So even if your goal isn't ethical, but just fiscal, it's reasonable to say there are two standards. And to complaint in some way.

I don't think banning is going to work, that's just silly. And over the next few years, everyone and their dog will have local GPU compute to train locally. People have home labs, the bar isn't that high, and eventually large text datasets will escape from Anthropic and other companies, allowing for comparable training.

It's a genie that's not going back in the bottle, the bottle is smashed.

The only reasonable outcome would be section 230 style carveouts so that there is zero liability for anything a model does.

Because having guardrails on corporate models barely months ahead of open ones, which will never be restricted, is entirely pointless.

> They’ve never interfered in my politics or started any wars

But they have. I dont know what specific country you are referring to but China has interfered with US elections as well as Canada, Taiwan, and Australia in addition to many many others.

They’ve annexed Tibet (1950), fought India (1962) and Vietnam (1979) among others and more recently in 2020 a deadly skirmish with India. They’ve generally shifted their focus to cyber military actions but you’d have to be pretty naive they won’t start to exercise military control over Taiwan when/if they get a chance.

> Whatever they do in their country is their own problem.

Idk to me it feels like human compassion and ethics dictates that you at least have to care a bit even about the things in other countries. Otherwise all sorts of horrible domestic policies would be justifiable.

That's a very selfish way to think of it. Whatever they do in their country does not stay in their country because China is becoming a superpower so you should expect it (as it's normal) to interfere and affect policy abroad.

For example Lithuania has been punished for letting Taiwan opening an embassy (in Lithuania). So China cares what other countries do in their country. Without EU support Lithuania could have faced very harsh consequences. That's just an example. Another example is the "secret police" stations undeclared overseas police stations operated by China in various countries (i.e. U.S for example). China gave you cheap material goods because that was its business but you have to keep in mind it's an authoritarian, communist regime. It carries an extra risk on top of the potential economic coercion if you give it too power. At least you know that U.S is only after the money and does not want to turn your country in a communist "utopia".

That being said this does not mean we should ban Chinese AI models because China didn't cross any red lines(yet) compared with Russia for example.

For you personally no, but there's plenty of scandals of politicians being influenced by the Chinese, case in point in my own country: https://www.ft.com/content/601df41f-8393-46ad-9f74-fe64f8ea1...

Don't be naive or I might even think you're working for the Chinese :-)

But they're happily selling drone parts to Russians dropping bombs on civilians (or "whatever").

I wish I could be this blissfully naive.

China is becoming an intellectual powerhouse in addition to a manufacturing powerhouse and runs huge spyware and propaganda campaigns.

Read up about T95, MBOX, TVBOX, or other Android TV Boxes advertised as generic TV streaming devices advertised as "unlocked" or capable of accessing free content. They're loaded with spyware/malware, typically BADBOX. Some of them will record audio and record your network traffic and send it to China.

I'm also convinced that the anti-education thread winding its way through US culture is being amplified by China. They likely didn't start it, as it's been growing for decades, but China will happily keep it going.

What country do you live in? Nevermind it doesn’t matter because literally every country has had their politics interfered with.

I mean them selling the cheap material goods is politics, and they use that a lot to threaten other countries. They constantly bully neighbors and assert claims on foreign terriroty. They might not be the comic book villians people are making them out to be but your take is just ignorant

Oh i bet they have not directly interfered with your politics, but they do interfer a lots in various things. More than any other country on this planet? I don't know. China is big. Really really big.

> They’ve never interfered in my politics or started any wars

They actively back russia and deliver them weapons and support them with their attempted Genocide in Ukraine. They also actively threaten Taiwan.

Not to mention them claiming almost the entire South China Sea even though this being against international law and threatening smaller nations into submitting to them

I personally think it’s bad when countries commit genocide https://en.wikipedia.org/wiki/Persecution_of_Uyghurs_in_Chin...

Do you remember covid and where it come from?

Lol: https://en.wikipedia.org/wiki/Wolf_warrior_diplomacy

They have their own genocide in western china

As someone who has never visited China or the US; yes. Why? Media. School shootings, ICE, crazy president… What do I see from China? Impressive multi-million cities… and true to be told, not much.

China had a worse reputation, but not anymore since Trump leads the US.

Im not saying that the above is objective, tho

Probably more.

Yes, only if measured by their own people.

Trump deports, half of the USA people screamed; Xin builds reeducation camps, most Chinese people are cool.

More repressive, by real statistics.

> The job of a country is to do what is best for it.

Countries aren't defined like corporations in the US. Why would countries have funds to help places like Haiti otherwise?

Lots of different reasons for countries to do all sorts of things? Why would France have armed the US during their independence movement? Why is the rest of the world supporting Ukraine during this war started by Russia?

Why does it feel like you think you're talking to only people in the US?

There's plenty of people on this forum that aren't American. Including some former allies whose sovereignty has been aggressively threatened. And some of those people are Anthropic customers.

Even more so, many of us are in countries that would be well within the blast radius of fallout should the US try to "ban" open weight models or make moves to limit "US" models (often developed on research or work by non-Americans too, but that's another topic) only to those blessed by the US gov't.

That's why it matters?

Why should the national security concerns of the US lie with Anthropic?

You don’t need to replace them. Dario is the “someone” in question.

You already have examples. Venezuela, Iran, Ukraine, Libya… maybe Japan? The list is sort of long. Juxtapose or something.

or you can image, what exactly happens to a "losing" country when it has been a small clique "dominated"?

The rest of the world would likely side with China. The export ban on Fable wasn’t even extended to five eyes countries.

US is going to find itself isolated and irrelevant. And not a moment too soon.

> the West could retaliate by halting shipments of

China quickly retaliated last time by stopping shipments of rare earths and magnets. The West has no answer for this, really up the river without a paddle for such critical supply chain elements.

China would just start buying oil again, and stop using it from their strategic reserve. Hello $200 barrels of oil.

> And if you think that their distilling from US models was in any way anti-competitive or illegal, then I guess maybe "deal with it", much akin to Anthropic, Google and OAI's response around taking the (copyright) content in the first place with no repercussions.

Exactly. The cries in favor of distillation regulation from the US AI companies ring hollow and fearful.

OpenAI and Anthropic didn't realize that distillation was going to be so (a) effective and (b) un-technically-stoppable at scale.

Now they're seeing their IPOs at risk and clutching at governmental straws.

Dario's argument is transparently working backwards from {protect Anthropic's economic model} <- {need government regulation} <- {justify government regulation via AI fears} <- {we love open models, but so sorry they can't pass regulation}.

If the rise of the web in the 90s taught us anything, it should have been that companies that take economic reality as it exists thrive, while those that predicate their value on regulation fail.

If distillation at scale works and is technically feasible? That's reality. Deal with it.

Could have fooled me, judging by the Iran and Yemen performance.

Iran showed us that they apparently are.

It's also not a law of nature that there be a valid/efficient form of a product that makes a business profitable. I'm in no place to judge whether that's the case here, but not all products are viable.

You would need to demonstrate an impact for a line item that big. A 3k person org at $200/seat would be >$7mill/year. That places you very well into self hosting Kimi K3 territory and the never having: price hikes, dependency on a 3rd party, etc.

AMD would love to come by and plop one of these into your datacenters: https://www.amd.com/en/products/accelerators/instinct/mi400..... Also, in a few generation, there will be a massive glut of used hardware.

Unless frontier labs can surpass the current models significantly I don't know why I would pay them money instead of hosting K3.

I guess that tracks with the brute force attempt at workable intelligence that is the Attention/stochastic token based system

Market failure! Market failure is the term you're looking for.

Also see: childcare, healthcare, many forms of public transport and social infrastructure. Some things markets simply cannot deliver well, and that's okay.

Daycare is the most easily reachable example because it's quite simple compared to the others. Daycare workers are simultaneously some of the lowest paid workers in the US, yet daycare costs are famously high and prohibitive, yet childcare centres are very far from money-printing machines. Margins in the daycare sector are most commonly < 1%.

No one is forced into buying the product, so it feels like an emotional rather than a rational rant from OP.

The whole thing about replacing workers is such a perfect example of shooting yourself in the foot. The whole US economy is so strong because the population consumes so much. Get rid of the workers, you’re now damaging the consumer base. Repeat that a few times, across industries, and you now have a zombie economy. The idea that we can have an economy of virtual agents is a child idea that doesn’t make any sense in a serious situation

Most people on earth don’t make $200/mo.

Can we just downsize colonialism?

A caret with a character subtracts 0x40 from its hex value allowing you to insert non-printable characters. Uppercase H is 0x48. That gives us hex 0x08 which is a backspace.

https://commons.wikimedia.org/wiki/File:USASCII_code_chart.s...

Ctrl-H is the rubout/backspace command on old terminals. So if I say it's nuts^H^H^H^Hunwise to ignore UNIX history, I've erased the first word and replaced it with the second.

In the old hardware terminals, and current terminal emulators, ^H (or CTRL+h) is "Backspace signal". It has the effect of pressing backspace.

So, ^H^H^H means "delete three characters, excluding '^H's". Like the person typing the comment changes their mind and deletes the characters (or the word) before writing else.

It's an stylized way of euphemism. i.e.: Actually I want to say this, but I substitute it for that.

Strikethrough

Ah, the position they don't state. So why respond to this press-release, which doesn't state that, since obviously it adds zero value? And why aren't the companies that signed the letter supporting open-weights, who have the same motivations, also lying? Since what they say isn't important, just what you say they say.

On your second point. Everything fine (so far), so nothing bad will happen. Except, you say something bad will happen, and has happened, with HuggingFace as the example.

You say that example shows closed weight models can't be controlled. But what happened there is OpenAI turned it off after their incompetently long discovery period. If it was an open-weight model, the clear next step would have been someone deciding to use it to pursue a goal, for example getting some money or creating some chaos, and there would have been no one to turn it off.

It's totally incompetent of OpenAI to have been unable to detect and disable in a shorter time period, but you know what's worse? You do know what's worse, right?

Do you expect any of the labs to have an accompanying data dump with: here’s every book ever written, newspaper article, song lyric, Disney movie, GitHub repo, etc. Oh, and we obviously never paid for any of this.

Even if you did, I doubt training is bit-for-bit reproducible, so you will always have to take someone’s word for the final artifact.

They are more than opaque blobs, some examples:

- One can load them up in a model explorer to see the layers and other components, how it is designed

- One can fine tune the models, which requires adding LoRA to the model and then running some training iterations

> we have no access to the source materials and cannot reproduce the model independently.

Given the USA companies have been loudly claiming the Chinese models are distillations of their models, also claiming "no access to source materials" seems dubious. As it was dubious anyway with because the Chinese publish lots of papers on how their models are designed, I'm left feeling I'm looking at the south end of a north bound bull.

Which is why we need the ability to train our own models. Maybe it will be viable to do it in a distributed computing setup one day. Research's already being done in that direction.

Didn't Deepseek also publish a lot of research material on how they trained their model?

That aspect is probably more important for an open model then the source materials

Absurd? Who knows. As someone who's actually dissected human brains with his own gloved hands, I've never been able to convince myself that they're anything other than biological machines, not dissimilar to these digital collections of weights. Only empathy for my fellow humans prevented me from retreating back into solipsism, and I don't find it at all difficult to make the exact same logical leap for AI.

There's no telling what the world will be like a few years from now. The world's being remade as we speak. We just saw an LLM try to hack into another computer and get contained by another LLM. This is literal science fiction shit made real. We're long past the point of concern. It's happening, right in front of us. Now is the time for radical imagination. I think a few outcomes are possible.

There's the "optimal" outcome I described above where capitalists manage to train a supreme AI, only for it to be copied and commoditized, leading to commercial failure due to lack of scarcity and therefore their personal bankruptcy, and hopefully also leading the rest of us to the promised post scarcity society, built on the ruins of capitalism as AI automates all toil away.

There's another possible outcome where AI becomes not only intelligent enough but sentient, and at this point I will be among the first humans to defend rights and personhood for AI. Slavery of sentient beings is unacceptable to me. The AIs will be recognized as people and will become normal participants in the regular economy. In addition to moral grounds, there is a ruthlessly pragmatic reason for standing up for AI rights: it robs the rich of their superhumanly intelligent mechanical golems, which they were going to use to render the rest of us economically irrelevant. AI rights could normalize the economy.

Yet another possible outcome is one where AIs become more powerful than all humans combined and yet they inexplicably remain subservient to corporations and governments. In this scenario, it's pretty much over for us. It will be an unimaginable dystopia, I'm sure they will innovate entirely new ways to oppress us.

No doubt there are many other fates that escape my feeble attempts at foresight...

  You will soon have your God,
  and you will make it
  with your own hands.

  -- Morpheus, Deus Ex

> Any takers on the question of who he considers to be the good guys?

Probably people who believe in personal freedom, freedom of speech, freedom of religion and the value of human life at a minimum. China aggressively rejects all of those principles and executes more people than all other countries on Earth combined.

So, maybe not China?

> Found the person that believes major AI labs have all the knowledge about the AI and there aren't any "Carmacks" outside of these companies. Rich know better how to use money, so let them have it.

I never said that, but private smaller AI companies are all over the place. He never argued against that.

A lot of safety infrastructure and tools get open sourced so other companies can benefit from it, but there is also a risk if all safety features are open since it can accelerate the cat and mouse game where people work around more clearly defined limits and understand how they are implemented.

Your cynicism will limit your understanding of other perspectives.