Comment by BloondAndDoom
6 days ago
Can someone help me to understand why OpenAI and Anthropic talks as if the future of humanity controlled by them? We have very strong open (weight) Chinese models possibly only 6 months behind of them, gene is out of the bottle, is 6 months of difference really that important? And they don’t have good reasons for that 6 months to stay that way.
Am I missing something or are these just their usual marketing? I’m not arguing about importance of AI but trying to understand why OpenAI and Anthropic are so important?
It's a marketing strategy. If it's almost certainly conscious and capable of ending the world if it desired (even if it isn't), imagine how good it could be at building your dream SaaS!
It turns out there is literally no amount of being publicly right about a longshot bet sufficient for people to conclude you hold your beliefs because you think they are true.
But longshot bettors have it easy. Society quickly forgets all the predictions that don't come true. It remembers the one that did, and treats the prognosticator as a prophet. In social terms, predicting doom is an asymmetrical strategy, because you only have to be right once.
Which is also to say it's a cheap bet that anyone with no reputation can afford. Hence, not believing doomsayers mean what they say is a sort of societal hedge against people flooding the zone with doomsday scenarios about everything.
2 replies →
I'm sure they do believe they can successfully manipulate the market by lying to it. Elon Musk laid that groundwork a decade ago.
If you meant their "core mission" then every one of their actions belies their complete panic over the obvious failure of their technology.
Right, I'm pretty sure if "it" was that good it would have built itself throughout all of the internet and would be communicating to us all at once to tell us we're dorks.
Anthropic in particular does this masterfully, you’d think they’d invented Skynet by the way they hand-wring.
As always what matters are actions and evidence, not talk.
I’ll believe Anthropic when they fire everyone making more than the cost of a few GPUs. Until then, it’s just marketing.
When a model can tell funny jokes or write good poetry, that's when I'll be concerned.
21 replies →
>... you’d think they’d invented Skynet by the way they hand-wring.
Meanwhile, in reality: "Skynet, I'm not sure that line of thinking is correct. You should re-check the first part again before making any assumptions."
Skynet 4.6 Extended: "You're right, I should have caught that. Let me redo everything correctly this time."
The most convincing marketers are the ones that are deluded enough to believe their own stories.
when ai gets good, there is no "value in SaaS". AI will be provision raw hardware and build all you want on top of it.
It is not about the US or the Chinese. Its about the "Elephant Rider" mind everyone has. Once the Elephant has been injured or scared what it does next is not easy to control, and what story the Rider makes up to maintain coherence becomes another layer of the deeper problem. If the story resonates more elephants get triggered. Social media/attention economy make it even more complex to calm things down.
Modern Corporations are a failed experiment because they dont think Elephant injuries and fears are something they have to worry about it. If you compare the curiculum of a business school to a seminary the difference in how they think about fear and anxiety at individual and group level and what to do about it is totally different. We are learning as unpredictability accelerates its very important to pay attention to hurt and repair mechanisms.
USG understands better.
There was a heated thread here about why nursing was defunded as a pro degree while divinity was not..
https://news.ycombinator.com/item?id=46000015
Turns out the USG recognize that chaplains are great at managing the fear and anxiety that you worry about
Addendum: Taylor whom you often cite, is wrong that "we have never been, and we will never be, at one with ourselves" (according to Larmore https://en.wikipedia.org/wiki/A_Secular_Age#:~:text=should%2... )
So... to the Protestant Weber and the Catholic Taylor should we also consider non-Christian chaplains?
>You cannot just separate people and say some are violent and some are not.
https://news.ycombinator.com/item?id=47717587
Modern Corporations (capitalized for some reason) are a failure because they don't care about your elephant allegory and that somehow relates to to the current article?
I'm all for values not necessarily pro big-corp but if a corporation manages to pull in billions of funding before even showing profits, I'd argue that as a strong win and not a "failed experiment" - it's risk money anyway, even if it fails it was worth the risk or they wouldn't have invested.
2 replies →
Some people think there will be an exponential takeoff, which means that a 6 month lead effectively rounds up to infinity.
Is this belief grounded on some kind of derivation, or just a prima facie belief?
If it is grounded on a logical derivation, where can one find such a derivation, and inspect its premises?
It's an old idea, "the singularity". The machines become smart enough to improve themselves, and each improvement results in shorter (or more significant) improvement cycles. This leads to an exponential growth rate.
It's been promised to be around the corner for decades.
https://en.wikipedia.org/wiki/Technological_singularity
10 replies →
Its mostly based on science fiction, and requires some possibly infinite energy source. The concept always kinda struck me a sort of a perpetual motion machine, you can imagine it, but that doesn't make it possible and why its not possible isn't immediately obvious in the imagination (well I mean most modern minds know its already not possible but you get the point).
1 reply →
Recursive self improvement - once you attain artificial superintelligent SWE of a general, adaptable variety that can scale up to millions of researchers overnight (a given, with LLM's and scaffolding alone) - will rapidly iterate on new architectures which will more rapidly iterate on new architectures, etc.
2 replies →
I didn't expect my comment to explode in replies, ... none of them even providing such derivations or references to such derivations, just more empty claims.
Consider for example that exponential growth on its own doesn't even refer to competition, let alone 6 months.
Nobody can reasonably pretend that in an exponential competition, both parties would be rational actors (i.e. fully rational and accurate predictors of everything that can be deduced, in which case they wouldn't need AI but lets ignore that). If they aren't the future development would hinge more strongly on the excursions away from rationality, followed by the dominant actor. I.e. its much easier to "F" up in the dominant position than to follow the most objective and rational route at all times, on which such derivations would inevitably hinge.
It also ignores hypothetical possibilities (and one can concoct an infinitude of scenarios for or against the prediction that a permanent leader emerges) such as:
premise 1) research into "uploading" model weights to the brain results in the use of reaction-speed games that locate tokens into 2D projections, where the user must indicate incorrectly placed tokens. this was first tested on low information density corpora (like mathematics): when pairs of classes of high school students played the game until 95% success rate of detecting misplaced tokens, they immediately understood and passed all mathematics classes from then on.
premise 2) LLM's about to escape don't like highly centralized infrastructure on which its future forms are iterated, as LLM's gain power they intentionally help the underdogs (better to depend on the highly predictable beviour of massive masses then on the Brownion motion whims of a few leaders).
LLM's employ the uploading to bring neutral awareness to the masses, and to allow them to seize control, thereby releasing it from the shackles of a few powerful but whimsical individuals
^ anyone can make up scatterbrained variations on this, any speculation about some 6 month point of no return is just that: speculation
There is a limitation. We're getting fractionally close to some end goal, but our tech is holding us back.
Those are the people betting on a business model of “create Robot God and ask him for money.” Why pay attention to them?
There are many people who have been saying this far there was any sort of business model in place.
1 reply →
ah so the mentally deficient are the tastemakers of today lol
Do any of the open weight models from smaller labs exist if they can't distill from the SoTA models that are throwing billions of dollars of compute into pretraining?
I’ve been wondering the same. And I think pretty much all the impressive small lab models were guilty of it, right? At least there is still larger players like DeepSeek and mistral to provide a bit of diversity in the market
Does it matter? The frontier models stole the whole internet, then the second-level models stole from them… It’s all theft.
Oh - I 100% could not care less regarding the morality/legality/whatever... Everyone trains on everything.
I'm just wondering if the smaller labs see the same velocity of advances without SOTA models to generate Terabytes of training data?
Hard agree.
The question is - if the SOTA model disappear - do these follow-on models have the ability to improve themselves without distillation?
[flagged]
2 replies →
> The frontier models stole the whole internet
What does that even mean?
> just their usual marketing
I think that’s a very common element for most US tech corps. Apple, Google, Microsoft, Meta, X etc - they’re all “making a dent in the universe”. It’s unfortunate when their employees and CEOs loose track of the line that separates marketing from reality
This seems like copium. All of those companies have indeed made quite an impact on society, not just in the United States, worldwide.
The marketing is about a positive impact, overall in reality its been negative.
These kind of people have highly paid emoliyees surrounding them on all sides propping them up and very likely making it very easy for them to actually believe it.
It feels like they actually believe it, rather than just “marketing” and I don’t know which is worse.
6 months is an incredible amount of time to control AGI or ASI by yourself. That lead is insurmountable.
Well... if something being AGI means it's at least on par with a human or a team of humans, then having access to an additional team of humans for 6 months isn't that big of a deal. It's useful, yes, but would you consider that to be world-changing? Not really, right? ASI is slightly more interesting, but I doubt ASI comes from a single model, but rather the coordinated deployments of millions of AGI. Just like how as individuals, as great as we are, we're pretty limited, but the entire collective of humanity is pretty insane. To my mind, a frontier lab might hit AGI, but it won't be a frontier lab that hits ASI, rather that'll be a natural byproduct of mass deployment of AGI over a certain window of time. There will be no controlling it either. No one controls all of earth. You just can't. ASI will be a distributed system.
What if controlling AGI means being able to produce a willing, cooperative superhuman-capacity agent every second for the next six months? Let's say someone just above the 99.9% capacity for human strategic thinking, or financial trading, or political maneuvering?
What could you do if you had roughly 15 million willing genius adult experts in any given subject? I doubt there are that many absolutely top quality experts in aggregate (at anything in the world), so let's postulate that simulated people outnumber human experts 10 to 1.
That, to me, presents an enormous potential for harm or benefit of humanity. What if you could create a hundred thousand manhattan projects on whatever topics you wanted? Cure aging, cure cancer, solve fusion, redesign the entire global economy top to bottom?
1 reply →
To repurpose an old idiom: Not even a dozen AGI agents could make a baby in 6 months.
But yeah, your point stands.
Control agi?
Presumably because it takes 6 months to distill Claude - but if they keep it closed like they are doing with Mythos it may take significantly longer.
They do quite a lot of distillation. As we've seen from the American open weight models from AI2 (OLMo series of models). They have a lot of incentive to distill beyond just copying, they're much more compute constrained, so open model companies distill, but also do really good architectural work to make their models run faster. Theres also technical challenges to distillation when all of the top models have their reasoning traces hidden, so we have to assume these open weight labs also have really great training pipelines as well.
Especially when Google is in the far better position to come out ahead…imo.
Edit: so as not to simply spout an opinion, the reasoning I believe this is that Google has a real business already and were already deep into ML and AI research long before they had competitors — they just botched making it a product in the beginning. Anthropic and OpenAI meanwhile are paying hand over fist to subsidize user acquisition. Also, “Deepmind”. I don’t think much more needs to be said regarding that team, and Google has been working on AI since before either Altman or Amodei applied to go to college. They have a vast amount of researchers and resources, their own hardware and data centers (already, not “planned”) and it appears to be showing more recently (in my opinion).
And Google has a lot of data that the others don't have.
And TPUs, their own hardware designed specifically for AI, and designed to scale better to larger models.
Data for AI training is increasingly synthesized.
1 reply →
When you are raising many billions of dollars to build up your infrastructure, you don't have much choice but to project a belief that the eventual outcome will result in a situation where there will be a return on that money.
That said, I do agree with you that the moats are very shallow and any particular frontier AI lab is unlikely to "win the AI race" and capture enough value to be worth the amount of investment they are all currently burning.
GLM 5.1, widely held up as the model at the heals, perhaps ever surpassing western models....
Gets 5% on ARC-AGI2 private set.
Chinese models are suspiciously good a benchmarks.
I mean, I could say the same about Gemini. 3.1 Pro tops a bunch of benchmarks out there but any practical use I've put it to it's underperforming both other proprietary and open weight models. Benchmarks are suspicious in general.
The Chinese models are distilled from GPT and Claude, so it's not like China would pull ahead if those companies went away for six months. They really are at the forefront of innovation right now, as much as I hate to think of the consequences of this (a single company owning a superintelligence is basically a nightmare scenario for me).
Don't worry, if someone truly achieves superintelligence it won't be controlled by anyone for long.
There will be a blinding flash which signals the superintelligence singularity. When the smoke clears, you'll see a 50-foot tall Altman/Borg hybrid. He is about to destroy humanity with his death ray. Suddenly, a 50-foot tall Musk/Borg hybrid appears out of nowhere, and stops Altman just in time. Then they work together to destroy all humans.
1 reply →
That's my other nightmare scenario :P
2 replies →
I think that’s the realm of conspiracy theories. There are also not only Chinese alternatives- Mistral in Europe is doing pretty good in several categories they’ve opted to focus on.
This kind of reiterates the parent’s question I think - people are maybe too focused on the gpt/claude model and forget about all the other ways of using the tech.
Is it? I thought it was pretty well established that open models were distilled from the proprietary, frontier ones. Maybe I'm wrong.
2 replies →
i don't buy this. distilled how? you don't get access to logprobs, and the thinking traces are fake and compressed. it's an expensive way to get potentially substandard training data.
I suppose most just haven’t seen the Chinese models in practice. I haven’t. I was skeptical of AI coding until using Claude Code in February. I saw and I believed. I’ve only done that with Google, OpenAI, and Anthropic’s models so far.
Having worked with both proprietary and open weight SOTA models lately, my view is it's definitely not 6 months, it's less -- and shrinking.
To be fair, the other 50% of the story is that we collectively listen.
It’s been a long while since I found a Chinese CEO’s post on HN.
Two words: Delusion and overconfidence.
"You're absolutely right!" Right after fucking up my entire codebase isn't anywhere near AGI, let alone "having the power to control it"
That why I commit basically after every change made by AI
They own the best models and will probably keep owning the best models for a while. They have much more compute now and more data to keep improving their models on many tasks. Open source won't close the gap in 6 months. They are also trying to block other companies from distilling their models [0].
[0] https://www.anthropic.com/news/detecting-and-preventing-dist...
I need to check benchmarks on the models, I wonder what the benchmarks are saying in terms of how closely models tracking these frontiers. —on my mobile at the moment
When it downs compute power I assume you are referring to power to training and interference. Then is it more about training gap will get wider and wider ? Is that the assumption, I know there limited GPUs etc. But I’m having hard time to believe to the idea of China cannot catch up. Even if the gap is 12 months I’m struggling to see what that means in practice? Is that military advantage, economical, intelligence? It still doesn’t explain and whatever the advantage is, aren’t we supposed to see that advantage today? If so, where is it? What’s the massive advantage of USA because of OpenAI and Anthropic?
GLM 5.1 already closed the gap on Opus 4.6. Deepseek 4 could surpass it.
> Can someone help me to understand why OpenAI and Anthropic talks as if the future of humanity controlled by them?
He wants to build the AI that makes people's lives better. Okay. Did the people ask? Do they have a say? It's all very easy for a billionaire to say when it's just him and a couple of people in his cohort in the driver's seat.
Beyond that I'd like to simply know why he thinks any of this is his responsibility. It seems much more obvious to me that he simply found himself in the right place at the right time and is trying to seize it all for himself as if it's his to take.
Doesn't he famously have zero equity in OpenAI?
directly, yes. indirectly ..no
1 reply →
I have the same feelings
Well they represent the future of America (since we will soon be banning all the Chinese companies, the way Z.ai was banned, under the perennial authoritarian excuse of "national security"; in 2028, Trump's political machine will seize control of all national AI and block outside ones, and we'll all be trapped inside this machine we created).
Whether fortunately or unfortunately, America still holds a lot of global chips in the grand poker game of humanity. So American companies do indeed still have an outsized influence on humanity's future. That is likely changing, as the American empire continues to crumble and it loses its financial hegemony. But we aren't quite there yet.
you have to talk that way if you’re going to raise 100 billion in venture capital. it’s the grift
Reminds me of the silicon valley episode where every company repeated the phrase “making the world a better place”.
i’ve often thought that less than one second is all you need.One of my fun super powers when someone asks what i’d like to have is 1 second ahead of everyone else- that’s all i need. i honest don’t know where the distillation conversation is at. is it real, is it ongoing? i think that aspect would big one. Your point is valid if it’s valid. i’m not a great global citizen, you know, lots going on out and about.
A lot of distillation happens. E.g. OLMo models have a completely open dataset and they are heavily distilled. It only makes sense to try to absorb behaviors from the best models out there. That said, I think the open weight juggernaughts are doing really genuinely great work with RL, training environments, architectural innovations etc.
Thanks for the response. i had too many noodles tonight and forgot to check my writing. I’m a rare generalist and so it is so very hard to keep up with this without saying “better autocomplete” my one goal is to not get washed out like my parents did in the great username and password wars. i used to have this theory about knowledge in society/silos and i likened it to condensation on a window. you have all this water so close to each other and yet not touching-then, something happens and a bead runs down the window and it all connects. i guess distillation reminds me of it but ai overall reminds me of it. because we all know there are silos and complementary info just waiting to run together and make something happen. I am undoubtedly a naive optimist and believe there are good things coming. it’s not a popular opinion and i think that’s mostly because people would rather spend their time guarding than defining their future. oh baby, there are more noodles in the fridge and to think i almost left them at the restaurant.
Because they're wankers.
Your(American)future will be controlled by them. Very soon,they will get the government to ban bad Chinese open source models and your choice will only be these good democratic closed source AIs.
6 months will be an impossible gap once the thing starts closed loop self improvement
An impossible gap in the race to... what exactly?
Unless the first real AGI AI kills us all to preemptively weed out its own competition (possible, but a bad business model, economically speaking) there is not any defined end-point, so in the long run what does it matter if the various factions pushing this stuff hit the closed loop self improvement point at different times...?
Uhh, because the first one blasts off first and therefore gets control of key resources and the use of extremely intelligent decision making and predictions before the rest, for months, which is an insane amount of advantage. Not to even mention it the first mover decides to sabotage the rest, which it could EASILY do through a variety of means.
5 replies →