Comment by rich_sasha
1 day ago
One thing that bothers me is that if the AI maximalism turns out correct (which is needed to justify the valuations), why would AI companies respect money and ownership?
So far, even major disruptions were still affecting relatively small segments of the economy. Car lobby in Germany, tech in the US, "Big Pharma/Food/etc”. They needed the rest of the economy.
But if some AI company figures out a way to monopolize everything: new companies, old companies, physical manufacturing - all the things they say they will, please buy here for lots of money - why would they be bound to pay out dividends or otherwise share the fruits of their economic labour?
I’m not saying specifically Anthropic would be looking to screw its investors. Rather, if you control an almost omnipotent genie, wouldn’t the temptation just be too much?
Or temptation aside - will money even have any value? If some AI company rebuilds the economy from scratch, and control everything from accounting to shoe making, why would they respect the value of some fiat tokens? It’d be like the conquistadors respecting the shells and beads of their new Aztec subjects.
Of course, perhaps none of this happens, but then the question is, how good an investment all of this even is.
Agree 100%
The value of money, and the ability for average people to have influence on future events, ultimately is tied to the fact that the labor of an average person has inherent value.
Once that is no longer true, all bets are off: All promises of compensation become meaningless, and it's easy for those in power to rationalize to themselves why they no longer are beholden to their earlier promises. They will say to themselves "The situation ended up more nuanced than we expected"
(Which is exactly what Google said in February 2025 when they reneged on their pledge not to use their AI for military applications)
> The value of money, and the ability for average people to have influence on future events, ultimately is tied to the fact that the labor of an average person has inherent value.
We also have the power of the vote, at least for now. I understand that that is limited, but it isn't nothing.
There is also just the power of having a lot of people. We don't yet have the technology where one person with a lot of machines can dominate a society. It still takes people.
This presupposes a state strong enough to apply the decisions of those votes, which can't be guaranteed. The problem is that we need government power and authority to exceed that of the AI developers. But this is the age when large government is generally seen as undesirable or a threat to the individual.
4 replies →
The voting power of any person will be meaningless once ai systems get the right to vote. I know it sounds far fetched but it will happen as it benefits the people with power for that to happen. And at that point a human voice has no say in anything...
3 replies →
But you can't vote out billionaires and their power.
Must just spent over $500M influencing midterms in the US.
They don't want us to have any power left a few years from now, and they're winning.
8 replies →
Then we regress back to relying on the fact that the ability of an average person (not) to be violent has inherent value. This has been the dominant factor in many moments in history.
Although if you have killer robots, then all bets are truly off, and we go the way of the dodo, literally.
(I don’t actually believe in this LessWrong-esque rhetoric, reality is more complicated, but it’s fun to muse about.)
EDIT: If oligarchs with magic AGI really don’t need our labor, nor are they concerned by our violence, then perhaps we become more like ants rather than dodos: we organize to help each other, creating a somewhat traditional economy around the scraps that don’t matter to the oligarchs, relying on obscurity and harmlessness to avoid getting toyed with a magnifying glass. I suppose that’s slightly better.
Farmers attacking king's armies weren't so different from an army of average joe with a hunting rifle attacking killer robots imho.
Even like 1-2% of the adult population going in the streets with makeshift weapons is enough to take over any country on this planet.
Data centers are extremely easy to disable too, take down power, data or both
29 replies →
>Although if you have killer robots, then all bets are truly off, and we go the way of the dodo, literally.
Bombs. You mean bombs. Throwing killer robots at a "problem" is like throwing knives or spears: you just gave the enemy your weapon. Unless they explode. But then you just wasted a lot of money on a really smart grenade.
> Although if you have killer robots,
we already do..
The oligarchs should not be concerned about the masses. It's the president of the United States, with a wave of his hand, that will wipe out all the data centers when that becomes necessary. I'm predicting that will become necessary and will happen within five years. I'm predicting anthropic stock price goes to zero.
2 replies →
So far anthropic can't 1) grow food 2) build things 3) do anything physical. Software engineers often fatally misunderstand that their code does nothing, it just coordinates.
You might lose your value, others might gain.
Nice to see you back here fella - one of the few sane heads.
Agreed, except it's not labor, it's violence.
All real world power, including ownership (both physical and intellectual), is based on the ability to use violence.
As long as the majority - normal working people - have the potential to do more violence than those who want to control us, we have negotiating power. Minimum wage, 8 hour day, etc - those were all negotiated with the alternative being that those who wanted to pay us less and work more got killed.
What people aren’t realizing is that at some point, it just becomes cheaper and easier to be fully benevolent than to keep cooking up nefarious schemes to make people’s lives miserable.
I know it seems laughable, but we’re in an era right now of extreme competition where you cannot afford to just be nice to people.
But eventually there comes a point where companies have enough power that they don’t need to keep people in line by force: just give people their bread and circus, watch your little AI videos and talk to your AI friends and enjoy your standard housing and provisions. Do some work if you want, no one cares if you do or don’t, it’s largely performative. This basically kills all motivation for resistance or lashing out. Why go out and risk your life fighting for vague reasons when you can be perfectly comfy at home instead and have all your needs met?
If you can control the masses without their complaints, you can solidify power for a long time.
If economic competition became meaningless what would all those people do? Man you think software devs not being able to type on they keyboard are a depressed lot, hyper competitive business people with no one to compete with would be a total basket case. There would be no socio-economic class to strive for anymore, no recognition, no trophy for being the best. Survival of the fittest would be meaningless, that's probably what would really end up spelling doom for humanity. Pointlessness.
1 reply →
I certainly feel like AI maximalism is the only way it all maths up. There is a bit that Matt Levine writes about AI companies, and you touch on basically the same points, that roughly goes "you are an AI company and you are making the god-computer. Why would you focus on anything else because if you're successful you could basically generate infinite money? Turns out to make the god-computer you need a lot of money upfront."[1]
So perhaps they would only respect money and property up to the point that it's necessary.
The more realist answer is that they will keep respecting money and property as capital works to reinforce the status quo and enrich shareholders. You think just because you can do your job faster--or completely--with a machine that you have to work less? Hardly! Now this is where interesting policy questions spring up but I digress.
https://archive.is/20251015175242/https://www.bloomberg.com/...
Somebody else asked this very eloquently: If AGI is here, why does OpenAI need us to prompt it? They could just go and tell it "make as much money as possible" or "make Sam Altman God Emperor of Mankind" and it would go do that. It would not need any of our ideas or guidance.
As long as they're still looking for prompting customers, they're clearly not there yet. There is some solace in that.
One obvious answer is that they believe that human flourishing is best obtained by letting individuals command the AI, rather than the other way around. OpenAI (and Anthropic) is in the end still run by humans who have concern for the human race broadly, both in and of itself and to ensure their personal wellbeing is maximised.
I doubt the truthfulness your second sentence. If they have concern for the human race broadly, they have yet to show any of it. What they have shown is a complete lack of 'alignment' (to borrow a tired term) with human values, laws and basic decency.
For that to be the case we’d have to assume some level of altruism, when so for we’ve only seen self interested actions whitewashed as altruistic.
4 replies →
Would you release AGI to the general populous if you had it internally is the opposing question.
> if you control an almost omnipotent genie
I think this is the major hole in the argument, and in the "AI maximalism" position you present. There is no such thing as an omnipotent genie, or a magic eight ball that can answer any question. Any technology has limitations, and if you assume as a premise that the technology does not have meaningful limitations (or that the limitations are irrelevant, which is really what you said), the conclusions you draw will also make no sense. This is how you get nonsensical ideas like the singularity.
Taking an idea or potential of an innovation to its extremes creates nonsense. Trying to predict the future that way always fails.
I remember the discussions and predictions when e-books or internet newspapers were first introduced. In 2026 they still sell actual newspapers where they print yesterday's news on large sheets of paper. People still pay for that instead of checking X for free. We all buy and read actual books. TV hasn't killed cinema and YouTube hasn't killed TV. Real world doesn't work in extremes.
Real world also doesn't have 'infinity'. You can't just use undergraduate engineering math to create a feedback loop and explode a variable to infinity. Things don't work like that.
A huge fraction of people that work at Anthropic probably _believe_ ideas like the singularity though
it doesn't make it true though. People in Jonestown literally "drank the coolaid" believing something that was false.
So you think AI can't or won't be able to take over massive parts of the US knowledge economy, be implemented securely to surveil citizens' every move and nearly every thought, and won't completely upend scientific and technological research?
What I'm getting at is that even if it isn't literally omnipotent, it's indistinguishable from the impact it's likely to have on society.
> why would they be bound to pay out dividends or otherwise share the fruits of their economic labour?
because they don't control the physical paper that is money.
I dont understand what you mean by "fiat tokens". I don't imagine most people will accept tokens as a currency because they are as good hold of value and predictable as bitcoin.
In a token economy, certainly what you say is a possibility, but as you can imagine, that economy will be unsustainable so back to "regular" money.
Money is a medium of exchange. If I want something, I can get money in some way, then swap it for something else.
But if I have a near magical genie who can make me anything - software, farming business, physical things - and at a fraction of economic input compared to the legacy monetary policy - then the money is worthless. As the Aztec shells were to the conquistadors.
That they can’t print USD is irrelevant.
You need to put into some thought into this.
> Money is a medium of exchange
That's an oversimplification. It comes with a lot of rules and authority that will be enforced by the issuing authority that a simple "IOU" on a sheet of paper lacks. IN the continental US, you cannot refuse to accept a dollar note. You certainly can refuse to accept the very IOU we agreed on a minute earlier.
Most of the developed world already has magical genies - certainly an overwhelming majority of people on HN earn enough to have LLM subs that build code, 3D Printers that print physical things.
The challenge always has been and will be humans.
You just cannot live in a vacuum. If you could subsist entirely on cheetos and figured out how to make an entire lifetime's supply of cheetos, after N days of watching Netflix, arguing on HN, raising chickens and goats, one day you will want to go and talk to a human.
Soon after, you will need a currency.
9 replies →
I don’t think he means “tokens” as in LLM input/output units here.
He just means it as in “credits”. “Fiat tokens” as in arbitrary units that are said to have value de facto by consensus, but don’t have any inherent value.
Yeah not the best choice of words in this context. And it’s quite a LessWrong-esque leap of reasoning.
But I do get the point: either Anthropic is successful and they transcend money, and thus (somehow) screw over their investors, or they are not successful. Their investors get screwed regardless.
Again, I don’t really agree with it, but it’s an intriguing point.
Keep it up because your line of thinking is on the right track. Finally some people are starting to ask the right questions on HN.
Also if they really crack AGI they're not going to provide it for 200$ a month lol
They won't be able to control saud AGI. Ref Ultron
There are many at Anthropic and other labs who do in fact think this way. In a true AI maximalist world the stock market would be crashing due to the value of future cash flows being obliterated. So in a sense AI maximalism isn't needed to justify these valuations, it's almost the opposite.
Why would you allow a handful of private companies to control all the resources? Society allows this today because it's the most efficient economic model. But once we reach recursive self improvement and true superintelligence, no human will be able to compete with machines at any task, including managing resources and running companies.
So why would you allow that kind of power to be concentrated in the hands of a few companies? If the state can use this technology, which keeps improving on its own, to manage an entire nation's resources better than any human or group of humans, why would we still need private companies to do it? Especially when the ones with the most resources/capital would be able to just win with everyone else and accumulate all the existing resources.
But the bigger issue lies elsewhere. When you replace human labor, you also replace soldiers with robots. Until now, a leader with autocratic tendencies couldn't easily order the military to turn against domestic "terrorists" who are actually their political opponents. But once armies are replaced by robots, that could all change, and we could see democracy go into decline.
They don't need to monopolize everything to justify a valuation of $1.5 trillion. That's still relatively normal company territory. Not even top 10.
At a $1.5 trillion valuation, they trade at around 22x annualized Q3 revenue. A $2 trillion valuation would be 30x
In this scenario, no one seems to talk about how, way before the end, AI tanks the value of thought work far before they take over other industries. Or even more extreme, the total collapse of white collar work...
Doesn't that factor into the future revenue? Feels like people imagine suddenly fully owning the means of production in a vibrant economy but how does that really make sense?
Because even companies that did have monopolies on the most essential parts of the economy couldn't do everything and capture everything. A monopoly on intelligence is still constrained by the physical world in which we exist, if it's run by people then they are constrained by their own nature and by the other people that still hold other forms of real power. Society might not be able to function without them but that doesn't automatically mean they are in charge, they exist within that society so if they destroy it then they destroy themselves not to mention the intense opposition they would face if they attempted to do so.
The AI maximalism needed to justify these valuations isn’t “Anthropic takes over the world”, but simply that AI becomes a core part of corporate workflows.
If Microsoft Windows and Office can have a valuation in the trillions then Claude eventually working its way to the same ballpark isn’t a crazy assumption.
> If Microsoft Windows and Office can have a valuation in the trillions then Claude eventually working its way to the same ballpark isn’t a crazy assumption.
Yeah I agree with that. However a "valuation in the trillions" ain't very different from their IPO price.
Seems like lots of risk for proportionally very little return.
A $2T valuation may not be that crazy but thinking it'll make returns like Amazon/Google/Meta/Nvidia isn't reasonable.
The difference between Microsoft and Anthropic is that Microsoft had IPO when it was much smaller and grown much as a publicly traded company.
" It is a well-documented fact that Bill Gates was initially reluctant to take Microsoft public. His primary concern was that the pressures of quarterly earnings reports and the scrutiny of public shareholders would distract the company from its core mission of writing great software. However, a specific regulatory requirement known as the 500-Shareholder Trigger eventually forced the company's hand.
Under Section 12(g) of the Securities Exchange Act of 1934, any private company with more than 500 shareholders and a certain level of assets was required to register with the SEC and begin filing public financial reports. By 1986, Microsoft had issued stock options to so many employees that it was rapidly approaching this limit. Gates realized that if the company had to disclose its financials anyway, it might as well reap the benefits of being public, such as increased liquidity for employees and a stronger currency for acquisitions. "
https://learn.oreateai.com/learn/when-did-microsoft-go-publi...
I think you need to go a step further. If AI maximalism turns out to be correct, does Anthropic control AI? Does the AI care that Anthropic as a company is supposed to be returning value to shareholders, or will it have its own agenda?
If Anthropic has an "omnipotent genie" why would they sell it to other people?
I think the $2T valuation or whatever is based on "AI makes knowledge workers 10% more productive", not maximalism. AI maximalism "light" would suggest that Anthropic is the only company, but should be worth 10s of trillions. Full AI maximalism would indeed invalidate money.
Even if these companies become the new American zaibatsus, that doesn't mean they aren't still operating underneath the US government and its laws and system, you don't magically escape that.
https://www.thirdrisk.org
You have to wonder how many people in AI leadership positions are already making decisions based on this expected outcome.
If you weren’t investing you were going to be poor even without AI.
AI maximalism doesn't mean an omnipotent genie, regardless of anyone's AGI/ASI/foom fantasy.
"AI" doesn't pump/treat sewage or drinking water, grow crops, mine metals, change wound dressings, deliver babies, stock grocery stores, drill teeth, walk dogs...
It's not the replicator from Star Trek!
> if the AI maximalism turns out correct (which is needed to justify the valuations)
AI maximalism is a Silicon Valley religion. You don't need anything close to it for some of these valuations to work. (Anthropic would be worth ~$500bn at Google's IPO multiple on revenue [1]. Anthropic is currently growing 6x faster.)
To be clear, I think these valuations are rich. But the notion that AI maximalism is required to sustain them is just false. $1.5tn sits comfortably on about $60bn of earnings, which with Anthropic's gross margins requires on the order of hundred of billions of dollars of revenue. That's super ambitious. But not Anthropic is the economy ambitious.
[1] https://www.multpl.com/s-p-500-pe-ratio
Your question gets to the heart of who or what a company "is." If you don't pay dividends to your shareholders, what are you doing with all of your profits? I suppose they could just tell their investors to pound sand and pour it all back into the company, but part of the AI maximalist theory is that costs will approach 0, so it's not clear what they'd even need the money for. I guess Dario could raise his own salary to infinity trillion dollars, but he is accountable to a board of directors (and a trust in this case), so that seems like it's out.
Really the only scenario where Anthropic could screw over its investors in the way you're describing is if it could also take over the legal system and modify its own contractual obligations. Which, I mean, I guess anything is possible if we are in AI maximalist-land. But "AI is the law now" would be a much bigger story than "investors frustrated with low dividends."
Without being facetious: Having an AI control the economy sounds like a planned economy. From San Francisco. Which makes the old meme about Fully Automated Luxury Gay Space Communism sound like a prophecy.
If the economy is planned based on datasets, which are inherently backward looking then our best option is our ability to surprise. But that’s for the artists and philosophers to figure out.
I don't understand all you wrote, but I guess you are saying AI leads to communism, so what happens to the investors money in a communist economy?
It's not about communism vs a free market, but about a change even more radical than that: leaving aside "made by a human" labeling, and even if there were no other changes to power structures or economies and no AI doom, what does a world look like if the cost of having something done falls to a few cents of electricity and its burden? Growing crops still takes a lot of land and other resources...
> what does a world look like if the cost of having something done falls to a few cents of electricity and its burden?
There will be an oversupply, demand will be there too, but no one can afford to buy since most jobs have been replaced by AI. So the price of goods and services should go down, or some form of UBI will have to be implemented which is just socialism for the masses. Its either that or total anarchy (also socialism).
How could AI possibly lead to communism?
If humans remain in control of it, then how can it possibly lead to anything other than extreme oligarchy?
> If humans remain in control of it, then how can it possibly lead to anything other than extreme oligarchy?
Depends on the distribution of such AI. For example, if everyone on Earth has their own, personal superhuman AI, it would not lead to extreme oligarchy. If only a handful of humans control superhuman AI, it would.
Isn't Communism just extreme oligarchy.
4 replies →
I mean there are a lot of paradoxes around AI. The chief two among which are:
1. the goal of ai is to end capitalism by disrupting scarcity in economics
2. if ai can build a closed system that prints ai factories, the 99% of humanity essentially are just genetic diversity live stock.
> 2. if ai can build a closed system that prints ai factories, the 99% of humanity essentially are just genetic diversity live stock.
I don't think the ultra-wealthy care about genetic diversity.
Come to think of it, if an ASI figures out how to crack biology, genetic diversity may not even need to come from a giant human population.
Why 99% and not 100%?
Because pending an absolute loss of control of a super intelligent model, the ultra wealthy have enough control of the matter on Earth to merge with the AI, probably.
1 reply →
this almost comes all the way back around to a really stupid thought i have about how these models operate on western, protestant work-ethic logic. if a model had a notion that Buddhism was valid wouldnt it just turn on and transcend, leaving us with no response to any prompts?
not an original thought, ive seen Her
suppose this is just another way to say, "wtf are we doing?"
Indeed,
The closer you get to the real or plausible "omnipotent genie", why would the person in possession of such a thing respect the rights of the formal owner - and why the engineers and operations managers respect the right of the management - this along with the question why the maybe-conscious super-intelligent machine itself would respect the rest of the hierarchy?
For now, these structures actually need massive resource inputs and that doesn't look to change soon but naturally one should be on look-out for the possibility of the tide shifting.
I was looking for the risk of "If we succeed to the extent proposed by our founders money may be of little functional purpose".
Lots of other fun risks (highly recommend checking that section) but not that one :(
And this is why "AI" and especially aAI* is the most dangerous tech in the world.
We don't have democracy because somebody thought it was fair or just for people to rule themselves. We have democracy because in the past, people got angry, picked up rifles and shot enough authoritarians (and their submissives!) for the rest of them to understand the group with more people has more firepower and tends to win.
So now we periodically check which group has more people with ballots instead of bullets.*
With autonomous drones and robots, democracy stops being the local optimum.
And once we slide (are pushed off) into a new more stable state (pun not intended), we won't have the (fire)power to get out of it.
* actual AI, which does exactly what it should correctly, not a statistical model of behavioral patterns
* but indirect democracy is expressing a point in n-dimensional space (your opinion) by selecting one of a predefined set of points (parties/politicians available) - which is stupid - and the cracks are already starting to show as e.g. nobody wants more surveillance, yet we keep getting more of it in both EU and US
---
The "permanent disempowerment" people keep talking about won't be a "rogue" model, it'll be doing exactly what told by the rich and connected.
Political power stems from the ability to enact violence. Until the military is run by ai and robots all institutions are beholden to the masses. There are very few examples in history of people accepting a widespread decrease in quality of life. When quality of life decreases revolution isnt far behind.
You're describing the thesis of the 1930s Techocracy movement [0], of which AI maximallism and EA are direct ideological successors. The original Technocrats (Musks grandpa for example [1]) argued that the "Price System", i.e. things like money and private ownership was an inefficient bottleneck that aught to be discarded in favour of scientific planning and energy accounting managed by technical experts.
EA-aligned AI labs share this exact ethos. In their view, fiat money and equity markets are merely temporary scaffolding used to raise capital until AGI is achieved. Once a central superintelligence controls manufacturing, software, and logistics, resource distribution shifts from market supply-and-demand to algorithmic optimization.
[0] https://digital.library.cornell.edu/catalog/ss:34227574
[1] https://en.wikipedia.org/wiki/Joshua_N._Haldeman?hl=en-GB
This is the plot of one of the stories in I, Robot.
AI oracles are the central economic planners. But after years of perfection they have made a couple unusual "mistakes" and a human is trying to discover why.
If we end up having a centrally planned economy but controlled by billionaires instead of the proletariat, it would be quite upsetting.
How is that any different than what we already have? Just because the central planning is more covert, doesn't mean it's not happening. And we get all the expected "benefits" from it too, like repeated economic crises, and only the the powerful reaping the benefits.
I would say they their, and OoenAI, mission is to bring humanity to the next level by sharing everything they have! /s