Comment by AJ007
1 day ago
Pretty easy to hit $100 an hour using Opus on API credits. The model providers are heavily subsidized, the datacenters appear to be too. If you look at the Coreweave stuff and the private datacenters it starts looking like the telecom bubble. Even Meta is looking to finance datacenter expansion - https://www.reuters.com/business/meta-seeks-29-billion-priva...
The reason they are talking about building new nuclear power plants in the US isn't just for a few training runs, its for inference. At scale the AI tools are going to be extremely expensive.
Also note China produces twice as much electricity as the United States. Software development and agent demand is going to be competitive across industries. You may think, oh I can just use a few hours of this a day and I got a week of work done (happens to me some days), but you are going to end up needing to match what your competitors are doing - not what you got comfortable with. This is the recurring trap of new technology (no capitalism required.)
There is a danger to independent developers becoming reliant on models. $100-$200 is a customer acquisition cost giveaway. The state of the art models probably will end up costing hourly what a human developer costs. There is also the speed and batching part. How willing is the developer to, for example, get 50% off but maybe wait twice as long for the output. Hopefully the good dev models end up only costing $1000-$2000 a month in a year. At least that will be more accessible.
Somewhere in the future these good models will run on device and just cost the price of your hardware. Will it be the AGI models? We will find out.
I wonder how this comment will age, will look back at it in 5 or 10 years.
Your excellent comments make me grateful that I am retired and just work part time on my own research and learning. I believe you when you say professional developers will need large inference compute budgets.
Probably because I am an old man, but I don’t personally vibe with full time AI assistant use, rather I will use the best models available for brief periods on specific problems.
Ironically, when I do use the best models available to me it is almost always to work on making weaker and smaller models running on Ollama more effective for my interests.
BTW, I have used neural network tech in production since 1985, and I am thrilled by the rate of progress, but worry about such externalities as energy use, environmental factors, and hurting the job market for many young people.
I've been around for a while (not quite retirement age) and this time is the closest to the new feeling I had using the internet and web in the early days. There are simultaneously infinite possibilities but also great uncertainty what pathways will be taken and how things will end up.
There are a lot of parts in the near term to dislike here, especially the consequences for privacy, adtech, energy use. I do have concerns that the greatest pitfalls in the short terms are being ignored while other uncertainties are being exaggerated. (I've been warning on deep learning model use for recommendation engines for years, and only a sliver of people seem to have picked up on that one, for example.)
On the other hand, if good enough models can run locally, humans can end up with a lot more autonomy and choice with their software and operating systems than they have today. The most powerful models might run on supercomputers and just be solving the really big science problems. There is a lot of fantastic software out there that does not improve by throwing infinite resources at it.
Another consideration is while the big tech firms are spending (what will likely approach) hundreds of billions of dollars in a race to "AGI", what matters to those same companies even more than winning is making sure that the winner isn't a winner takes all. In that case, hopefully the outcome looks more like open source.
The SOTA models will always run in data centers, because they have 5x or more VRAM and 10-100x the compute allowance. Plus, they can make good use of scaling w/ batch inference which is a huge power savings, and which a single developer machine doesn’t make full use of.
> Pretty easy to hit $100 an hour
I don’t see how that can be true, but if it is…
Either you, or I are definitely use Claude Code incorrectly.
It’s definitely easy with an API key I hit 200$ in an evening. I didn’t think that could be possible. Horrifying
To be clear, this is a lot of full-scale reading and (re)writing, without any rules, promots, "agents"/code to limit your resource usage, right?
Nobody's asking for $200 in single-line diffs in less than a day - right?
4 replies →
This is around what what Cursor was costing me with Claude 4 Opus before I switched to Claude Code. Sonnet works fine for some things, but for some projects it spews unusable garbage, unless the specification is so detailed that it's almost the implementation already.
> unless the specification is so detailed that it's almost the implementation already.
You mean… it’s almost exactly like working with interns and jr developers? ;)
1 reply →
Why “no capitalism required”? Competition of this kind is only possible with capitalism.
Have you been human before? competition for resources and status is an instinctive trait.
It rears its head regardless of what sociopolitical environment you place us in.
You’re either competing to offer better products or services to customers…or you’re competing for your position in the breadline or politburo via black markets.
Even in the Soviet Union there were multiple design bureaus competing for designs of things like aircraft. Tupolec, Ilyushin, Sukhoi, Mikoyan-Gyurevich (MiG), Yakolev, Mil. There were quite a lot. Several (not all, they had their specialisations) provided designs when a requirement was raised. Not too different from the US yet not capitalist.
Unfortunately it's called war and it appears to be part of human nature.
Not really, it's possible with any market economy, even a hypothetical socialist one (that is, one where all market actors are worker-owned co-ops).
And, since there is no global super-state, the world economy is a market economy, so even if every state were a state-owned planned economy, North Korea style, still there would exist this type of competition between states.
Worker owned coops are not socialist unless the government forces it.
2 replies →
I mean, if you wanna get technical, many companies in Silicon Valley are worker-owned (equity compensation)
4 replies →