Comment by simonw
1 day ago
I believe that API pricing runs at a healthy margin, at least compared to the server and energy costs used to serve the tokens.
Recent conversation here on that topic: https://news.ycombinator.com/item?id=47062534#47063134
> There exist a large number of people who are absolutely convinced that LLM providers are all running inference at a loss in order to capture the market and will drive the prices up sky high as soon as everyone is hooked.
> I think this is often a mental excuse for continuing to avoid engaging with this tech, in the hope that it will all go away.
Thanks for that psychological explanation. I was wondering why people were simply ignoring the math that shows that inference at API pricing can be quite profitable, e.g. published here for DeepSeek V3/R1 with 545% profitability: https://github.com/deepseek-ai/open-infra-index/blob/main/20...
There isn't a single thing about how the AI companies are operating that looks like a normal business. I know people who were in the room when Scott Sullivan, CFO of Worldcom, assured everyone that the future was bright at Worldcom days before they collapsed. So you'll have to excuse me if I don't believe the words of someone whose sole job is to justify hundreds of billions of dollars being thrown at Anthropic when he says their future is bright.
I agree that the amount of investment thrown at these companies is absurd.
But I also think that their API token pricing represents a real margin over the inference costs for serving those tokens.
Both things can be true at once.