Comment by simonw
10 days ago
I don't believe that's true on inference - I think most if not all of the major providers are selling inference at a (likely very small) margin over what it costs to serve them (hardware + energy).
They likely lose money when you take into account the capital cost of training the model itself, but that cost is at least fixed: once it's trained you can serve traffic from it for as long as you chose to keep the model running in production.
Some companies like Google, Facebook, Microsoft, and OpenAI are definitely losing money providing free inference to millions of users daily. Companies where most users are using their API, like Anthropic, are probably seeing good margins since most of their users are paying users.
yes I would generally agree; although I don't have a have source for this, I've heard whispers of Anthropic running at a much higher margin compared to the other labs