It's not clear at all that subscription was ever a loss leader.
The purpose of the high API prices is, in addition to trying to get more money, to anchor the high price, and then make the subscriptions seem like a great deal. But inference is just not that expensive, not when provided at scale and with huge batches.
You can see this with the various Chinese providers doing inference on open models, which are unlikely to cost less to run than the models at the frontier AI labs. A single 8-GPU GB300 cluster doing high-throughput batch inference on Kimi K3.1 Pro, which is the largest open weights model available, can do something like 300 000 tokens per second, that is, 10T tokens per year. That cluster, with current insane inflated prices, costs something like $1M to buy, and $50k to run per year in electricity. At $10/Mtok, that pays itself back in 4 days. Do you think that that's a reasonable price?
The subscriptions were probably priced with reasonable positive margins.
The huge expenses are in training and pushing the models forwards, with the gigantic RLAIF runs we know they are doing. Then once you got the model, you can scale up inference at low cost and want to serve as many people as possible.
No, inference as whole, including all the subscriptions.
The API Gross margin is even higher.
Fair. Perhaps it would be better to say a fully utilized subscription is a loss leader
It's not clear at all that subscription was ever a loss leader.
The purpose of the high API prices is, in addition to trying to get more money, to anchor the high price, and then make the subscriptions seem like a great deal. But inference is just not that expensive, not when provided at scale and with huge batches.
You can see this with the various Chinese providers doing inference on open models, which are unlikely to cost less to run than the models at the frontier AI labs. A single 8-GPU GB300 cluster doing high-throughput batch inference on Kimi K3.1 Pro, which is the largest open weights model available, can do something like 300 000 tokens per second, that is, 10T tokens per year. That cluster, with current insane inflated prices, costs something like $1M to buy, and $50k to run per year in electricity. At $10/Mtok, that pays itself back in 4 days. Do you think that that's a reasonable price?
The subscriptions were probably priced with reasonable positive margins.
The huge expenses are in training and pushing the models forwards, with the gigantic RLAIF runs we know they are doing. Then once you got the model, you can scale up inference at low cost and want to serve as many people as possible.