Comment by bsder
14 hours ago
> $200 in rent every month into the large model providers
We all know that is hugely subsidized, and I guarantee that OpenAI and Anthropic are looking to enshittify that ASAP.
The enterprise users, however, are not subsidized like that. They pay per token. And some developers in those companies are chewing down a lot of tokens. Self-hosting an open weight model could be a massive savings very quickly. It also gives them negotiation leverage when talking to OpenAI and Anthropic.
We all ASSUME that.
For all we know, inference might be dirt cheap, they might just be hiking the API prices so high for us to think subscriptions are subsidized.
Well, it is always possible that OpenAI and Anthropic have some secret sauce that they are hiding that nobody knows about. However, the pricing of tokens on places like HuggingFace and OpenRouter suggests that probably isn't true. In addition, the lack of reliability and general slowness of Opus and Fable at various times also suggests that.
Now, the one wildcard in all of this could be Google. They are on the eighth generation of their TPU and have been holding their cards extremely close to the vest. I don't think anybody has a good read on exactly how much capacity they have. Most things you can kind of figure out the overall business numbers and what's going on in Google--the TPU area is one of the exceptions. I know a couple of big customers and even they don't have any visibility on that front.