Comment by pjc50
8 hours ago
.. conversely, all the cloud LLMs are being subsidized by their investors in addition to massive economies of scale.
8 hours ago
.. conversely, all the cloud LLMs are being subsidized by their investors in addition to massive economies of scale.
It is false to say that all cloud LLMs are subsidized. The open weights models are hosted through numerous third party providers on OpenRouter that are operating as hosting businesses. They aren’t spending investor money to provide tokens for you at below-cost rates. They’re operating as hosting businesses.
economies of scale are enough to explain the entire price difference. Running 8 concurrent requests at 100 token/s on $100k hardware is a lot cheaper than running one concurrent request at 20 token/s on $20k hardware