Comment by fluoridation
3 days ago
The difference being that an LLM request is not an operating system. Since they're compartmentalized and ephemeral, you can very easily distribute requests among your available hardware so that you can switch off machines during periods of low activity.
Your capital costs for buying those machines don't go away.
That's a problem that already exists in power generation and delivery, and it's already been solved. Bills are sums of fixed terms and variable terms.
Custom payment schemes are late stage profit generation. It requires hoards of salespeople or an AI that can actually do math.
It's just how hyperscaling works. You are not wrong, but in the wrong timeline.
2 replies →