Comment by cortesoft
11 hours ago
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
It doesn't just matter that inference is profitable - cash from profitable inference needs to outweigh cash burned by training. If training costs even $1 more than your profits from inference on a model, you lost money on the whole thing.