Comment by parineum

There is also the cost of the inference hardware that gets ignored because they already have it from training.

The main problem with the scenario of just doing inference is it relies on nobody else training models better than yours. As long as people are training private models that are better than yours, just inference isn't a viable business model.