Comment by ThrowawayTestr
9 hours ago
If inference wasn't profitable why are consumer GPUs that are only good for running open models so expensive? Why and how would these companies sell access to open models if it wasn't profitable? You can rent your own GPU and make money if your electricity is cheap enough.
Consumer GPUs can also be used for things like gaming, remember? And they have gotten so expensive because Nvidia has sacrificed that market in order to build more datacenter GPUs.
Selling at a loss is far from unusual: Uber did it to capture a market, farmers do it to recoup at least some of their investment rather than losing all of it.
>You can rent your own GPU and make money if your electricity is cheap enough.
I would be curious to know whether you'd make more profit from the same kWh of electricity mining bitcoin versus serving inference. I know they use different hardware, but since the "inference is profitable" discussion ignores the purchase price of the GPU, let's pretend you could have opted to buy an ASIC instead.