Comment by jacquesm

Well... that's a tricky one. I bought a lot of these second hand. I can sell them for more than they originally cost new, so indeed, it is not 'cheap'.

Then again, I run a business and for the business it is absolutely manageable, it is roughly the price of a single job. Privately this would be a different matter but I assume that if you would do this privately you'd simply get a single (but powerful) GPU second hand and offload to RAM. Not as fast but with llama.cpp, 'ninfer', DwarfStar and various other clones of those two you can get some pretty decent performance out of single GPUs.

I've been buying powerful hardware since the 90's, starting with SGI and then ever larger Linux boxes. I make my living with them so it's never been a huge decision to invest some of the proceeds back into the tools. But on an absolute scale, yes, it is a lot of money.