Comment by speff
2 years ago
A few months ago there was a post on HN about catching up to the current state of LLM dev and learning how to use it. In it there were recommendations for hardware - the lowest tier being a 3090/4090. When looking at the decision tree for even cheaper options, it basically said to find another hobby.
Not sure if that's changed but ever since seeing that line, I've been put off of that world. I still occasionally click on HN links advertising new methods which can be run on "consumer" cards and every time it's just a 3090/4090...
I don't have that much money.
For training or inference?
The p40 was on the inference side of the tree and you can get one on eBay for $200 or less.
If we're still entertaining the above analogy, a home kitchen costs more than a PC with a 3090/4090.
But when I got into software in the 90s, it was about $4000 for just a run of the mill desktop PC, which is about $8000 in today's money. And it didn't even have 3d acceleration.
In the grand scheme of things, a 3090/4090 is not expensive.
A gas camping stove is like $50. A toaster oven is another $50-100. The best reviewed chefs knife on the market for commercial kitchens is $30.
You don’t need thousands of dollars to start cooking.
I have friends in the who cook on this stuff at home all the time. We in the west have elevated our kitchens into something luxurious. Nothing wrong with that, but it’s overkill for many people.
You can get started with cooking on a gas camping stove like you can get started with AI projects on a Coral Edge TPU accelerator.