Comment by llm_nerd
5 days ago
Your post makes sense if you bought the hardware for other reasons, and maybe run models occasionally as a novelty.
That isn't the case for many, though, and there is a whole social media space where people are hyping up the latest homebrew options for running models, believing it frees them from the yoke of big AI.
Millions of people are buying big $ maxed-out hardware like the Mac Studios or DGX specifically to run LLMs. Someone rationally running the numbers is a good thing.
Let's not get ahead of ourselves. Millions, really? I can believe there are a lot of enthusiasts doing this, but "millions" needs a citation.
This is HN; it has probably never occurred to half the people here that the average person even in first world countries doesn't even have the financial capacity to make an impulse five-figure USD purchase, even if on credit.
No one said anything about this being an "impulse" purchase -- it would usually be perceived as a "career investment" purchase, given that many feel they need to race to be with "it" or be left behind -- nor does "five figure" have anything to do with this -- DGX options are available at $3000 -- there's a certain irony that you are posting a comment that is basically "people in my country/circle can't, therefore no one can", while dismissing my comment that many people can.
Millions of people are paying thousands of dollars a year to buy a slightly upgraded entertainment package in their car. There are 60 million or so millionaires alone, including 6 million+ in China.
There are a lot of people with a lot of wealth on the planet. A lot. Millions...it isn't that unfounded, friend.
So doing this "this is HN" snide jerk act, and then basically projecting your lot on the planet is...I don't know if you intended it, but it's rather amazing.
> Millions of people are buying big $ maxed-out hardware like the Mac Studios or DGX specifically to run LLMs.
What's your source for this?
Not just one, but two replies completely hung up on that. Like, why even reply when you already saw the other guy doing the same tired pedantry? Just wanted to feel like you contributed?
Ignoring that it was just tosser hyperbole (that absolutely zero reasonable people need to question), yes, enormous numbers of people are buying GPUs or hardware with the explicit goal of running local LLMs, and social media is full of people hyping various setups and models. Mac Minis are almost impossible to find, and that alone is selling at a clip of about 300,000 every four months. Large memory GPUs are basically a myth at this point. All so people can pay more to get a worse result than commercial options, which is precisely the point of the submission.
These local setups only ever make sense if you have something that confidential, or you're doing something that ToS of the majors would ban you for.
Now given this pedantry horseshit, you'll probably demand that I specifically show a citation on DGX or Studio sales, which...rofl.