Comment by jliptzin
10 days ago
Yea not sure who would put money into an OpenAI or Anthropic IPO at this point. After this episode I promptly bought the hardware I needed to run local models (~$25k) and am extremely impressed. The price tag alone is worth the peace of mind knowing I will always have a locally running LLM that nobody can take away. I don't need Mythos-level intelligence to do 99% of my day to day work; Opus 4.8 was more than enough and I am pretty close to that with the open source models. So what happens when this hardware inevitably gets cheaper and cheaper and people realize they can run the models they actually need locally and without handing all their data over to these companies?
Only if you're doing cutting edge research or some highly, highly niche project would you need the frontier models.
Can I ask what did you buy? I don't have that money now but I wouldn't knpw what to buy anyway.
I can run most models 100B and under on my MacBook Pro with Ultra 3 and 128GB of RAM at 25-70tok/sec-ish, and it was around $5k.
I don’t think I can run GLM 5.2 since it requires around 256GB of memory and the inference is probably too slow, but the future (planned) Apple M7 may be able to. The leaks say it will support up to ~700GB of RAM.
The models under 100B are kind of dumb as a brick and aren’t that useful unless you’re really bad at coding imo. They can’t really be trusted not to hallucinate so they’re not even good for data processing.
I'll probably buy something like this in the future, but given the rapid advances and that I bought a laptop recently, I don't think is urgent.
Blackwell 96gb + 128gb system ram. Not powerful enough for GLM 5.2 but I may upgrade.
Thanks! What a beast.