Comment by whizzter
4 hours ago
My bet is that the prices will crash once OpenAI (and/or Antrophic) IPO's have happened.
Right now the biggest threat to their IPO's is that people realize that local models are good enough for whatever they're peddling, what's the most important factor to even running good enough models? RAM since you want the models in memory to not be total slogs.
But remember that markets can stay irrational longer than anyone can hold his breath. If they get more funding there's a good chance they'll invest more in the destruction of the remaining production capacity. Admitting that with normal pricing anyone could have a decent AI-machine for 2K is hard - prices for acceptable AI-machines most likely will go >10K first.
You are saying there will be even more demand for RAM and that will cause the prices to crash?
The chance is that they cache out during IPO, and will lose interest to increase capacity, some/many contracts will be canceled, and demand being reduced.
Perhaps all hobbyst developers interested in local AI combined is a smaller demand than AI companies hoarding parts. That would make demand decrease.
If local models are good enough, doesn't that increase demand for DRAM as everyone buys DRAM for their poorly utilized local machines?
Surely it is a more efficient use of DRAM to run inference on shared hardware with large batch sizes and more utilization.
Luckily very few people can configure and are interested in local models. But your nearby datacenter running Chinese open-weight models is also good enough.
My point is that dram demand is mostly orthogonal to whether everyone is using open weight models or secret weight models. Heavy demand for local models (whether secret or open weight) will require even more aggregate DRAM than for shared.
Demand will only go down if people reduce their use of these AI tools. Given how much folks here complain about quotas, I'm very skeptical that will happen willingly.
2 replies →
My bet is that we're not gonna see any adjustments in RAM pricing until one of the planned data center projects collapses in a spectacular way.
One theory: they will need to throw away all these Nvidia cards in the trash at some point right ?
Because what to do with power-consuming outdated hardware ? let's say 5 years from now ?
They will need new RAM.
I wonder.
I’d gladly take a few of these self-contained rack-clusters off their hands when they do.
I’d even get a house with a garage or something just for that.
The billionaires locked in a race to spend effectively unlimited funds on AI CapEx will have to be convinced by markets and/or their advisers that there aren't enough profits and that cutting losses (like with Metaverse) in their quixotic quest is necessary.
And honestly, we will have much bigger problems if that bubble pops in a spectacular fashion.
Which problems would that be? Nasdaq crashing by a few percent and a major player to go under. Seems almost inevitable at some point.
2 replies →
> that local models are good enough for whatever they're peddling
they are not. Unless you are satisfied with plausible, but mostly garbage output.
They are actually quite a bit better than you might think. Qwen3.6 27B is pretty capable at coding.
For non-coding work, they are more than good enough. A lot of the ways my non-technical family members have interacted with AI would be perfectly served by using a local model.
After all, people were more than satisfied with the results from GPT 3. That has long since been surpassed by open weight models.
I'm sure there are things local models are good enough at in non-coding work, but for anything complex I do not find this to be the case.
I'd say local models are fairly capable of even somewhat complex coding execution. For complex non-coding work (research, in-depth analysis, assembly of complex info-dense documents) I'd rather do it by hand than switch from Opus 4.7 to anything I could even theoretically run locally.
I don't know what kind of coding, but for my case it's been useless. Not working code almost every time. It's much quicker to just write it by hand than use that model.
1 reply →
I get results comparable to the saas. Maybe Anthropic sold you too much crack tokens.
isn't that literally all output an LLM generates?
Honestly, that's the output I get from non-local models, anyway. If I'm going to get plausible nonsense either way, I may as well run it on my own hardware.
[dead]