Comment by SecretDreams
3 days ago
Improvements in LLM efficiency should be driving hardware to get cheaper.
I agree with everything you've said, I'm just not seeing any material benefit to the statement as of now.
3 days ago
Improvements in LLM efficiency should be driving hardware to get cheaper.
I agree with everything you've said, I'm just not seeing any material benefit to the statement as of now.
Inference costs falling 2x doesn’t decrease hardware prices when demand for tokens has increased 10x.
It's the ratio. If revenue goes up 10x you can afford 10x more hardware if you can afford to do it all.