Comment by echelon
11 hours ago
Y'all aren't seeing the same future I am, I guess.
- Our career is reaching the end of the line
- 99.9999% of users will be using the cloud
- if we don't have strong open source models, we're going to be locked into hyperscaler APIs for life
- piddly little home GPUs don't do squat against this
Why are you building for hobby uses?
Build for freedom of the ability to make and scale businesses. To remain competitive. To have options in the future independent of hyperscalers.
We're going to be locked out of the game soon.
Everyone should be panicking about losing the ability to participate.
Play with your RTXes all you like. They might as well be raspberry pis. They're toys.
Our future depends on our ability to run and access large scale, competitive, open weights. Not stuff you run with LM Studio or ComfyUI as a hobby.
I don't agree that we are being left behind with regards to AI, I believe it's simply not worth participating in. I hope it all comes crashing down.
That's not the right perspective to have.
Also, the only thing crashing down will be the economic participation of everyday people if we don't have ownership over the means of creation. Hyperscalers will be just fine.
Eventually, we are going to figure out how to do more inference with less RAM. There is simply no way that current transformer-based LLMs are the right thing to do. LLMs still rely on emergent properties that no one fully understands, where the sheer quantity of weights and duration of training are the dominant factors driving performance.
There is no reason on God's green earth why a coding model should need to ingest all of Shakespeare, five dozen gluten-free cookbooks, the complete works of Stephen King, and 30 GB of bad fanfic from alt.binaries.furry. Yet for reasons nobody understands, all of that crap is somehow needed in order to achieve the best output quality and accuracy in unrelated fields. This state of ignorance can't last. Language models shouldn't need 10% of the RAM they are taking now.
Every other point you raise is very valid, but I really don't think hardware is going to be the problem that everybody assumes it will be.