Comment by atomicnumber3
18 hours ago
I know I'm just one person, but I absolutely detest all of the rhetoric coming from these companies and what it means for software freedom - the FSF kind. I ABHOR the idea that free software developers have to either use the hosted (controlled) service, or go way slower than proprietary software devs.
So I'm doing what I can - I only use local models for 100% of my personal coding. (And their capability is amazing! Qwen 3.8 Flash Next leaves me wanting for nothing at this point.)
And I know open weights aren't enough. I'm ready to support open training if/when we get around to that. I hope once hardware prices ease, we can see homelabbers building small AI hosts (small by homelab standards yknow. So let's say 512GB host ram and 1-2 32GB cards, though i hope we also start to see AI cards with lots of vram but less compute cores). Then maybe we can do folding@home style distributed training.
In a time when software freedom is perhaps more threatened than ever, I'm happy to see that information still WANTS to be free, perhaps even more than ever. Look at how easy it is to distill claude! Despite, presumably, anthropic working very hard to prevent it.
It's interesting, because licensing is based on copyright, and copyright doesn't apply to AI code.
In theory, things will become even more free, because you can copy whatever code you like, it's all in the public domain because it was all written by an AI (or enough of it was that a court will decide that copying the whole code base is fair).
And the ability of AI's to reverse-engineer binaries means that even hiding the code won't be enough.
Companies may write T&Cs that try to cover some of this, but they'll still have lost the biggest protection they have.
So, in theory, we move to what the FOSS movement have always wanted: all the code available, all the time, with anyone able to change whatever they like about the software they run.
Somehow I don't think anyone is going to be happy about this, though.
Homelabbers pooling would be a very nice thing to do and I would definitely consider joining in.
As a homelabber, that 5090 I bought a couple years ago for 2k just sold for 9k and I paid off my car.
Heh, good for you. It's insane. A friend of mine just got ripped off here (I posted about it the other day but it never got traction), he is a student and bought what he thought was a GPU but it was actually an empty shell with the GPU and the RAM removed from the board.
https://jacquesmattheij.com/beware-gpu-scammers/
Crazy times!
Congrats on getting rid of the debt, that's really a great feeling.
2 replies →
What are you using to host Qwen Flash Next right now?
Unsloth studio, on a server that I built in 2024 that was, at the time, ludicrously oversized, but is now just ok-sized to run the q4 quant of qwen flash next. (128GB ram, 16GB vram).
I really want to build something less scuffed, but, yknow. So I'm making do atm.
There were a bunch of high performance computing efforts from the 1990s and on where, if you could work out that you'd need 5 years of "today's" compute, you should obviously hang on to the money for 4 years, then spend it on the correspondingly improved hardware and finish the job in a under a year. I've been considering building a box to run OLMO (one of the only models with training data "that you wouldn't have to defend in court" as far as I can tell?) and even with Sam Altman's supply chain attack, stalling for bigger hardware still seems like the right path...
1 reply →