Comment by lrvick
2 days ago
I wonder how long before people figure out that you can run alliterated models locally on consumer hardware capable of doing all this without help from Sam or Dario.
2 days ago
I wonder how long before people figure out that you can run alliterated models locally on consumer hardware capable of doing all this without help from Sam or Dario.
I think the performance of these models on consumer hardware have a long way to go before abliterated* models have the potential to go mainstream.
Damn spell check. I typed it correctly the first time.
Anyway, I have 8 r9700 GPUs racked up that can run a dozen or so sessions of qwen 3.8 flash next at once at > 100tps each.
I build operating systems and fix compilers with them and reverse engineer anything I want. Never given money to Sam or Dario or anyone else for tokens. If consumer gpus and models never got any better, I am all set.
So in this case it may not be that people haven't "figured out" that this is possible but that they don't have $20k lying around to experiment with.
2 replies →
I very much want this to be true, I would shift as much of my workload as possible to local models. But is qwen 3.8 really good enough to do reverse engineering work? Is it because you have many sessions running at once, so it can "brute force" its way through problems?
2 replies →
"8 r9700 GPUs" is not a consumer-level system.
3 replies →