Comment by golem14
7 hours ago
Interesting thought, because it's a yield question. How tolerant are models today to a few broken weights.
If tolerant, they could churn out many cheaper chips, some perhaps with slight abnormal tendencies ;)
7 hours ago
Interesting thought, because it's a yield question. How tolerant are models today to a few broken weights.
If tolerant, they could churn out many cheaper chips, some perhaps with slight abnormal tendencies ;)
> How tolerant are models today to a few broken weights.
Extremely! You can remove entire layers and the model will still work just fine, with barely perceptible capability losses.
I've cut/bypassed ~15% of total parameters out of Gemma 4 31B on a pod once. Still got perfectly coherent responses out of it. Certain layers are a lot more important than others, particularly early and late ones; but it's honestly astonishing how much can be cut out from the middle without destroying the model's coherence.
I didn't run any meaningful benchmarks, so I have no idea what the capability loss looks like exactly. But "produce coherent and sensible English in response to a wide variety of prompts" was definitely not among the things the model unlearned.
Brings to mind the scene in '2001' where Bowman is pulling out individual pieces of hardware that represent the mind of HAL, and it becomes increasingly incoherent as more physical hardware is detached.
https://www.youtube.com/watch?v=UwCFY6pmaYY
I wonder if you had a few percent of problems in the yield, if it would be functionally equivalent to the difference between a unsloth-published Q6 standard size GGUF vs. the nearly perfect precision of an unsloth Q8-K-XL. Or more like Q4 vs Q8 where a lot is lost.