Comment by willy_k
10 hours ago
Because deleting model weights after training is likely to cause knock-on effects in model knowledge and/or behavior. Targetting it might mitigate this but it’s
a) not guaranteed that only censor-ey parameters get removed, and b) likely that removing those parameters still has effects on the effectiveness of related parameters.
Considering these are essentially document completion engines[0], can't you just start the task with the version that doesn't refuse and then continue with the version that would refuse but now has to keep going after it accepted the task? :-P
[0] in the sense that the "discussion" is basically a turn based game between you and the LLM filling a chat transcript document
The weights aren't deleted, it's just additional fine tuning, is my understanding.
There is no question model quality is degraded by this though.
It's altered, sure. I think inherent degradation is a step too far though.