Comment by mapontosevenths
9 hours ago
Thats not what it means at all. It uses SVD[0] to map the subspace in which the refusal happens. Its all pretty standard stuff with some hype on top to make it an interesting read.
Its basically using a compression technique to figure out which logits are the relevant ones and then zeroing them.
[0] https://en.wikipedia.org/wiki/Singular_value_decomposition
You are also not quite correct, IMO. See my comment at https://github.com/elder-plinius/OBLITERATUS?tab=readme-ov-f...) of the network as an "Ablation Study" to attempt to determine the semantics of these components.
However, all these methods are marked as "Novel", I.e., maybe just BS made up by the author. IMO I don't see how they can work based on how they are named, they are way too dumb and clunky. But proper abliteration like you mentioned can definitely work.
You got me there. I missed the wackier antics further down. Mea culpa.
So did I initially until I saw a few more things from others here.