Comment by djmips
2 hours ago
A question that comes up in my mind, since I don't fully understand how this works, is how does this affect runtime performance. It feels like abliterated weight models would work faster than some extra runtime operations?
The vector that needs to be checked is length n while computing that vector requires n^2 operations. I haven't benchmarked it but I expect the performance overhead to be a rounding error.