← Back to context Comment by wren6991 7 hours ago Yeah, orthogonalisation is a special case of a rank-1 LoRA. 2 comments wren6991 Reply ziofill 3 hours ago I don’t think this is true. wren6991 3 hours ago Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
ziofill 3 hours ago I don’t think this is true. wren6991 3 hours ago Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
wren6991 3 hours ago Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
I don’t think this is true.
Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?