there is absolutely no sense in which the SVD/PCA decomposition is just a rotation matrix. you should probably review your linear algebra textbook (hint: scaling is extremely important).
PCA is an orthogonal transformation of the covariance matrix, so like all orthogonal transformations, it’s _literally a rotation_ in N-dimensional space.
SVD is more complex but ultimately it’s just another useful decomposition of a matrix.
I’m not sure why you’re both negative and dismissive. Transformation matrices in graphics are a good and approachable way to get used to linear transformations, which turn out to be useful pretty much everywhere.
Whether or not that helps you with ML depends more on what you’re doing in ML. FAANG doesn’t have a monopoly on ML or on interesting work in ML.
> PCA is an orthogonal transformation of the covariance matrix
Yes you're now the second person the literally repeat the same thing I've already stated extremely clearly and succinctly: PCA is not just rotation (hint: you also need to understand covariance).
> I’m not sure why you’re both negative and dismissive. Transformation matrices in graphics are a good and approachable way to get used to linear transformations, which turn out to be useful pretty much everywhere.
I've already literally drawn the analogy/metaphor that I've drawn: if you think 2d/3d rotation matrices as they are used in graphics is any kind of introduction to the matrices in ML (modeling linear transformations or otherwise) then you're probably the type of person that believes that cash registers any kind of introduction to finance.
My point is not that hard to understand. Graphics in no way, way, shape, or form prepares you for ML. I don't understand why this is so controversial.
i don't understand who is having trouble reading the dialogue here you or i;
> there is absolutely no sense in which the SVD/PCA decomposition is just a rotation matrix... (hint: scaling is extremely important)
...
> SVD is the decomposition of a matrix into two rotation matrices and a scaling matrix, by definition:
yes that's exactly what i was implying when i said SVD more than just rotation, scaling is also important.
my point, which is my same original point, is that if you think learning about rotation/euler matrices is going to prepare you in any way, shape, or form for ML (vis-a-vis SVD/PCA or RoPE or anything else) you're in for a very rude awakening.
there is absolutely no sense in which the SVD/PCA decomposition is just a rotation matrix. you should probably review your linear algebra textbook (hint: scaling is extremely important).
Cognoboffin is exactly right. SVD decomposed matrix into a sequence of rotation, scaling and unrotation matrices.
If anyone needs a review it's not cognoboffin.
You led with the claim you have never seen a rotation matrix in ML. I am having doubts about whether you have the ability to recognise one.
I suspect new hires get a free pass as long as they can talk a storm about backpropagation these days.
PCA is an orthogonal transformation of the covariance matrix, so like all orthogonal transformations, it’s _literally a rotation_ in N-dimensional space.
SVD is more complex but ultimately it’s just another useful decomposition of a matrix.
I’m not sure why you’re both negative and dismissive. Transformation matrices in graphics are a good and approachable way to get used to linear transformations, which turn out to be useful pretty much everywhere.
Whether or not that helps you with ML depends more on what you’re doing in ML. FAANG doesn’t have a monopoly on ML or on interesting work in ML.
> PCA is an orthogonal transformation of the covariance matrix
Yes you're now the second person the literally repeat the same thing I've already stated extremely clearly and succinctly: PCA is not just rotation (hint: you also need to understand covariance).
> I’m not sure why you’re both negative and dismissive. Transformation matrices in graphics are a good and approachable way to get used to linear transformations, which turn out to be useful pretty much everywhere.
I've already literally drawn the analogy/metaphor that I've drawn: if you think 2d/3d rotation matrices as they are used in graphics is any kind of introduction to the matrices in ML (modeling linear transformations or otherwise) then you're probably the type of person that believes that cash registers any kind of introduction to finance.
My point is not that hard to understand. Graphics in no way, way, shape, or form prepares you for ML. I don't understand why this is so controversial.
3 replies →
SVD is the decomposition of a matrix into two rotation matrices and a scaling matrix, by definition:
https://en.wikipedia.org/wiki/Singular_value_decomposition
i don't understand who is having trouble reading the dialogue here you or i;
> there is absolutely no sense in which the SVD/PCA decomposition is just a rotation matrix... (hint: scaling is extremely important)
...
> SVD is the decomposition of a matrix into two rotation matrices and a scaling matrix, by definition:
yes that's exactly what i was implying when i said SVD more than just rotation, scaling is also important.
my point, which is my same original point, is that if you think learning about rotation/euler matrices is going to prepare you in any way, shape, or form for ML (vis-a-vis SVD/PCA or RoPE or anything else) you're in for a very rude awakening.
1 reply →