Mixing
PCA, SVD, t-SNE - which is linear and which is non-linear?
Clash Royale CLAN TAG#URR8PPP
up vote
2
down vote
favorite
I am quite confused because a colleague of mine recently told me that he preferred using SVD instead of PCA (by eigendecomposition) because, contrary to the latter, the former is non-linear so it can identify also some non-linear patterns.
However, I cannot see exactly in what way SVD is non-linear since I have the impression that it simply applies a series of linear matrix multiplications (see also this StackExchange answer).
I know that t-SNE is certainly non-linear and for this reason it is sometimes called as non-linear PCA.
Is SVD non-linear while PCA (by eigendecompostion) is linear?
machine-learning unsupervised-learning
asked 2 hours ago
Poete Maudit
32813
add a comment |Â
up vote
2
down vote
favorite
I am quite confused because a colleague of mine recently told me that he preferred using SVD instead of PCA (by eigendecomposition) because, contrary to the latter, the former is non-linear so it can identify also some non-linear patterns.
However, I cannot see exactly in what way SVD is non-linear since I have the impression that it simply applies a series of linear matrix multiplications (see also this StackExchange answer).
I know that t-SNE is certainly non-linear and for this reason it is sometimes called as non-linear PCA.
Is SVD non-linear while PCA (by eigendecompostion) is linear?
machine-learning unsupervised-learning
asked 2 hours ago
Poete Maudit
32813
add a comment |Â
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I am quite confused because a colleague of mine recently told me that he preferred using SVD instead of PCA (by eigendecomposition) because, contrary to the latter, the former is non-linear so it can identify also some non-linear patterns.
However, I cannot see exactly in what way SVD is non-linear since I have the impression that it simply applies a series of linear matrix multiplications (see also this StackExchange answer).
I know that t-SNE is certainly non-linear and for this reason it is sometimes called as non-linear PCA.
Is SVD non-linear while PCA (by eigendecompostion) is linear?
machine-learning unsupervised-learning
asked 2 hours ago
Poete Maudit
32813
I am quite confused because a colleague of mine recently told me that he preferred using SVD instead of PCA (by eigendecomposition) because, contrary to the latter, the former is non-linear so it can identify also some non-linear patterns.
However, I cannot see exactly in what way SVD is non-linear since I have the impression that it simply applies a series of linear matrix multiplications (see also this StackExchange answer).
I know that t-SNE is certainly non-linear and for this reason it is sometimes called as non-linear PCA.
Is SVD non-linear while PCA (by eigendecompostion) is linear?
machine-learning unsupervised-learning
machine-learning unsupervised-learning
asked 2 hours ago
Poete Maudit
32813
asked 2 hours ago
Poete Maudit
32813
asked 2 hours ago
Poete Maudit
32813
asked 2 hours ago
Poete Maudit
32813
asked 2 hours ago
Poete Maudit
32813
32813
add a comment |Â
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
2
down vote
To the best of my knowledge no.
SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc.
About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! So be careful putting all these next to each other. In other words, they are all dimensionality reduction algorithms however, PCA and SVD can be used for feature extraction but t-SNE can not. All can be used for visualization purposes (in EDA).
I certainly recommend reading this answer. Probably the fact that "the square roots of the eigenvalues of $XX^⊤$ are the singular values of $X$" confused your friend that it's a nonlinear method.
Hope it helps. Good Luck!
answered 1 hour ago
Kasra Manshaei
3,1111035
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
To the best of my knowledge no.
SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc.
About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! So be careful putting all these next to each other. In other words, they are all dimensionality reduction algorithms however, PCA and SVD can be used for feature extraction but t-SNE can not. All can be used for visualization purposes (in EDA).
I certainly recommend reading this answer. Probably the fact that "the square roots of the eigenvalues of $XX^⊤$ are the singular values of $X$" confused your friend that it's a nonlinear method.
Hope it helps. Good Luck!
answered 1 hour ago
Kasra Manshaei
3,1111035
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
add a comment |Â
up vote
2
down vote
To the best of my knowledge no.
SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc.
About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! So be careful putting all these next to each other. In other words, they are all dimensionality reduction algorithms however, PCA and SVD can be used for feature extraction but t-SNE can not. All can be used for visualization purposes (in EDA).
I certainly recommend reading this answer. Probably the fact that "the square roots of the eigenvalues of $XX^⊤$ are the singular values of $X$" confused your friend that it's a nonlinear method.
Hope it helps. Good Luck!
answered 1 hour ago
Kasra Manshaei
3,1111035
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
add a comment |Â
up vote
2
down vote
up vote
2
down vote
To the best of my knowledge no.
SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc.
About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! So be careful putting all these next to each other. In other words, they are all dimensionality reduction algorithms however, PCA and SVD can be used for feature extraction but t-SNE can not. All can be used for visualization purposes (in EDA).
I certainly recommend reading this answer. Probably the fact that "the square roots of the eigenvalues of $XX^⊤$ are the singular values of $X$" confused your friend that it's a nonlinear method.
Hope it helps. Good Luck!
answered 1 hour ago
Kasra Manshaei
3,1111035
To the best of my knowledge no.
SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc.
About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! So be careful putting all these next to each other. In other words, they are all dimensionality reduction algorithms however, PCA and SVD can be used for feature extraction but t-SNE can not. All can be used for visualization purposes (in EDA).
I certainly recommend reading this answer. Probably the fact that "the square roots of the eigenvalues of $XX^⊤$ are the singular values of $X$" confused your friend that it's a nonlinear method.
Hope it helps. Good Luck!
answered 1 hour ago
Kasra Manshaei
3,1111035
answered 1 hour ago
Kasra Manshaei
3,1111035
answered 1 hour ago
Kasra Manshaei
3,1111035
answered 1 hour ago
Kasra Manshaei
3,1111035
3,1111035
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
add a comment |Â
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
Thank you for your nice answer(upvote). I agree with all your points at it.
– Poete Maudit
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
I am glad it helped :)
– Kasra Manshaei
1 hour ago
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f40260%2fpca-svd-t-sne-which-is-linear-and-which-is-non-linear%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
