Can off-diagonals be nonzero for covariance matrix after PCA?
$begingroup$
I have some data for which I found the covariance matrix for:
$$Sigma = begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \ −1.00 & 1.58 & −1.92 & −13.92 \ 3.33 & −1.92 & 62.92 & −23.42 \ 33.00 & −13.92 & −23.42 & 398.92 end{bmatrix}$$
After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:
$$Sigma = begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \ 0.42 & 60.25 & -0.42 & 0.00 \ -3.92 & -0.42 & 0.92 & 0.00 \ 0.00 & 0.00 & 0.00 & 0.00 end{bmatrix}$$
From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?
statistics vectors covariance variance
asked Dec 18 '18 at 12:00
gatorgator
66411339
$endgroup$
add a comment |
$begingroup$
I have some data for which I found the covariance matrix for:
$$Sigma = begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \ −1.00 & 1.58 & −1.92 & −13.92 \ 3.33 & −1.92 & 62.92 & −23.42 \ 33.00 & −13.92 & −23.42 & 398.92 end{bmatrix}$$
After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:
$$Sigma = begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \ 0.42 & 60.25 & -0.42 & 0.00 \ -3.92 & -0.42 & 0.92 & 0.00 \ 0.00 & 0.00 & 0.00 & 0.00 end{bmatrix}$$
From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?
statistics vectors covariance variance
asked Dec 18 '18 at 12:00
gatorgator
66411339
$endgroup$
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11
add a comment |
$begingroup$
I have some data for which I found the covariance matrix for:
$$Sigma = begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \ −1.00 & 1.58 & −1.92 & −13.92 \ 3.33 & −1.92 & 62.92 & −23.42 \ 33.00 & −13.92 & −23.42 & 398.92 end{bmatrix}$$
After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:
$$Sigma = begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \ 0.42 & 60.25 & -0.42 & 0.00 \ -3.92 & -0.42 & 0.92 & 0.00 \ 0.00 & 0.00 & 0.00 & 0.00 end{bmatrix}$$
From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?
statistics vectors covariance variance
asked Dec 18 '18 at 12:00
gatorgator
66411339
$endgroup$
I have some data for which I found the covariance matrix for:
$$Sigma = begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \ −1.00 & 1.58 & −1.92 & −13.92 \ 3.33 & −1.92 & 62.92 & −23.42 \ 33.00 & −13.92 & −23.42 & 398.92 end{bmatrix}$$
After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:
$$Sigma = begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \ 0.42 & 60.25 & -0.42 & 0.00 \ -3.92 & -0.42 & 0.92 & 0.00 \ 0.00 & 0.00 & 0.00 & 0.00 end{bmatrix}$$
From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?
statistics vectors covariance variance
statistics vectors covariance variance
asked Dec 18 '18 at 12:00
gatorgator
66411339
asked Dec 18 '18 at 12:00
gatorgator
66411339
asked Dec 18 '18 at 12:00
gatorgator
66411339
asked Dec 18 '18 at 12:00
gatorgator
66411339
asked Dec 18 '18 at 12:00
gatorgator
66411339
66411339
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11
add a comment |
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:
$$Sigma - lambda I = 0$$
answered Dec 18 '18 at 14:10
gatorgator
66411339
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3045081%2fcan-off-diagonals-be-nonzero-for-covariance-matrix-after-pca%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:
$$Sigma - lambda I = 0$$
answered Dec 18 '18 at 14:10
gatorgator
66411339
$endgroup$
add a comment |
$begingroup$
I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:
$$Sigma - lambda I = 0$$
answered Dec 18 '18 at 14:10
gatorgator
66411339
$endgroup$
add a comment |
$begingroup$
I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:
$$Sigma - lambda I = 0$$
answered Dec 18 '18 at 14:10
gatorgator
66411339
$endgroup$
I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:
$$Sigma - lambda I = 0$$
answered Dec 18 '18 at 14:10
gatorgator
66411339
answered Dec 18 '18 at 14:10
gatorgator
66411339
answered Dec 18 '18 at 14:10
gatorgator
66411339
answered Dec 18 '18 at 14:10
gatorgator
66411339
66411339
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3045081%2fcan-off-diagonals-be-nonzero-for-covariance-matrix-after-pca%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Have you centered the data ?
$endgroup$
– nicomezi
Dec 18 '18 at 12:06
$begingroup$
@nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration.
$endgroup$
– gator
Dec 18 '18 at 12:08
$begingroup$
It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902
$endgroup$
– nicomezi
Dec 18 '18 at 12:11