How to transform an expression into a form involving the trace of a product of two matrices












3












$begingroup$


In page 594 of Bishop's PRML, the following equation is implied:



$$
-frac{1}{2}sum(mathbf{x}_n-mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n-mathbf{bar{x}}) = -frac{N}{2}mathrm{Tr}(mathbf{C}^{-1}mathbf{S})
$$



where



$$
mathbf{S} = frac{1}{N}sum(mathbf{x}_n-mathbf{bar{x}})(mathbf{x}_n-mathbf{bar{x}})^T
$$
,$mathbf{C}$ is a symmetric matrix and $mathbf{bar{x}} = frac{sum_{n=1}^Nmathbf{x}_n}{N}$.



I want to derive this equation myself. But I'm not sure how to do it. Could someone show why the equation holds?






linear-algebra







share|cite|improve this question











$endgroup$












  • $begingroup$
    I have taken the liberty to modify your title which was "uninformative".
    $endgroup$
    – Jean Marie
    Dec 31 '18 at 19:48
















3












$begingroup$


In page 594 of Bishop's PRML, the following equation is implied:



$$
-frac{1}{2}sum(mathbf{x}_n-mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n-mathbf{bar{x}}) = -frac{N}{2}mathrm{Tr}(mathbf{C}^{-1}mathbf{S})
$$



where



$$
mathbf{S} = frac{1}{N}sum(mathbf{x}_n-mathbf{bar{x}})(mathbf{x}_n-mathbf{bar{x}})^T
$$
,$mathbf{C}$ is a symmetric matrix and $mathbf{bar{x}} = frac{sum_{n=1}^Nmathbf{x}_n}{N}$.



I want to derive this equation myself. But I'm not sure how to do it. Could someone show why the equation holds?






linear-algebra







share|cite|improve this question











$endgroup$












  • $begingroup$
    I have taken the liberty to modify your title which was "uninformative".
    $endgroup$
    – Jean Marie
    Dec 31 '18 at 19:48














3












3








3





$begingroup$


In page 594 of Bishop's PRML, the following equation is implied:



$$
-frac{1}{2}sum(mathbf{x}_n-mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n-mathbf{bar{x}}) = -frac{N}{2}mathrm{Tr}(mathbf{C}^{-1}mathbf{S})
$$



where



$$
mathbf{S} = frac{1}{N}sum(mathbf{x}_n-mathbf{bar{x}})(mathbf{x}_n-mathbf{bar{x}})^T
$$
,$mathbf{C}$ is a symmetric matrix and $mathbf{bar{x}} = frac{sum_{n=1}^Nmathbf{x}_n}{N}$.



I want to derive this equation myself. But I'm not sure how to do it. Could someone show why the equation holds?






linear-algebra







share|cite|improve this question











$endgroup$




In page 594 of Bishop's PRML, the following equation is implied:



$$
-frac{1}{2}sum(mathbf{x}_n-mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n-mathbf{bar{x}}) = -frac{N}{2}mathrm{Tr}(mathbf{C}^{-1}mathbf{S})
$$



where



$$
mathbf{S} = frac{1}{N}sum(mathbf{x}_n-mathbf{bar{x}})(mathbf{x}_n-mathbf{bar{x}})^T
$$
,$mathbf{C}$ is a symmetric matrix and $mathbf{bar{x}} = frac{sum_{n=1}^Nmathbf{x}_n}{N}$.



I want to derive this equation myself. But I'm not sure how to do it. Could someone show why the equation holds?






linear-algebra




linear-algebra






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited Dec 31 '18 at 19:46









Jean Marie

31.6k42355




31.6k42355










asked Dec 31 '18 at 12:16









SandiSandi

262112




262112












  • $begingroup$
    I have taken the liberty to modify your title which was "uninformative".
    $endgroup$
    – Jean Marie
    Dec 31 '18 at 19:48


















  • $begingroup$
    I have taken the liberty to modify your title which was "uninformative".
    $endgroup$
    – Jean Marie
    Dec 31 '18 at 19:48
















$begingroup$
I have taken the liberty to modify your title which was "uninformative".
$endgroup$
– Jean Marie
Dec 31 '18 at 19:48




$begingroup$
I have taken the liberty to modify your title which was "uninformative".
$endgroup$
– Jean Marie
Dec 31 '18 at 19:48










2 Answers
2






active

oldest

votes


















4












$begingroup$

Guide:



Notice that $(x_n - bar{x})^TC^{-1}(x_n - bar{x})$ is a scalar,



hence $$(x_n - bar{x})^TC^{-1}(x_n - bar{x})= operatorname{Tr}left[(x_n - bar{x})^TC^{-1}(x_n - bar{x})right]=operatorname{Tr}left[C^{-1}(x_n - bar{x})(x_n - bar{x})^Tright]$$



since $operatorname{Tr}(AB)=operatorname{Tr}(BA)$.
Hopefully you can take it from here.






share|cite|improve this answer











$endgroup$





















    4












    $begingroup$

    With the help of Siong Thye Goh, I did the following:



    begin{align}
    -frac{1}{2}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}}) &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})]\
    &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
    &= -frac{1}{2}mathrm{Tr}[sum^N_{n=1}mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
    &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
    &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}Nmathbf{S}] = -frac{N}{2}mathrm{Tr}[mathbf{C}^{-1}mathbf{S}]
    end{align}






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
      $endgroup$
      – Siong Thye Goh
      Dec 31 '18 at 13:00












    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3057651%2fhow-to-transform-an-expression-into-a-form-involving-the-trace-of-a-product-of-t%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    4












    $begingroup$

    Guide:



    Notice that $(x_n - bar{x})^TC^{-1}(x_n - bar{x})$ is a scalar,



    hence $$(x_n - bar{x})^TC^{-1}(x_n - bar{x})= operatorname{Tr}left[(x_n - bar{x})^TC^{-1}(x_n - bar{x})right]=operatorname{Tr}left[C^{-1}(x_n - bar{x})(x_n - bar{x})^Tright]$$



    since $operatorname{Tr}(AB)=operatorname{Tr}(BA)$.
    Hopefully you can take it from here.






    share|cite|improve this answer











    $endgroup$


















      4












      $begingroup$

      Guide:



      Notice that $(x_n - bar{x})^TC^{-1}(x_n - bar{x})$ is a scalar,



      hence $$(x_n - bar{x})^TC^{-1}(x_n - bar{x})= operatorname{Tr}left[(x_n - bar{x})^TC^{-1}(x_n - bar{x})right]=operatorname{Tr}left[C^{-1}(x_n - bar{x})(x_n - bar{x})^Tright]$$



      since $operatorname{Tr}(AB)=operatorname{Tr}(BA)$.
      Hopefully you can take it from here.






      share|cite|improve this answer











      $endgroup$
















        4












        4








        4





        $begingroup$

        Guide:



        Notice that $(x_n - bar{x})^TC^{-1}(x_n - bar{x})$ is a scalar,



        hence $$(x_n - bar{x})^TC^{-1}(x_n - bar{x})= operatorname{Tr}left[(x_n - bar{x})^TC^{-1}(x_n - bar{x})right]=operatorname{Tr}left[C^{-1}(x_n - bar{x})(x_n - bar{x})^Tright]$$



        since $operatorname{Tr}(AB)=operatorname{Tr}(BA)$.
        Hopefully you can take it from here.






        share|cite|improve this answer











        $endgroup$



        Guide:



        Notice that $(x_n - bar{x})^TC^{-1}(x_n - bar{x})$ is a scalar,



        hence $$(x_n - bar{x})^TC^{-1}(x_n - bar{x})= operatorname{Tr}left[(x_n - bar{x})^TC^{-1}(x_n - bar{x})right]=operatorname{Tr}left[C^{-1}(x_n - bar{x})(x_n - bar{x})^Tright]$$



        since $operatorname{Tr}(AB)=operatorname{Tr}(BA)$.
        Hopefully you can take it from here.







        share|cite|improve this answer














        share|cite|improve this answer



        share|cite|improve this answer








        edited Dec 31 '18 at 12:28









        Bernard

        125k743119




        125k743119










        answered Dec 31 '18 at 12:19









        Siong Thye GohSiong Thye Goh

        105k1469121




        105k1469121























            4












            $begingroup$

            With the help of Siong Thye Goh, I did the following:



            begin{align}
            -frac{1}{2}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}}) &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})]\
            &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[sum^N_{n=1}mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}Nmathbf{S}] = -frac{N}{2}mathrm{Tr}[mathbf{C}^{-1}mathbf{S}]
            end{align}






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
              $endgroup$
              – Siong Thye Goh
              Dec 31 '18 at 13:00
















            4












            $begingroup$

            With the help of Siong Thye Goh, I did the following:



            begin{align}
            -frac{1}{2}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}}) &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})]\
            &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[sum^N_{n=1}mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}Nmathbf{S}] = -frac{N}{2}mathrm{Tr}[mathbf{C}^{-1}mathbf{S}]
            end{align}






            share|cite|improve this answer











            $endgroup$













            • $begingroup$
              +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
              $endgroup$
              – Siong Thye Goh
              Dec 31 '18 at 13:00














            4












            4








            4





            $begingroup$

            With the help of Siong Thye Goh, I did the following:



            begin{align}
            -frac{1}{2}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}}) &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})]\
            &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[sum^N_{n=1}mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}Nmathbf{S}] = -frac{N}{2}mathrm{Tr}[mathbf{C}^{-1}mathbf{S}]
            end{align}






            share|cite|improve this answer











            $endgroup$



            With the help of Siong Thye Goh, I did the following:



            begin{align}
            -frac{1}{2}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}}) &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[(mathbf{x}_n - mathbf{bar{x}})^Tmathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})]\
            &= -frac{1}{2}sum_{n=1}^Nmathrm{Tr}[mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[sum^N_{n=1}mathbf{C}^{-1}(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}sum_{n=1}^N(mathbf{x}_n - mathbf{bar{x}})(mathbf{x}_n - mathbf{bar{x}})^T]\
            &= -frac{1}{2}mathrm{Tr}[mathbf{C}^{-1}Nmathbf{S}] = -frac{N}{2}mathrm{Tr}[mathbf{C}^{-1}mathbf{S}]
            end{align}







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited Jan 1 at 14:34

























            answered Dec 31 '18 at 12:43









            SandiSandi

            262112




            262112












            • $begingroup$
              +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
              $endgroup$
              – Siong Thye Goh
              Dec 31 '18 at 13:00


















            • $begingroup$
              +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
              $endgroup$
              – Siong Thye Goh
              Dec 31 '18 at 13:00
















            $begingroup$
            +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
            $endgroup$
            – Siong Thye Goh
            Dec 31 '18 at 13:00




            $begingroup$
            +1 great job! just a minor careless step at the very beginning, there's a negative there. ;)
            $endgroup$
            – Siong Thye Goh
            Dec 31 '18 at 13:00


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3057651%2fhow-to-transform-an-expression-into-a-form-involving-the-trace-of-a-product-of-t%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            Bundesstraße 106

            Image
            Dieser Artikel beschreibt die Bundesstraße 106 in Deutschland. Zur gleichnamigen Straße in Österreich siehe Mölltal Straße. Vorlage:Infobox hochrangige Straße/Wartung/DE-B Bundesstraße 106 in Deutschland Karte Basisdaten Betreiber: Deutschland  Bundesrepublik Deutschland Straßenbeginn: Wismar ( 53° 54′  N , 11° 24′  O 53.900536 11.396875 ) Straßenende: Schwerin ( 53° 36′  N , 11° 24′  O 53.592785 11.404892 ) Gesamtlänge: 40,7 km Bundesland: Mecklenburg-Vorpommern Ausbauzustand: zweistreifig [1] Bundesstraße 106 in Schwerin (Umgehungsstraße) Straßenverlauf Land Mecklenburg-Vorpommern Landkreis Nordwestmecklenburg Ortsumgehung Wismar Wallensteingraben über   Dorf Mecklenburg Wallensteingraben Groß Stieten Bad Kleinen OT Niendorf Bahnstrecke Lübeck–Bad Kleinen Zickhusen Lübstorf Klein Trebbow OT Kirch Stück ...
            Read more

            Verónica Boquete

            Image
            Verónica Boquete im Trikot der Nationalmannschaft (2013) Personalia Name Verónica Boquete Giadans Geburtstag 9. April 1987 Geburtsort Santiago de Compostela, Spanien Größe 159 cm Position Mittelfeld / Sturm Juniorinnen Jahre Station 2001–2005 Xuventú Aguiño Frauen Jahre Station Spiele (Tore) 1 2005–2008 Prainsa Zaragoza 83 (40) 2008–2010 Espanyol Barcelona 53 (30) 2010 Buffalo Flash 9 0 (9) 2010 Chicago Red Stars 3 0 (1) 2010–2011 Espanyol Barcelona 26 (41) 2011 Philadelphia Independence 13 0 (5) 2011 Energija Woronesch 5 0 (1) 2012–2014 Tyresö FF 39 (14) 2014 Portland Thorns FC 15 0 (4) 2014–2015 1. FFC Frankfurt 23 0 (7) 2015–2016 FC Bayern München 9 0 (0) 2016–2018 Paris Saint-Germain 31 0 (8) 2018 Peking BG Phoenix 2019– Utah Royals FC 0 0 (0) Nationalmannschaft Jahre Auswahl ...
            Read more

            Ida-Boy-Ed-Garten

            Image
            Die Lage des Ida-Boy-Ed-Gartens, rot markiert Der Ida-Boy-Ed-Garten Der Ida-Boy-Ed-Garten, Blick von der Wakenitzmauer durch das Tor in der Stadtmauer Der Ida-Boy-Ed-Garten ist eine Straße der Lübecker Altstadt. Inhaltsverzeichnis 1 Lage 2 Geschichte 3 Literatur 4 Weblinks Lage | Der etwa 90 Meter lange Ida-Boy-Ed-Garten befindet sich an der nördlichen Spitze der Altstadtinsel, vor dem Burgtor. Die in einem Bogen durch eine Grünanlage hangaufwärts führende Straße verbindet die Wakenitzmauer mit der Burgtorbrücke. Geschichte | Der heutige Ida-Boy-Ed-Garten wurde ursprünglich als Verlängerung der Wakenitzmauer angelegt. 1903 wurde in den erhaltenen Teil der mittelalterlichen Stadtmauer, an dem die Wakenitzmauer bei der Einmündung der Kaiserstraße begann, ein Tor mit einem großen Bogen für die Durchführung der Straße und zwei kleineren seitlichen Öffnungen für Fußgänger gebrochen, um für die neu errichteten Wohnhäuser in diesem Bereich ...
            Read more