Why is the conjugate of a function always convex?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP











up vote
2
down vote

favorite
1












The conjugate of a function $f$ is given (for some $y in operatornamedom(f)$) as:



$$f^*(y) = sup_x in operatornamedom(f)left(y^Tx - f(x)right)$$



It is known that $f^*$ is convex even if $f$ is not. I would like to know how prove this.






convex-analysis convex-optimization







share|cite|improve this question

























    up vote
    2
    down vote

    favorite
    1












    The conjugate of a function $f$ is given (for some $y in operatornamedom(f)$) as:



    $$f^*(y) = sup_x in operatornamedom(f)left(y^Tx - f(x)right)$$



    It is known that $f^*$ is convex even if $f$ is not. I would like to know how prove this.






    convex-analysis convex-optimization







    share|cite|improve this question























      up vote
      2
      down vote

      favorite
      1









      up vote
      2
      down vote

      favorite
      1






      1





      The conjugate of a function $f$ is given (for some $y in operatornamedom(f)$) as:



      $$f^*(y) = sup_x in operatornamedom(f)left(y^Tx - f(x)right)$$



      It is known that $f^*$ is convex even if $f$ is not. I would like to know how prove this.






      convex-analysis convex-optimization







      share|cite|improve this question













      The conjugate of a function $f$ is given (for some $y in operatornamedom(f)$) as:



      $$f^*(y) = sup_x in operatornamedom(f)left(y^Tx - f(x)right)$$



      It is known that $f^*$ is convex even if $f$ is not. I would like to know how prove this.






      convex-analysis convex-optimization




      convex-analysis convex-optimization






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked 3 hours ago









      Nurmister

      909




      909




















          2 Answers
          2






          active

          oldest

          votes

















          up vote
          3
          down vote



          accepted










          For any given $x in operatornamedom f$, the function:
          $$y mapsto y^top x - f(x)$$
          is an affine function, which is convex and lower semicontinuous (and, indeed, continuous). The epigraph of these functions is therefore closed and convex.



          Then, $f^*$ is the pointwise supremum of the above functions, and its epigraph is the intersection of the above affine functions' epigraphs. Each are closed and convex, proving the epigraph of $f^*$ is closed and convex, thus $f^*$ is convex and lower semicontinuous.






          share|cite|improve this answer




















          • I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
            – Nurmister
            1 hour ago

















          up vote
          2
          down vote













          As I wrote this question I recalled the following fact, and have attempted to prove this property of conjugate functions using it.




          If $h(y,,x)$ is convex in $y$ for each $x in mathcalA$, then $g(y) = sup_x in mathcalAh(y,,x)$ (the pointwise supremum) is convex.




          Let us try to apply this fact to our problem, finding the "equivalent terms":




          1. $f^*(y)$ is $g(y)$


          2. $mathcalA$ is $operatornamedom(f)$


          3. $y^Tx - f(x)$ is $h(y,,x)$

          Do these "equivalent" terms meet the conditions they need to for us to be able to use this fact? I.e.,




          $forall x in operatornamedom(f),,textis,h(y,,x) := y^Tx - f(x)$ convex in $y$?




          Let us try to prove this using Jensen's inequality.



          For some $x,,a,,b in operatornamedom(f),,theta in [0,,1]$ consider:
          beginequation*
          beginaligned
          & h(theta a + (1 - theta) b,,x)\
          & = (theta a + (1 - theta) b)^Tx - f(x)\
          & = theta a^T x + (1 - theta) b^T x - f(x)\
          & = theta a^T x - theta f(x) + (1 - theta) b^T x - (1 - theta) f(x)\
          & = theta (a^T x - f(x)) + (1 - theta) (b^T x - f(x))\
          & = theta h(a,,x) + (1 - theta) h(b,,x)\
          endaligned
          endequation*



          Since equality always holds, $h(y,,x)$ is convex in $y$.



          Now that I have done this, I realize the following: $h(y,,x)$ was just an affine function in $y$, meaning that the pointwise supremum, i.e. the conjugate of $f$, will also be convex.






          share|cite|improve this answer




















            Your Answer




            StackExchange.ifUsing("editor", function ()
            return StackExchange.using("mathjaxEditing", function ()
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            );
            );
            , "mathjax-editing");

            StackExchange.ready(function()
            var channelOptions =
            tags: "".split(" "),
            id: "69"
            ;
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function()
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled)
            StackExchange.using("snippets", function()
            createEditor();
            );

            else
            createEditor();

            );

            function createEditor()
            StackExchange.prepareEditor(
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: false,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            noCode: true, onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            );



            );













             

            draft saved


            draft discarded


















            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2940385%2fwhy-is-the-conjugate-of-a-function-always-convex%23new-answer', 'question_page');

            );

            Post as a guest

















            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            3
            down vote



            accepted










            For any given $x in operatornamedom f$, the function:
            $$y mapsto y^top x - f(x)$$
            is an affine function, which is convex and lower semicontinuous (and, indeed, continuous). The epigraph of these functions is therefore closed and convex.



            Then, $f^*$ is the pointwise supremum of the above functions, and its epigraph is the intersection of the above affine functions' epigraphs. Each are closed and convex, proving the epigraph of $f^*$ is closed and convex, thus $f^*$ is convex and lower semicontinuous.






            share|cite|improve this answer




















            • I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
              – Nurmister
              1 hour ago














            up vote
            3
            down vote



            accepted










            For any given $x in operatornamedom f$, the function:
            $$y mapsto y^top x - f(x)$$
            is an affine function, which is convex and lower semicontinuous (and, indeed, continuous). The epigraph of these functions is therefore closed and convex.



            Then, $f^*$ is the pointwise supremum of the above functions, and its epigraph is the intersection of the above affine functions' epigraphs. Each are closed and convex, proving the epigraph of $f^*$ is closed and convex, thus $f^*$ is convex and lower semicontinuous.






            share|cite|improve this answer




















            • I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
              – Nurmister
              1 hour ago












            up vote
            3
            down vote



            accepted







            up vote
            3
            down vote



            accepted






            For any given $x in operatornamedom f$, the function:
            $$y mapsto y^top x - f(x)$$
            is an affine function, which is convex and lower semicontinuous (and, indeed, continuous). The epigraph of these functions is therefore closed and convex.



            Then, $f^*$ is the pointwise supremum of the above functions, and its epigraph is the intersection of the above affine functions' epigraphs. Each are closed and convex, proving the epigraph of $f^*$ is closed and convex, thus $f^*$ is convex and lower semicontinuous.






            share|cite|improve this answer












            For any given $x in operatornamedom f$, the function:
            $$y mapsto y^top x - f(x)$$
            is an affine function, which is convex and lower semicontinuous (and, indeed, continuous). The epigraph of these functions is therefore closed and convex.



            Then, $f^*$ is the pointwise supremum of the above functions, and its epigraph is the intersection of the above affine functions' epigraphs. Each are closed and convex, proving the epigraph of $f^*$ is closed and convex, thus $f^*$ is convex and lower semicontinuous.







            share|cite|improve this answer












            share|cite|improve this answer



            share|cite|improve this answer










            answered 2 hours ago









            Theo Bendit

            14.6k12045




            14.6k12045











            • I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
              – Nurmister
              1 hour ago
















            • I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
              – Nurmister
              1 hour ago















            I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
            – Nurmister
            1 hour ago




            I think my answer covers it, but your use of epigraphs to show convexity is a good technique I forgot about.
            – Nurmister
            1 hour ago










            up vote
            2
            down vote













            As I wrote this question I recalled the following fact, and have attempted to prove this property of conjugate functions using it.




            If $h(y,,x)$ is convex in $y$ for each $x in mathcalA$, then $g(y) = sup_x in mathcalAh(y,,x)$ (the pointwise supremum) is convex.




            Let us try to apply this fact to our problem, finding the "equivalent terms":




            1. $f^*(y)$ is $g(y)$


            2. $mathcalA$ is $operatornamedom(f)$


            3. $y^Tx - f(x)$ is $h(y,,x)$

            Do these "equivalent" terms meet the conditions they need to for us to be able to use this fact? I.e.,




            $forall x in operatornamedom(f),,textis,h(y,,x) := y^Tx - f(x)$ convex in $y$?




            Let us try to prove this using Jensen's inequality.



            For some $x,,a,,b in operatornamedom(f),,theta in [0,,1]$ consider:
            beginequation*
            beginaligned
            & h(theta a + (1 - theta) b,,x)\
            & = (theta a + (1 - theta) b)^Tx - f(x)\
            & = theta a^T x + (1 - theta) b^T x - f(x)\
            & = theta a^T x - theta f(x) + (1 - theta) b^T x - (1 - theta) f(x)\
            & = theta (a^T x - f(x)) + (1 - theta) (b^T x - f(x))\
            & = theta h(a,,x) + (1 - theta) h(b,,x)\
            endaligned
            endequation*



            Since equality always holds, $h(y,,x)$ is convex in $y$.



            Now that I have done this, I realize the following: $h(y,,x)$ was just an affine function in $y$, meaning that the pointwise supremum, i.e. the conjugate of $f$, will also be convex.






            share|cite|improve this answer
























              up vote
              2
              down vote













              As I wrote this question I recalled the following fact, and have attempted to prove this property of conjugate functions using it.




              If $h(y,,x)$ is convex in $y$ for each $x in mathcalA$, then $g(y) = sup_x in mathcalAh(y,,x)$ (the pointwise supremum) is convex.




              Let us try to apply this fact to our problem, finding the "equivalent terms":




              1. $f^*(y)$ is $g(y)$


              2. $mathcalA$ is $operatornamedom(f)$


              3. $y^Tx - f(x)$ is $h(y,,x)$

              Do these "equivalent" terms meet the conditions they need to for us to be able to use this fact? I.e.,




              $forall x in operatornamedom(f),,textis,h(y,,x) := y^Tx - f(x)$ convex in $y$?




              Let us try to prove this using Jensen's inequality.



              For some $x,,a,,b in operatornamedom(f),,theta in [0,,1]$ consider:
              beginequation*
              beginaligned
              & h(theta a + (1 - theta) b,,x)\
              & = (theta a + (1 - theta) b)^Tx - f(x)\
              & = theta a^T x + (1 - theta) b^T x - f(x)\
              & = theta a^T x - theta f(x) + (1 - theta) b^T x - (1 - theta) f(x)\
              & = theta (a^T x - f(x)) + (1 - theta) (b^T x - f(x))\
              & = theta h(a,,x) + (1 - theta) h(b,,x)\
              endaligned
              endequation*



              Since equality always holds, $h(y,,x)$ is convex in $y$.



              Now that I have done this, I realize the following: $h(y,,x)$ was just an affine function in $y$, meaning that the pointwise supremum, i.e. the conjugate of $f$, will also be convex.






              share|cite|improve this answer






















                up vote
                2
                down vote










                up vote
                2
                down vote









                As I wrote this question I recalled the following fact, and have attempted to prove this property of conjugate functions using it.




                If $h(y,,x)$ is convex in $y$ for each $x in mathcalA$, then $g(y) = sup_x in mathcalAh(y,,x)$ (the pointwise supremum) is convex.




                Let us try to apply this fact to our problem, finding the "equivalent terms":




                1. $f^*(y)$ is $g(y)$


                2. $mathcalA$ is $operatornamedom(f)$


                3. $y^Tx - f(x)$ is $h(y,,x)$

                Do these "equivalent" terms meet the conditions they need to for us to be able to use this fact? I.e.,




                $forall x in operatornamedom(f),,textis,h(y,,x) := y^Tx - f(x)$ convex in $y$?




                Let us try to prove this using Jensen's inequality.



                For some $x,,a,,b in operatornamedom(f),,theta in [0,,1]$ consider:
                beginequation*
                beginaligned
                & h(theta a + (1 - theta) b,,x)\
                & = (theta a + (1 - theta) b)^Tx - f(x)\
                & = theta a^T x + (1 - theta) b^T x - f(x)\
                & = theta a^T x - theta f(x) + (1 - theta) b^T x - (1 - theta) f(x)\
                & = theta (a^T x - f(x)) + (1 - theta) (b^T x - f(x))\
                & = theta h(a,,x) + (1 - theta) h(b,,x)\
                endaligned
                endequation*



                Since equality always holds, $h(y,,x)$ is convex in $y$.



                Now that I have done this, I realize the following: $h(y,,x)$ was just an affine function in $y$, meaning that the pointwise supremum, i.e. the conjugate of $f$, will also be convex.






                share|cite|improve this answer












                As I wrote this question I recalled the following fact, and have attempted to prove this property of conjugate functions using it.




                If $h(y,,x)$ is convex in $y$ for each $x in mathcalA$, then $g(y) = sup_x in mathcalAh(y,,x)$ (the pointwise supremum) is convex.




                Let us try to apply this fact to our problem, finding the "equivalent terms":




                1. $f^*(y)$ is $g(y)$


                2. $mathcalA$ is $operatornamedom(f)$


                3. $y^Tx - f(x)$ is $h(y,,x)$

                Do these "equivalent" terms meet the conditions they need to for us to be able to use this fact? I.e.,




                $forall x in operatornamedom(f),,textis,h(y,,x) := y^Tx - f(x)$ convex in $y$?




                Let us try to prove this using Jensen's inequality.



                For some $x,,a,,b in operatornamedom(f),,theta in [0,,1]$ consider:
                beginequation*
                beginaligned
                & h(theta a + (1 - theta) b,,x)\
                & = (theta a + (1 - theta) b)^Tx - f(x)\
                & = theta a^T x + (1 - theta) b^T x - f(x)\
                & = theta a^T x - theta f(x) + (1 - theta) b^T x - (1 - theta) f(x)\
                & = theta (a^T x - f(x)) + (1 - theta) (b^T x - f(x))\
                & = theta h(a,,x) + (1 - theta) h(b,,x)\
                endaligned
                endequation*



                Since equality always holds, $h(y,,x)$ is convex in $y$.



                Now that I have done this, I realize the following: $h(y,,x)$ was just an affine function in $y$, meaning that the pointwise supremum, i.e. the conjugate of $f$, will also be convex.







                share|cite|improve this answer












                share|cite|improve this answer



                share|cite|improve this answer










                answered 3 hours ago









                Nurmister

                909




                909



























                     

                    draft saved


                    draft discarded















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function ()
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2940385%2fwhy-is-the-conjugate-of-a-function-always-convex%23new-answer', 'question_page');

                    );

                    Post as a guest
































































                    Comments

                    Post a Comment

                    Popular posts from this blog

                    What does second last employer means? [closed]

                    Image
                    Clash Royale CLAN TAG #URR8PPP .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0; up vote -3 down vote favorite Say, for example I have worked in the following companies, Company A => 2010 - 2011 Company B => 2011 - 2012 Company C => 2012 - Till Date Company D => Have offer Now assume Company D is asking to submit documents from my second last employer. So from the list of companies given, which company would be qualified as second last employer? software-industry employer share | improve this question asked Aug 10 '14 at 5:16 Rajaprabhu Aravindasamy 559 1 6 13 closed as off-topic by Jim G., jmort253 ♦ Aug 11 '14 at 1:58 This question appears to be off-topic. The users who voted to close gave this specific reason: "Questions seeking advice on company-specific regulations, agreements, or policies should be directed to your manager or HR department. Questions that address...
                    Read more

                    List of Gilmore Girls characters

                    Image
                    The season three cast—(Back row): Lane, Michel, Paris, Emily, Richard, Sookie, Miss Patty, Kirk; Front row: Jess, Luke, Lorelai, Rory, Dean This is a list of characters for the comedy-drama television series Gilmore Girls . Contents 1 Main characters 1.1 Lorelai Gilmore 1.2 Rory Gilmore 1.3 Sookie St. James 1.4 Lane Kim 1.5 Michel Gerard 1.6 Luke Danes 1.7 Emily Gilmore 1.8 Richard Gilmore 1.9 Paris Geller 1.10 Dean Forester 1.11 Jess Mariano 1.12 Kirk Gleason 1.13 Jason Stiles 1.14 Logan Huntzberger 2 Major recurring characters 2.1 Christopher Hayden 2.2 Jackson Belleville 2.3 Mrs. Kim 2.4 Tristin DuGray 2.5 Max Medina 2.6 Taylor Doose 2.7 Dave Rygalski 2.8 Marty 3 Recurring characters 3.1 Stars Hollow 3.2 Chilton 3.3 Yale 3.4 Other 4 Notable guest stars 5 Unseen characters 6 Notes 7 References Main characters Actor Character Appearances Season 1 Season 2 Season 3 Season 4 Season 5 Season 6 Season 7 A Year in the Life Lauren Graham Lorelai Gilmore...
                    Read more

                    Confectionery

                    Image
                    Prepared foods made with a lot of sugar or other cabohydrates "Sweetmeat" redirects here. For the racehorse, see Sweetmeat (horse). Not to be confused with Sweetbread. This Kransekake is a traditional Scandinavian baker's confection. Confectionery is the art of making confections , which are food items that are rich in sugar and carbohydrates. Exact definitions are difficult. [1] In general, though, confectionery is divided into two broad and somewhat overlapping categories, bakers' confections and sugar confections. [2] Bakers' confectionery, also called flour confections , includes principally sweet pastries, cakes, and similar baked goods. Sugar confectionery includes candies ( sweets in British English), candied nuts, chocolates, chewing gum, bubble gum, pastillage, and other confections that are made primarily of sugar. In some cases, chocolate confections (confections made of chocolate) are treated as a separate category, as are sugar-free versions of ...
                    Read more