By definition of the conjugate,

f^*(\xi)=\sup_{z\in E}\{\langle \xi,z\rangle-f(z)\}\ge \langle \xi,x\rangle-f(x),

which rearranges to

f(x)+f^*(\xi)\ge \langle \xi,x\rangle.

Assume now that f is proper, closed, and convex. Equality holds if and only if

f^*(\xi)=\langle \xi,x\rangle-f(x).

By the definition of f^*, this is equivalent to

\forall z\in E,\qquad \langle \xi,z\rangle-f(z)\le \langle \xi,x\rangle-f(x),

that is,

\forall z\in E,\qquad f(z)\ge f(x)+\langle \xi,z-x\rangle.

This is exactly the statement that \xi\in \partial f(x).

For every x\in E and every \xi\in E^*, Lemma 4.1 gives

\langle \xi,x\rangle-f^*(\xi)\le f(x).

Taking the supremum over \xi yields

f^{**}(x)\le f(x) \qquad \forall x\in E.

By Lemma 4.2, it remains to prove the reverse inequality

f(x)\le f^{**}(x) \qquad \forall x\in E.

Fix x_0\in E and t_0<f(x_0). Then (x_0,t_0)\notin \operatorname{epi}(f). Because f is proper, closed, and convex, the epigraph \operatorname{epi}(f) is a nonempty closed convex subset of E\times \mathbb{R}. By Theorem 2.6, there exists a closed halfspace containing \operatorname{epi}(f) but not (x_0,t_0).

Since \operatorname{epi}(f) is upward closed in the t-direction, we may take that halfspace in the form

H_{\xi,a}:=\{(x,t)\in E\times \mathbb{R}: t\ge \langle \xi,x\rangle-a\}

for some \xi\in E^* and a\in \mathbb{R}. Thus

\operatorname{epi}(f)\subseteq H_{\xi,a} \qquad\text{and}\qquad t_0<\langle \xi,x_0\rangle-a.

The containment \operatorname{epi}(f)\subseteq H_{\xi,a} is equivalent to saying that the affine function

\ell_{\xi,a}(x):=\langle \xi,x\rangle-a

is a global affine lower bound for f, that is,

\ell_{\xi,a}(x)\le f(x) \qquad \forall x\in E.

For fixed slope \xi, the smallest admissible intercept is exactly f^*(\xi), because

\ell_{\xi,a}(x)\le f(x)\ \forall x \iff a\ge \sup_{x\in E}\{\langle \xi,x\rangle-f(x)\}=f^*(\xi).

Hence every affine lower bound of slope \xi lies below the tight affine function of the same slope,

x\mapsto \langle \xi,x\rangle-f^*(\xi).

In particular,

t_0<\langle \xi,x_0\rangle-a\le \langle \xi,x_0\rangle-f^*(\xi)\le f^{**}(x_0).

Since this holds for every t_0<f(x_0), we conclude that

f(x_0)\le f^{**}(x_0).

Combined with Lemma 4.2, this gives

\forall x\in E,\qquad f^{**}(x)=f(x).

By definition,

f^*(\xi)=\sup_{x\in \mathbb{R}}\left\{x\xi-\frac{|x|^p}{p}\right\}.

The maximizer has the same sign as \xi, so it is enough to consider x\ge 0 and \xi\ge 0. For fixed \xi\ge 0, set \varphi_\xi(x):=x\xi-\frac{x^p}{p} on x\ge 0. Since \varphi_\xi'(x)=\xi-x^{p-1}, the unique critical point is x=\xi^{1/(p-1)}=\xi^{q-1}. Substituting gives

f^*(\xi) = \xi\cdot \xi^{q-1}-\frac{\xi^{p(q-1)}}{p} = \xi^q-\frac{\xi^q}{p} = \frac{\xi^q}{q}.

By symmetry, f^*(\xi)=\frac{|\xi|^q}{q} for all \xi\in \mathbb{R}. Applying Lemma 4.1 to this pair then gives the classical Young inequality stated above.

Because \Phi is convex, its epigraph is a convex subset of X\times U\times \mathbb{R}. The epigraph of p is the projection

\operatorname{epi}(p)=\{(u,t)\in U\times \mathbb{R}:\exists x\in X\text{ with }\Phi(x,u)\le t\}.

Hence \operatorname{epi}(p) is convex, so p is convex.

Fix y\in U^*. By definition,

\begin{aligned} p^*(y) &=\sup_{u\in U}\{\langle y,u\rangle-p(u)\} \\ &=\sup_{u\in U}\sup_{x\in X}\{\langle y,u\rangle-\Phi(x,u)\} \\ &=\sup_{x\in X,\ u\in U}\{\langle 0,x\rangle+\langle y,u\rangle-\Phi(x,u)\} \\ &=\Phi^*(0,y). \end{aligned}

This proves the identity in part (1). For every u\in U, the definition of p^*(y) gives

p^*(y)=\sup_{v\in U}\{\langle y,v\rangle-p(v)\}\ge \langle y,u\rangle-p(u).

Using p^*(y)=\Phi^*(0,y), we obtain

p(u)+\Phi^*(0,y)\ge \langle y,u\rangle,

which is exactly

p(u)\ge -\Phi^*(0,y)+\langle y,u\rangle.

Assume now that p(0)=-\infty. For every M>0, choose x_M\in X such that

\Phi(x_M,0)\le -M.

Then, for every y\in U^*,

\Phi^*(0,y)\ge \langle 0,x_M\rangle+\langle y,0\rangle-\Phi(x_M,0)\ge M.

Since M is arbitrary, \Phi^*(0,y)=+\infty for every y, and hence

\sup_{y\in U^*}\{-\Phi^*(0,y)\}=-\infty.

This proves part (2).

Assume next that p(0)>-\infty and choose y\in \partial p(0). Then

\forall u\in U,\qquad p(u)\ge p(0)+\langle y,u\rangle.

Equivalently,

\forall u\in U,\qquad \langle y,u\rangle-p(u)\le -p(0).

Taking the supremum over u gives

p^*(y)\le -p(0).

On the other hand, evaluating at u=0 gives

p^*(y)\ge \langle y,0\rangle-p(0)=-p(0).

Hence

p^*(y)=-p(0).

Using part (1) of Theorem 4.4,

p(0)=-p^*(y)=-\Phi^*(0,y).

Combined with the weak-duality inequality from part (1), this yields

p(0)=\max_{v\in U^*}\{-\Phi^*(0,v)\},

with the maximum attained at the chosen y. This proves part (3).

Finally, define

D:=\{u\in U:\exists x\in X\text{ such that }\Phi(x,u)<+\infty\}.

If u\in D, then there exists x\in X such that \Phi(x,u)<+\infty, so

p(u)=\inf_{x'\in X}\Phi(x',u)\le \Phi(x,u)<+\infty,

hence u\in \operatorname{dom} p. Conversely, if u\in \operatorname{dom} p, then p(u)<+\infty, so not all values \Phi(x,u) can equal +\infty; therefore there exists x\in X such that \Phi(x,u)<+\infty, and hence u\in D. Thus

D=\operatorname{dom} p.

Assume now that p(0)\in \mathbb{R} and 0\in \operatorname{ri}(D)=\operatorname{ri}(\operatorname{dom} p). By convexity of p, it is enough to show that p never takes the value -\infty on \operatorname{dom} p. Fix u\in \operatorname{dom} p. Since 0\in \operatorname{ri}(\operatorname{dom} p), there exists \lambda\in(0,1) such that \lambda u\in \operatorname{dom} p. Then p(0) and p(\lambda u) are both finite, and convexity gives

p(\lambda u)\le \lambda p(u)+(1-\lambda)p(0).

Therefore p(u)>-\infty. Hence p:U\to \mathbb{R}\cup\{+\infty\} is proper and convex, with 0\in \operatorname{ri}(\operatorname{dom} p). By Theorem 2.10, we obtain \partial p(0)\neq\varnothing. Part (3) now applies and proves part (4).

The formula for p(u) is immediate from the definition of \Phi. For y\in \mathbb{R}^m,

\begin{aligned} \Phi^*(0,y) &=\sup_{x\in \mathbb{R}^n,\ u\in \mathbb{R}^m} \{y^\top u-c^\top x-\delta_{\mathbb{R}_+^n}(x)-\delta_{\mathbb{R}_+^m}(Ax-b-u)\}. \end{aligned}

If some coordinate of y is negative, then the constraint Ax-b-u\in \mathbb{R}_+^m only imposes an upper bound u\le Ax-b, so sending the corresponding coordinate of u to -\infty shows that \Phi^*(0,y)=+\infty. Hence we may restrict to y\ge 0. For such y, the supremum over u is attained at the largest feasible choice u=Ax-b, so

\Phi^*(0,y) =\sup_{x\ge 0}\{y^\top(Ax-b)-c^\top x\} =-b^\top y+\sup_{x\ge 0}\{x^\top(A^\top y-c)\}.

If A^\top y\le c, then x^\top(A^\top y-c)\le 0 for every x\ge 0, so the supremum is 0, attained at x=0. If instead some coordinate of A^\top y-c is positive, then scaling the corresponding basis vector shows that the supremum is +\infty. This proves the displayed formula for \Phi^*(0,y).

Part (1) of Theorem 4.4 then gives

\sup_{y\in \mathbb{R}^m}\{-\Phi^*(0,y)\}\le p(0),

which is exactly the weak-duality inequality

\sup\{b^\top y:A^\top y\le c,\ y\ge 0\} \le \inf\{c^\top x:Ax\ge b,\ x\ge 0\}.

If in addition 0\in \operatorname{ri}(\operatorname{dom} p) and p(0)\in \mathbb{R}, then part (4) of Theorem 4.4 yields the claimed equality and dual attainment.

Set

g(u):=\lambda\|u\|, \qquad \Phi(x,u):=f(x)+g(Ax+u).

Define the marginal value function

p(u):=\inf_{x\in X}\Phi(x,u)=\inf_{x\in X}\{f(x)+\lambda\|Ax+u\|\}.

Because g is finite everywhere on Y, one has \operatorname{dom} g=Y. Since f is proper, there exists x_0\in X with f(x_0)<+\infty, so for every u\in Y,

p(u)\le f(x_0)+\lambda\|Ax_0+u\|<+\infty.

Therefore

\operatorname{dom} p=Y, \qquad 0\in \operatorname{ri}(\operatorname{dom} p).

For y\in Y^*,

\begin{aligned} \Phi^*(0,y) &=\sup_{x\in X,\ u\in Y}\{\langle y,u\rangle-f(x)-g(Ax+u)\} \\ &=\sup_{x\in X,\ z\in Y}\{\langle y,z-Ax\rangle-f(x)-g(z)\} \\ &=\sup_{x\in X}\{\langle -A^*y,x\rangle-f(x)\} +\sup_{z\in Y}\{\langle y,z\rangle-g(z)\} \\ &=f^*(-A^*y)+g^*(y). \end{aligned}

We now compute g^*. For y\in Y^*,

g^*(y)=\sup_{u\in Y}\{\langle y,u\rangle-\lambda\|u\|\}.

If \|y\|_*\le \lambda, then for every u\in Y,

\langle y,u\rangle\le \|y\|_*\|u\|\le \lambda\|u\|,

so

\langle y,u\rangle-\lambda\|u\|\le 0.

Taking u=0 shows that the supremum is exactly 0.

If instead \|y\|_*>\lambda, then by the definition of the dual norm there exists u_0\in Y such that

\langle y,u_0\rangle>\lambda\|u_0\|.

For every t>0,

\langle y,tu_0\rangle-\lambda\|tu_0\| = t\bigl(\langle y,u_0\rangle-\lambda\|u_0\|\bigr)\to +\infty.

Hence g^*(y)=+\infty. Therefore

g^*(y)= \begin{cases} 0,&\|y\|_*\le \lambda,\\ +\infty,&\|y\|_*>\lambda. \end{cases}

If p(0)=-\infty, then part (2) of Theorem 4.4 gives

\sup_{y\in Y^*}\{-\Phi^*(0,y)\}=-\infty.

Since 0\in Y^* and \|0\|_*=0, the feasible set \{y\in Y^*:\|y\|_*\le \lambda\} is nonempty, so the supremum is then a maximum, and the desired formula follows.

If instead p(0)>-\infty, then p(0)\in \mathbb{R} because p(0)<+\infty, and part (4) of Theorem 4.4 yields

p(0)=\max_{y\in Y^*}\{-\Phi^*(0,y)\}.

In this case, since p(0)=\inf_{x\in X}\{f(x)+\lambda\|Ax\|\}, substituting the formula for \Phi^*(0,y) and then the formula for g^*(y) yields

\inf_{x\in X}\{f(x)+\lambda\|Ax\|\} = \max_{\substack{y\in Y^*\\ \|y\|_*\le \lambda}} \{-f^*(-A^*y)\}.