2. Lecture 2: Separation and Duality🔗

Lecture 1 established the slogan that convexity turns local information into global information. Lecture 2 starts from convex functions, but very quickly it switches to sets. The point of the epigraph is exactly to make that switch. Once a convex function is encoded as a subset of E \times \mathbb{R}, the right tools are projection, separation, and supporting hyperplanes. After that, we translate the geometry back into function language through subgradients.

Lecture 2 turns the slogan of Lecture 1 that convexity turns local information into global information into one geometric picture:

\operatorname{cl}(\operatorname{conv}(S)) = \bigcap\{H(\xi,b) : S \subseteq H(\xi,b)\}.

The two separation theorems to remember are the point-to-closed-convex-set theorem and the cone-separation theorem. Everything else in the lecture is either a reformulation of that picture or a way of bringing it back to convex functions.

2.1. Extended Real and Epigraph🔗

We begin by enlarging the value space from \mathbb{R} to \mathbb{R}\cup\{+\infty\}. The point is not to do arithmetic with +\infty for its own sake. The point is that +\infty lets us encode the effective domain inside the function itself. That makes the epigraph construction completely uniform.

Definition

2.1 Extended real line used in these notes

We write

\mathbb{R}\cup\{+\infty\}

for the one-sided extended real line. Its order is the usual order on \mathbb{R}, extended by

a\le +\infty \qquad\text{for every }a\in \mathbb{R}\cup\{+\infty\}.

In Lecture 2 we use the conventions

a+(+\infty)=+\infty,\qquad \lambda(+\infty)=+\infty\ (\lambda>0),\qquad 0\cdot(+\infty):=0,\qquad \inf\varnothing=+\infty.

Similar properties hold for -\infty. The expression +\infty-\infty is undefined, and we will not use it.

Definition

2.2 Extended-value function and effective domain

Let f : E \to \mathbb{R}\cup\{+\infty\}. Its effective domain is

\operatorname{dom} f := \{x\in E : f(x)<+\infty\}.

The value +\infty is used to encode points outside the effective domain.

Definition

2.3 Convex extended-value function

Let f : E \to \mathbb{R}\cup\{+\infty\}. We say that f is convex if

\forall x,y\in E,\ \forall \theta\in[0,1],\qquad f(\theta x+(1-\theta)y)\le \theta f(x)+(1-\theta)f(y).

Remark

2.1 Compatibility with the Lecture 1 notion of convexity

Let C \subseteq E be convex, and let h : C \to \mathbb{R}. Define the ambient extension \tilde h : E \to \mathbb{R}\cup\{+\infty\} by

\tilde h(x):= \begin{cases} h(x), & x\in C,\\ +\infty, & x\notin C. \end{cases}

Then h is convex in the sense of Lecture 1 if and only if \tilde h is convex in the extended-value sense above. So Lecture 2 is not changing the notion of convexity. It is only changing the packaging.

Definition

2.4 Epigraph

Let f : E \to \mathbb{R}\cup\{+\infty\}. Its epigraph is

\operatorname{epi}(f):=\{(x,t)\in E\times \mathbb{R}: t\ge f(x)\}.

The epigraph is the first place where the function-to-set translation becomes concrete. The next lemma says that convexity of f is exactly convexity of \operatorname{epi}(f).

After the epigraph criterion, the lecture switches to convex sets. The first geometric input is Euclidean projection. Projection is what manufactures separating hyperplanes, and later those separating hyperplanes will come back as subgradients.

2.2. Separation Theorems for Convex Sets🔗

Definition

2.5 Closed halfspace

Let \xi\in E^\ast\setminus\{0\} and b\in \mathbb{R}. The associated closed halfspace is

H(\xi,b):=\{x\in E : \langle \xi,x\rangle\le b\}.

Definition

2.6 Interior and closure

Let S\subseteq E. Its interior is

\operatorname{int}(S):= \{x\in S:\exists r>0\text{ such that }\{y\in E:\|y-x\|<r\}\subseteq S\}.

Its closure is

\operatorname{cl}(S):= \bigcap\{F\subseteq E:F\text{ is closed and }S\subseteq F\}.

Equivalently,

x\in \operatorname{cl}(S) \iff \forall r>0,\ \{y\in E:\|y-x\|<r\}\cap S\neq\varnothing.

Remark

2.2 Basic properties of interior and closure

For every S\subseteq E, the set \operatorname{int}(S) is open, the set \operatorname{cl}(S) is closed, and

\operatorname{int}(S)\subseteq S\subseteq \operatorname{cl}(S).

Moreover, S is open if and only if \operatorname{int}(S)=S, and S is closed if and only if \operatorname{cl}(S)=S.

Why is the closedness assumption needed? The proof below reduces separation to Euclidean projection, and projection may fail to exist for a convex set that is not closed. The statement itself can also fail: for example, if C=(0,1)\subseteq \mathbb{R} and z=1, then z\notin C but no strict separating halfspace exists. So closedness is not just a proof artifact.

Proof idea. Project z onto C, call the projection p, and use the normal vector z-p to define the separating hyperplane. The next theorem provides exactly this projection step.

Projection has now produced the first genuine separation statement. The next step is to let exterior points approach a boundary point of C; the limiting argument gives the supporting hyperplane theorem.

2.3. Duality of Convex Sets and Cones🔗

Once supporting hyperplanes are available, the natural global question is whether a closed convex set can be reconstructed from all the halfspaces that contain it. To state this cleanly, we first introduce convex hull and closed convex hull.

Definition

2.7 Convex hull

Let S\subseteq E. The convex hull of S is

\operatorname{conv}(S):= \bigcap\{C\subseteq E : C\text{ is convex and }S\subseteq C\}.

Definition

2.8 Closed convex hull

Let S\subseteq E. The closed convex hull of S is

\operatorname{cl}\operatorname{conv}(S):= \bigcap\{C\subseteq E : C\text{ is closed and convex and }S\subseteq C\}.

Proposition

2.5 Basic properties of closure, convex hull, and closed convex hull

Let S\subseteq E. Then the following hold:

\operatorname{cl}(S) is closed, \operatorname{conv}(S) is convex, and S\subseteq \operatorname{cl}(S)\cap \operatorname{conv}(S).
\operatorname{cl}\operatorname{conv}(S) = \operatorname{cl}(\operatorname{conv}(S)).
\operatorname{cl}\operatorname{conv}(S) is closed and convex, and S\subseteq \operatorname{cl}\operatorname{conv}(S).
If C\subseteq E is closed and convex, then \operatorname{cl}\operatorname{conv}(C)=C.
\operatorname{cl}\operatorname{conv}(\operatorname{cl}\operatorname{conv}(S)) = \operatorname{cl}\operatorname{conv}(S).
There exists a closed set S\subseteq \mathbb{R}^2 such that

\operatorname{conv}(\operatorname{cl}(S)) \subsetneq \operatorname{cl}\operatorname{conv}(S).

Proof

The set \operatorname{cl}(S) is an intersection of closed sets, so it is closed, and every set in that intersection contains S, so S\subseteq \operatorname{cl}(S). Likewise, \operatorname{conv}(S) is an intersection of convex sets, so it is convex, and again S\subseteq \operatorname{conv}(S). Hence S\subseteq \operatorname{cl}(S)\cap \operatorname{conv}(S).
First note that \operatorname{cl}(\operatorname{conv}(S)) is closed and convex and contains S. By the defining minimality of \operatorname{cl}\operatorname{conv}(S), this gives

\operatorname{cl}\operatorname{conv}(S) \subseteq \operatorname{cl}(\operatorname{conv}(S)).

Conversely, by definition of \operatorname{conv}(S), one has \operatorname{conv}(S)\subseteq \operatorname{cl}\operatorname{conv}(S). By item (3) below, \operatorname{cl}\operatorname{conv}(S) is closed, so taking closures yields

\operatorname{cl}(\operatorname{conv}(S)) \subseteq \operatorname{cl}\operatorname{conv}(S).

Therefore

\operatorname{cl}\operatorname{conv}(S) = \operatorname{cl}(\operatorname{conv}(S)).
By definition,

\operatorname{cl}\operatorname{conv}(S) = \bigcap\{C\subseteq E : C\text{ is closed and convex and }S\subseteq C\}.

An intersection of closed convex sets is again closed and convex, and every set in the intersection contains S. Hence \operatorname{cl}\operatorname{conv}(S) is closed and convex, and S\subseteq \operatorname{cl}\operatorname{conv}(S).
Let C\subseteq E be closed and convex. Since C is one of the sets over which the defining intersection for \operatorname{cl}\operatorname{conv}(C) is taken, we have

\operatorname{cl}\operatorname{conv}(C)\subseteq C.

On the other hand, item (3) with S=C gives C\subseteq \operatorname{cl}\operatorname{conv}(C). Therefore

\operatorname{cl}\operatorname{conv}(C)=C.
By item (3), the set \operatorname{cl}\operatorname{conv}(S) is itself closed and convex. Applying item (4) with C:=\operatorname{cl}\operatorname{conv}(S), we obtain

\operatorname{cl}\operatorname{conv}(\operatorname{cl}\operatorname{conv}(S)) = \operatorname{cl}\operatorname{conv}(S).
Consider the closed set

S := \{(0,0)\}\cup\{(x,y)\in \mathbb{R}^2 : x>0,\ y\ge 1/x\}.

We claim that

\operatorname{conv}(S) = \{(0,0)\}\cup\{(u,v)\in \mathbb{R}^2 : u>0,\ v>0\}.

First, let u>0 and v>0. Choose t\in(0,1) so small that t^2\le uv, and set

a := u/t,\qquad b := v/t.

Then a>0, and

b = v/t \ge 1/a.

Hence (a,b)\in S, and

(u,v)=t(a,b)+(1-t)(0,0)\in \operatorname{conv}(S).

Conversely, every point of S\setminus\{(0,0)\} has both coordinates strictly positive, so every nonzero convex combination of points of S also has both coordinates strictly positive. This proves the claim.

Since S is already closed, we have \operatorname{conv}(\operatorname{cl}(S))=\operatorname{conv}(S). On the other hand,

\operatorname{cl}\operatorname{conv}(S) = \{(u,v)\in \mathbb{R}^2 : u\ge 0,\ v\ge 0\},

which strictly contains \operatorname{conv}(S); for example, (0,1)\in \operatorname{cl}\operatorname{conv}(S)\setminus \operatorname{conv}(S). Therefore

\operatorname{conv}(\operatorname{cl}(S)) = \operatorname{conv}(S) \subsetneq \operatorname{cl}\operatorname{conv}(S).

Theorem

2.6 Closed convex hull equals the intersection of all containing halfspaces

Let S\subseteq E. Then

\operatorname{cl}\operatorname{conv}(S) = \bigcap\{H(\xi,b): \xi\in E^\ast\setminus\{0\},\ b\in \mathbb{R},\ S\subseteq H(\xi,b)\}.

In particular, if S is nonempty, closed, and convex, then

S=\operatorname{cl}\operatorname{conv}(S) =\bigcap\{H(\xi,b): \xi\in E^\ast\setminus\{0\},\ b\in \mathbb{R},\ S\subseteq H(\xi,b)\}.

Proof

Define

\mathcal H(S):= \bigcap\{H(\xi,b): \xi\in E^\ast\setminus\{0\},\ b\in \mathbb{R},\ S\subseteq H(\xi,b)\}.

Every closed halfspace is closed and convex, so \operatorname{cl}\operatorname{conv}(S)\subseteq \mathcal H(S).

Conversely, let z\in E\setminus \operatorname{cl}\operatorname{conv}(S). Applying Theorem 2.2 to the closed convex set \operatorname{cl}\operatorname{conv}(S) and the point z, we obtain \xi\neq 0 and b\in \mathbb{R} such that

\langle \xi,z\rangle>b \qquad\text{and}\qquad \forall y\in \operatorname{cl}\operatorname{conv}(S),\ \langle \xi,y\rangle\le b.

In particular, S\subseteq H(\xi,b), whereas z\notin H(\xi,b). Hence z\notin \mathcal H(S). This proves \mathcal H(S)\subseteq \operatorname{cl}\operatorname{conv}(S), and therefore \mathcal H(S)=\operatorname{cl}\operatorname{conv}(S). If S is additionally nonempty, closed, and convex, then \operatorname{cl}\operatorname{conv}(S)=S by Proposition 2.5, so the final displayed identity follows immediately.

Formal Statement and Proof

Lean theorem: Lecture02.thm_l2_separation.

theorem Lecture02_thm_l2_separation
    {E : Type*} [NormedAddCommGroup E] [NormedSpace ℝ E] [FiniteDimensional ℝ E]
    {S : Set E} :
    closure (convexHull ℝ S)
      = ⋂ (p : Covector E × ℝ) (_ : ∀ x ∈ S, p.1 x ≤ p.2),
          closedHalfspace p.1 p.2
    ∧
    (S.Nonempty → IsClosed S → Convex ℝ S →
      S = closure (convexHull ℝ S)
      ∧
      S = ⋂ (p : Covector E × ℝ) (_ : ∀ x ∈ S, p.1 x ≤ p.2),
            closedHalfspace p.1 p.2) := byE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E⊢ closure ((convexHull ℝ) S) = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2 ∧
  (S.Nonempty →
    IsClosed S →
      Convex ℝ S → S = closure ((convexHull ℝ) S) ∧ S = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2)
  refine ⟨(Lecture02.thm_l2_separation (S := S)).1, ?_⟩E:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E⊢ S.Nonempty →
  IsClosed S →
    Convex ℝ S → S = closure ((convexHull ℝ) S) ∧ S = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2
  intro _hS_ne hS_closedE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E_hS_ne:S.NonemptyhS_closed:IsClosed S⊢ Convex ℝ S → S = closure ((convexHull ℝ) S) ∧ S = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2 hS_convexE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E_hS_ne:S.NonemptyhS_closed:IsClosed ShS_convex:Convex ℝ S⊢ S = closure ((convexHull ℝ) S) ∧ S = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2
  have hS_eq_clconv : closure (convexHull ℝ S) = S := byE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E_hS_ne:S.NonemptyhS_closed:IsClosed ShS_convex:Convex ℝ S⊢ closure ((convexHull ℝ) S) = S
    exact (prop_l2_cch (S := S)).2.2.2.2.2.2.2.1 hS_closed hS_convexAll goals completed! 🐙
  refine ⟨hS_eq_clconv.symm, ?_⟩E:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ ES:Set E_hS_ne:S.NonemptyhS_closed:IsClosed ShS_convex:Convex ℝ ShS_eq_clconv:closure ((convexHull ℝ) S) = S := prop_l2_cch.right.right.right.right.right.right.right.left hS_closed hS_convex⊢ S = ⋂ p, ⋂ (_ : ∀ x ∈ S, p.1 x ≤ p.2), closedHalfspace p.1 p.2
  calc
    S = closure (convexHull ℝ S) := hS_eq_clconv.symm
    _ = ⋂ (p : Covector E × ℝ) (_ : ∀ x ∈ S, p.1 x ≤ p.2),
          closedHalfspace p.1 p.2 :=
      (Lecture02.thm_l2_separation (S := S)).1

At this point the set-level duality picture is in place: \operatorname{cl}\operatorname{conv}(S) is recovered by intersecting all containing halfspaces. The next step is to specialize that picture to convex cones. That cone version will be used again later, especially in Lecture 4 and Lecture 6.

Definition

2.9 Convex cone, dual cone, and polar cone

A set K\subseteq E is a convex cone if

\forall x_1,x_2\in K,\ \forall \alpha_1,\alpha_2\in [0,+\infty),\qquad \alpha_1 x_1+\alpha_2 x_2\in K.

Its dual cone and polar cone are

K^\ast:=\{\xi\in E^\ast:\forall x\in K,\ \langle \xi,x\rangle\ge 0\}, \qquad K^\circ:=\{\xi\in E^\ast:\forall x\in K,\ \langle \xi,x\rangle\le 0\}.

The cone theorem is the last set-level specialization in this lecture. Before returning to functions, we record affine hull and relative interior, because the final subgradient existence theorem naturally lives on \operatorname{ri}(\operatorname{dom} f).

2.4. Affine Hull and Relative Interior🔗

Definition

2.10 Affine sets and affine spaces

Let A\subseteq E. We say that A is affine if

\forall x,y\in A,\ \forall \theta\in \mathbb{R},\qquad \theta x+(1-\theta)y\in A.

In these notes, an affine space means an affine subset of some ambient finite-dimensional real vector space, equipped with the induced affine operations.

Definition

2.11 Affine hull and relative interior

Let S\subseteq E. Its affine hull is

\operatorname{aff}(S):= \bigcap\{A\subseteq E : A\text{ is affine and }S\subseteq A\}.

If C\subseteq E, then the relative interior of C is

\operatorname{ri}(C):= \{x\in C:\exists r>0\text{ such that }\{y\in \operatorname{aff}(C):\|y-x\|<r\}\subseteq C\}.

Remark

2.3 Why affine spaces appear here

Convexity and the separation theorems above are fundamentally affine notions: they only use affine combinations and halfspaces, not a preferred origin in the ambient vector space. One could therefore formulate much of this lecture directly on affine spaces. To keep the terminology lighter, we do not globalize that viewpoint throughout the course. Instead, we work globally in a vector space E, and pass locally to affine subspaces when needed. The proof of Theorem 2.10 is the first place where this local affine viewpoint becomes essential.

2.5. Subgradient and Its Existence🔗

We now translate the geometry back into function language. A supporting hyperplane to \operatorname{epi}(f) through (x,f(x)) is exactly a global affine lower bound on f, and that is what a subgradient is. The only new point is that the relevant ambient space is not always all of E, but \operatorname{aff}(\operatorname{dom} f)\times \mathbb{R}.

Definition

2.12 Subgradient and subdifferential

Let f : E \to \mathbb{R}\cup\{+\infty\}, and let x\in \operatorname{dom} f. A covector g\in E^\ast is a subgradient of f at x if

\forall y\in E,\qquad f(y)\ge f(x)+\langle g,y-x\rangle.

The set of all such covectors is denoted by \partial f(x).

Definition

2.13 Proper extended-value function

Let f : E \to \mathbb{R}\cup\{+\infty\}. We say that f is proper if

\operatorname{dom} f\neq \varnothing.

Equivalently, f is proper if there exists x\in E such that f(x)<+\infty. In treatments based on the full extended real line \mathbb{R}\cup\{\pm\infty\}, one also requires f(x)>-\infty for every x. We do not impose that extra condition here because our present extended-value convention has already excluded the value -\infty.

Theorem

2.10 Existence of a subgradient on the relative interior

Let f : E \to \mathbb{R}\cup\{+\infty\} be proper and convex. Then

\forall x\in \operatorname{ri}(\operatorname{dom} f),\qquad \partial f(x)\neq \varnothing.

Proof

Let x\in \operatorname{ri}(\operatorname{dom} f), set A:=\operatorname{aff}(\operatorname{dom} f), and let L:=A-x. Consider the convex set

\mathcal E:=\{(y,t)\in A\times \mathbb{R} : f(y)\le t\}.

Since \mathcal E=\operatorname{epi}(f)\cap (A\times \mathbb{R}), Lemma 2.1 implies that \mathcal E is convex. Translate \mathcal E so that (x,f(x)) becomes the origin:

\widetilde{\mathcal E}:= \{(y-x,t-f(x))\in L\times \mathbb{R}:(y,t)\in \mathcal E\}.

Then \widetilde{\mathcal E} is a convex subset of the finite-dimensional vector space L\times \mathbb{R}, and (0,0)\in \widetilde{\mathcal E}. Moreover, (0,0)\notin \operatorname{int}(\widetilde{\mathcal E}): indeed, for every \varepsilon>0, we have (0,-\varepsilon)\notin \widetilde{\mathcal E}.

Apply Theorem 2.4 in the finite-dimensional vector space L\times \mathbb{R}. We obtain (\eta,\alpha)\in L^\ast\times \mathbb{R}, not both zero, such that

\forall (v,s)\in \widetilde{\mathcal E},\qquad \langle \eta,v\rangle+\alpha s\le 0.

Since (v,s)\in \widetilde{\mathcal E} implies (v,s+r)\in \widetilde{\mathcal E} for every r\ge 0, we must have \alpha\le 0. We claim that \alpha<0. If \alpha=0, then

\forall y\in \operatorname{dom} f,\qquad \langle \eta,y-x\rangle\le 0.

Because x\in \operatorname{ri}(\operatorname{dom} f), there exists r>0 such that x+v\in \operatorname{dom} f for every v\in L with \|v\|<r. Fix v\in L, and choose t>0 so small that t\|v\|<r. Then x+tv,x-tv\in \operatorname{dom} f, so the displayed inequality gives

\langle \eta,tv\rangle\le 0 \qquad\text{and}\qquad \langle \eta,-tv\rangle\le 0.

Hence \langle \eta,v\rangle=0. Since v\in L was arbitrary, \eta=0, a contradiction. Thus \alpha<0.

Set g_L:=-\eta/\alpha\in L^\ast. For every y\in \operatorname{dom} f, the point (y,f(y)) belongs to \mathcal E, so

(y-x,f(y)-f(x))\in \widetilde{\mathcal E}.

Therefore

f(y)\ge f(x)+\langle g_L,y-x\rangle.

By Lemma 2.12, extend g_L to some covector g\in E^\ast. Using y-x\in L, we still have

\forall y\in \operatorname{dom} f,\qquad f(y)\ge f(x)+\langle g,y-x\rangle.

For y\notin \operatorname{dom} f, the same inequality is again automatic, so g\in \partial f(x). Equivalently, by Lemma 2.8, (g,-1) defines a supporting hyperplane of \operatorname{epi}(f) at (x,f(x)).

Formal Statement and Proof

Lean theorem: Lecture02.thm_l2_subgrad_exist.

theorem Lecture02_thm_l2_subgrad_exist
    {E : Type*} [NormedAddCommGroup E] [NormedSpace ℝ E] [FiniteDimensional ℝ E]
    {f : E → EReal}
    (hproper : IsProper f)
    (hconv : EConvexOn Set.univ f)
    {x : E}
    (hx : x ∈ intrinsicInterior ℝ (effectiveDomain f)) :
    ∃ g : Covector E, IsERealSubgradient f x g := byE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ Ef:E → ERealhproper:IsProper fhconv:EConvexOn Set.univ fx:Ehx:x ∈ intrinsicInterior ℝ (effectiveDomain f)⊢ ∃ g, IsERealSubgradient f x g
  obtain ⟨g, hg⟩ :=
    exists_ereal_subgradientWithin_of_mem_intrinsicInterior_effectiveDomain
      hproper hconv hxE:Type u_1inst✝²:NormedAddCommGroup Einst✝¹:NormedSpace ℝ Einst✝:FiniteDimensional ℝ Ef:E → ERealhproper:IsProper fhconv:EConvexOn Set.univ fx:Ehx:x ∈ intrinsicInterior ℝ (effectiveDomain f)g:Covector Ehg:IsERealSubgradientWithin f (effectiveDomain f) x g⊢ ∃ g, IsERealSubgradient f x g
  exact ⟨g, hg.of_effectiveDomain⟩All goals completed! 🐙

Remark

2.4 Why the (relative) interior condition is needed

The hypothesis x\in \operatorname{ri}(\operatorname{dom} f) cannot be weakened to x\in \operatorname{cl}(\operatorname{dom} f). Consider

f(x):= \begin{cases} x\log x,& x>0,\\ 0,& x=0,\\ +\infty,& x<0. \end{cases}

Then f is proper, closed, and convex, with \operatorname{dom} f=[0,\infty), so 0\in \operatorname{cl}(\operatorname{dom} f)\setminus \operatorname{ri}(\operatorname{dom} f). However, \partial f(0)=\varnothing. Indeed, if g\in \partial f(0), then for every y>0,

y\log y=f(y)\ge f(0)+gy=gy,

so \log y\ge g for every y>0, which is impossible because \log y\to -\infty as y\downarrow 0.

2.6. Proof-Local Lemmas🔗

Lemma

2.11 Closure preserves convexity and interior for convex sets

Let C\subseteq E be convex. Then \operatorname{cl}(C) is convex and

\operatorname{int}(\operatorname{cl}(C))=\operatorname{int}(C).

Proof

To prove convexity of \operatorname{cl}(C), let u,v\in \operatorname{cl}(C) and \theta\in[0,1]. Choose sequences u_n,v_n\in C such that u_n\to u and v_n\to v. Then

\theta u_n+(1-\theta)v_n\in C

for every n, and

\theta u_n+(1-\theta)v_n\to \theta u+(1-\theta)v.

Hence \theta u+(1-\theta)v\in \operatorname{cl}(C).

The inclusion \operatorname{int}(C)\subseteq \operatorname{int}(\operatorname{cl}(C)) is immediate from C\subseteq \operatorname{cl}(C). For the reverse inclusion, let x_0\in \operatorname{int}(\operatorname{cl}(C)). Choose r>0 such that

\{y\in E:\|y-x_0\|<r\}\subseteq \operatorname{cl}(C).

Let d:=\dim(E), fix a basis e_1,\dots,e_d of E, and choose \varepsilon>0 so small that the points

y_i:=x_0+\varepsilon e_i \qquad (1\le i\le d), \qquad y_0:=x_0-\varepsilon\sum_{i=1}^d e_i

all belong to \{y\in E:\|y-x_0\|<r\}. Then y_0,\dots,y_d\in \operatorname{cl}(C), they are affinely independent, and

x_0=\frac{1}{d+1}\sum_{i=0}^d y_i,

so x_0\in \operatorname{int}(\operatorname{conv}\{y_0,\dots,y_d\}). Choose z_0,\dots,z_d\in C sufficiently close to y_0,\dots,y_d that z_0,\dots,z_d remain affinely independent and the barycentric coordinates of x_0 with respect to z_0,\dots,z_d remain strictly positive. Then

x_0\in \operatorname{int}(\operatorname{conv}\{z_0,\dots,z_d\})\subseteq C,

so x_0\in \operatorname{int}(C). Therefore \operatorname{int}(\operatorname{cl}(C))\subseteq \operatorname{int}(C), proving the claim.

Formal Statement and Proof

Lean theorem: Lecture02.lem_l2_int_cl_convex.

theorem ConvexOptV3.Lectures.Lecture02.lem_l2_int_cl_convex.{u_2} : ∀ {E : Type u_2} [inst : NormedAddCommGroup E]
  [inst_1 : NormedSpace ℝ E] [FiniteDimensional ℝ E] {C : Set E},
  Convex ℝ C → Convex ℝ (closure C) ∧ interior (closure C) = interior C :=
fun {E} [NormedAddCommGroup E] [NormedSpace ℝ E] [FiniteDimensional ℝ E] {C} hC_convex =>
  ⟨Convex.closure hC_convex,
    if hC_int : (interior C).Nonempty then Convex.interior_closure_eq_interior_of_nonempty_interior hC_convex hC_int
    else
      have h_affine_closure :=
        le_antisymm
          (affineSpan_le_of_subset_coe
            (closure_minimal (subset_affineSpan ℝ C) (AffineSubspace.closed_of_finiteDimensional (affineSpan ℝ C))))
          (affineSpan_mono ℝ subset_closure);
      have hIntClosure_empty :=
        Set.eq_empty_iff_forall_notMem.mpr fun x hx =>
          have h_affine_top_closure :=
            (Convex.interior_nonempty_iff_affineSpan_eq_top (Convex.closure hC_convex)).mp (Exists.intro x hx);
          have h_affine_top := Eq.mp (congrFun' (congrArg Eq h_affine_closure) ⊤) h_affine_top_closure;
          hC_int ((Convex.interior_nonempty_iff_affineSpan_eq_top hC_convex).mpr h_affine_top);
      have hInt_empty := Set.not_nonempty_iff_eq_empty.mp hC_int;
      Eq.mpr (id (congrArg (fun _a => _a = interior C) hIntClosure_empty))
        (Eq.mpr (id (congrArg (fun _a => ∅ = _a) hInt_empty)) (Eq.refl ∅))⟩#print Lecture02.lem_l2_int_cl_convex

2.7. Exercises🔗

Prove Proposition 2.5. Hint for item 6: try T:=\{(x,y)\in \mathbb{R}^2:x>0,\ y\ge 1/x\}\cup\{(0,0)\}.
Fix the auxiliary Euclidean norm \|\cdot\|_2 from the ambient convention and let f(x)=\|x\|_2 on E. Compute \partial f(0) and \partial f(x) for x\neq 0.