# Subgradients **STATS 606:** Computation and Optimization Methods in Statistics University of Michigan
## Subgradients **Def:** $g\in\reals^n$ is a subgradient of $f:\reals^n\to\reals$ iff $$f(y) \ge f(x) + g^\top(y-x)\text{ for all }y\in\reals^n.$$ * $f(x) + g^\top(y-x)$ is a *global* minorant of $f$ * $g^\top(y-x)$ *under*-estimates $f(y) - f(x)$ The set of all subgradient of $f$ at $x$ is called the **subdifferential** of $f$ at $x$; this set is typically denoted $\partial f(x)$. **Obs:** $x_\*\in\argmin_xf(x)$ iff $0\in\partial f(x_*)$.
## Ex: subdifferential of $|x|$
## Subgradient are supporting hyperplanes to $\epi(f)$ **Obs:** $g\in\partial f(x)$ iff $(g,-1)\in\reals^n\times\reals$ defines a supporting hyperplane to $\epi(f)$ at $(x,f(x))$; i.e. $$\begin{bmatrix}g \\\\ -1\end{bmatrix}^\top\left(\begin{bmatrix}y \\\\ t\end{bmatrix} - \begin{bmatrix}x\\\\f(x)\end{bmatrix}\right)\le 0\text{ for all }(y,t)\in\epi(f).$$ Thus subgradients exist because supporting hyperplanes exist: **Lem:** $\partial f(x)$ is non-empty (i.e. $f$ is subdifferentiable) at any $x$ in the interior of $\dom f$.
## Subgradients properties Let $x$ be in the interior of $\dom f$: 1. $\partial f(x)$ is compact 2. If $f$ is convex and differentiable at $x$, then $\partial f(x) = \\{\nabla f(x)\\}$. 3. If $f$ is convex and $\partial f(x)$ is a singleton, then $f$ is differentiable at $x$ (and the single element in $\partial f(x)$ is $\nabla f(x)$). 4. The directional derivative of $f$ at $x$ in the direction $d$ is $f'(x;d) = \sup_{g\in\partial f(x)} g^\top d$.