UA SIE545 優化理論基礎2 凸函數 概念 理論 總結

---

凸函數的概念與簡單性質

Convex function $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. Call $f$ a convex function on $S$ if $?x_1,x_2\in S$, $\lambda \in (0,1)$
$$f(\lambda x_1+(1?\lambda )x_2)\le \lambda f(x_1)+(1?\lambda )f(x_2)$$

Call $f$ a strictly convex function on $S$ if $?x_1,x_2\in S$, $\lambda \in (0,1)$
$$f(\lambda x_1+(1?\lambda )x_2)<\lambda f(x_1)+(1?\lambda )f(x_2)$$

Call $f$ a (strictly) concave function on $S$ if $?f$ is (strictly) convex.

Level set

Lower-level set: $S_{\alpha }=\{x\in S:f(x)\le \alpha \}$

Upper-level set: $\{x\in S:f(x)\ge \alpha \}$

Properties

If $f$ is a convex function, $S_{\alpha }$ is a convex set.

$f\in C(intS)$ ($f$ is continuous on the interior of $S$)

For ${\mathbb{R}}^n$ convex function $f$, any non-zero directional derivative exists. $$?\underset{\lambda \to 0^{+}}{\lim }\frac{f(\bar{x}+\lambda d)?f(\bar{x})}{\lambda },\bar{x}\in S,\bar{x}+\lambda d\in S$$

---

次梯度(sub-gradient)

epigraph $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set.
$$epif=\{(x,y):x\in S,y\in \mathbb{R},y\ge f(x)\}$$

hypograph $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set.
$$hypf=\{(x,y):x\in S,y\in \mathbb{R},y\le f(x)\}$$

Property $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is convex iff $epif$ is convex.

subgradient $f:S\to \mathbb{R}$ is convex where $S$ is a nonempty convex set. Then $\xi$ is called subgradient of $f$ at $\bar{x}$ if
$$f(x)\ge f(\bar{x})+{\xi }^T(x?\bar{x}),?x\in S$$

$f:S\to \mathbb{R}$ is concave where $S$ is a nonempty convex set. Then $\xi$ is called subgradient of $f$ at $\bar{x}$ if
$$f(x)\le f(\bar{x})+{\xi }^T(x?\bar{x}),?x\in S$$

Property

$f:S\to \mathbb{R}$ is convex where $S$ is a nonempty convex set. $?\bar{x}\in intS$, $?\xi$ such that
$$f(x)\ge f(\bar{x})+{\xi }^T(x?\bar{x}),?x\in S$$and the hyperplane
$$H=\{(x,y):y=f(\bar{x})+{\xi }^T(x?\bar{x})\}$$supports $epif$ at $(\bar{x},f(\bar{x}))$.

$f:S\to \mathbb{R}$ is strictly convex where $S$ is a nonempty convex set. $?\bar{x}\in intS$, $?\xi$ such that
$$f(x)>f(\bar{x})+{\xi }^T(x?\bar{x}),?x\in S$$

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. If $?\bar{x}\in intS$, $?\xi$ such that
$$f(x)\ge f(\bar{x})+{\xi }^T(x?\bar{x}),?x\in S$$then $f$ is convex.

---

可微的凸函數

differentiable $f:S\to \mathbb{R}$ where $S$ is a nonempty set. Say $f$ is differentiable at $\bar{x}\in intS$ if $?\beta$ such that
$$\begin{gathered} f(x)=f(\bar{x})+{\beta }^T(x?\bar{x})+o(\Vert x?\bar{x}\Vert ) \\ \beta =?f(\bar{x}) \end{gathered}$$

Property

$f:S\to \mathbb{R}$ is convex where $S$ is a nonempty convex set. If $f$ is differentiable at $\bar{x}$, then $?f(\bar{x})$ is subgradient.

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is differentiable at $\bar{x}$, then $f$ is convex iff $?\bar{x}\in S$, $$f(x)\ge f(\bar{x})+?f(\bar{x}{)}^T(x?\bar{x}),?x\in S$$

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is differentiable on $S$, then $f$ is convex iff $?x_1,x_2\in S$, $$[?f(x_2)??f(x_1){]}^T(x_2?x_1)\ge 0$$

PSD positive semi-definite, $?x\in {\mathbb{R}}^n$, $x^{T}Hx\ge 0$
PD positive definite, $?x\in {\mathbb{R}}^n$, $x^{T}Hx>0$

Tips for checking PSD/PD

$?i,H_{ii}<0$, $H$ is not PSD; $?i,H_{ii}\le 0$, $H$ is not PD;

main sub-matrix is PSD/PD then $H$ is PSD/PD

if $H^T=H$, PD = PSD+nonsingular

if $H$ is $2\times 2$, $H_{11}>0,H_{22}>0,|H|>0$ means $H$ is PD

Property

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is twice differentiable on $S$, then $f$ is convex iff $Hf(\bar{x})$ is PSD.

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is twice differentiable on $S$, then if $Hf(\bar{x})$ is PD, $f$ is strictly convex; if $f$ is strictly convex, $Hf(\bar{x})$ is PSD; if $f$ is strictly convex and quadratic, $Hf(\bar{x})$ is PD

$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $f$ is infinitely differentiable on $S$, then $f$ is strictly convex at $\bar{x}$ iff $f^{(n)}(\bar{x})>0$ and $f^{(j)}(x)=0,?1<j<n$

---

凸函數的推廣

quasiconvex $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $?x_1,x_2\in S$, $$f(\lambda x_1+(1?\lambda )x_2)\le \max (f(x_1),f(x_2))$$

Property

$f$ is quasiconvex iff $?\alpha$, $S_{\alpha }$ is convex

$S$ is nonempty compact polyhedral then $?\bar{x}$ that is both an extreme point of $S$ and also the optimal solution to ${\min }_{S}f(x)$

$S$ is open convex, $f$ is differentiable. $f$ is quasiconvex iff one of the following is correct: if $x_1,x_2\in S$, $f(x_1)\le f(x_2)$, then $?f(x_2{)}^T(x_1?x_2)\le 0$ or if $x_1,x_2\in S$, $?f(x_2{)}^T(x_1?x_2)\le 0$, then $f(x_1)\le f(x_2)$

strict quasiconvex $f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $?x_1,x_2\in S$, $f(x_1)=f(x_2)$, $$f(\lambda x_1+(1?\lambda )x_2)<\max (f(x_1),f(x_2))$$

Property

If $\bar{x}$ is local minimum to ${\min }_{S}f(x)$ where $S$ is convex, then $\bar{x}$ is global minimum.

strong quasiconvex
$f:S\to \mathbb{R}$ where $S$ is a nonempty convex set. $?x_1,x_2\in S$, $x_1=x_2$, $$f(\lambda x_1+(1?\lambda )x_2)<\max (f(x_1),f(x_2))$$

Property

strong quasiconvex leads to strict quasiconvex

If $\bar{x}$ is local minimum to ${\min }_{S}f(x)$ where $S$ is convex, then $\bar{x}$ is unique global minimum.

Pseudoconvex $?x_1,x_2\in X$ such that $?f(x_1{)}^T(x_2?x_1)\ge 0$, we have $f(x_2)\ge f(x_1)$.

Property

Pseudoconvex + differentiable = strict quasiconvex

Strict Pseudoconvex + differentiable = strong quasiconvex

總結

以上是生活随笔為你收集整理的UA SIE545 优化理论基础2 凸函数 概念 理论 总结的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。