矩阵与矢量求导
Definition D.l：
設:f(x)f(x)f(x)是標量,
x=(x1…xN)Tx=\left(x_{1} \ldots x_{N}\right)^{T}x=(x1?…xN?)T
,那么：
?f(x)?x=(?f(x)?x1?f(x)?x2??f(x)?xN)\frac{\partial f(x)}{\partial x}=\left(\begin{array}{c}{\frac{\partial f(x)}{\partial x_{1}}} \\ {\frac{\partial f(x)}{\partial x_{2}}} \\ {\vdots} \\ {\frac{\partial f(x)}{\partial x_{N}}}\end{array}\right)?x?f(x)?=????????x1??f(x)??x2??f(x)???xN??f(x)?????????
(?f(x)?x)T=?f(x)?xT=(?f(x)?x1?f(x)?x2…?f(x)?xN)\left(\frac{\partial f(x)}{\partial x}\right)^{T}=\frac{\partial f(x)}{\partial x^{T}}=\left(\begin{array}{cc}{\frac{\partial f(x)}{\partial x_{1}}} & {\frac{\partial f(x)}{\partial x_{2}}}\end{array} \ldots \frac{\partial f(x)}{\partial x_{N}}\right)(?x?f(x)?)T=?xT?f(x)?=(?x1??f(x)???x2??f(x)??…?xN??f(x)?)
--------------------------------------------------------------------------------------
Definition D.2
假設f(x)f(x)f(x)二階可導：
?2f(x)?x?xT=(?2f(x)?x12?2f(x)?x1?x2…?2f(x)?x1?xN?2f(x)?x2?x1?2f(x)?x22?????2f(x)?xN?x1???2f(x)?xN2)\frac{\partial^{2} f(x)}{\partial x \partial x^{T}}=\left(\begin{array}{cccc}{\frac{\partial^{2} f(x)}{\partial x_{1}^{2}}} & {\frac{\partial^{2} f(x)}{\partial x_{1} \partial x_{2}}} & {\dots} & {\frac{\partial^{2} f(x)}{\partial x_{1} \partial x_{N}}} \\ {\frac{\partial^{2} f(x)}{\partial x_{2} \partial x_{1}}} & {\frac{\partial^{2} f(x)}{\partial x_{2}^{2}}} & {} & {\vdots} \\ {\vdots} & {} & {\ddots} & {\vdots} \\ {\frac{\partial^{2} f(x)}{\partial x_{N} \partial x_{1}}} & {\cdots} & {\cdots} & {\frac{\partial^{2} f(x)}{\partial x_{N}^{2}}}\end{array}\right)?x?xT?2f(x)?=??????????x12??2f(x)??x2??x1??2f(x)???xN??x1??2f(x)???x1??x2??2f(x)??x22??2f(x)???…????x1??xN??2f(x)????xN2??2f(x)???????????
在假設
?2f(x)/?xp?xq=?2f(x)/?xq?xp\partial^{2} f(x) / \partial x_{p} \partial x_{q}=\partial^{2} f(x) / \partial x_{q} \partial x_{p}?2f(x)/?xp??xq?=?2f(x)/?xq??xp?成立的情況下，
該Hessian矩陣對稱。
--------------------------------------------------------------------------------------
Dehition D3：
假設f(x)f(x)f(x)是KX1的矢量函數：
f(x)=(f1(x)f2(x)?fK(x))f(x)=\left(\begin{array}{c}{f_{1}(x)} \\ {f_{2}(x)} \\ {\vdots} \\ {f_{K}(x)}\end{array}\right)f(x)=??????f1?(x)f2?(x)?fK?(x)???????
x=(x1…xN)Tx=\left(x_{1} \ldots x_{N}\right)^{T}x=(x1?…xN?)T
則f(x)f(x)f(x)的Jacobian矩陣是：
?f(x)?xT=(?f1(x)?x1?f1(x)?x2…?f1(x)?xL?f2(x)?x1?f2(x)?x2…?f2(x)?xL????fK(x)?x1?fK(x)?x2…?fK(x)?xL)\frac{\partial f(x)}{\partial x^{T}}=\left(\begin{array}{cccc}{\frac{\partial f_{1}(x)}{\partial x_{1}}} & {\frac{\partial f_{1}(x)}{\partial x_{2}}} & {\dots} & {\frac{\partial f_{1}(x)}{\partial x_{L}}} \\ {\frac{\partial f_{2}(x)}{\partial x_{1}}} & {\frac{\partial f_{2}(x)}{\partial x_{2}}} & {\dots} & {\frac{\partial f_{2}(x)}{\partial x_{L}}} \\ {\vdots} & {\vdots} & {} & {\vdots} \\ {\frac{\partial f_{K}(x)}{\partial x_{1}}} & {\frac{\partial f_{K}(x)}{\partial x_{2}}} & {\dots} & {\frac{\partial f_{K}(x)}{\partial x_{L}}}\end{array}\right)?xT?f(x)?=????????x1??f1?(x)??x1??f2?(x)???x1??fK?(x)???x2??f1?(x)??x2??f2?(x)???x2??fK?(x)??………??xL??f1?(x)??xL??f2?(x)???xL??fK?(x)?????????
該Jacobian矩陣的轉置是：
(?f(x)?xT)T=?fT(x)?x\left(\frac{\partial f(x)}{\partial x^{T}}\right)^{T}=\frac{\partial f^{T}(x)}{\partial x}(?xT?f(x)?)T=?x?fT(x)?
--------------------------------------------------------------------------------------
定理4我感覺是十分顯而易見，所以就沒有轉載。
Reference:
[1]APPENDIX D VECTOR AND MATRIX DIFFERENTIATION
總結
- 上一篇： 奇异值的几何意义
- 下一篇： 如何一次性复制带有markdown/ma