黑塞矩陣

海森矩陣（德語：Hesse-Matrix；英語：Hessian matrix 或 Hessian），又譯作黑塞矩陣、海塞（賽）矩陣或海瑟矩陣等，是一個由多變量實值函數的所有二階偏導數組成的方陣，由德國數學家奧托·黑塞引入並以其命名。

定義

假設有一實值函數${\displaystyle f(x_{1},x_{2},\dots ,x_{n})\,}$，如果 ${\displaystyle f\,}$的所有二階偏導數都存在並在定義域內連續，那麼函數${\displaystyle f\,}$的黑塞矩陣為

$${\displaystyle \mathbf {H} ={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}\,}$$

或使用下標記號表示為

$${\displaystyle \mathbf {H} _{ij}={\frac {\partial ^{2}f}{\partial x_{i}\partial x_{j}}}}$$

顯然黑塞矩陣 ${\displaystyle \mathbf {H} \,}$是一個${\displaystyle n\times n\,}$方陣。黑塞矩陣的行列式被稱爲黑塞式（英語：Hessian），而需注意的是英語環境下使用Hessian一詞時可能指上述矩陣也可能指上述矩陣的行列式^[1]。

性質

由高等數學知識可知，若一元函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點的某個鄰域內具有任意階導數，則函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點處的泰勒展開式為

$${\displaystyle f(x)=f(x_{0})+f'(x_{0})\Delta x+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,}$$

其中，${\displaystyle \Delta x=x-x_{0}\,}$。

同理，二元函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的泰勒展開式為

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+f_{x_{1}}(x_{10},x_{20})\Delta x_{1}+f_{x_{2}}(x_{10},x_{20})\Delta x_{2}+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{10},x_{20})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{10},x_{20})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{10},x_{20})\Delta x_{2}^{2}]+\cdots \,}$$

其中，${\displaystyle \Delta x_{1}=x_{1}-x_{10}\,}$，${\displaystyle \Delta x_{2}=x_{2}-x_{20}\,}$，${\displaystyle f_{x_{1}}={\frac {\partial f}{\partial x_{1}}}\,}$，${\displaystyle f_{x_{2}}={\frac {\partial f}{\partial x_{2}}}\,}$，${\displaystyle f_{x_{1}x_{1}}={\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\,}$，${\displaystyle f_{x_{2}x_{2}}={\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\,}$，${\displaystyle f_{x_{1}x_{2}}={\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}\,}$。

將上述展開式寫成矩陣形式，則有

$${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots }$$

其中，${\displaystyle \Delta x={\begin{bmatrix}\Delta x_{1}\\\\\Delta x_{2}\end{bmatrix}}\,}$，${\displaystyle \Delta x^{\mathrm {T} }={\begin{bmatrix}\Delta x_{1}&\Delta x_{2}\end{bmatrix}}\,}$是${\displaystyle \Delta x}$的轉置，${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}\\\\{\frac {\partial f}{\partial x_{2}}}\end{bmatrix}}\,}$是函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$的梯度，矩陣

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

即函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的${\displaystyle 2\times 2\,}$黑塞矩陣。它是由函數${\displaystyle f(x_{1},x_{2})}$在${\displaystyle x_{0}(x_{10},x_{20})}$點處的所有二階偏導數所組成的方陣。

由函數的二次連續性，有

$${\displaystyle {\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}}$$

所以，黑塞矩陣${\displaystyle G(x_{0})\,}$為對稱矩陣。

將二元函數的泰勒展開式推廣到多元函數，函數${\displaystyle f(x_{1},x_{2},\cdots ,x_{n})\,}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$點處的泰勒展開式為

$${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots \,}$$

其中，$${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}\,}$$ 為函數${\displaystyle f(x)}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$點的梯度，

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

為函數${\displaystyle f(x)\,}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$點的${\displaystyle n\times n\,}$黑塞矩陣。若函數有${\displaystyle n\,}$次連續性，則函數的${\displaystyle n\times n\,}$黑塞矩陣是對稱矩陣。

說明：在優化設計領域中，黑塞矩陣常用${\displaystyle G\,}$表示，且梯度有時用${\displaystyle g\,}$表示。^[2]

函數${\displaystyle f\,}$的黑塞矩陣和雅可比矩陣有如下關係：

$${\displaystyle \mathrm {H} (f)=\mathrm {J} (\nabla f)^{T}\,}$$

即函數${\displaystyle f\,}$的黑塞矩陣等於其梯度的雅可比矩陣。

應用

函數的極值條件

對於一元函數${\displaystyle f(x)\,}$，在給定區間內某${\displaystyle x=x_{0}\,}$點處可導，並在${\displaystyle x=x_{0}\,}$點處取得極值，其必要條件是

$${\displaystyle f'(x_{0})=0\,}$$

即函數${\displaystyle f(x)\,}$的極值必定在駐點處取得，或者說可導函數${\displaystyle f(x)\,}$的極值點必定是駐點；但反過來，函數的駐點不一定是極值點。檢驗駐點是否為極值點，可以採用二階導數的正負號來判斷。根據函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點處的泰勒展開式，考慮到上述極值必要條件，有

$${\displaystyle f(x)=f(x_{0})+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,}$$

若${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點處取得極小值，則要求在${\displaystyle x=x_{0}\,}$某一鄰域內一切點${\displaystyle x\,}$都必須滿足

$${\displaystyle f(x)-f(x_{0})>0\,}$$

即要求

$${\displaystyle {\frac {f''(x_{0})}{2!}}\Delta x^{2}>0\,}$$

亦即要求

$${\displaystyle f''(x_{0})>0\,}$$

${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點處取得極大值的討論與之類似。於是有極值充分條件：

設一元函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$點處具有二階導數，且${\displaystyle f'(x_{0})=0\,}$，${\displaystyle f''(x_{0})\neq 0\,}$，則

1. 當${\displaystyle f''(x_{0})>0\,}$時，函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$處取得極小值；
2. 當${\displaystyle f''(x_{0})<0\,}$時，函數${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$處取得極大值。

而當${\displaystyle f''(x_{0})=0\,}$時，無法直接判斷，還需要逐次檢驗其更高階導數的正負號。由此有一個規律：若其開始不為零的導數階數為偶數，則駐點是極值點；若為奇數，則為拐點，而不是極值點。

對於二元函數${\displaystyle f(x_{1},x_{2})\,}$，在給定區域內某${\displaystyle x_{0}(x_{10},x_{20})\,}$點處可導，並在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處取得極值，其必要條件是

$${\displaystyle f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,}$$

即

$${\displaystyle \nabla f(x_{0})=0\,}$$

同樣，這只是必要條件，要進一步判斷${\displaystyle x_{0}(x_{10},x_{20})\,}$是否為極值點需要找到取得極值的充分條件。根據函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的泰勒展開式，考慮到上述極值必要條件，有

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{0})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{0})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{0})\Delta x_{2}^{2}]+\cdots \,}$$

設${\displaystyle A=f_{x_{1}x_{1}}(x_{0})\,}$，${\displaystyle B=f_{x_{1}x_{2}}(x_{0})\,}$，${\displaystyle C=f_{x_{2}x_{2}}(x_{0})\,}$，則

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[A\Delta x_{1}^{2}+2B\Delta x_{1}\Delta x_{2}+C\Delta x_{2}^{2}]+\cdots \,}$$

或

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]+\cdots \,}$$

若${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處取得極小值，則要求在${\displaystyle x_{0}(x_{10},x_{20})\,}$某一鄰域內一切點${\displaystyle x\,}$都必須滿足

$${\displaystyle f(x_{1},x_{2})-f(x_{10},x_{20})>0\,}$$

即要求

$${\displaystyle {\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]>0\,}$$

亦即要求${\displaystyle A>0\,}$，${\displaystyle AC-B^{2}>0\,}$

即
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}-({\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}})^{2}\end{bmatrix}}_{x_{0}}>0\,}$$

此條件反映了${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的黑塞矩陣${\displaystyle G(x_{0})\,}$的各階主子式都大於零，即對於

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

要求
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle |G(x_{0})|={\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

${\displaystyle f((x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處取得極大值的討論與之類似。於是有極值充分條件：

設二元函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點的鄰域內連續且具有一階和二階連續偏導數，又有${\displaystyle f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,}$，同時令${\displaystyle A=f_{x_{1}x_{1}}(x_{0})\,}$，${\displaystyle B=f_{x_{1}x_{2}}(x_{0})\,}$，${\displaystyle C=f_{x_{2}x_{2}}(x_{0})\,}$，則

1. 當${\displaystyle A>0\,}$，${\displaystyle AC-B^{2}>0\,}$時，函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$處取得極小值；
2. 當${\displaystyle A<0\,}$，${\displaystyle AC-B^{2}>0\,}$時，函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$處取得極大值。

此外可以判斷，當${\displaystyle AC-B^{2}<0\,}$時，函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處沒有極值，此點稱爲鞍點。而當${\displaystyle AC-B^{2}=0\,}$時，無法直接判斷，對此，補充一個規律：當${\displaystyle AC-B^{2}=0\,}$時，如果有${\displaystyle A\equiv 0\,}$，那麼函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$有極值，且當${\displaystyle C>0\,}$有極小值，當${\displaystyle C<0\,}$有極大值。

由線性代數的知識可知，若矩陣${\displaystyle G(x_{0})\,}$滿足
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

則矩陣${\displaystyle G(x_{0})\,}$是正定矩陣，或者說矩陣${\displaystyle G(x_{0})\,}$正定。

若矩陣${\displaystyle G(x_{0})\,}$滿足
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}<0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

則矩陣${\displaystyle G(x_{0})\,}$是負定矩陣，或者說矩陣${\displaystyle G(x_{0})\,}$負定。^[3]

於是，二元函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處取得極值的條件表述為：二元函數${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的黑塞矩陣正定，則取得極小值；在${\displaystyle x_{0}(x_{10},x_{20})\,}$點處的黑塞矩陣負定，則取得極大值。

對於多元函數${\displaystyle f(x_{1},x_{2},\cdots ,x_{n})\,}$，若在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$點處取得極值，則極值存在的必要條件為

$${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}=0\,}$$

取得極小值的充分條件為

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

正定，即要求${\displaystyle G(x_{0})\,}$的各階主子式都大於零，即
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

$${\displaystyle \vdots }$$

$${\displaystyle |G(x_{0})|>0\,}$$
取得極大值的充分條件為

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

負定。^[4][5][6]

拓展閱讀

• 雅可比矩陣
• 梯度