黑塞矩阵

海森矩阵（德语：Hesse-Matrix；英语：Hessian matrix 或 Hessian），又译作黑塞矩阵、海塞（赛）矩阵或海瑟矩阵等，是一个由多变量实值函数的所有二阶偏导数组成的方阵，由德国数学家奥托·黑塞引入并以其命名。

定义

假设有一实值函数${\displaystyle f(x_{1},x_{2},\dots ,x_{n})\,}$，如果 ${\displaystyle f\,}$的所有二阶偏导数都存在并在定义域内连续，那么函数${\displaystyle f\,}$的黑塞矩阵为

$${\displaystyle \mathbf {H} ={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}\,}$$

或使用下标记号表示为

$${\displaystyle \mathbf {H} _{ij}={\frac {\partial ^{2}f}{\partial x_{i}\partial x_{j}}}}$$

显然黑塞矩阵 ${\displaystyle \mathbf {H} \,}$是一个${\displaystyle n\times n\,}$方阵。黑塞矩阵的行列式被称为黑塞式（英语：Hessian），而需注意的是英语环境下使用Hessian一词时可能指上述矩阵也可能指上述矩阵的行列式^[1]。

性质

由高等数学知识可知，若一元函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点的某个邻域内具有任意阶导数，则函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点处的泰勒展开式为

$${\displaystyle f(x)=f(x_{0})+f'(x_{0})\Delta x+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,}$$

其中，${\displaystyle \Delta x=x-x_{0}\,}$。

同理，二元函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的泰勒展开式为

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+f_{x_{1}}(x_{10},x_{20})\Delta x_{1}+f_{x_{2}}(x_{10},x_{20})\Delta x_{2}+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{10},x_{20})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{10},x_{20})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{10},x_{20})\Delta x_{2}^{2}]+\cdots \,}$$

其中，${\displaystyle \Delta x_{1}=x_{1}-x_{10}\,}$，${\displaystyle \Delta x_{2}=x_{2}-x_{20}\,}$，${\displaystyle f_{x_{1}}={\frac {\partial f}{\partial x_{1}}}\,}$，${\displaystyle f_{x_{2}}={\frac {\partial f}{\partial x_{2}}}\,}$，${\displaystyle f_{x_{1}x_{1}}={\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\,}$，${\displaystyle f_{x_{2}x_{2}}={\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\,}$，${\displaystyle f_{x_{1}x_{2}}={\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}\,}$。

将上述展开式写成矩阵形式，则有

$${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots }$$

其中，${\displaystyle \Delta x={\begin{bmatrix}\Delta x_{1}\\\\\Delta x_{2}\end{bmatrix}}\,}$，${\displaystyle \Delta x^{\mathrm {T} }={\begin{bmatrix}\Delta x_{1}&\Delta x_{2}\end{bmatrix}}\,}$是${\displaystyle \Delta x}$的转置，${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}\\\\{\frac {\partial f}{\partial x_{2}}}\end{bmatrix}}\,}$是函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$的梯度，矩阵

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

即函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的${\displaystyle 2\times 2\,}$黑塞矩阵。它是由函数${\displaystyle f(x_{1},x_{2})}$在${\displaystyle x_{0}(x_{10},x_{20})}$点处的所有二阶偏导数所组成的方阵。

由函数的二次连续性，有

$${\displaystyle {\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}}$$

所以，黑塞矩阵${\displaystyle G(x_{0})\,}$为对称矩阵。

将二元函数的泰勒展开式推广到多元函数，函数${\displaystyle f(x_{1},x_{2},\cdots ,x_{n})\,}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$点处的泰勒展开式为

$${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots \,}$$

其中，$${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}\,}$$ 为函数${\displaystyle f(x)}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$点的梯度，

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

为函数${\displaystyle f(x)\,}$在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$点的${\displaystyle n\times n\,}$黑塞矩阵。若函数有${\displaystyle n\,}$次连续性，则函数的${\displaystyle n\times n\,}$黑塞矩阵是对称矩阵。

说明：在优化设计领域中，黑塞矩阵常用${\displaystyle G\,}$表示，且梯度有时用${\displaystyle g\,}$表示。^[2]

函数${\displaystyle f\,}$的黑塞矩阵和雅可比矩阵有如下关系：

$${\displaystyle \mathrm {H} (f)=\mathrm {J} (\nabla f)^{T}\,}$$

即函数${\displaystyle f\,}$的黑塞矩阵等于其梯度的雅可比矩阵。

应用

函数的极值条件

对于一元函数${\displaystyle f(x)\,}$，在给定区间内某${\displaystyle x=x_{0}\,}$点处可导，并在${\displaystyle x=x_{0}\,}$点处取得极值，其必要条件是

$${\displaystyle f'(x_{0})=0\,}$$

即函数${\displaystyle f(x)\,}$的极值必定在驻点处取得，或者说可导函数${\displaystyle f(x)\,}$的极值点必定是驻点；但反过来，函数的驻点不一定是极值点。检验驻点是否为极值点，可以采用二阶导数的正负号来判断。根据函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点处的泰勒展开式，考虑到上述极值必要条件，有

$${\displaystyle f(x)=f(x_{0})+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,}$$

若${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点处取得极小值，则要求在${\displaystyle x=x_{0}\,}$某一邻域内一切点${\displaystyle x\,}$都必须满足

$${\displaystyle f(x)-f(x_{0})>0\,}$$

即要求

$${\displaystyle {\frac {f''(x_{0})}{2!}}\Delta x^{2}>0\,}$$

亦即要求

$${\displaystyle f''(x_{0})>0\,}$$

${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点处取得极大值的讨论与之类似。于是有极值充分条件：

设一元函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$点处具有二阶导数，且${\displaystyle f'(x_{0})=0\,}$，${\displaystyle f''(x_{0})\neq 0\,}$，则

1. 当${\displaystyle f''(x_{0})>0\,}$时，函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$处取得极小值；
2. 当${\displaystyle f''(x_{0})<0\,}$时，函数${\displaystyle f(x)\,}$在${\displaystyle x=x_{0}\,}$处取得极大值。

而当${\displaystyle f''(x_{0})=0\,}$时，无法直接判断，还需要逐次检验其更高阶导数的正负号。由此有一个规律：若其开始不为零的导数阶数为偶数，则驻点是极值点；若为奇数，则为拐点，而不是极值点。

对于二元函数${\displaystyle f(x_{1},x_{2})\,}$，在给定区域内某${\displaystyle x_{0}(x_{10},x_{20})\,}$点处可导，并在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处取得极值，其必要条件是

$${\displaystyle f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,}$$

即

$${\displaystyle \nabla f(x_{0})=0\,}$$

同样，这只是必要条件，要进一步判断${\displaystyle x_{0}(x_{10},x_{20})\,}$是否为极值点需要找到取得极值的充分条件。根据函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的泰勒展开式，考虑到上述极值必要条件，有

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{0})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{0})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{0})\Delta x_{2}^{2}]+\cdots \,}$$

设${\displaystyle A=f_{x_{1}x_{1}}(x_{0})\,}$，${\displaystyle B=f_{x_{1}x_{2}}(x_{0})\,}$，${\displaystyle C=f_{x_{2}x_{2}}(x_{0})\,}$，则

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[A\Delta x_{1}^{2}+2B\Delta x_{1}\Delta x_{2}+C\Delta x_{2}^{2}]+\cdots \,}$$

或

$${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]+\cdots \,}$$

若${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处取得极小值，则要求在${\displaystyle x_{0}(x_{10},x_{20})\,}$某一邻域内一切点${\displaystyle x\,}$都必须满足

$${\displaystyle f(x_{1},x_{2})-f(x_{10},x_{20})>0\,}$$

即要求

$${\displaystyle {\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]>0\,}$$

亦即要求${\displaystyle A>0\,}$，${\displaystyle AC-B^{2}>0\,}$

即
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}-({\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}})^{2}\end{bmatrix}}_{x_{0}}>0\,}$$

此条件反映了${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的黑塞矩阵${\displaystyle G(x_{0})\,}$的各阶主子式都大于零，即对于

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

要求
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle |G(x_{0})|={\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

${\displaystyle f((x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处取得极大值的讨论与之类似。于是有极值充分条件：

设二元函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点的邻域内连续且具有一阶和二阶连续偏导数，又有${\displaystyle f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,}$，同时令${\displaystyle A=f_{x_{1}x_{1}}(x_{0})\,}$，${\displaystyle B=f_{x_{1}x_{2}}(x_{0})\,}$，${\displaystyle C=f_{x_{2}x_{2}}(x_{0})\,}$，则

1. 当${\displaystyle A>0\,}$，${\displaystyle AC-B^{2}>0\,}$时，函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$处取得极小值；
2. 当${\displaystyle A<0\,}$，${\displaystyle AC-B^{2}>0\,}$时，函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$处取得极大值。

此外可以判断，当${\displaystyle AC-B^{2}<0\,}$时，函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处没有极值，此点称为鞍点。而当${\displaystyle AC-B^{2}=0\,}$时，无法直接判断，对此，补充一个规律：当${\displaystyle AC-B^{2}=0\,}$时，如果有${\displaystyle A\equiv 0\,}$，那么函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$有极值，且当${\displaystyle C>0\,}$有极小值，当${\displaystyle C<0\,}$有极大值。

由线性代数的知识可知，若矩阵${\displaystyle G(x_{0})\,}$满足
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

则矩阵${\displaystyle G(x_{0})\,}$是正定矩阵，或者说矩阵${\displaystyle G(x_{0})\,}$正定。

若矩阵${\displaystyle G(x_{0})\,}$满足
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}<0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

则矩阵${\displaystyle G(x_{0})\,}$是负定矩阵，或者说矩阵${\displaystyle G(x_{0})\,}$负定。^[3]

于是，二元函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处取得极值的条件表述为：二元函数${\displaystyle f(x_{1},x_{2})\,}$在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的黑塞矩阵正定，则取得极小值；在${\displaystyle x_{0}(x_{10},x_{20})\,}$点处的黑塞矩阵负定，则取得极大值。

对于多元函数${\displaystyle f(x_{1},x_{2},\cdots ,x_{n})\,}$，若在${\displaystyle x_{0}(x_{1},x_{2},\cdots ,x_{n})\,}$点处取得极值，则极值存在的必要条件为

$${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}=0\,}$$

取得极小值的充分条件为

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

正定，即要求${\displaystyle G(x_{0})\,}$的各阶主子式都大于零，即
$${\displaystyle \left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,}$$

$${\displaystyle {\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,}$$

$${\displaystyle \vdots }$$

$${\displaystyle |G(x_{0})|>0\,}$$
取得极大值的充分条件为

$${\displaystyle G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,}$$

负定。^[4][5][6]

拓展阅读

• 雅可比矩阵
• 梯度