矩阵求导

$$\frac{\partial(\boldsymbol{x}^T \boldsymbol{a})}{\partial \boldsymbol{x}} = \frac{\partial(\boldsymbol{a}^T \boldsymbol{x})}{\partial \boldsymbol{x}} = \boldsymbol{a}$$

其中，$\boldsymbol{a}$ 为常数向量，$\boldsymbol{a} = (a_1, a_2, \cdots, a_n)^T$

证明

$$\frac{\partial(\boldsymbol{x}^T \boldsymbol{a})}{\partial \boldsymbol{x}} = \frac{\partial(\boldsymbol{a}^T \boldsymbol{x})}{\partial \boldsymbol{x}}$$ $$= \frac{\partial(a_1 x_1 + a_2 x_2 + \cdots + a_n x_n)}{\partial \boldsymbol{x}}$$ $$= \begin{bmatrix}\frac{\partial(a_1 x_1 + a_2 x_2 + \cdots + a_n x_n)}{\partial x_1} \\\frac{\partial(a_1 x_1 + a_2 x_2 + \cdots + a_n x_n)}{\partial x_2} \\\vdots \\\frac{\partial(a_1 x_1 + a_2 x_2 + \cdots + a_n x_n)}{\partial x_n}\end{bmatrix}$$ $$= \begin{bmatrix}a_1 \\a_2 \\\vdots \\a_n\end{bmatrix}$$ $$= \boldsymbol{a}$$

$$\frac{\partial(\boldsymbol{x}^T\boldsymbol{x})}{\partial\boldsymbol{x}} = 2\boldsymbol{x}$$

证明

$$\frac{\partial(\boldsymbol{x}^T\boldsymbol{x})}{\partial\boldsymbol{x}} = \frac{\partial(x_1^2 + x_2^2 + \cdots + x_n^2)}{\partial\boldsymbol{x}}$$ $$= \begin{bmatrix} \frac{\partial(x_1^2 + x_2^2 + \cdots + x_n^2)}{\partial x_1} \\ \frac{\partial(x_1^2 + x_2^2 + \cdots + x_n^2)}{\partial x_2} \\ \vdots \\ \frac{\partial(x_1^2 + x_2^2 + \cdots + x_n^2)}{\partial x_n} \end{bmatrix}$$ $$= \begin{bmatrix} 2x_1 \\ 2x_2 \\ \vdots \\ 2x_n \end{bmatrix}$$ $$= 2\begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}$$ $$= 2\boldsymbol{x}$$

$$\frac{\partial(\boldsymbol{x}^{T}A\boldsymbol{x})}{\partial \boldsymbol{x}} = A\boldsymbol{x} + A^{T}\boldsymbol{x}$$

其中， $A_{n\times n}$ 是常数矩阵， $A_{n\times n} = (a_{ij})_{i=1,j=1}^{n,n}$ 。

证明

$$\frac{\partial (\boldsymbol{x}^T A \boldsymbol{x})}{\partial \boldsymbol{x}} = \frac{\begin{aligned}\partial (&a_{11}x_1 x_1 + a_{12}x_1 x_2 + \cdots + a_{1n}x_1 x_n \\+\ &a_{21}x_2 x_1 + a_{22}x_2 x_2 + \cdots + a_{2n}x_2 x_n \\+\ &\cdots \\+\ &a_{n1}x_n x_1 + a_{n2}x_n x_2 + \cdots + a_{nn}x_n x_n )\end{aligned} }{\partial x}$$ $$= \begin{bmatrix} \frac{\begin{aligned}\partial (&a_{11}x_1 x_1 + a_{12}x_1 x_2 + \cdots + a_{1n}x_1 x_n \\+\ &a_{21}x_2 x_1 + a_{22}x_2 x_2 + \cdots + a_{2n}x_2 x_n \\+\ &\cdots \\+\ &a_{n1}x_n x_1 + a_{n2}x_n x_2 + \cdots + a_{nn}x_n x_n )\end{aligned} }{\partial x_1} \\ \frac{\begin{aligned}\partial (&a_{11}x_1 x_1 + a_{12}x_1 x_2 + \cdots + a_{1n}x_1 x_n \\+\ &a_{21}x_2 x_1 + a_{22}x_2 x_2 + \cdots + a_{2n}x_2 x_n \\+\ &\cdots \\+\ &a_{n1}x_n x_1 + a_{n2}x_n x_2 + \cdots + a_{nn}x_n x_n )\end{aligned} }{\partial x_2} \\ \vdots \\ \frac{\begin{aligned}\partial (&a_{11}x_1 x_1 + a_{12}x_1 x_2 + \cdots + a_{1n}x_1 x_n \\+\ &a_{21}x_2 x_1 + a_{22}x_2 x_2 + \cdots + a_{2n}x_2 x_n \\+\ &\cdots \\+\ &a_{n1}x_n x_1 + a_{n2}x_n x_2 + \cdots + a_{nn}x_n x_n )\end{aligned} }{\partial x_n} \end{bmatrix}$$ $$\begin{aligned}&=\begin{bmatrix}(a_{11}x_1+a_{12}x_2+\cdots+a_{1n}x_n)+(a_{11}x_1+a_{21}x_2+\cdots+a_{n1}x_n)\\(a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n)+(a_{12}x_1+a_{22}x_2+\cdots+a_{n2}x_n)\\\vdots\\(a_{n1}x_1+a_{n2}x_2+\cdots+a_{nn}x_n)+(a_{1n}x_1+a_{2n}x_2+\cdots+a_{nn}x_n)\end{bmatrix}\\&=\begin{bmatrix}a_{11}x_1+a_{12}x_2+\cdots+a_{1n}x_n\\a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n\\\vdots\\a_{n1}x_1+a_{n2}x_2+\cdots+a_{nn}x_n\end{bmatrix}+\begin{bmatrix}a_{11}x_1+a_{21}x_2+\cdots+a_{n1}x_n\\a_{12}x_1+a_{22}x_2+\cdots+a_{n2}x_n\\\vdots\\a_{1n}x_1+a_{2n}x_2+\cdots+a_{nn}x_n\end{bmatrix}\\&=\begin{bmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\a_{21}&a_{22}&\cdots&a_{2n}\\\vdots&\vdots&\ddots&\vdots\\a_{n1}&a_{n2}&\cdots&a_{nn}\end{bmatrix}\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}+\begin{bmatrix}a_{11}&a_{21}&\cdots&a_{n1}\\a_{12}&a_{22}&\cdots&a_{n2}\\\vdots&\vdots&\ddots&\vdots\\a_{1n}&a_{2n}&\cdots&a_{nn}\end{bmatrix}\begin{bmatrix}x_1\\x_2\\\vdots\\x_n\end{bmatrix}\\&=A\boldsymbol{x}+A^T\boldsymbol{x}\end{aligned}$$