Kronecker Products and Stack Operator

目录

• 定义
  • Stack Operator
  • Kronecker Product
• 性质
  • Stack Operator
  • Kronecker Product
    • 半线性

Whitcomb L. Notes on Kronecker Products.

定义

Stack Operator

对于任意的矩阵\(A \in \mathbb{R}^{m \times n}\),

\[vec(A) := [A_{00}, A_{10}, \ldots, A_{m-1,n-1}]^T \in \mathbb{R}^{mn},
\]

即按列展开.

Kronecker Product

对于任意的矩阵\(A \in \mathbb{R}^{m\times n }, B \in \mathbb{R}^{p \times q}\),

\[A \otimes B :=
\left [
\begin{array}{ccc}
A_{00} \cdot B & \cdots & A_{0n-1} \cdot B \\
\vdots & \ddots & \vdots \\
A_{m-1,0} \cdot B & \cdots & A_{m-1,n-1} \cdot B
\end{array} \right ] \in \mathbb{R}^{mp \times nq}
\]

性质

Stack Operator

\[\mathrm{Tr}(A^TB) = vec(A)^T vec(B).
\]

Kronecker Product

易知,

\[[A \otimes B]_{ip+s,jq+t} = A_{i,j} \cdot B_{s,t}, \quad i\in [m], s\in[p], j \in [n], t \in [q],
\]

这里\([m] = \{0, 1, \ldots, m-1\}\).

\(a \otimes b = vec(b a^T)\)

• \(a \in \mathbb{R}^m, b \in \mathbb{R}^n\), 则
  \[a \otimes b = vec(b a^T)
  \]

\((A \otimes B)^T = (A^T \otimes B^T)\)

\((A \otimes B)^T = (A^T \otimes B^T)\)是显然的.

故

\[[A \otimes B]_{*, jq+t} = A_{*,j} \otimes B_{*, t} = vec(B_{*, t} A_{*,j}^T) \\
[A \otimes B]_{ip+s, *}^T = A_{i,*}^T \otimes B_{s,*}^T = vec(B^T_{s,*} A_{i,*}).
\]

半线性

• \(A \otimes \alpha B = \alpha A \otimes B = \alpha (A \otimes B).\)

• \[(A+B) \otimes C = A \otimes C +B \otimes C \\
  A \otimes (B+C) = A\otimes B + A \otimes C.
  \]

• \((A \otimes B) \otimes C=A \otimes (B\otimes C)\):

  \[\begin{array}{ll}
  (A \otimes B) \otimes C &= [A_{i,j} \cdot B_{s,t} \cdot C ]\\
  &= A \otimes (B \otimes C).
  \end{array}
  \]

• 通常 \((A \otimes B) \not= (B \otimes A)\).

\((A \otimes B) (C\otimes D) = (AC \otimes BD)\)

\[\begin{array}{ll}
[(A \otimes B) (C\otimes D)]_{ip+s, jq+t}
&= [A \otimes B]_{ip+s, *} [C\otimes D]_{*,jq+t} \\
&= vec(B_{s, *}^TA_{i,*})^T vec(D_{*,t} C_{*,j}^T) \\
&= \mathrm{Tr}(A_{i,*}^TB_{s,*}D_{*,t} C_{*,j}^T) \\
&= \mathrm{Tr}(C_{*,j}^TA_{i,*}^TB_{s,*}D_{*,t}) \\
&= A_{i, *}C_{*,j} \cdot B_{s, *} D_{*,t} \\
&= [AC]_{ij} \cdot [BD]_{st} \\
&= [AC \otimes BD]_{ip+s,jq+t}.
\end{array}
\]

\((A \otimes B)^{-1} = (A^{-1} \otimes B^{-1})\)

条件自然是A, B为满秩方阵:

\[(A \otimes B) (A^{-1} \otimes B^{-1}) = (AA^{-1} \otimes BB^{-1}) = I
\]

\(\mathrm{det}(A_{n\times n} \otimes B_{m \times m}) = \mathrm{det}(A)^m \cdot \mathrm{det}(B)^n\)

就像用普通的高斯消去法将矩阵化为对角型一样, 在对\(A_{n\times n } \otimes B_{m\times m}\)消去的过程中可以发现, \(B\)不会产生丝毫的影响, 结果便是显而易见的了.

\(\mathrm{Tr}(A \otimes B) = \mathrm{Tr}(A) \cdot \mathrm{Tr}(B)\)

\[\mathrm{Tr}(A \otimes B) = \sum_{i=1}^m \sum_{j=1}^n A_iB_j = \mathrm{Tr}(A) \cdot \mathrm{Tr}(B).
\]

\(vec(ABC) = (C^T \otimes A) vec(B)\)

设\(A \in \mathbb{R}^{m\times n}, B \in \mathbb{R}^{n \times p}, C \in \mathbb{R}^{p \times q}\),

\[[vec(ABC)]_{jm+i} = [ABC]_{i,j} = \mathrm{Tr}(A_{i,*}BC_{*,j}) = \mathrm{Tr}(C_{*,j}A_{i,*}B)=vec(A_{i,*}^TC_{*j}^T)^T vec(B) = [C^T \otimes A]_{jm+i,*} vec(B)
\]

特例:

\[Ax = IAx = vec(IAx) = (x^T \otimes I)vec(A)
\]

这个在处理梯度的时候会比较有用:

\[y = Ax
\]

则

\[\mathrm{d}y = (\mathrm{d}A)x + A\mathrm{d}x = (x^T \otimes I) vec(\mathrm{d}A) + A \mathrm{d}x.
\]