More Tensor Lingo

Continuing from last time, we will be talking more about tensor notations.

n-mode Vectors

Let $\mathcal{A}$ be an $I_1 \times I_2 \times \dotsb \times I_N$ tensor. An $n$ -mode vector of $\mathcal{A}$ is an $I_n$ -vector obtained from $\mathcal{A}$ by fixing indices $i_1, i_2, \dotsc, i_{n-1}, i_{n+1}, \dotsc, i_N$ but let $i_n$ vary.

Example 1: Let $\mathcal{A}$ be a $2\times 2 \times 2$ tensor with $a_{111} = 0$ , $a_{112} = 1$ , $a_{121} = 2$ , $a_{122} = 3$ , $a_{211} = 4$ , $a_{212} = 5$ , $a_{221} = 6$ , and $a_{222} = 7$ .

Fixing $i_1 = 2$ , $i_2 = 1$ , and letting $i_3$ vary, we get the 3-mode vector $(4,5)^T$ .

Fixing $i_2 = 1$ , $i_3 = 1$ , and letting $i_1$ vary, we get the 1-mode vector $(0,4)^T$ .

Fixing $i_3 = 2$ , $i_1 = 2$ , and letting $i_2$ vary, we get the 2-mode vector $(5,7)^T$ .

Matrix Unfolding

Matrix unfolding is a way to represent a tensor as a matrix. The concept is easy to grasp, but tedious to describe mathematically. So let us start with some examples on 3-mode tensors.

Let $\mathcal{A}$ be an $I_1 \times I_2 \times I_3$ tensor. It has three different matrix unfoldings $A_{(1)}, A_{(2)},$ and $A_{(3)}$ , corresponding to its three modes. $A_{(1)}$ has $I_1$ rows and $I_2I_3$ columns where the columns are the 1-mode vectors of $\mathcal{A}$ . The columns are ordered so that the index $i_3$ varies faster than $i_2$ . So, the first column corresponds to the 1-mode vector where $i_2 = i_3 = 1$ . The second column corresponds to $i_2 = 1$ and $i_3 = 2$ . The $(I_3+1)$ -st column corresponds to $i_2 = 2$ and $i_3 = 1$ . In general, the 1-mode vector with $i_2 = a$ and $i_3 = b$ appears as the $((a-1)I_2 + b)$ -th column.

Analogously, $A_{(2)}$ has $I_2$ rows and $I_3 I_1$ columns. Here, the 2-mode vector with $i_3 = a$ and $i_1 = b$ appears as the $((a-1)I_3 + b)$ -th column. Notice that $i_1$ varies faster than $i_3$ . In general, when we say that $A_{(2)}$ (or any other unfolded matrices) is of size $I_2 \times I_3 I_1$ , we mean that $i_1$ (the number to the right) varies faster than $i_3$ (the number of the left).

A similar situation goes for $A_{(3)}$ . It has $I_3$ row and $I_1 I_2$ columns with index $i_1$ varies more slowly than $i_2$ .

Example 2: Let $\mathcal{A}$ be the tensor in Example 1. The three matrix unfoldings of $\mathcal{A}$ are given below:

$A_{(1)} = \begin{bmatrix} 0 & 1 & 2 & 3\\ 4 & 5 & 6 & 7\end{bmatrix}$

$A_{(2)} = \begin{bmatrix} 0 & 4 & 1 & 5\\ 2 & 6 & 3 & 7\end{bmatrix}$

$A_{(3)} = \begin{bmatrix} 0 & 2 & 4 & 6\\ 1 & 3 & 5 & 7\end{bmatrix}$

In general, a matrix unfolding of an $I_1 \times I_2 \times \dotsb \times I_N$ tensor $\mathcal{A}$ along the $n$ -mode is an $I_n \times I_{n+1} I_{n+2} \dotsm I_N I_1 I_2 \dotsm I_{n-1}$ matrix $A_{(n)}$ whose columns are the $n$ -mode vectors, and the columns are arranged such that the index $i_{n-1}$ varies the fastest and $i_{n+1}$ the slowest. To be pedantic, we can specify exactly where the element $a_{i_1 i_2 \dotsm i_N}$ appears in $\mathcal{A}$ . It appears on Row $i_n$ and on Column

$\begin{matrix} (i_{n+1}-1) I_{n+2} I_{n+3} I_{n+4} \dotsm I_N I_1 I_2 \dotsm I_{n-1} + \\ (i_{n+2}-1) I_{n+3} I_{n+4} \dotsm I_N I_1 I_2 \dotsm I_{n-1} + \\ (i_{n+3}-1) I_{n+4} \dotsm I_N I_1 I_2 \dotsm I_{n-1} + \\ \vdots \\ (i_{n-3}-1) I_{n-2} I_{n-1}+ \\ (i_{n-2}-1) I_{n-1} + \\ i_{n-1} \end{matrix}$