Tensor train compression and truncation error — Examples

Set $W=e_2\otimes e_1\otimes e_1+e_1\otimes e_2\otimes e_1+e_1\otimes e_1\otimes e_2$. It has an explicit three-term CP expression, so its CP rank is at most three. Its first-mode slices are $W_1=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right)$ and $W_2=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)$. They are linearly independent, and $\det (x{W}_1+y{W}_2)=-x^2$.

Suppose $W$ had a two-term expression $a\otimes b\otimes c+a^{\mathrm{\prime }}\otimes b^{\mathrm{\prime }}\otimes c^{\mathrm{\prime }}$. The first-mode slices being independent forces $a,a^{\mathrm{\prime }}$ to be independent. If $b,b^{\mathrm{\prime }}$ or $c,c^{\mathrm{\prime }}$ were dependent, every linear combination of the two slices would have matrix rank at most one, so its determinant would vanish identically, contrary to $\det (x{W}_1+y{W}_2)=-x^2$. Thus both pairs are independent. For any $x,y$, the pencil slice is $(x{a}_1+y{a}_2)b{c}^{\mathsf{T}}+(x{a}_1^{\mathrm{\prime }}+y{a}_2^{\mathrm{\prime }})b^{\mathrm{\prime }}{c}^{\mathrm{\prime }\mathsf{T}}$. Its determinant is a nonzero constant times $(x{a}_1+y{a}_2)(x{a}_1^{\mathrm{\prime }}+y{a}_2^{\mathrm{\prime }})$. Independence of $a,a^{\mathrm{\prime }}$ makes the two linear factors nonproportional, so this determinant has two distinct zero lines; it cannot equal $-x^2$, which vanishes on the repeated line $x=0$. This contradiction proves ${\mathrm{rank}}_{CP}(W)>2$, hence it is exactly three.

For every $\epsilon >0$, define $W_{\epsilon }={\epsilon }^{-1}((e_1+\epsilon e_2{)}^{\otimes 3}-e_1^{\otimes 3})$. This is a difference of two elementary tensors, with the scalar absorbed into one factor, so its CP rank is at most two. Expanding gives $W_{\epsilon }=W+\epsilon (e_2\otimes e_2\otimes e_1+e_2\otimes e_1\otimes e_2+e_1\otimes e_2\otimes e_2)+{\epsilon }^2{e}_2^{\otimes 3}$, so $\mathrm{\parallel }{W}_{\epsilon }-W{\mathrm{\parallel }}_F\to 0$. Thus the infimal rank-two approximation error is zero. Since step 2.1 shows $W$ itself has rank three, no rank-at-most-two tensor attains that infimum.

At cut $k$, the unfolding is the outer product of the vector with entries ${\prod }_{j=1}^k{u}_{i_j}^{(j)}$ and the vector with entries ${\prod }_{j=k+1}^d{u}_{i_j}^{(j)}$. Both vectors are nonzero, so this matrix has rank one.

Set $G_k(1,i_k,1)=u_{i_k}^{(k)}$ for each core. Their contraction is ${\prod }_k{u}_{i_k}^{(k)}=A_{i_1,\dots ,i_d}$, so this is an exact TT with every internal bond size one. By [F3] these sizes are minimal; full-rank TT-SVD therefore also returns this exact rank-one structure.

With columns ordered by $(i_2,i_3)=(1,1),(1,2),(2,1),(2,2)$, the first unfolding is $\left(\begin{array}{cccc}3& 0& 0& 1\\ 0& 2& 0& 0\end{array}\right)$. Its rows are orthogonal with norms $\sqrt{10}$ and $2$, so the leading singular value is $\sqrt{10}$ and the first discarded tail is ${\delta }_1=2$. The retained left vector is $(1,0{)}^{\mathsf{T}}$, and the remainder ${\mathrm{\Sigma }}_1{V}_1^{\ast }$ reshaped as a $2\times 2$ matrix is $\left(\begin{array}{cc}3& 0\\ 0& 1\end{array}\right)$.

The second remainder has singular values $3$ and $1$, hence its rank-one truncation discards ${\delta }_2=1$ and returns the tensor $\widehat{A}=3e_1\otimes e_1\otimes e_1$. The omitted entries of $A-\widehat{A}$ are $e_1\otimes e_2\otimes e_2$ and $2e_2\otimes e_1\otimes e_2$, which are orthogonal and have squared norms $1$ and $4$. Therefore $\mathrm{\parallel }A-\widehat{A}{\mathrm{\parallel }}_F=\sqrt{5}=\sqrt{{\delta }_1^2+{\delta }_2^2}$, realizing equality in [F3].

For each cut $k$, only the $n$ row tuples $(j,\dots ,j)$ and the matching $n$ column tuples $(j,\dots ,j)$ contain nonzero entries of $A^{⟨k⟩}$. The resulting matrix has $n$ ones in distinct rows and columns and zeros elsewhere, so its rank is exactly $n$.

By [F3], every exact TT representation has bond size at least $n$ at every cut. Such a representation is explicit: the first core enforces ${\alpha }_1=i_1$, each interior core enforces ${\alpha }_{k-1}=i_k={\alpha }_k$, and the last core enforces ${\alpha }_{d-1}=i_d$. Contracting these Kronecker-delta entries yields $1$ exactly when all mode indices are the same and $0$ otherwise, which is the stated tensor.