Numerical Algorithms for Computing an Arbitrary Singular Value of a Tensor Sum

Abstract

We consider computing an arbitrary singular value of a tensor sum: T:=In⊗Im⊗A+In⊗B⊗Iℓ+C⊗Im⊗Iℓ∈Rℓmn×ℓmn, where A∈Rℓ×ℓ, B∈Rm×m, C∈Rn×n. We focus on the shift-and-invert Lanczos method, which solves a shift-and-invert eigenvalue problem of (TTT−σ˜2Iℓmn)−1, where σ˜ is set to a scalar value close to the desired singular value. The desired singular value is computed by the maximum eigenvalue of the eigenvalue problem. This shift-and-invert Lanczos method needs to solve large-scale linear systems with the coefficient matrix TTT−σ˜2Iℓmn. The preconditioned conjugate gradient (PCG) method is applied since the direct methods cannot be applied due to the nonzero structure of the coefficient matrix. However, it is difficult in terms of memory requirements to simply implement the shift-and-invert Lanczos and the PCG methods since the size of T grows rapidly by the sizes of A, B, and C. In this paper, we present the following two techniques: (1) efficient implementations of the shift-and-invert Lanczos method for the eigenvalue problem of TTT and the PCG method for TTT−σ˜2Iℓmn using three-dimensional arrays (third-order tensors) and the n-mode products, and (2) preconditioning matrices of the PCG method based on the eigenvalue and the Schur decomposition of T. Finally, we show the effectiveness of the proposed methods through numerical experiments.