Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Abstract

Machine learning and data mining algorithms are becoming increasingly\nimportant in analyzing large volume, multi-relational and multi--modal\ndatasets, which are often conveniently represented as multiway arrays or\ntensors. It is therefore timely and valuable for the multidisciplinary research\ncommunity to review tensor decompositions and tensor networks as emerging tools\nfor large-scale data analysis and data mining. We provide the mathematical and\ngraphical representations and interpretation of tensor networks, with the main\nfocus on the Tucker and Tensor Train (TT) decompositions and their extensions\nor generalizations.\n Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker\nmodels, tensor train (TT) decompositions, matrix product states (MPS), matrix\nproduct operators (MPO), basic tensor operations, multiway component analysis,\nmultilinear blind source separation, tensor completion, linear/multilinear\ndimensionality reduction, large-scale optimization problems, symmetric\neigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations,\npseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis\n(CCA) (This is Part 1)\n