An Online and Scalable Model for Generalized Sparse Nonnegative Matrix Factorization in Industrial Applications on Multi-GPU

Abstract

Generalized sparse nonnegative matrix factorization (SNMF) has been proven useful in extracting information and representing sparse data with various types of probabilistic distributions from industrial applications, e.g., recommender systems and social networks.However, current solution approaches for generalized SNMF are based on the manipulation of whole sparse matrices and factor matrices, which will result in large-scale intermediate data.Thus, these approaches cannot describe the high-dimensional and sparse matrices in mainstream industrial and big data platforms, e.g., graphics processing unit (GPU) and multi-GPU, in an online and scalable manner. To overcome these issues, an online, scalable, and single-thread-based SNMF for CUDA parallelization on GPU (CUSNMF) and multi-GPU (MCUSNMF) is proposed in this article. First, theoretical derivation is conducted, which demonstrates that the CUSNMF depends only on the products and sums of the involved feature tuples. Next, the compactness, which can follow the sparsity pattern of sparse matrices, endows the CUSNMF with online learning capability and the fine granularity gives it high parallelization potential on GPU and multi-GPU. Finally, the performance results on several real industrial datasets demonstrate the linear scalability of the time overhead and the space requirement and the validity of the extension to online learning. Moreover, CUSNMF obtains speedup of 7X on a P100 GPU compared to that of the state-of-the-art parallel approaches on a shared memory platform.