arXiv (Cornell University) · 2018 · 14 citations · 6 references
ComputingComputer ScienceData StructuresMachine LearningParallel ComputingData ParallelismDeep LearningMemory Hierarchy OptimizationNeural NetworksParallel ArchitecturesPattern RecognitionAccuracyConvolutional Neural NetworksRuntime PerformanceSparse LearningSparse ModelsStructural ConstraintsUnstructured Models
Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies are much lower than that of dense DNNs on regular parallel hardware such as TPU. This inefficiency leads to poor/no performance benefits for sparse DNNs. Performance issue for sparse DNNs can be alleviated by bringing structure to the sparsity and leveraging it for improving runtime efficiency. But such structural constraints often lead to suboptimal accuracies. In this work, we jointly address both accuracy and performance of sparse DNNs using our proposed class of sparse neural networks called HBsNN (Hierarchical Block sparse Neural Networks). For a given sparsity, HBsNN models achieve better runtime performance than unstructured sparse models and better accuracy than highly structured sparse models.
6
Rethinking the Inception Architecture for Computer Vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe et al. · 2016 · 30.9K citations
Yann LeCun, John S. Denker, Sara A. Solla · 1989 · 2.6K citations
Song Han, Xingyu Liu, Huizi Mao et al. · ACM SIGARCH Computer Architecture News · 2016 · 2K citations
Angshuman Parashar, Minsoo Rhu, Anurag Mukkara et al. · 2017 · 927 citations