2023 · 22 citations · 27 references
ComputingComputer EngineeringComputer ScienceDistributed SystemsHardware-software Co-designMachine LearningDeep LearningDistributed Data ProcessingDistributed Heterogeneous SystemsHeterogeneous System ArchitectureRecommendation SystemsScalable Machine LearningDeep Learning For RecommendationDeep Learning On GpusDistributed Machine LearningEmbedded Hardware Co-designHardware-software Co-optimizationHigh Bandwidth MemoryMemory BandwidthScalable Recommendation Systems
Deep Learning Recommendation Models (DLRMs) are important applications in various domains and have evolved into one of the largest and most important machine learning applications. With their trillions of parameters necessarily exceeding the high bandwidth memory (HBM) capacity of GPUs, ever more massive DLRMs require large-scale multi-node systems for distributed training and inference. However, these all suffer from the all-to-all communication bottleneck, which limits scalability.
27
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Martı́n Abadi, Ashish Agarwal, Paul Barham et al. · arXiv (Cornell University) · 2016 · 9.8K citations · Full text
Deep Neural Networks for YouTube Recommendations
Paul Covington, Jay Adams, Emre Sargin · 2016 · 3.4K citations · Full text
Two Decades of Recommender Systems at Amazon.com
Brent Smith, Greg Linden · IEEE Internet Computing · 2017 · 652 citations
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Yujun Lin, Song Han, Huizi Mao et al. · arXiv (Cornell University) · 2017 · 647 citations · Full text