2019 · 45 citations · 18 references
ComputingComputer EngineeringComputer ScienceAlgorithmsDistributed SystemsServerless ComputingDistributed AlgorithmsDistributed ComputingDistributed Data ProcessingGradient-based OptimizationGradient Descent
Coded computation techniques provide robustness against straggling servers in distributed computing, with the following limitations: First, they increase decoding complexity. Second, they ignore computations carried out by straggling servers; and they are typically designed to recover the full gradient, and thus, cannot provide a balance between the accuracy of the gradient and per-iteration completion time. Here we introduce a hybrid approach, called coded partial gradient computation (CPGC), that benefits from the advantages of both coded and uncoded computation schemes, and reduces both the computation time and decoding complexity.
18
Speeding Up Distributed Machine Learning Using Codes
Kangwook Lee, Maximilian Lam, Ramtin Pedarsani et al. · IEEE Transactions on Information Theory · 2017 · 866 citations · Full text
Revisiting Distributed Synchronous SGD
Jianmin Chen, Rajat Monga, Samy Bengio et al. · arXiv (Cornell University) · 2017 · 610 citations · Full text