Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems

Abstract

The following topics are dealt with: learning (artificial intelligence); parallel processing; neural nets; convolutional neural nets; data analysis; stochastic processes; gradient methods; image classification; message passing; evolutionary computation.