MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems

TLDR

MXNet is a multi‑language, computation‑ and memory‑efficient ML library that facilitates deep neural network development across heterogeneous systems from mobile devices to distributed GPU clusters. This paper describes the API design and system implementation of MXNet, explaining how symbolic expressions and tensor operations are unified in a single framework. MXNet embeds symbolic expressions and imperative tensor operations in the host language, providing auto‑differentiation for gradient computation. Preliminary experiments show promising results on large‑scale deep neural network applications using multiple GPU machines.

Abstract

MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends declarative symbolic expression with imperative tensor computation. It offers auto differentiation to derive gradients. MXNet is computation and memory efficient and runs on various heterogeneous systems, ranging from mobile devices to distributed GPU clusters. This paper describes both the API design and the system implementation of MXNet, and explains how embedding of both symbolic expression and tensor operation is handled in a unified fashion. Our preliminary experiments reveal promising results on large scale deep neural network applications using multiple GPU machines.