2017 · 778 citations · 17 references
ComputingComputer ScienceComputer SystemsDistributed SystemsMachine LearningSensor FusionEdge ArchitectureEdge DevicesEdge-cloud ArchitectureEdge-cloud IntegrationFog ComputingSensor NetworksDistributed Sensor FusionDistributed Training At The EdgeFault Tolerance MechanismsFog Computing ArchitectureOn-device InferenceSensor Data Processing
DDNNs enable fast, localized inference on edge and end devices while also supporting full DNN inference in the cloud, and can scale in network size and geographic reach when built on a scalable distributed hierarchy. The paper proposes distributed deep neural networks that span cloud, edge, and end devices. DDNNs are implemented by mapping DNN sections onto the hierarchy and jointly training them to reduce communication and resource use while maximizing feature usefulness for cloud processing. Experiments show DDNNs improve sensor fusion, fault tolerance, and data privacy, boost object recognition accuracy by leveraging geographic diversity, and cut communication costs by over 20× while maintaining high accuracy.
We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. In implementing a DDNN, we map sections of a DNN onto a distributed computing hierarchy. By jointly training these sections, we minimize communication and resource usage for devices and maximize usefulness of extracted features which are utilized in the cloud. The resulting system has built-in support for automatic sensor fusion and fault tolerance. As a proof of concept, we show a DDNN can exploit geographical diversity of sensors to improve object recognition accuracy and reduce communication cost. In our experiment, compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.
17
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text