Backpropagation in linear arrays-a performance analysis and optimization

Abstract

Neural networks are valuable tools for the support of a wide range of image processing applications. For video-rate operation, special-purpose parallel hardware is often necessary. One of the most common architectures used for this purpose is the linear systolic array. The design and implementation of multi-layer neural networks in linear systolic arrays can be complex, however. This paper demonstrates that the smallest network is not necessarily the best in terms of learning or recall times. Furthermore, this paper shows that the manner in which networks are mapped into a particular hardware structure affects both the performance of the application and the efficiency with which the hardware resources are used. We analyze and identify how to best structure neural networks to optimize network performance for throughput, latency and the efficiency with which the hardware is used. We use the HANNIBAL neural network processor as a research vehicle for these investigations and demonstrate the value of the proposed techniques by a number of example applications.