Enhancing both generalization and fault tolerance of multilayer neural networks

Abstract

In this paper, we propose the method to enhance both generalization ability and fault tolerance of multilayer neural networks. Many methods that enhance either generalization ability or fault tolerance have been proposed, but very few methods enhance both of them. We discuss the combination of the method for good generalization and the method for high fault tolerance. Avoiding the interference, we propose local augmentation method (LAUG) to enhance fault tolerance. It duplicates hidden units according to the importance of each unit. Since it manipulates a trained network keeping the input-output relation of the network, LAUG does not interfere with any training algorithms to enhance generalization ability. Finally, we show the effectiveness of our method through some experiments.