Back Propagation is Sensitive to Initial Conditions

Abstract

This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the back-propagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of back propagation to initial weight configuration. Back Propagation is Sensitive to Initial Conditions John F. Kolen Jordan B. Pollack Laboratory for Artificial Intelligence Research Computer and Information Science Department The Ohio State University Columbus, Ohio 43210, USA [e-mail], [e-mail] TR 90-JK-BPSIC Introduction Back Propagation (Rumelhart, Hinton, & Williams, 1986) is the network training method of choice for many neural network...