Inference of a rule by a neural network with thermal noise

Abstract

Learning and generalization by a perceptron are described within a statistical-mechanical framework. In the specific case considered here, the goal of learning is to infer the properties of a reference perceptron from examples. As the number of examples is increased a transition to optimal learning at finite temperature is found: The generalization error can be decreased by adding thermal noise to the synaptic coupling parameters. Although the transition is weak, significant improvement can be achieved further beyond the threshold.