A geometric approach to learning in neural networks

Abstract

A geometric view is presented of how information is processed in feedforward networks of linear threshold units. The role played by the hidden units is described in terms of the complementary notions of contraction and dimension expansion. A tight sufficient condition for the representation of an arbitrary Boolean function is given by introducing regular partitionings. Learning is interpreted as a general search procedure seeking a custom-made minimal architecture for a given but otherwise arbitrary function or set of examples. A new class of learning algorithms is introduced, which provide a suboptimal solution in a polynomial number of steps. The results of several experiments on the storage and on the rule extraction abilities of three-layer perceptrons are presented. When the input patterns are strongly correlated, simple neuronal structures with good generalization properties emerge.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>