Feature Discovery by Competitive Learning*

TLDR

Competitive learning is a nonassociative statistical scheme that discovers salient general features for pattern classification and may be complemented by other learning mechanisms in more complete networks. This study examines the unsupervised Competitive Learning paradigm through simulation and formal analysis to demonstrate its capacity to enable parallel neuron‑like networks to accomplish useful learning tasks and to deepen understanding of feature discovery in adaptive networks. A simple competitive mechanism identifies feature detectors from stimulus patterns, which can be stacked into multilayer systems for categorizing nonlinearly separable sets, and correlated stimuli act as teaching inputs to develop otherwise unattainable detectors. Competitive learning allows parallel neuron‑like networks to perform many useful learning tasks, though the authors note it is not the sole learning principle.

Abstract

This paper reports the results of our studies with an unsupervised learning paradigm which we have called “Competitive Learning.” We have examined competitive learning using both computer simulation and formal analysis and have found that when it is applied to parallel networks of neuron‐like elements, many potentially useful learning tasks can be accomplished. We were attracted to competitive learning because it seems to provide a way to discover the salient, general features which can be used to classify a set of patterns. We show how a very simply competitive mechanism can discover a set of feature detectors which capture important aspects of the set of stimulus input patterns. We also show how these feature detectors can form the basis of a multilayer system that can serve to learn categorizations of stimulus sets which are not linearly separable. We show how the use of correlated stimuli con serve as a kind of “teaching” input to the system to allow the development of feature detectors which would not develop otherwise. Although we find the competitive learning mechanism a very interesting and powerful learning principle, we do not, of course, imagine thot it is the only learning principle. Competitive learning is an essentially nonassociative statistical learning scheme. We certainly imagine that other kinds of learning mechanisms will be involved in the building of associations among patterns of activation in a more complete neural network. We offer this analysis of these competitive learning mechanisms to further our understanding of how simple adaptive networks can discover features important in the description of the stimulus environment in which the system finds itself.