Network Computation in Neural Systems · 1996 · 490 citations · 16 references
Life SciencesNeuroscienceVision ScienceComputational NeuroscienceVisual NeuroscienceVisual PerceptionNeural CodingNeural NetworksRepresentation LearningVisual PhysiologyVisual Receptive FieldsVisual SystemNatural Image StatisticsNatural Scene StatisticsNeural Representation LearningPrincipal ComponentsReceptive Field EncodingResponse PropertiesVisual Signal ProcessingVisual System Physiology
Natural images exhibit characteristic statistical regularities, including higher‑order structures that linear models cannot capture, making efficient coding a challenge. The study aims to characterize natural image structure and relate it to early visual neuron responses, proposing sparseness maximization as an objective for efficient coding. A sparse‑coding network trained on natural scenes learns localized, oriented, bandpass receptive fields resembling mammalian striate cortex neurons. The network successfully develops such receptive fields, supporting the sparseness objective for efficient coding of natural scenes.
Natural images contain characteristic statistical regularities that set them apart from purely random images. Understanding what these regularities are can enable natural images to be coded more efficiently. In this paper, we describe some of the forms of structure that are contained in natural images, and we show how these are related to the response properties of neurons at early stages of the visual system. Many of the important forms of structure require higher-order (i.e. more than linear, pairwise) statistics to characterize, which makes models based on linear Hebbian learning, or principal components analysis, inappropriate for finding efficient codes for natural images. We suggest that a good objective for an efficient coding of natural scenes is to maximize the sparseness of the representation, and we show that a network that learns sparse codes of natural scenes succeeds in developing localized, oriented, bandpass receptive fields similar to those in the mammalian striate cortex.This paper was presented at the Workshop on Information Theory and the Brain, held at the University of Stirling, UK, on 4–5 September 1995.
16