IEEE Signal Processing Magazine · 2017 · 78 citations · 7 references
ComputingComputer ScienceComputer VisionFeature EngineeringMachine LearningDeep LearningNeural NetworksPattern RecognitionRepresentation LearningSupervised LearningBackpropagationConvolutional Neural NetworksMachine Learning Classification
There is a resurging interest in developing a neural-network-based solution to the supervised machine-learning problem. The convolutional neural network (CNN) will be studied in this lecture note. We introduce a rectified-correlations on a sphere (RECOS) [1] transform as a basic building block of CNNs. It consists of two main concepts: 1) data clustering on a sphere and 2) rectification. We then interpret a CNN as a network that implements the guided multilayer RECOS transform with two highlights. First, we compare the traditional single-layer and modern multilayer signal-analysis approaches, point out key areas that enable the multilayer approach, and provide a full explanation to the operating principle of CNNs. Second, we discuss how guidance is provided by labels through backpropagation (BP) in the training.
7
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Multilayer feedforward networks are universal approximators
Kurt Hornik, Maxwell B. Stinchcombe, Halbert White · Neural Networks · 1989 · 21.2K citations