Publication | Open Access
The Singular Values of Convolutional Layers
59
Citations
0
References
2018
Year
Convolutional Neural NetworkEngineeringMachine LearningBatch NormalizationFunctional AnalysisDeblurringImage AnalysisSparse Neural NetworkVideo TransformerConvolutional LayerMachine VisionInverse ProblemsDeconvolutionMedical Image ComputingDeep LearningComputer VisionSingularly Perturbed ProblemDeep Residual NetworkConvolutional Layers
We characterize the singular values of the linear transformation associated with a standard 2D multi-channel convolutional layer, enabling their efficient computation. This characterization also leads to an algorithm for projecting a convolutional layer onto an operator-norm ball. We show that this is an effective regularizer; for example, it improves the test error of a deep residual network using batch normalization on CIFAR-10 from 6.2\% to 5.3\%.