IET Computer Vision · 2021 · 64 citations · 20 references
Convolutional Neural NetworkEngineeringMachine LearningNeural Networks (Machine Learning)Recurrent Neural NetworkSocial SciencesMobile Neural NetworksImage ClassificationImage AnalysisPattern RecognitionSparse Neural NetworkEmbedded Machine LearningVision RecognitionMachine VisionImage Classification (Visual Culture Studies)Abstract LightweightComputer EngineeringComputer ScienceNeural Networks (Computational Neuroscience)High Convergence SpeedNeural Architecture SearchComputer VisionSmooth Activation FunctionDeep Neural NetworksCategorizationBrain-like ComputingLightweight Neural NetworksImage Classification (Electrical Engineering)
Abstract Lightweight or mobile neural networks used for real‐time computer vision tasks contain fewer parameters than normal networks, which lead to a constrained performance. Herein, a novel activation function named as Tanh Exponential Activation Function (TanhExp) is proposed which can improve the performance for these networks on image classification task significantly. The definition of TanhExp is f ( x ) = x tanh( e x ). The simplicity, efficiency, and robustness of TanhExp on various datasets and network models is demonstrated and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour also remains stable even with noise added and dataset altered. It is shown that without increasing the size of the network, the capacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters added.
20
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 214.9K citations · Full text
Image Classification, Deep Neural Networks, Machine Vision +14
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 56.5K citations · Full text
Engineering, Machine Learning, Multilayer Neural Networks +17
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.2K citations
Squeeze-and-Excitation Networks
Jie Hu, Li Shen, Gang Sun · 2018 · 26.8K citations
Convolutional Neural Network, Machine Vision, Machine Learning +13