ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL · 2020 · 54 citations · 24 references
ComputingComputer ScienceData ScienceAlgorithmsArtificial IntelligenceMachine LearningDeep LearningGradient-based OptimizationNeural NetworksOptimization For Machine LearningPattern RecognitionAccuracyAdam OptimizationComparative Data AnalysisNeural Network ModelingOptimization Algorithms
Adopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimization algorithms namely: sgd, rmsprop, adagrad, adadelta, adam, adamax and nadam on four unrelated datasets discretely, to conclude which one dispenses the best accuracy, efficiency and performance to our deep neural network. This work will provide insightful analysis to a data scientist in choosing the best optimizer while modelling their deep neural network.
24
A Stochastic Approximation Method
Herbert Robbins, Sutton Monro · The Annals of Mathematical Statistics · 1951 · 9.6K citations · Full text
Review of Deep Learning Algorithms and Architectures
Ajay Shrestha, Ausif Mahmood · IEEE Access · 2019 · 1.8K citations · Full text