Procedia Computer Science · 2021 · 15 citations · 16 references
ComputingComputer ScienceElectrical EngineeringAlgorithmsMachine LearningOptimizationLoss FunctionNeural NetworksAdam OptimizationElectrical Impedance TomographyExponential DecayExponential Decay ModelsHeuristic AlgorithmsMoment EstimationOptimization AlgorithmsSupport Vector Machine Classification
The Fish School Search (FSS) algorithm is a heuristic technique for finding globally optimal solutions. This algorithm is characterized by its simplicity in implementation, and high performance. Since the first mention of FSS, this effective optimization algorithm has been of a great interest among researches and practitioners around the globe. Modifications of FSS exist, applied to solve practical problems, including image reconstruction in electrical impedance tomography, finding optimal solutions in assembly line balancing problems, neural network structure optimization. In this paper, we consider a modification of the FSS algorithm, which uses chaos theory to generate uniformly distributed pseudorandom numbers, and incorporates exponential step decay. The described modified optimization algorithm is known as ETFSS, and is characterized by faster convergence speed and better performance. In order to further investigate the performance of the novel optimization algorithm, we apply ETFSS to neural network loss function optimization. In addition, we compare the described approach with other machine learning techniques, such as the support vector machine (SVM) algorithm, k-nearest neighbors (KNN) algorithm and back propagation-based neural network, trained using the adaptive moment estimation (Adam) optimizer. We visualize classification results using T-distributed stochastic neighbor embedding (TSNE) method, and uniform manifold approximation and projection (UMAP) method, in order to provide more details considering classification performance and dataset shape. The obtained results confirm, that ETFSS can produce slightly more accurate classifications when compared to backpropagation.
16
Laurens van der Maaten, Geoffrey E. Hinton · Journal of Machine Learning Research · 2008 · 35.7K citations
Yuhai Wu, Vladimir Vapnik · Technometrics · 1999 · 27K citations
UMAP: Uniform Manifold Approximation and Projection
Leland McInnes, John Healy, Nathaniel Saul et al. · The Journal of Open Source Software · 2018 · 9.6K citations · Full text
Simple mathematical models with very complicated dynamics
Robert M. May · Nature · 1976 · 6.9K citations