Genetic algorithm for automatic tuning of neural network hyperparameters

Abstract

Artificial neural networks affect our everyday life. But every neural network depends on the appropriate training set and setting of internal properties with hyperparameters. Even accurate and complete training set doesnt imply high performance of neural network algorithm. Tuning of hyperparameters is crucial for correct functionality, fast learning and high accuracy of neural networks. The hyperparameter selection relies on manual fine-tuning based on multiple full training trials. There are a lot of neural network implementation available for public and commercial use, but the setting of hyperparameters is often a neglected problem. Choosing the best structure and hyperparameters is the primary challenge in designing a neural network. This article describes a genetic algorithm for automatic selection of hyperparameters and their tuning for increasing the performance of neural networks without human interaction. The optimization algorithm accelerates the discovery of configuration, which is otherwise a time-consuming task. We evaluate the results of optimizations in comparison to naïve approach and compare pro and cons of different techniques.