DeepAI AI Chat
Log In Sign Up

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

by   Xueli Xiao, et al.
Georgia State University

Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with deep knowledge, and trials and errors. Genetic algorithms have been used in hyperparameter optimizations. However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth. As the depth increases, the number of hyperparameters grows exponentially, and searching becomes exponentially harder. It is important to have an efficient algorithm that can find a good model in reasonable time. In this article, we propose to use a variable length genetic algorithm (GA) to systematically and automatically tune the hyperparameters of a CNN to improve its performance. Experimental results show that our algorithm can find good CNN hyperparameters efficiently. It is clear from our experiments that if more time is spent on optimizing the hyperparameters, better results could be achieved. Theoretically, if we had unlimited time and CPU power, we could find the optimized hyperparameters and achieve the best results in the future.


Hybrid Genetic Algorithm and Hill Climbing Optimization for the Neural Network

In this paper, we propose a hybrid model combining genetic algorithm and...

Deep Genetic Network

Optimizing a neural network's performance is a tedious and time taking p...

Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

Most learning algorithms require the practitioner to manually set the va...

A Study of Genetic Algorithms for Hyperparameter Optimization of Neural Networks in Machine Translation

With neural networks having demonstrated their versatility and benefits,...

HOAX: A Hyperparameter Optimization Algorithm Explorer for Neural Networks

Computational chemistry has become an important tool to predict and unde...

Genetic CNN

The deep Convolutional Neural Network (CNN) is the state-of-the-art solu...

Genetic Algorithm based hyper-parameters optimization for transfer Convolutional Neural Network

Hyperparameter optimization is a challenging problem in developing deep ...