Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

06/23/2020
by   Xueli Xiao, et al.
0

Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with deep knowledge, and trials and errors. Genetic algorithms have been used in hyperparameter optimizations. However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth. As the depth increases, the number of hyperparameters grows exponentially, and searching becomes exponentially harder. It is important to have an efficient algorithm that can find a good model in reasonable time. In this article, we propose to use a variable length genetic algorithm (GA) to systematically and automatically tune the hyperparameters of a CNN to improve its performance. Experimental results show that our algorithm can find good CNN hyperparameters efficiently. It is clear from our experiments that if more time is spent on optimizing the hyperparameters, better results could be achieved. Theoretically, if we had unlimited time and CPU power, we could find the optimized hyperparameters and achieve the best results in the future.

READ FULL TEXT
research
08/24/2023

Hybrid Genetic Algorithm and Hill Climbing Optimization for the Neural Network

In this paper, we propose a hybrid model combining genetic algorithm and...
research
11/05/2018

Deep Genetic Network

Optimizing a neural network's performance is a tedious and time taking p...
research
07/19/2018

Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

Most learning algorithms require the practitioner to manually set the va...
research
09/15/2020

A Study of Genetic Algorithms for Hyperparameter Optimization of Neural Networks in Machine Translation

With neural networks having demonstrated their versatility and benefits,...
research
02/01/2023

HOAX: A Hyperparameter Optimization Algorithm Explorer for Neural Networks

Computational chemistry has become an important tool to predict and unde...
research
03/04/2017

Genetic CNN

The deep Convolutional Neural Network (CNN) is the state-of-the-art solu...
research
02/26/2021

Genetic Algorithm based hyper-parameters optimization for transfer Convolutional Neural Network

Hyperparameter optimization is a challenging problem in developing deep ...

Please sign up or login with your details

Forgot password? Click here to reset