A Surrogate-Assisted Highly Cooperative Coevolutionary Algorithm for Hyperparameter Optimization in Deep Convolutional Neural Network

by   An Chen, et al.

Convolutional neural networks (CNNs) have gained remarkable success in recent years. However, their performance highly relies on the architecture hyperparameters, and finding proper hyperparameters for a deep CNN is a challenging optimization problem owing to its high-dimensional and computationally expensive characteristics. Given these difficulties, this study proposes a surrogate-assisted highly cooperative hyperparameter optimization (SHCHO) algorithm for chain-styled CNNs. To narrow the large search space, SHCHO first decomposes the whole CNN into several overlapping sub-CNNs in accordance with the overlapping hyperparameter interaction structure and then cooperatively optimizes these hyperparameter subsets. Two cooperation mechanisms are designed during this process. One coordinates all the sub-CNNs to reproduce the information flow in the whole CNN and achieve macro cooperation among them, and the other tackles the overlapping components by simultaneously considering the involved two sub-CNNs and facilitates micro cooperation between them. As a result, a proper hyperparameter configuration can be effectively located for the whole CNN. Besides, SHCHO also employs the well-performing surrogate technique to assist in the hyperparameter optimization of each sub-CNN, thereby greatly reducing the expensive computational cost. Extensive experimental results on two widely-used image classification datasets indicate that SHCHO can significantly improve the performance of CNNs.


page 1

page 2

page 3

page 4


Improving Multi-fidelity Optimization with a Recurring Learning Rate for Hyperparameter Tuning

Despite the evolution of Convolutional Neural Networks (CNNs), their per...

Evaluating Generic Auto-ML Tools for Computational Pathology

Image analysis tasks in computational pathology are commonly solved usin...

Multi-level CNN for lung nodule classification with Gaussian Process assisted hyperparameter optimization

This paper investigates lung nodule classification by using deep neural ...

SA-CNN: Application to text categorization issues using simulated annealing-based convolutional neural network optimization

Convolutional neural networks (CNNs) are a representative class of deep ...

Surrogate-assisted cooperative signal optimization for large-scale traffic networks

Reasonable setting of traffic signals can be very helpful in alleviating...

Surrogate Model Assisted Cooperative Coevolution for Large Scale Optimization

It has been shown that cooperative coevolution (CC) can effectively deal...

Hyperparameter Optimization with Differentiable Metafeatures

Metafeatures, or dataset characteristics, have been shown to improve the...

Please sign up or login with your details

Forgot password? Click here to reset