A New Compensatory Genetic Algorithm-Based Method for Effective Compressed Multi-function Convolutional Neural Network Model Selection with Multi-Objective Optimization

06/08/2019
by   Luna M. Zhang, et al.
0

In recent years, there have been many popular Convolutional Neural Networks (CNNs), such as Google's Inception-V4, that have performed very well for various image classification problems. These commonly used CNN models usually use the same activation function, such as RELU, for all neurons in the convolutional layers; they are "Single-function CNNs." However, SCNNs may not always be optimal. Thus, a "Multi-function CNN" (MCNN), which uses different activation functions for different neurons, has been shown to outperform a SCNN. Also, CNNs typically have very large architectures that use a lot of memory and need a lot of data in order to be trained well. As a result, they tend to have very high training and prediction times too. An important research problem is how to automatically and efficiently find the best CNN with both high classification performance and compact architecture with high training and prediction speeds, small power usage, and small memory size for any image classification problem. It is very useful to intelligently find an effective, fast, energy-efficient, and memory-efficient "Compressed Multi-function CNN" (CMCNN) from a large number of candidate MCNNs. A new compensatory algorithm using a new genetic algorithm (GA) is created to find the best CMCNN with an ideal compensation between performance and architecture size. The optimal CMCNN has the best performance and the smallest architecture size. Simulations using the CIFAR10 dataset showed that the new compensatory algorithm could find CMCNNs that could outperform non-compressed MCNNs in terms of classification performance (F1-score), speed, power usage, and memory usage. Other effective, fast, power-efficient, and memory-efficient CMCNNs based on popular CNN architectures will be developed for image classification problems in important real-world applications, such as brain informatics and biomedical imaging.

READ FULL TEXT
research
05/30/2018

Multi-function Convolutional Neural Networks for Improving Image Classification Performance

Traditional Convolutional Neural Networks (CNNs) typically use the same ...
research
09/29/2019

GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm

Convolutional Neural Networks (CNNs) have gained a significant attractio...
research
10/16/2017

Entanglement Entropy of Target Functions for Image Classification and Convolutional Neural Network

The success of deep convolutional neural network (CNN) in computer visio...
research
11/20/2019

Localized Compression: Applying Convolutional Neural Networks to Compressed Images

We address the challenge of applying existing convolutional neural netwo...
research
02/13/2023

A Domain Decomposition-Based CNN-DNN Architecture for Model Parallel Training Applied to Image Recognition Problems

Deep neural networks (DNNs) and, in particular, convolutional neural net...
research
12/07/2017

Distributed learning of CNNs on heterogeneous CPU/GPU architectures

Convolutional Neural Networks (CNNs) have shown to be powerful classific...

Please sign up or login with your details

Forgot password? Click here to reset