A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions

by   Pengzhen Ren, et al.
Northwest University

Deep learning has made major breakthroughs and progress in many fields. This is due to the powerful automatic representation capabilities of deep learning. It has been proved that the design of the network architecture is crucial to the feature representation of data and the final performance. In order to obtain a good feature representation of data, the researchers designed various complex network architectures. However, the design of the network architecture relies heavily on the researchers' prior knowledge and experience. Therefore, a natural idea is to reduce human intervention as much as possible and let the algorithm automatically design the architecture of the network. Thus going further to the strong intelligence. In recent years, a large number of related algorithms for Neural Architecture Search (NAS) have emerged. They have made various improvements to the NAS algorithm, and the related research work is complicated and rich. In order to reduce the difficulty for beginners to conduct NAS-related research, a comprehensive and systematic survey on the NAS is essential. Previously related surveys began to classify existing work mainly from the basic components of NAS: search space, search strategy and evaluation strategy. This classification method is more intuitive, but it is difficult for readers to grasp the challenges and the landmark work in the middle. Therefore, in this survey, we provide a new perspective: starting with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then giving solutions for subsequent related research work. In addition, we conducted a detailed and comprehensive analysis, comparison and summary of these works. Finally, we give possible future research directions.


page 7

page 13

page 14

page 18


Poisoning the Search Space in Neural Architecture Search

Deep learning has proven to be a highly effective problem-solving tool f...

Efficient Evaluation Methods for Neural Architecture Search: A Survey

Neural Architecture Search (NAS) has received increasing attention becau...

TextNAS: A Neural Architecture Search Space tailored for Text Representation

Learning text representation is crucial for text classification and othe...

Neural Architecture Search For Fault Diagnosis

Data-driven methods have made great progress in fault diagnosis, especia...

Automated Deep Learning: Neural Architecture Search Is Not the End

Deep learning (DL) has proven to be a highly effective approach for deve...

A Comprehensive Survey on Hardware-Aware Neural Architecture Search

Neural Architecture Search (NAS) methods have been growing in popularity...

Meta-Learning of NAS for Few-shot Learning in Medical Image Applications

Deep learning methods have been successful in solving tasks in machine l...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset