Abstract

Neural networks have proven effective at solving difficult problems but designing their architectures can be challenging, even for image classification problems alone. Our goal is to minimize human participation, so we employ evolutionary algorithms to discover such networks automatically. Despite significant computational requirements, we show that it is now possible to evolve models with accuracies within the range of those published in the last year. Specifically, we employ simple evolutionary techniques at unprecedented scales to discover models for the CIFAR-10 and CIFAR-100 datasets, starting from trivial initial conditions and reaching accuracies of 94.6% (95.6% for ensemble) and 77.0%, respectively. To do this, we use novel and intuitive mutation operators that navigate large search spaces; we stress that no human participation is required once evolution starts and that the output is a fully-trained model. Throughout this work, we place special emphasis on the repeatability of results, the variability in the outcomes and the computational requirements.

https://arxiv.org/pdf/1703.01041.pdf

 

 

 

 

 

 

 

 

Key points:

1. Need many controllers, so-called Workers in the paper, to guild the evolution process (e.g., selection,  mutation)
2. The controllers are working in a distributed way

Benefits: The proposed method is simple and it is able to generate a fully trained network requiring no post-processing.

concerns:

The paper is interesting in the way that it helps to discover a fully automated DNN architecture for solving complex tasks without human participation. Although the authors claim that their method is scalable, only companies owning large-scale platform can employ this method and as long as there is no more economical implementation, we cannot see it as a scalable solution. However, it is a good starting point for automating the scalable architectural design of DNNs besides other solutions such as reinforcement learning. [other comment]