Conference proceeding
EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching From Scratch
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol.2019-, pp.9051-9060
06/2019
DOI: 10.1109/CVPR.2019.00927
Abstract
Designing the structure of neural networks is considered one of the most challenging tasks in deep learning, especially when there is few prior knowledge about the task domain. In this paper, we propose an Ecologically-Inspired GENetic (EIGEN) approach that uses the concept of succession, extinction, mimicry, and gene duplication to search neural network structure from scratch with poorly initialized simple network and few constraints forced during the evolution, as we assume no prior knowledge about the task domain. Specifically, we first use primary succession to rapidly evolve a population of poorly initialized neural network structures into a more diverse population, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Extinction is applied in both stages to reduce computational cost. Mimicry is employed during the entire evolution process to help the inferior networks imitate the behavior of a superior network and gene duplication is utilized to duplicate the learned blocks of novel structures, both of which help to find better network structures. Experimental results show that our proposed approach can achieve similar or better performance compared to the existing genetic approaches with dramatically reduced computation cost. For example, the network discovered by our approach on CIFAR-100 dataset achieves 78.1% test accuracy under 120 GPU hours, compared to 77.0% test accuracy in more than 65, 536 GPU hours in [35].
Details
- Title: Subtitle
- EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching From Scratch
- Creators
- Jian Ren - Rutgers, The State University of New JerseyZhe Li - University of IowaJianchao Yang - ByteDance (China)Ning Xu - Amazon (United States)Tianbao Yang - University of IowaDavid J Foran - Rutgers, The State University of New Jersey
- Resource Type
- Conference proceeding
- Publication Details
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol.2019-, pp.9051-9060
- DOI
- 10.1109/CVPR.2019.00927
- ISSN
- 1063-6919
- eISSN
- 2575-7075
- Publisher
- IEEE
- Language
- English
- Date published
- 06/2019
- Academic Unit
- Computer Science
- Record Identifier
- 9984259424402771
Metrics
6 Record Views