Similarity surrogate-assisted evolutionary neural architecture search with dual encoding strategy

Abstract

<abstract><p>Neural architecture search (NAS), a promising method for automated neural architecture design, is often hampered by its overwhelming computational burden, especially the architecture evaluation process in evolutionary neural architecture search (ENAS). Although there are surrogate models based on regression or ranking to assist or replace the neural architecture evaluation process in ENAS to reduce the computational cost, these surrogate models are still affected by poor architectures and are not able to accurately find good architectures in a search space. To solve the above problems, we propose a novel surrogate-assisted NAS approach, which we call the similarity surrogate-assisted ENAS with dual encoding strategy (SSENAS). We propose a surrogate model based on similarity measurement to select excellent neural architectures from a large number of candidate architectures in a search space. Furthermore, we propose a dual encoding strategy for architecture generation and surrogate evaluation in ENAS to improve the exploration of well-performing neural architectures in a search space and realize sufficiently informative representations of neural architectures, respectively. We have performed experiments on NAS benchmarks to verify the effectiveness of the proposed algorithm. The experimental results show that SSENAS can accurately find the best neural architecture in the NAS-Bench-201 search space after only 400 queries of the tabular benchmark. In the NAS-Bench-101 search space, it can also get results that are comparable to other algorithms. In addition, we conducted a large number of experiments and analyses on the proposed algorithm, showing that the surrogate model measured via similarity can gradually search for excellent neural architectures in a search space.</p></abstract>