EvoREP: Evolving Reliable Ensemble of Zero-Cost Proxies for Zero-Shot Neural Architecture Search

Zero-shot neural architecture search (NAS) offers a significant boost to the efficiency of automated network design, where zero-cost proxies play a key role in evaluating the quality of candidate networks without training. However, most existing proxies measure network capability by a single specific characteristic, which is insufficient to achieve a high correlation with the final performance. Although some studies attempt to use multiple proxies for better estimation performance, they often fall short in considering the inherently multi-dimensional properties of neural networks and/or rely on simple combinations of these proxies. This work presents an effective nonlinear ensemble method to aggregate nine representative zero-cost proxies aiming at various network characteristics for comprehensive network evaluations. To obtain a reliable aggregation of proxies on a given task and efficiently utilize candidate base proxies, a diversity-aware adaptive particle swarm optimization (DAA-PSO) approach is developed to evolve the ensemble proxy. DAA-PSO maintains a pool of best solutions with varying sparsity levels to enhance the diversity of particles and employs an adaptive solution sparsification strategy to effectively identify the most suitable base proxies for the ensemble. The proposed method, named EvoREP (evolving reliable ensemble of zero-cost proxies), can efficiently discover the best ensemble proxy for the target task. Extensive experiments on popular NAS benchmarks and open-domain search spaces demonstrate the effectiveness and promising transferability of EvoREP in evolving ensemble proxies, exhibiting remarkably high correlation with test accuracy.

Recommended citation: J. Huang, B. Xue, Y. Sun, M. Zhang, and G. G. Yen, “EvoREP: Evolving reliable ensemble of zero-cost proxies for zero-shot neural architecture search,” IEEE Transactions on Evolutionary Computation, pp. 1–15, 2025.
Download Paper | Download Slides | Download Bibtex