Evolving Comprehensive Proxies for Zero-Shot Neural Architecture Search

Neural architecture search (NAS) has emerged as a promising technology for automatically designing deep neural network (DNN) architectures. However, its development is significantly constrained by the prohibitively high computational cost of architecture evaluations. Recently, zero-shot NAS has addressed this challenge by employing zero-cost proxies to evaluate candidate architectures without expensive gradient training, effectively mitigating the timeintensive nature of NAS. However, a major limitation is that most existing zero-cost proxies focus narrowly on a single aspect of DNNs, resulting in biased evaluations with generally weak correlations to the network’s performance. In this work, we address this issue by assembling four distinct zero-cost proxies in a nonlinear fashion to provide a comprehensive evaluation of DNNs across multiple dimensions, including expressivity, convergence, generalization, and parameter saliency. Furthermore, we develop an adaptive particle swarm optimization-based approach to effectively evolve the coefficients of each base proxy in the ensemble for task-specific optimization. Extensive experimental results on various NAS benchmarks and open-domain search spaces demonstrate the effectiveness of the proposed method. Our findings show that the complementarity of zero-cost proxies greatly improves the reliability of performance evaluation, thereby enabling zero-shot NAS to identify more promising network architectures.

Recommended citation: Junhao Huang, Bing Xue, Yanan Sun, and Mengjie Zhang. 2025. Evolving Comprehensive Proxies for Zero-Shot Neural Architecture Search. In Genetic and Evolutionary Computation Conference (GECCO), July 14-18, 2025, Malaga, Spain. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3712256.3726315
Download Paper | Download Slides | Download Bibtex