Activation features empowered: An efficient gradient-free proxy to zero-shot neural architecture search

Zero-shot neural architecture search offers a training-free paradigm for automated architecture design by employing zero-cost proxies. Recently, the minimum eigenvalue of correlation (MeCo) on feature maps has been identified as a strong gradient-free indicator of architecture performance. However, MeCo computation relies on the feature maps of each and every layer in the network, resulting in significant computational overhead. In this work, we reveal the computational redundancy of MeCo, showing that an accurate assessment of network capacity can be achieved without feature maps of all layers. Strengthened by a theoretical analysis, we propose Faster MeCo (FMeCo), which focuses exclusively on activation features for MeCo calculation. In addition, we integrate a simplified scoring method for architecture’s input discrimination capability, also based on activation features, into FMeCo, to create FMeCo-SWOT. Experimental results on various NAS benchmarks, datasets, and search spaces demonstrate the superiority and efficiency of our proposed proxies. Notably, both FMeCo and FMeCo-SWOT take merely 1/3 of the time required by MeCo for evaluating 1000 architectures in NAS-Bench-201 with higher accuracy. Relying only on activation features, our proposed proxies also achieve strong performance in evaluating both CNN and ViT architectures.

Recommended citation: J. Huang, B. Xue, Y. Sun, M. Zhang, and G. G. Yen, “Activation features empowered: An efficient gradient-free proxy to zero-shot neural architecture search,” Pattern Recognition, vol. 179, p. 113917, 2026.
Download Paper | Download Slides | Download Bibtex