Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · 2022 · 14 citations · 19 references
ComputingComputer ScienceArtificial IntelligenceComputer ArchitectureMachine LearningDeep LearningGraph-based LearningNeural NetworksPattern RecognitionRepresentation LearningSelf-supervised LearningArchitecture SearchAutoencodersGraph AutoencodersGraph Neural NetworksGraph Self-supervised LearningNeural Architecture SearchSearch StrategiesSelf-supervised Pretraining
Performance estimation of neural architecture is a crucial component of neural architecture search (NAS). Meanwhile, neural predictor is a current mainstream performance estimation method. However, it is a challenging task to train the predictor with few architecture evaluations for efficient NAS. In this paper, we propose a graph masked autoencoder (GMAE) enhanced predictor, which can reduce the dependence on supervision data by self-supervised pre-training with untrained architectures. We compare our GMAE-enhanced predictor with existing predictors in different search spaces, and experimental results show that our predictor has high query utilization. Moreover, GMAE-enhanced predictor with different search strategies can discover competitive architectures in different search spaces. Code and supplementary materials are available at https://github.com/kunjing96/GMAENAS.git.
19
Neural Architecture Search with Reinforcement Learning
Barret Zoph, Quoc V. Le · arXiv (Cornell University) · 2016 · 3.9K citations · Full text
Regularized Evolution for Image Classifier Architecture Search
Esteban Real, Alok Aggarwal, Yanping Huang et al. · 2019 · 2.7K citations
Graph Contrastive Learning with Augmentations
Yuning You, Tianlong Chen, Yongduo Sui et al. · arXiv (Cornell University) · 2020 · 865 citations · Full text