Designing Neural Network Architectures using Reinforcement Learning

TLDR

Designing CNN architectures currently requires human expertise and labor, with new architectures handcrafted or modified from existing networks. We introduce MetaQNN, a reinforcement‑learning based meta‑modeling algorithm that automatically generates high‑performing CNN architectures for a given learning task. The learning agent employs Q‑learning with ε‑greedy exploration and experience replay to sequentially select CNN layers from a finite architecture space, iteratively discovering designs with improved performance. On image‑classification benchmarks, the agent‑designed networks using only standard layers beat existing networks of the same type, rival state‑of‑the‑art methods that use more complex layers, and outperform other meta‑modeling approaches.

Abstract

At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using $Q$-learning with an $\epsilon$-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.