Fast A3RL: Aesthetics-Aware Adversarial Reinforcement Learning for Image Cropping

Abstract

Image cropping aims at improving the quality of images by removing unwanted outer areas, which is widely used in the photography and printing industry. Most previous cropping methods that don't need bounding box supervision rely on the sliding window mechanism. The sliding window method results in fixed aspect ratios and limits the shape of the cropping region. Moreover, the sliding window method usually produces lots of candidates on the input image, which is very time-consuming. Motivated by these challenges, we formulate image cropping as a sequential decision-making process and propose a reinforcement learning based framework to address this problem, namely Fast Aesthetics-Aware Adversarial Reinforcement Learning (Fast A3RL). Particularly, the proposed method develops an aesthetics-aware reward function, which is dedicated for image cropping. Similar to human's decisionmaking process, we use a comprehensive state representation including both the current observation and historical experience. We train the agent using the actor-critic architecture in an end-to-end manner. The adversarial learning process is also applied during the training stage. The proposed method is evaluated on several popular cropping datasets, in which the images are unseen during training. Experiment results show that our method achieves state-of-the-art performance with much fewer candidate windows and much less time compared with related methods.