Collaborative Deep Reinforcement Learning for Image Cropping

Abstract

An automatic photo composition method based on collaborative deep reinforcement learning(called CDRL-RC) is proposed in this paper. Our method models photo composition as a markov decision-making process by reinforcement learning and generates cropping result through a series of moving and zooming actions. Emotional attention information is added to the composition task, which was trained by eye-tracking datasets to consider the relationship and importance between objects. In order to sufficiently use the emotional attention map and original image for image cropping, they are processed as inputs to two collaborative agents. For the collaborative composition of two agents, we design an information interaction module, which allows inter-agents to exchange information and give advice to each other, and finally predict the action together. In addition, we add attention weight to the traditional IoU to efficiently evaluate the cropping quantity in the reward function. Experiment results show that our CDRL-RC model achieved the state-of-the-art photo composition performance on a variety of datasets.