Use of a Capsule Network to Detect Fake Images and Videos

TLDR

Advances in GPU and TPU hardware have accelerated AI, enabling widespread use of computer‑generated images and videos, but also facilitating malicious deepfake attacks that existing countermeasures struggle to detect across domains. This work proposes a capsule network to detect a broad spectrum of attacks, including printed image presentation, replayed video, and deep learning‑generated fake videos. The capsule network employs a novel architecture that captures spatial hierarchies, and the authors provide the first theoretical analysis and visualization of capsule networks for forensic detection. The model achieves comparable performance to conventional convolutional neural networks while using far fewer parameters.

Abstract

The revolution in computer hardware, especially in graphics processing units and tensor processing units, has enabled significant advances in computer graphics and artificial intelligence algorithms. In addition to their many beneficial applications in daily life and business, computer-generated/manipulated images and videos can be used for malicious purposes that violate security systems, privacy, and social trust. The deepfake phenomenon and its variations enable a normal user to use his or her personal computer to easily create fake videos of anybody from a short real online video. Several countermeasures have been introduced to deal with attacks using such videos. However, most of them are targeted at certain domains and are ineffective when applied to other domains or new attacks. In this paper, we introduce a capsule network that can detect various kinds of attacks, from presentation attacks using printed images and replayed videos to attacks using fake videos created using deep learning. It uses many fewer parameters than traditional convolutional neural networks with similar performance. Moreover, we explain, for the first time ever in the literature, the theory behind the application of capsule networks to the forensics problem through detailed analysis and visualization.