Revelio: A Modular and Effective Framework for Reproducible Training and Evaluation of Morphing Attack Detectors

Abstract

Morphing Attack, i.e. the elusion of face verification systems through a facial morphing operation between a criminal and an accomplice, has recently emerged as a serious security threat. Despite the importance of this kind of attack, the development and comparison of Morphing Attack Detection (MAD) methods is still a challenging task, especially with deep learning approaches. Specifically, the lack of public datasets, the absence of common training and validation protocols, and the limited release of public source code hamper the reproducibility and objective comparison of new MAD systems. Usually, these aspects are mainly due to privacy concerns, that limit data transfers and storage, and to the recent introduction of the MAD task. Therefore, in this paper, we propose and publicly release <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Revelio</i> , a modular framework for the reproducible development and evaluation of MAD systems. We include an overview of the modules, and describe the plugin system providing the possibility of extending native components with new functionalities. An extensive cross-datasets experimental evaluation is conducted to validate the framework and the performance of trained models on several publicly-released datasets, and to deeply analyze the main challenges in the MAD task based on single input images. We also propose a new metric, namely WAED, to summarize in a single value the error-based metrics commonly used in the MAD task, computed over different datasets, thus facilitating the comparative evaluation of different approaches. Finally, by exploiting Revelio, a new state-of-the-art MAD model (on SOTAMD single-image benchmark) is proposed and released.