Detecting and classifying scars, marks, and tattoos found in the wild

Abstract

Within the forensics community, there is a growing interest in automatic biometric-based approaches for describing subjects in an image. By labeling scars, marks and tattoos, a collection of these discriminative attributes can be assigned to images and used to assist in large-scale person search and identification. Typically, the imagery considered in a forensics context consists to some degree of uncontrolled, unprofessionally generated photographs. Recent work has shown that it is quite feasible to detect scars and marks, as well as categorize tattoos, presuming that the source imagery is controlled in some manner. In this work, we introduce a new methodology for detecting and classifying scars, marks and tattoos found in unconstrained imagery typical of forensics scenarios. Novel approaches for initial feature detection and automatic segmentation are described. We also consider the “open set” nature of the classification problem, and describe an appropriate machine learning methodology that addresses it. An extensive series of experiments for representative unconstrained data is presented, highlighting the effectiveness of our approach for images found “in the wild”.