MFC Datasets: Large-Scale Benchmark Datasets for Media Forensic Challenge Evaluation

Haiying Guan, Mark Kozak, Eric Robertson, Yooyoung Lee, Amy N. Yates, Andrew Delgado, Daniel F. Zhou, Timothée Kheyrkhah, Jeff Smith, Jonathan G. Fiscus

2019 · 13 citations · 22 references

Concepts

TL;DR

The paper introduces a benchmark for Media Forensics Challenge evaluations, outlining objectives, challenges, and methods for building and annotating large‑scale forensic datasets. The dataset includes over 176,000 high‑provenance images and 11,000 videos, plus more than 100,000 manipulated images and 4,000 videos, and an infrastructure that selects customized evaluation subsets based on specified queries. Using these datasets, the authors assessed progress and analyzed system performance across media forensics tasks over the past two years, presenting evaluation results from recent challenges.

Abstract

We provide a benchmark for digital Media Forensics Challenge (MFC) evaluations. Our comprehensive data comprises over 176,000 high provenance (HP) images and 11,000 HP videos; more than 100,000 manipulated images and 4,000 manipulated videos; 35 million internet images and 300,000 video clips. We have designed and generated a series of development, evaluation, and challenge datasets, and used them to assess the progress and thoroughly analyze the performance of diverse systems on a variety of media forensics tasks in the past two years. In this paper, we first introduce the objectives, challenges, and approaches to building media forensics evaluation datasets. We then discuss our approaches to forensic dataset collection, annotation, and manipulation, and present the design and infrastructure to effectively and efficiently build the evaluation datasets to support various evaluation tasks. Given a specified query, we build an infrastructure that selects the customized evaluation subsets for the targeted analysis report. Finally, we demonstrate the evaluation results in the past evaluations.

References

22