Methods and Metrics for Evaluating Analytic Insider Threat Tools

Abstract

The insider threat is a prime security concern for government and industry organizations. As insider threat programs come into operational practice, there is a continuing need to assess the effectiveness of tools, methods, and data sources, which enables continual process improvement. This is particularly challenging in operational environments, where the actual number of malicious insiders in a study sample is not known. The present paper addresses the design of evaluation strategies and associated measures of effectiveness; several quantitative/statistical significance test approaches are described with examples, and a new measure, the Enrichment Ratio, is proposed and described as a means of assessing the impact of proposed tools on the organization's operations.