2002 · 126 citations · 10 references
ComputingComputer EngineeringComputer ScienceComputer SecurityDecision AnalyticsNetwork SecurityComputer IntrusionIntrusion DetectionSecurity MetricsSystem SecurityComputer Intrusion DetectionCost Performance MetricsExpected Cost MinimizationGoal-performance RelationshipIntrusion Detection Evaluation MetricsNetwork Intrusion DetectionOperating Characteristics EvaluationPerformance Goal Setting
We present a method of analysis for evaluating intrusion detection systems. The method can be used to compare the performance of intrusion detectors, to evaluate performance goals for intrusion detectors, and to determine the best configuration of an intrusion detector for a given environment. The method uses a decision analysis that integrates and extends ROC (receiver operating characteristics) and cost analysis methods to provide an expected cost metric. We provide general results and illustrate the method in several numerical examples that cover a range of detectors that meet a performance goal and two actual detectors operating in a realistic environment. We demonstrate that, contrary to common advice, the value of an intrusion detection system and the optimal operation of that system depend not only on the system's ROC curve, but also on cost metrics and the hostility of the operating environment as summarized by the probability of intrusion. Extensions of the method are outlined, and conclusions are drawn.
10
Decisions with Multiple Objectives
Ralph L. Keeney, Howard Raiffa · Cambridge University Press eBooks · 1993 · 1.8K citations
Introduction to Statistical Decision Theory.
Mark J. Schervish, John W. Pratt, Howard Raiffa et al. · Journal of the American Statistical Association · 1996 · 717 citations
Cost-based modeling for fraud and intrusion detection: results from the JAM project
Salvatore J. Stolfo, Wei Fan, Wenke Lee et al. · 2002 · 611 citations · Full text