IEEE Transactions on Knowledge and Data Engineering · 2018 · 45 citations · 52 references
BusinessComputer ScienceInformation SystemsBusiness IntelligenceBusiness Process ManagementData MiningBusiness Process AnalysisBusiness Process AutomationProcess AnalysisProcess Flow AnalysisProcess MiningProcess ModelingComparative EvaluationProcess DiscoveryProcess Model Evaluation
Process mining uses event logs to reveal actual process performance, and automated process discovery—one of its most studied operations—has produced many methods that trade scalability, accuracy, and model complexity, yet their evaluation has been ad‑hoc and often incomparable. This article systematically reviews and comparatively evaluates automated process discovery methods using an open‑source benchmark. The authors evaluate the methods on 33 real‑life event logs—12 publicly available, 12 proprietary, and 9 quality metrics—within an open‑source benchmark framework. The study reveals gaps such as limited scalability of some methods and significant performance divergence across quality metrics.
Process mining allows analysts to exploit logs of historical executions of business processes to extract insights regarding the actual performance of these processes. One of the most widely studied process mining operations is automated process discovery. An automated process discovery method takes as input an event log, and produces as output a business process model that captures the control-flow relations between tasks that are observed in or implied by the event log. Various automated process discovery methods have been proposed in the past two decades, striking different tradeoffs between scalability, accuracy, and complexity of the resulting models. However, these methods have been evaluated in an ad-hoc manner, employing different datasets, experimental setups, evaluation measures, and baselines, often leading to incomparable conclusions and sometimes unreproducible results due to the use of closed datasets. This article provides a systematic review and comparative evaluation of automated process discovery methods, using an open-source benchmark and covering 12 publicly-available real-life event logs, 12 proprietary real-life event logs, and nine quality metrics. The results highlight gaps and unexplored tradeoffs in the field, including the lack of scalability of some methods and a strong divergence in their performance with respect to the different quality metrics used.
52
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Ron Kohavi · 1995 · 10.7K citations · Full text
Procedures for Performing Systematic Reviews
Barbara Kitchenham · 2004 · 4.8K citations
Conducting research literature reviews: from the Internet to paper
Choice Reviews Online · 2005 · 1.9K citations