Automated Discovery of Process Models from Event Logs: Review and Benchmark

Adriano Augusto, Raffaele Conforti, Marlon Dumas, Marcello La Rosa, Fabrizio Maria Maggi, Andrea Marrella, Massimo Mecella, Allar Soo

IEEE Transactions on Knowledge and Data Engineering · 2018 · 45 citations · 52 references

DOIFull text

Open access

Concepts

TL;DR

Process mining uses event logs to reveal actual process performance, and automated process discovery—one of its most studied operations—has produced many methods that trade scalability, accuracy, and model complexity, yet their evaluation has been ad‑hoc and often incomparable. This article systematically reviews and comparatively evaluates automated process discovery methods using an open‑source benchmark. The authors evaluate the methods on 33 real‑life event logs—12 publicly available, 12 proprietary, and 9 quality metrics—within an open‑source benchmark framework. The study reveals gaps such as limited scalability of some methods and significant performance divergence across quality metrics.

Abstract

Process mining allows analysts to exploit logs of historical executions of business processes to extract insights regarding the actual performance of these processes. One of the most widely studied process mining operations is automated process discovery. An automated process discovery method takes as input an event log, and produces as output a business process model that captures the control-flow relations between tasks that are observed in or implied by the event log. Various automated process discovery methods have been proposed in the past two decades, striking different tradeoffs between scalability, accuracy, and complexity of the resulting models. However, these methods have been evaluated in an ad-hoc manner, employing different datasets, experimental setups, evaluation measures, and baselines, often leading to incomparable conclusions and sometimes unreproducible results due to the use of closed datasets. This article provides a systematic review and comparative evaluation of automated process discovery methods, using an open-source benchmark and covering 12 publicly-available real-life event logs, 12 proprietary real-life event logs, and nine quality metrics. The results highlight gaps and unexplored tradeoffs in the field, including the lack of scalability of some methods and a strong divergence in their performance with respect to the different quality metrics used.

References

52