ACM SIGOPS Operating Systems Review · 2010 · 100 citations · 12 references
ComputingComputer SystemsBayesian Data AnalysisData MiningDistributed SystemsDependability AnalysisDependency ParsingDistributed Systems ArchitectureFault DetectionLog AnalysisCo-occurrence AnalysisDependency DiscoveryDistributed LoggingRoot Cause MiningSystem Log AnalysisTime ConsistencyUnstructured Logging
Dependencies among system components are crucial to locating root errors in a distributed system. In this paper, we propose an approach to mine intercomponent dependencies from unstructured logs. The technique requires neither additional system instrumentation nor any application specific knowledge. In the approach, we first parse each log message into its log key and parameters. Then, we find dependent log key pairs belong to different components by leveraging co-occurrence analysis and parameter correspondence. After that, we use Bayesian decision theory to estimate the dependency direction of each dependent log key pair. We further apply time delay consistency to remove false positive detections. Case studies on Hadoop show that the technique successfully identifies the dependencies among the distributed system components.
12
Sanjay Ghemawat, Howard Gobioff, Shun-Tak A. Leung · 2003 · 1.4K citations
Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis
Qiang Fu, Jian–Guang Lou, Yi Wang et al. · 2009 · 557 citations