What Supercomputers Say: A Study of Five System Logs

TLDR

Real deployed system logs are needed to detect and diagnose failures in large‑scale computer systems, but empirical data has been inaccessible, hindering progress. The study examines system logs from five supercomputers to provide insight and guide future research on log‑based failure detection. The authors collected logs from Blue Gene/L, Red Storm, Thunderbird, Spirit, and Liberty, identified alerts, and introduced a simpler filtering algorithm and an expanded operational context. This is the first study of raw system logs from multiple supercomputers.

Abstract

If we hope to automatically detect and diagnose failures in large-scale computer systems, we must study real deployed systems and the data they generate. Progress has been hampered by the inaccessibility of empirical data. This paper addresses that dearth by examining system logs from five supercomputers, with the aim of providing useful insight and direction for future research into the use of such logs. We present details about the systems, methods of log collection, and how alerts were identified; propose a simpler and more effective filtering algorithm; and define operational context to encompass the crucial information that we found to be currently missing from most logs. The machines we consider (and the number of processors) are: Blue Gene/L (131072), Red Storm (10880), Thunderbird (9024), Spirit (1028), and Liberty (512). This is the first study of raw system logs from multiple supercomputers.