2021 · 22 citations · 41 references
ComputingComputer ScienceHigh-performance ComputingOperating SystemsParallel ComputingStorage SystemsBottleneck IdentificationCpu SchedulingPerformance Bottleneck AnalysisPerformance ModelingHigh-performance Computing SystemsPerformance Anomaly DetectionPerformance Bottleneck DetectionPerformance MonitoringPerformance VariabilityResource ContentionResource Contention AnalysisWorkload Characterization
Monitoring and analyzing I/O behaviors is critical to the efficient utilization of parallel storage systems. Unfortunately, with increasing I/O requirements and resource contention, I/O performance variability is becoming a significant concern. This paper investigates I/O behavior and performance variability on a large-scale high-performance computing (HPC) system using a novel methodology that identifies similarity across jobs from the same application leveraging an I/O characterization tool and then, detects potential I/O performance variability across jobs of the same application. We demonstrate and discuss how our unique methodology can be used to perform temporal and feature analyses to detect interesting I/O performance variability patterns in production HPC systems, and their implications for operating/managing large-scale systems.
41
Scikit-learn: Machine Learning in Python
Fabián Pedregosa, Gaël Varoquaux, Alexandre Gramfort et al. · arXiv (Cornell University) · 2012 · 63.7K citations · Full text
Scikit-learn: Machine Learning in Python
PedregosaFabian, VaroquauxGaël, GramfortAlexandre et al. · Journal of Machine Learning Research · 2011 · 8.2K citations
24/7 Characterization of petascale I/O workloads
Philip Carns, Robert Latham, Robert Ross et al. · 2009 · 212 citations