2015 · 31 citations · 15 references
ComputingComputer EngineeringComputer ScienceComputer ArchitectureMachine LearningParallel ComputingMemory HierarchyMemory Hierarchy DesignParallel ArchitecturesPerformance ModelingPerformance OptimizationPerformance TuningAnalytical ModelsComplexityHybrid Performance ModelingI/o OptimizationParallel Performance ModelingRobustnessShared StorageStorage Hierarchy
The optimization of parallel I/O has become challenging because of the increasing storage hierarchy, performance variability of shared storage systems, and the number of factors in the hardware and software stacks that impact performance. In this paper, we perform an in-depth study of the complexity involved in I/O autotuning and performance modeling, including the architecture, software stack, and noise. We propose a novel hybrid model combining analytical models for communication and storage operations and black-box models for the performance of the individual operations. The experimental results show that the hybrid approach performs significantly better and shows a higher robustness to noise than state-of-the-art machine learning approaches, at the cost of a higher modeling complexity.
15
Pattern Recognition and Machine Learning
Journal of Electronic Imaging · 2007 · 22.1K citations
Pattern recognition and machine learning
Choice Reviews Online · 2007 · 2.7K citations
GPFS: A Shared-Disk File System for Large Computing Clusters
Frank Schmuck, Roger Haskin · 2002 · 1.2K citations
Data sieving and collective I/O in ROMIO
Rajeev Thakur, William Gropp, Ewing Lusk · 1999 · 498 citations · Full text