2021 · 21 citations · 17 references
ComputingComputer ScienceComputer SystemsOperating SystemsParallel ComputingStorage SystemsBottleneck IdentificationFile System AnalysisParallel File SystemsParallel Performance AnalysisPerformance Bottleneck AnalysisPerformance TuningBottleneck DetectionFile System Performance EvaluationFile System Performance TuningI/o Bottleneck AnalysisI/o OptimizationKernel ProfilingPerformance Bottleneck DetectionSystem Bottleneck Detection
Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.
17
Data sieving and collective I/O in ROMIO
Rajeev Thakur, William Gropp, Ewing Lusk · 1999 · 498 citations · Full text
Felipe Pezoa, Juan L. Reutter, Fernando Suárez et al. · 2016 · 412 citations
Taming parallel I/O complexity with auto-tuning
Babak Behzad, Huong Luu, Ruth Aydt et al. · 2013 · 112 citations · Full text