Proceedings - IEEE International Parallel and Distributed Processing Symposium · 2008 · 83 citations · 25 references
ComputingComputer ScienceComputer SystemsComputer ArchitectureParallel ComputingStorage SystemsData ParallelismMemory Hierarchy OptimizationParallel ArchitecturesParallel Computing SystemsParallel Performance AnalysisBandwidthHierarchyHigh-performance Computing SystemsI/o OptimizationParallel I/o ArchitecturePerformance CharacterizationStorage HierarchySupercomputer Architecture
This paper presents an extensive characterization, tuning, and optimization of parallel I/O on the Cray XT supercomputer, named Jaguar, at Oak Ridge National Laboratory. We have characterized the performance and scalability for different levels of storage hierarchy including a single Lustre object storage target, a single S2A storage couplet, and the entire system. Our analysis covers both data- and metadata-intensive I/O patterns. In particular, for small, non-contiguous data- intensive I/O on Jaguar, we have evaluated several parallel I/O techniques, such as data sieving and two- phase collective I/O, and shed light on their effectiveness. Based on our characterization, we have demonstrated that it is possible, and often prudent, to improve the I/O performance of scientific benchmarks and applications by tuning and optimizing I/O. For example, we demonstrate that the I/O performance of the S3D combustion application can be improved at large scale by tuning the I/O system to avoid a bandwidth degradation of 49% with 8192 processes when compared to 4096 processes. We have also shown that the performance of Flash I/O can be improved by 34% by tuning the collective I/O parameters carefully.
25
Myrinet: a gigabit-per-second local area network
N.J. Boden, D. Cohen, Robert E. Felderman et al. · IEEE Micro · 1995 · 1.7K citations
Data sieving and collective I/O in ROMIO
Rajeev Thakur, William Gropp, Ewing Lusk · 1999 · 498 citations · Full text
The Quadrics network: high-performance clustering technology
Fabrizio Petrini, Wu-chun Feng, Adolfy Hoisie et al. · IEEE Micro · 2002 · 322 citations
Size Scaling of Turbulent Transport in Magnetically Confined Plasmas
Zhihong Lin, S. Ethier, T. S. Hahm et al. · Physical Review Letters · 2002 · 237 citations