IEEE Transactions on Parallel and Distributed Systems · 2021 · 20 citations · 33 references
ComputingComputer EngineeringComputer ScienceComputer ArchitectureHeterogeneous ComputingParallel ComputingAccelerator ArchitectureGraphics Processing Unit ArchitectureGraphics Processing Unit ComputingGraphics Processing UnitsHeterogeneous System ArchitectureParallel ArchitecturesExecution Algorithm DesignFloating-point ArithmeticFloating-point InstructionsGpu ArchitectureGpu Parallel AlgorithmsHeterogeneous System Performance ModelingParallel SystemSystem Architecture Design
There is a widening gap between GPU and other components (CPU, PCIe bus and communication network) in heterogeneous parallel system. The gap forces us to orchestrate cooperative execution among these components much more carefully than ever before. By taking the LINPACK benchmark as a case study, this article proposes a fine-grained pipelining algorithm on large-scale CPU-GPU heterogeneous cluster systems. First, we build an algorithmic model that reveals a new approach to GPU-centric and fine-grained pipelining algorithm design. Then, we present four model-driven pipelining algorithms that incrementally squeeze bubbles in the pipeline so that it is occupied by more useful floating-point calculations. The algorithms are implemented on both the AMD and NVIDIA GPU platforms. The finally optimized LINPACK program achieves 107 PFlops on 25, 600 GPUs (70 percent floating-point efficiency). Several insights have been drawn to suggest tradeoff of algorithm design, programming support, and architecture design.
33
Benchmarking GPUs to tune dense linear algebra
В. М. Волков, James Demmel · 2008 · 726 citations
Chi-Keung Luk, Sunpyo Hong, Hyesoon Kim · 2009 · 512 citations
Benchmarking GPUs to tune dense linear algebra
J. Demmel · 2008 · 475 citations
Dense linear algebra solvers for multicore with GPU accelerators
Stanimire Tomov, Rajib Nath, Hatem Ltaief et al. · 2010 · 251 citations