IEEE Transactions on Parallel and Distributed Systems · 2019 · 25 citations · 27 references
EngineeringComputer EngineeringComputer ScienceComputer ArchitectureParallel ComputingSchedulingCpu SchedulingGraphics Processing Unit ArchitectureGraphics Processing Unit ComputingHardware AccelerationKernel DesignParallel Computing SystemsCo-schedulingExecution Time AnalysisGpu ArchitectureGpu MicroarchitectureGpu Performance ProfilingHardware ResourcesKernel Resource ManagementThread Block Scheduling
While GPUs are meantime omnipresent for many scientific and technical computations, they still continue to evolve as processors. An important recent feature is the ability to execute multiple kernels concurrently via queue streams. However, experiments show that different parameters including the behavior of kernels, the order of kernel launches and other execution configurations, e.g., the number of concurrent thread blocks, may result in different execution time for concurrent kernel execution. Since kernels may have different resource requirements, they can be classified into different classes, which are traditionally assumed as either memory-bound or compute-bound. However, a kernel may belong to the different classes on different hardware according to the hardware resources. In this paper, the definition of kernel mix intensity is introduced. Based on this, a scheduling framework called concurrent CUDA (cCUDA) is proposed to co-schedule the concurrent kernels more efficiently. It first profiles and ranks kernels with different execution behaviors and then takes the kernel resource requirements into account to partition thread blocks of different kernels and overlap them to better utilize the GPU resources. Experimental results on real hardware demonstrate performance improvement in terms of execution time of up to 1.86x, and an average speedup of 1.28x for a wide range of kernels. cCUDA is available at https://github.com/kshekofteh/cCUDA.
27
A bridging model for parallel computation
Leslie G. Valiant · Communications of the ACM · 1990 · 3.7K citations · Full text
Samuel Williams, Andrew Waterman, David A. Patterson · Communications of the ACM · 2009 · 2.4K citations · Full text
Benchmarking GPUs to tune dense linear algebra
В. М. Волков, James Demmel · 2008 · 726 citations
Benchmarking GPUs to tune dense linear algebra
J. Demmel · 2008 · 475 citations