Publication | Closed Access
TVM: an automated end-to-end optimizing compiler for deep learning
909
Citations
0
References
2018
Year
EngineeringMachine LearningCompiler TechnologyComputer ArchitectureEmbedded SystemsEmbedded Machine LearningCompilersParallel ComputingCompiler SupportComputer EngineeringComputer ScienceHardware DevicesDeep LearningOptimizing CompilerHardware AccelerationOptimization ChallengesProgram AnalysisMemory Latency HidingDomain-specific Accelerator
Machine learning must run on a wide variety of hardware, yet current frameworks depend on vendor‑specific operator libraries and are tuned only for server‑class GPUs, making deployment to mobile, embedded, or accelerator devices labor‑intensive. TVM is a compiler that exposes graph‑level and operator‑level optimizations to achieve performance portability for deep‑learning workloads across diverse hardware back‑ends. It addresses deep‑learning optimization challenges—such as high‑level operator fusion, mapping to arbitrary hardware primitives, and memory‑latency hiding—by automating low‑level code optimization with a learning‑based cost model for rapid exploration. Experiments show TVM attains performance competitive with hand‑tuned libraries on low‑power CPUs, mobile GPUs, and server GPUs, can target new accelerators like FPGA, and is open‑source and used in production by major companies.
There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms - such as mobile phones, embedded devices, and accelerators (e.g., FPGAs, ASICs) - requires significant manual effort. We propose TVM, a compiler that exposes graph-level and operator-level optimizations to provide performance portability to deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives, and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of code optimizations. Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of-the-art, hand-tuned libraries for low-power CPU, mobile GPU, and server-class GPUs. We also demonstrate TVM's ability to target new accelerator back-ends, such as the FPGA-based generic deep learning accelerator. The system is open sourced and in production use inside several major companies.