ML Processors Are Going Multi-Core: A performance dream or a scheduling nightmare?

Abstract

Applications of machine learning (ML) increasingly penetrate into our daily routines, our work, and our living environments. In this way, more complex machine intelligence algorithms fundamentally change and enhance the way we live, work, and interact. However, the real-time deployment of these algorithms toward the dreams of smart spaces, digital twins, the metaverse, or personalized health care requires a powerful compute continuum from cloud to (extreme) edge, capable of efficiently executing the compute-hungry ML workloads, such as deep neural networks (NNs). To enable the required real-time responsiveness at affordable energy or power budgets, many ML-optimized custom processors (also called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">accelerators</i> ) have been presented over the past decade, as depicted in <xref ref-type="fig" rid="fig1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Figure 1</xref> .