Near-Data Processing: Insights from a MICRO-46 Workshop

TLDR

Data movement costs in big‑data systems drive renewed interest in near‑data processing, a concept first explored in the 1990s that has recently regained attention after a decade of dormancy. This article aims to highlight why near‑data processing is compelling today and to outline key challenges for its future realization. A well‑attended NDP workshop was organized at MICRO‑46, and its key insights were captured in this article for broad dissemination.

Abstract

The cost of data movement in big-data systems motivates careful examination of near-data processing (NDP) frameworks. The concept of NDP was actively researched in the 1990s, but gained little commercial traction. After a decade-long dormancy, interest in this topic has spiked. A workshop on NDP was organized at MICRO-46 and was well attended. Given the interest, the organizers and keynote speakers have attempted to capture the key insights from the workshop into an article that can be widely disseminated. This article describes the many reasons why NDP is compelling today and identifies key upcoming challenges in realizing the potential of NDP.