SupermarQ: A Scalable Quantum Benchmark Suite

TLDR

Quantum computing’s emergence has sparked speculation about its revolutionary impact, yet its infancy and diverse architectures hinder reliable performance measurement. The authors present SupermarQ, a scalable, hardware‑agnostic benchmark suite that employs application‑level metrics to assess quantum performance. SupermarQ adapts classical benchmarking methods to quantum hardware by defining feature vectors, selecting representative applications across domains, and gathering results from IBM, IonQ, and AQT@LBNL platforms. SupermarQ is introduced as a pivotal step toward a large, open‑source, evolving quantum benchmarking ecosystem.

Abstract

The emergence of quantum computers as a new computational paradigm has been accompanied by speculation concerning the scope and timeline of their anticipated revolutionary changes. While quantum computing is still in its infancy, the variety of different architectures used to implement quantum computations make it difficult to reliably measure and compare performance. This problem motivates our introduction of SupermarQ, a scalable, hardware-agnostic quantum benchmark suite which uses application-level metrics to measure performance. SupermarQ is the first attempt to systematically apply techniques from classical benchmarking methodology to the quantum domain. We define a set of feature vectors to quantify coverage, select applications from a variety of domains to ensure the suite is representative of real workloads, and collect benchmark results from the IBM, IonQ, and AQT@LBNL platforms. Looking forward, we envision that quantum benchmarking will encompass a large cross-community effort built on open source, constantly evolving benchmark suites. We introduce SupermarQ as an important step in this direction.