QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation

TLDR

Rapid advances in NISQ‑era quantum computing create an urgent need for low‑level benchmark suites and evaluation metrics to assess device properties, compiler efficiency, and simulator performance. This work introduces QASMBench, a low‑level, OpenQASM‑based benchmark suite designed to fill that gap. QASMBench aggregates common quantum routines from chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, and cryptography, defines four circuit metrics—gate density, retention lifespan, measurement density, and entanglement variance—to assess execution efficiency, error susceptibility, and hardware‑specific optimization potential, and can be run on IBM‑Q, Rigetti, IonQ, and Quantinuum devices, with its fidelity evaluated on 12 IBM‑Q machines via density‑matrix tomography and compared to IonQ and Rigetti platforms. QASMBench is available at http://github.com/pnnl/QASMBench.

Abstract

The rapid development of quantum computing (QC) in the NISQ era urgently demands a low-level benchmark suite and insightful evaluation metrics for characterizing the properties of prototype NISQ devices, the efficiency of QC programming compilers, schedulers and assemblers, and the capability of quantum system simulators in a classical computer. In this work, we fill this gap by proposing a low-level, easy-to-use benchmark suite called QASMBench based on the OpenQASM assembly representation. It consolidates commonly used quantum routines and kernels from a variety of domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, and so on, trading-off between generality and usability. To analyze these kernels in terms of NISQ device execution, in addition to circuit width and depth, we propose four circuit metrics including gate density, retention lifespan, measurement density, and entanglement variance, to extract more insights about the execution efficiency, the susceptibility to NISQ error, and the potential gain from machine-specific optimizations. Applications in QASMBench can be launched and verified on several NISQ platforms, including IBM-Q, Rigetti, IonQ and Quantinuum. For evaluation, we measure the execution fidelity of a subset of QASMBench applications on 12 IBM-Q machines through density matrix state tomography, comprising 25K circuit evaluations. We also compare the fidelity of executions among the IBM-Q machines, the IonQ QPU and the Rigetti Aspen M-1 system. QASMBench is released at: http://github.com/pnnl/QASMBench .