Defining and detecting quantum speedup

TLDR

The rise of small‑scale quantum devices prompts the need to fairly assess and compare their computational power with classical devices, particularly to detect quantum speedup. The study aims to define and measure quantum speedup across scenarios while highlighting pitfalls that could mask or fake it. The authors illustrate their framework using data from a randomized benchmark on a D‑Wave Two device with up to 503 qubits. The benchmark on random spin‑glass instances with limited precision shows no evidence of quantum speedup across the full data set, inconclusive results on subsets, and suggests that speedup may be problem‑dependent and elusive.

Abstract

The development of small-scale digital and analog quantum devices raises the question of how to fairly assess and compare the computational power of classical and quantum devices, and of how to detect quantum speedup. Here we show how to define and measure quantum speedup in various scenarios, and how to avoid pitfalls that might mask or fake quantum speedup. We illustrate our discussion with data from a randomized benchmark test on a D-Wave Two device with up to 503 qubits. Comparing the performance of the device on random spin glass instances with limited precision to simulated classical and quantum annealers, we find no evidence of quantum speedup when the entire data set is considered, and obtain inconclusive results when comparing subsets of instances on an instance-by-instance basis. Our results for one particular benchmark do not rule out the possibility of speedup for other classes of problems and illustrate that quantum speedup is elusive and can depend on the question posed.