Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Amsterdam Benchmark Suite, a standardized framework for evaluating the efficiency of classical simulators for non-Clifford quantum circuits. By providing a consistent dataset of circuits and virtual hardware, it addresses the fragmentation in benchmarking practices that has hindered fair comparisons across different simulation methods. The key result is that this suite enables researchers to quantitatively assess and compare the performance of various simulation techniques, fostering advancements in the field.
A standardized benchmarking suite reveals significant disparities in the efficiency of classical simulators for non-Clifford quantum circuits, paving the way for more meaningful comparisons and improvements.
Recent years have seen a rapid growth in literature presenting new methods for simulating non-Clifford quantum circuits with classical hardware. These methods span a range of approaches, including stabiliser decomposition and tensor contraction techniques, varying in efficiency depending on circuit class, depth, non-Clifford gate count, and other metrics. A notable limitation of this literature is the lack of a standardised approach to benchmarking, with each new paper outlining its own specification, simulating its own set of circuits on the authors' own hardware. This paper seeks to address this issue by presenting a standardised and canonical benchmark suite and infrastructure for quantifying the efficiency of non-Clifford classical simulators, with a consistent dataset of circuits and providing consistent (virtual) hardware, thereby enabling a fair comparison of results.