Measuring what matters: A scalable framework for application-level quantum benchmarking
Abstract
As quantum computing systems continue to mature, there is an increasing need for benchmarking methodologies that capture performance in terms of meaningful, application-level metrics. In this work, we present a scalable framework for application-level quantum benchmarking that is designed to support internal system evaluation and cross-platform comparison across technology providers. Our framework is guided by a set of core principles, including measurability, simplicity, scalability, and extensibility. We present 13 benchmark families that reflect realistic workloads across multiple domains. This enables the systematic evaluation of the quality of solutions, the total execution time, total used energy, as well as Time-to-Solution. The benchmarks are designed to be reproducible, interpretable across stakeholder groups, and adaptable to evolving system capabilities. The framework aims to bridge the gap between low-level performance metrics and real-world value, providing a unified approach to assessing quantum systems. The resulting benchmarks support development and validation and contribute to the foundation of industry-wide benchmarking standards.
Cite
@article{arxiv.2604.11781,
title = {Measuring what matters: A scalable framework for application-level quantum benchmarking},
author = {Willie Aboumrad and Claudio Girotto and Joshua Goings and Luning Zhao and Miguel Angel Lopez-Ruiz and Daiwei Zhu and Ananth Kaushik and Sayonee Ray and Samwel Sekwao and Jason Iaconis and Andrew Arrasmith and Andrii Maksymov and Yvette de Sereville and Felix Tripier and Far McKon and Coleman Collins and Evgeny Epifanovsky and Masako Yamada and Martin Roetteler},
journal= {arXiv preprint arXiv:2604.11781},
year = {2026}
}