Related papers: Logical Randomized Benchmarking
In this paper, we introduce a new approach to quantum benchmarking inspired by quantum verification motivating new paradigms of quantum benchmarking. Our proposed benchmark not only serves as a robust indicator of computational capability…
Quantum error correction in general is experimentally challenging as it requires significant expansion of the size of quantum circuits and accurate performance of quantum gates to fulfill the error threshold requirement. Here we propose a…
We propose network benchmarking: a procedure to efficiently benchmark the quality of a quantum network link connecting quantum processors in a quantum network. This procedure is based on the standard randomized benchmarking protocol and…
The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks. For example, recent work in programming education has used LLMs to generate learning…
The rapid development of large language models (LLMs) has necessitated the creation of benchmarks to evaluate their performance. These benchmarks resemble human tests and surveys, as they consist of sets of questions designed to measure…
As evaluation designs of large language models may shape our trajectory toward artificial general intelligence, comprehensive and forward-looking assessment is essential. Existing benchmarks primarily assess static knowledge, while…
A minimal depth quantum circuit implementing 5-qubit quantum error correction in a manner optimized for a linear nearest neighbor architecture is described. The canonical decomposition is used to construct fast and simple gates that…
There have been a huge number of benchmarks proposed to evaluate how large language models (LLMs) behave for logic inference tasks. However, it remains an open question how to properly evaluate this ability. In this paper, we provide a…
Large Language Models (LLMs) achieve strong performance on logical reasoning benchmarks, yet their reliability remains uncertain. Existing evaluations rely on static benchmarks, which fail to assess robustness under logically equivalent…
Phase estimation is a quantum algorithm for measuring the eigenvalues of a Hamiltonian. We propose and rigorously analyse a randomized phase estimation algorithm with two distinctive features. First, our algorithm has complexity independent…
Context: Software engineering has a problem in that when we empirically evaluate competing prediction systems we obtain conflicting results. Objective: To reduce the inconsistency amongst validation study results and provide a more formal…
A quantum measurement is logically reversible if the premeasurement density operator of the measured system can be calculated from the postmeasurement density operator and from the outcome of the measurement. This paper analyzes why many…
Human evaluation for natural language generation (NLG) often suffers from inconsistent user ratings. While previous research tends to attribute this problem to individual user preferences, we show that the quality of human judgements can…
While fundamental scientific researchers are eagerly anticipating the breakthroughs of quantum computing both in theory and technology, the current quantum computer, i.e. noisy intermediate-scale quantum (NISQ) computer encounters a…
Quantum Computing (QC) is undergoing a high rate of development, investment and research devoted to its improvement.However, there is little consensus in the industry and wider literature as to what improvement might consist of beyond…
Symbolic logical reasoning is a critical yet underexplored capability of large language models (LLMs), providing reliable and verifiable decision-making in high-stakes domains such as mathematical reasoning and legal judgment. In this…
Benchmarking methods that can be adapted to multi-qubit systems are essential for assessing the overall or "holistic" performance of nascent quantum processors. The current industry standard is Clifford randomized benchmarking (RB), which…
Large language models (LLMs) are increasingly deployed in settings where reasoning, such as multi-step problem solving and chain-of-thought, is essential. Yet, current evaluation practices overwhelmingly report single-run accuracy while…
Reliability and failure detection of large language models (LLMs) is critical for their deployment in high-stakes, multi-step reasoning tasks. Prior work explores confidence estimation for self-evaluating LLM-scorer systems, with confidence…
We perform randomized benchmarking on neutral atomic quantum bits (qubits) confined in an optical lattice. Single qubit gates are implemented using microwaves, resulting in a measured error per randomized computational gate of 1.4(1) x…