Related papers: QAMA: Scalable Quantum Annealing Multi-Head Attent…

Chunked prefill has become a widely adopted serving strategy for long-context large language models, but efficient attention computation in this regime remains challenging. Existing sparse attention methods are primarily designed for…

Large language models (LLMs) now support context windows of hundreds of thousands to millions of tokens, enabling applications such as long-document summarization, large-scale code synthesis, multi-document question answering and persistent…

We introduce the reinforcement quantum annealing (RQA) scheme in which an intelligent agent interacts with a quantum annealer that plays the stochastic environment role of learning automata and tries to iteratively find better Ising…

The field of Quantum Computing has gathered significant popularity in recent years and a large number of papers have studied its effectiveness in tackling many tasks. We focus in particular on Quantum Annealing (QA), a meta-heuristic solver…

Large Language Models (LLMs) have emerged as a pivotal research area, yet the attention module remains a critical bottleneck in LLM inference, even with techniques like KVCache to mitigate redundant computations. While various top-$k$…

We propose the Quantum Graph Attention Network (QGAT), a hybrid graph neural network that integrates variational quantum circuits into the attention mechanism. At its core, QGAT employs strongly entangling quantum circuits with…

Detecting active users in a non-orthogonal multiple access (NOMA) network poses a significant challenge for 5G/6G applications. Traditional algorithms tackling this task, relying on classical processors, have to make a compromise between…

We present Learning-Driven Annealing (LDA), a framework that links individual quantum annealing evolutions into a global solution strategy to mitigate hardware constraints such as short annealing times and integrated control errors. Unlike…

In a recent study (Ref. [1]), quantum annealing was reported to exhibit a scaling advantage for approximately solving Quadratic Unconstrained Binary Optimization (QUBO). However, this claim critically depends on the choice of classical…

Quantum Annealing (QA) is one of the most promising frameworks for quantum optimization. Here, we focus on the problem of minimizing complex classical cost functions associated with prototypical discrete neural networks, specifically the…

Multi-Head Attention (MHA) is the core computational primitive underlying modern Large Language Models (LLMs). However, MHA suffers from a fundamental linear scaling limitation: $H$ attention heads produce exactly $H$ independent attention…

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable…

Long-context modeling is crucial for next-generation language models, yet the high computational cost of standard attention mechanisms poses significant computational challenges. Sparse attention offers a promising direction for improving…

Quantum annealing is a computational paradigm in which optimisation problems are mapped onto the energy landscape of an interacting quantum system and explored through its dynamical evolution. By continuously transforming a simple initial…

Long-context understanding is crucial for many NLP applications, yet transformers struggle with efficiency due to the quadratic complexity of self-attention. Sparse attention methods alleviate this cost but often impose static, predefined…

Attention serves as the fundamental mechanism for long-context modeling in large language models (LLMs), yet dense attention becomes structurally prohibitive for long sequences due to its quadratic complexity. Consequently, sparse attention…

The quadratic complexity of standard attention mechanisms poses a significant scalability bottleneck for large language models (LLMs) in long-context scenarios. While hybrid attention strategies that combine sparse and full attention within…

The success of the self-attention mechanism in classical machine learning models has inspired the development of quantum analogs aimed at reducing computational overhead. Self-attention integrates learnable query and key matrices to…

We assess the prospects for algorithms within the general framework of quantum annealing (QA) to achieve a quantum speedup relative to classical state of the art methods in combinatorial optimization and related sampling tasks. We argue for…

Quantization-aware training (QAT) has achieved remarkable success in low-bit ($\leq$4-bit) quantization for classification networks. However, when applied to more complex visual tasks such as object detection and image segmentation,…