相关论文: Self-Attention as Transport: Limits of Symmetric S…

We review recent results on the anomalous transport in one-dimensional and quasi-one-dimensional systems with bulk and surface disorder. Main attention is paid to the role of long-range correlations in random potentials for the bulk…

The attention interaction matrix $QK^{\top}$ contains two entangled computations: a skew-symmetric component that redistributes information between positions (routing) and a symmetric component that scales mutual relevance (filtering). We…

Trajectory prediction models in autonomous driving are vulnerable to perturbations from non-causal agents whose actions should not affect the ego-agent's behavior. Such perturbations can lead to incorrect predictions of other agents'…

We present a training-free method for detecting valid mathematical reasoning in large language models through spectral analysis of attention patterns. By treating attention matrices as adjacency matrices of dynamic graphs over tokens, we…

We introduce an operator-theoretic framework for analyzing directional dependence in multivariate time series based on order-constrained spectral non-invariance. Directional influence is defined as the sensitivity of second-order dependence…

We study self-attention dynamics on the unit sphere as an interacting particle system arising from an idealized Transformer-type update. Under a symmetry assumption on weight matrices given by $Q^\top K=V=V^\top$, the flow admits a…

Large language models can follow complex instructions in a single turn, yet over long multi-turn interactions they often lose the thread of instructions, persona, and rules. This degradation has been measured behaviorally but not…

Pedestrian intention prediction needs to be accurate for autonomous vehicles to navigate safely in urban environments. We present a lightweight, socially informed architecture for pedestrian intention prediction. It fuses four behavioral…

Multimodal large language models (MLLMs) frequently hallucinate objects that are absent from the visual input, often because attention during decoding is disproportionately drawn to visually dominant or frequently occurring content. We…

Transformer-based architectures have become the dominant paradigm for Continuous-Time Dynamic Graph (CTDG) learning, yet their performance remains limited on temporally shifted datasets. In this work, we identify attention dispersion as a…

Vision transformers using self-attention or its proposed alternatives have demonstrated promising results in many image related tasks. However, the underpinning inductive bias of attention is not well understood. To address this issue, this…

Contextual hallucinations -- statements unsupported by given context -- remain a significant challenge in AI. We demonstrate a practical interpretability insight: a generator-agnostic observer model detects hallucinations via a single…

We introduce directional routing, a lightweight mechanism that gives each transformer attention head learned suppression directions controlled by a shared router, at 3.9% parameter cost. We train a 433M-parameter model alongside an…

We study electron transport properties of a monoatomic graphite layer (graphene) with different types of disorder at half filling. We show that the transport properties of the system depend strongly on the symmetry of disorder. We find that…

The autocorrelation function of spectral determinants (ASD) is used to characterize the discrete spectrum of a phase coherent quasi- 1- dimensional, disordered wire as a function of its length L in a finite, weak magnetic field. An…

We study interference effects in the dynamics of a spin-$1/2$ particle propagating in two dimensions in a disordered potential and subject to a generalized spin-orbit coupling. With the particle initially in a spin-polarized plane wave…

This paper studies the two-user Gaussian interference channel with half-duplex causal cognition. This channel model consists of two source-destination pairs sharing a common wireless channel. One of the sources, referred to as the…

Modality differences have led to the development of heterogeneous architectures for vision and language models. While images typically require 2D non-causal modeling, texts utilize 1D causal modeling. This distinction poses significant…

As wireless networks transition toward 6G, high mobility, clustered scattering, and hardware impairments increasingly challenge classical assumptions on channel sparsity, resolvability, and stationarity. In these regimes, performance…

Self-initiated attention shifts play a critical role in voluntary behavior but are difficult to study due to the absence of explicit temporal markers. While previous studies have examined their neural correlates, it remains unclear how…