Related papers: Spectral Selection in Symmetric Self-Attention Dyn…
The aim of this paper is to study the convergence of the primal-dual dynamics pertaining to Support Vector Machines (SVM). The optimization routine, used for determining an SVM for classification, is first formulated as a dynamical system.…
This paper proposes a variational self-attention model (VSAM) that employs variational inference to derive self-attention. We model the self-attention vector as random variables by imposing a probabilistic distribution. The self-attention…
The scale-free nature of gravitational interaction in both Newtonian gravity and the general theory of relativity gives rise to the concept of self-similarity, where solutions are scale invariant. As a result of this property, the governing…
We present the first systematic study of weight matrix singular value spectra \emph{during} transformer pretraining, tracking full SVD decompositions of every weight matrix at 25-step intervals across three model scales (30M--285M…
We consider model of a complex particle that consists of a rigid shell and a nucleus with spatial asymmetric interaction. The particle's dynamics with the nucleus driven by a periodic excitation is considered. It is shown that…
State-space analysis is widely employed for examining power system dynamics but faces challenges in large-scale power systems integrated with numerous inverter-based resources (IBRs), where the significant increase of system states…
The asymptotic analysis of kinetic models describing the behavior of particles interacting through alignment is performed. We will analyze the asymptotic regime corresponding to large alignment frequency where the alignment effects are…
The self-adjoint matrix Sturm-Liouville operator on a finite interval with a boundary condition in the general form is studied. We obtain asymptotic formulas for the eigenvalues and the weight matrices of the considered operator. These…
We investigate the coupled dynamics of concentration and charge in asymmetric 1:1 electrolytes, focusing on the interplay between diffusion asymmetry and external electric fields. Using Brownian dynamics simulations and linearized…
We describe some general results that constrain the dynamical fluctuations that can occur in non-equilibrium steady states, with a focus on molecular dynamics. That is, we consider Hamiltonian systems, coupled to external heat baths, and…
A heterogeneous medium of constituents with vastly different mechanical properties, whose inhomogeneities are in close proximity to each other, is considered. The gradient of the solution to the corresponding problem exhibits singular…
The full spectrum of transfer matrices of the general eight-vertex model on a square lattice is obtained by numerical diagonalization. The eigenvalue spacing distribution and the spectral rigidity are analyzed. In non-integrable regimes we…
Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…
We have proposed a method for the dynamic simulation of a collection of self-propelled particles in a viscous Newtonian fluid. We restrict attention to particles whose size and velocity are small enough that the fluid motion is in the…
We consider the hydrodynamic theory of an active fluid of self-propelled particles with nematic aligning interactions. This class of materials has polar symmetry at the microscopic level, but forms macrostates of nematic symmetry. We…
We consider the problem of asymptotic stability of a self-similar attractor for a simple semilinear radial wave equation which arises in the study of the Yang-Mills equations in 5+1 dimensions. Our analysis consists of two steps. In the…
The aim of this article is to investigate the convergence properties of a heterogeneous consensus model on Stiefel manifolds. We consider each agent, without interaction, moving according to the flow determined by the fundamental vector…
Kernel-based learning methods can dramatically increase the storage capacity of Hopfield networks, yet the dynamical mechanisms behind this enhancement remain poorly understood. We address this gap by combining a geometric characterization…
Large language model pre-training typically exhibits a two-phase trajectory: a fast initial loss drop followed by a prolonged slow improvement. We identify an underlying spectral phenomenon, Stability of Singular Distribution (SoSD), where…
Ensembles of quantum chaotic systems are expected to exhibit energy eigenvalues with random-matrix-like level repulsion between pairs of energies separated by less than the inverse Thouless time. Recent research has shown that exact and…