Related papers: Beyond Random Matrix Theory for Deep Networks
In statistics, assuming samples are independent is reasonable. However, this property can fail to hold for the features, a distinction that has led to several lines of work aiming to remove the latter assumption of independence present in…
The wave propagation in random medium plays a critical role in optics and quantum physics. Multiple scattering of coherent wave in a random medium determines the transport procedure. Brownian motions of the scatterers perturb each…
In this paper we consider a new normalization of matrices obtained by choosing distinct codewords at random from linear codes over finite fields and find that under some natural algebraic conditions of the codes their empirical spectral…
Although the spectra of random networks have been studied for a long time, the influence of network topology on the dense limit of network spectra remains poorly understood. By considering the configuration model of networks with four…
We investigate the local spectral statistics of the loss surface Hessians of artificial neural networks, where we discover excellent agreement with Gaussian Orthogonal Ensemble statistics across several network architectures and datasets.…
Deep neural networks (DNNs) have brought significant advancements in various applications in recent years, such as image recognition, speech recognition, and natural language processing. In particular, Vision Transformers (ViTs) have…
The paper is concerned with deformed Wigner random matrices. These matrices are closely related to Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly…
The eigenvalues and eigenvectors of the connectivity matrix of complex networks contain information about its topology and its collective behavior. In particular, the spectral density $\rho(\lambda)$ of this matrix reveals important network…
We study the universality of spectral statistics of large random matrices. We consider $N\times N$ symmetric, hermitian or quaternion self-dual random matrices with independent, identically distributed entries (Wigner matrices) where the…
The celebrated Mar\v{c}enko-Pastur law, that considers the asymptotic spectral density of random covariance matrices, has found a great number of applications in physics, biology, economics, engineering, among others. Here, using techniques…
In this work, we consider the weighted difference of two independent complex Wishart matrices and derive the joint probability density function of the corresponding eigenvalues in a finite-dimension scenario using two distinct approaches.…
It is a classical result of Wigner that for an hermitian matrix with independent entries on and above the diagonal, the mean empirical eigenvalue distribution converges weakly to the semicircle law as matrix size tends to infinity. In this…
We study spectral behavior of sparsely connected random networks under the random matrix framework. Sub-networks without any connection among them form a network having perfect community structure. As connections among the sub-networks are…
We investigate the level density for several ensembles of positive random matrices of a Wishart--like structure, $W=XX^{\dagger}$, where $X$ stands for a nonhermitian random matrix. In particular, making use of the Cauchy transform, we…
The scaled standard Wigner matrix (symmetric with mean zero, variance one i.i.d. entries), and its limiting eigenvalue distribution, namely the semi-circular distribution, has attracted much attention. The $2k$th moment of the limit equals…
One of the distinguishing characteristics of modern deep learning systems is that they typically employ neural network architectures that utilize enormous numbers of parameters, often in the millions and sometimes even in the billions.…
We study an "inner-product kernel" random matrix model, whose empirical spectral distribution was shown by Xiuyuan Cheng and Amit Singer to converge to a deterministic measure in the large $n$ and $p$ limit. We provide an interpretation of…
We investigate random matrices whose entries are obtained by applying a nonlinear kernel function to pairwise inner products between $n$ independent data vectors, drawn uniformly from the unit sphere in $\mathbb{R}^d$. This study is…
We study statistical properties of a class of band random matrices which naturally appears in systems of interacting particles. The local spectral density is shown to follow the Breit-Wigner distribution in both localized and delocalized…
This work analyzes singular-value spectra of weight matrices in pretrained transformer models to understand how information is stored at both ends of the spectrum. Using Random Matrix Theory (RMT) as a zero information hypothesis, we…