Related papers: Beyond Random Matrix Theory for Deep Networks
Random matrix theory is a well-developed area of probability theory that has numerous connections with other areas of mathematics and its applications. Much of the literature in this area is concerned with matrices that possess many exact…
A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems and to express different statistical operations such as measurement,…
We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and outlier spectral dynamics for spiked…
We study statistical properties of energy spectra of a tight-binding model on the two-dimensional quasiperiodic Ammann-Beenker tiling. Taking into account the symmetries of finite approximants, we find that the underlying universal…
The class of norm-dependent Random Matrix Ensembles is studied in the presence of an external field. The probability density in those ensembles depends on the trace of the squared random matrices, but is otherwise arbitrary. An exact…
We consider fully connected and feedforward deep neural networks with dependent and possibly heavy-tailed weights, as introduced in [26], to address limitations of the standard Gaussian prior. It has been proved in [26] that, as the number…
Our goal is to study statistical properies of "dielectric resonances" which are poles of conductance of a large random $LC$ network. Such poles are a particular example of eigenvalues $\lambda_n$ of matrix pencils ${\bf H}-\lambda {\bf W}$,…
Random Matrix Theory (RMT) has successfully modeled diverse systems, from energy levels of heavy nuclei to zeros of $L$-functions; this correspondence has allowed RMT to successfully predict many number theoretic behaviors. However there…
We investigate the asymptotic behavior of the eigenvalues of spiked perturbations of Wigner matrices when the dimension goes to infinity. The entries of the Hermitian Wigner matrix have a distribution which is symmetric and satisfies a…
We introduce kernel estimators for the semicircle law. In this first part of our general theory on the estimators, we prove the consistency and conduct simulation study to show the performance of the estimators. We also point out that…
For a class of symmetric random matrices whose entries are martingale differences adapted to an increasing filtration, we prove that under a Lindeberg-like condition, the empirical spectral distribution behaves asymptotically similarly to a…
We apply the method of determinants to study the distribution of the largest singular values of large $ m \times n $ real rectangular random matrices with independent Cauchy entries. We show that statistical properties of the (rescaled by a…
We compute the spectral statistics of the sum H of two independent complex Wishart matrices, each of which is correlated with a different covariance matrix. Random matrix theory enjoys many applications including sums and products of random…
We consider a random symmetric matrix ${\bf X} = [X_{jk}]_{j,k=1}^n$ with upper triangular entries being i.i.d. random variables with mean zero and unit variance. We additionally suppose that $\mathbb E |X_{11}|^{4 + \delta} =:…
We introduce a one-parameter deformation of the Wishart-Laguerre or chiral ensembles of positive definite random matrices with Dyson index beta=1,2 and 4. Our generalised model has a fat-tailed distribution while preserving the invariance…
Mixture Density Networks (MDNs) can be used to generate probability density functions of model parameters $\boldsymbol{\theta}$ given a set of observables $\mathbf{x}$. In some applications, training data are available only for discrete…
This paper investigates efficient deep neural networks (DNNs) to replace dense unstructured weight matrices with structured ones that possess desired properties. The challenge arises because the optimal weight matrix structure in popular…
Neural networks have achieved remarkable performance in various application domains. Nevertheless, a large number of weights in pre-trained deep neural networks prohibit them from being deployed on smartphones and embedded systems. It is…
We construct a diffusive matrix model for the $\beta$-Wishart (or Laguerre) ensemble for general continuous $\beta\in [0,2]$, which preserves invariance under the orthogonal/unitary group transformation. Scaling the Dyson index $\beta$ with…
The model of heavy Wigner matrices generalizes the classical ensemble of Wigner matrices: the sub-diagonal entries are independent, identically distributed along to and out of the diagonal, and the moments its entries are of order 1/N,…