Related papers: Beyond Random Matrix Theory for Deep Networks

We consider Gaussian ensembles of m N x N complex matrices. We identify an enhanced symmetry in the system and the resultant closed subsector, which is naturally associated with the radial sector of the theory. The density of radial…

The paper deals with distribution of singular values of product of random matrices arising in the analysis of deep neural networks. The matrices resemble the product analogs of the sample covariance matrices, however, an important…

We extend the proof of the local semicircle law for generalized Wigner matrices given in [4] to the case when the matrix of variances has an eigenvalue $ -1 $. In particular, this result provides a short proof of the optimal local…

We study a $q$-deformed random unitary ensemble associated with the little-$q$ Laguerre weight, which provides a discrete analogue of the classical Laguerre unitary ensemble. In the double scaling regime $q=e^{-\lambda/N}$, where $N$ is the…

We study the effect of mini-batching on the loss landscape of deep neural networks using spiked, field-dependent random matrix theory. We demonstrate that the magnitude of the extremal values of the batch Hessian are larger than those of…

We investigate an additive perturbation of a complex Wishart random matrix and prove that a large deviation principle holds for the spectral measures. The rate function is associated to a vector equilibrium problem coming from logarithmic…

In recent years there has been increased interest in understanding the interplay between deep generative models (DGMs) and the manifold hypothesis. Research in this area focuses on understanding the reasons why commonly-used DGMs succeed or…

Artificial neural networks are functions depending on a finite number of parameters typically encoded as weights and biases. The identification of the parameters of the network from finite samples of input-output pairs is often referred to…

Dot product kernels, such as polynomial and exponential (softmax) kernels, are among the most widely used kernels in machine learning, as they enable modeling the interactions between input features, which is crucial in applications like…

As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping inputs to correct outputs are extremely difficult to…

Deep neural networks have amply demonstrated their prowess but estimating the reliability of their predictions remains challenging. Deep Ensembles are widely considered as being one of the best methods for generating uncertainty estimates…

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal…

We consider $N\times N$ symmetric or hermitian random matrices with independent, identically distributed entries where the probability distribution for each matrix element is given by a measure $\nu$ with a subexponential decay. We prove…

The universal eigenvalue distribution characterizing the Gram matrix of semiclassical ensembles of black hole microstates is recognized as the Marchenko-Pastur distribution, which plays a prominent role as the universal limit distribution…

This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight matrix is modeled as the sum of signal and noise matrices. The…

This paper studies the asymptotic spectral properties of the sample covariance matrix for high dimensional compositional data, including the limiting spectral distribution, the limit of extreme eigenvalues, and the central limit theorem for…

We consider ensembles of Wigner matrices, whose entries are (up to the symmetry constraints) independent and identically distributed random variables. We show the convergence of the Stieltjes transform towards the Stieltjes transform of the…

A new universality class distinct from the standard Wigner-Dyson ones is identified. This class is realized by putting a metallic quantum dot in contact with a superconductor, while applying a magnetic field so as to make the pairing field…

It has previously been hypothesized, and supported with some experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation. We study the following related…

We define a class of "algebraic" random matrices. These are random matrices for which the Stieltjes transform of the limiting eigenvalue distribution function is algebraic, i.e., it satisfies a (bivariate) polynomial equation. The Wigner…