Related papers: Beyond Random Matrix Theory for Deep Networks
We consider the singular vectors of any $m \times n$ submatrix of a rectangular $M \times N$ Gaussian matrix and study their asymptotic overlaps with those of the full matrix, in the macroscopic regime where $N \,/\, M\,$, $m \,/\, M$ as…
We introduce deep tensor networks, which are exponentially wide neural networks based on the tensor network representation of the weight matrices. We evaluate the proposed method on the image classification (MNIST, FashionMNIST) and…
Learning in Deep Neural Networks (DNN) takes place by minimizing a non-convex high-dimensional loss function, typically by a stochastic gradient descent (SGD) strategy. The learning process is observed to be able to find good minimizers…
It is impossible today to pretend that the practice of machine learning is always compatible with the idea that training and testing data follow the same distribution. Several authors have recently used ensemble techniques to show how…
Most of exotic resonances observed in the past decade appear as peak structure near some threshold. These near-threshold phenomena can be interpreted as genuine resonant states or enhanced threshold cusps. Apparently, there is no…
We consider a deep matrix factorization model of covariance matrices trained with the Bures-Wasserstein distance. While recent works have made advances in the study of the optimization problem for overparametrized low-rank matrix…
We consider large Hermitian matrices whose entries are defined by evaluating the exponential function along orbits of the skew-shift $\binom{j}{2} \omega+jy+x \mod 1$ for irrational $\omega$. We prove that the eigenvalue distribution of…
In the current paper we consider a Wigner matrix and consider an analytic function of polynomial growth on a set containing the support of the semicircular law in its interior. We prove that the linear spectral statistics corresponding to…
Vinberg cones and the ambient vector spaces are important in modern statistics of sparse models and of graphical models. The aim of this paper is to study eigenvalue distributions of Gaussian, Wigner and covariance matrices related to…
Ensembling neural networks is an effective way to increase accuracy, and can often match the performance of individual larger models. This observation poses a natural question: given the choice between a deep ensemble and a single neural…
We consider a versatile matrix model of the form ${\bf A}+i {\bf B}$, where ${\bf A}$ and ${\bf B}$ are real random circulant matrices with independent but, in general, nonidentically distributed Gaussian entries. For this model, we derive…
The spectral form factor of random matrix theory plays a key role in the description of disordered and chaotic quantum systems. While its moments are known to be approximately Gaussian, corrections subleading in the matrix dimension, $D$,…
In novelty detection, the goal is to decide if a new data point should be categorized as an inlier or an outlier, given a training dataset that primarily captures the inlier distribution. Recent approaches typically use deep encoder and…
We study here the so-called spiked Wigner and Wishart models, where one observes a low-rank matrix perturbed by some Gaussian noise. These models encompass many classical statistical tasks such as sparse PCA, submatrix localization,…
We review recent developments on quantum scattering from mesoscopic systems. Various spatial geometries whose closed analogs shows diffusive, localized or critical behavior are considered. These are features that cannot be described by the…
Consider a random symmetric matrix with i.i.d.~entries on and above its diagonal that are products of Bernoulli random variables and random variables with sub-Gaussian tails. Such a matrix will be called a sparse Wigner matrix and can be…
Wishart ensembles of random matrix theory have been useful in modeling positive definite matrices encountered in classical and quantum chaotic systems. We consider nonzero means for the entries of the constituting matrix A which defines the…
Deep learning has been applied to various tasks in the field of machine learning and has shown superiority to other common procedures such as kernel methods. To provide a better theoretical understanding of the reasons for its success, we…
We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables. We demonstrate…
We study the fundamental limits of detecting the presence of an additive rank-one perturbation, or spike, to a Wigner matrix. When the spike comes from a prior that is i.i.d. across coordinates, we prove that the log-likelihood ratio of the…