Related papers: A Random Matrix Approach to Neural Networks

Let $X_N$ be an $N\ts N$ random symmetric matrix with independent equidistributed entries. If the law $P$ of the entries has a finite second moment, it was shown by Wigner \cite{wigner} that the empirical distribution of the eigenvalues of…

Given a locally finite graph $\Gamma$, an amenable subgroup $G$ of graph automorphisms acting freely and almost transitively on its vertices, and a $G$-invariant activity function $\lambda$, consider the free energy $f_G(\Gamma,\lambda)$ of…

We introduce the notion of a random matrix-valued multiplicative function, generalizing Rademacher random multiplicative functions to matrices. We provide an asymptotic for the second moment based on a linear recurrence property for…

We extend results known for the randomized Gauss-Seidel and the Gauss-Southwell methods for the case of a Hermitian and positive definite matrix to certain classes of non-Hermitian matrices. We obtain convergence results for a whole range…

Exciting new work on the generalization bounds for neural networks (NN) given by Neyshabur et al. , Bartlett et al. closely depend on two parameter-depenedent quantities: the Lipschitz constant upper-bound and the stable rank (a softer…

We propose an Gaussian Mixture Model (GMM) learning algorithm, based on our previous work of GMM expansion idea. The new algorithm brings more robustness and simplicity than classic Expectation Maximization (EM) algorithm. It also improves…

We obtain nonasymptotic bounds on the spectral norm of random matrices with independent entries that improve significantly on earlier results. If $X$ is the $n\times n$ symmetric matrix with $X_{ij}\sim N(0,b_{ij}^2)$, we show that…

"Toeplitzification" or "redundancy (spatial) averaging", the well-known routine for deriving the Toeplitz covariance matrix estimate from the standard sample covariance matrix, recently regained new attention due to the important Random…

We consider graph classes $\mathcal G$ in which every graph has components in a class $\mathcal{C}$ of connected graphs. We provide a framework for the asymptotic study of $\lvert\mathcal{G}_{n,N}\rvert$, the number of graphs in…

Given an $n\times n$ symmetric matrix $W\in [0,1]^{[n]\times [n]}$, let $\mathcal{G}(n,W)$ be the random graph obtained by independently including each edge $jk$ with probability $W_{jk}$. Given a degree sequence ${\bf d}=(d_1,\ldots,…

This paper is concerned with the asymptotic empirical eigenvalue distribution of a non linear random matrix ensemble. More precisely we consider $M= \frac{1}{m} YY^*$ with $Y=f(WX)$ where $W$ and $X$ are random rectangular matrices with…

Within a random-matrix-theory approach, we use the nearest-neighbor energy level spacing distribution $P(s)$ and the entropic eigenfunction localization length $\ell$ to study spectral and eigenfunction properties (of adjacency matrices) of…

We consider the problem of learning a one-hidden-layer neural network: we assume the input $x\in \mathbb{R}^d$ is from Gaussian distribution and the label $y = a^\top \sigma(Bx) + \xi$, where $a$ is a nonnegative vector in $\mathbb{R}^m$…

These lecture notes provide a comprehensive, self-contained introduction to the analysis of Wishart matrix moments. This study may act as an introduction to some particular aspects of random matrix theory, or as a self-contained exposition…

We compute spectra of large stochastic matrices $W$, defined on sparse random graphs, where edges $(i,j)$ of the graph are given positive random weights $W_{ij}>0$ in such a fashion that column sums are normalized to one. We compute spectra…

A semi-parametric, non-linear regression model in the presence of latent variables is applied towards learning network graph structure. These latent variables can correspond to unmodeled phenomena or unmeasured agents in a complex system of…

The limiting distribution of eigenvalues of N x N random matrices has many applications. One of the most studied ensembles are real symmetric matrices with independent entries iidrv; the limiting rescaled spectral measure (LRSM)…

Consider the sample covariance matrix $$\Sigma^{1/2}XX^T\Sigma^{1/2}$$ where $X$ is an $M\times N$ random matrix with independent entries and $\Sigma$ is an $M\times M$ diagonal matrix. It is known that if $\Sigma$ is deterministic, then…

We study the universality of spectral statistics of large random matrices. We consider $N\times N$ symmetric, hermitian or quaternion self-dual random matrices with independent, identically distributed entries (Wigner matrices) where the…

We introduce a random matrix model where the entries are dependent across both rows and columns. More precisely, we investigate matrices of the form $\X=(X_{(i-1)n+t})_{it}\in\R^{p\times n}$ derived from a linear process $X_t=\sum_j c_j…