Related papers: A Random Matrix Approach to Neural Networks
We present a distributed asynchronous algorithm for approximating a single component of the solution to a system of linear equations $Ax = b$, where $A$ is a positive definite real matrix, and $b \in \mathbb{R}^n$. This is equivalent to…
The high complexity of many-body quantum dynamics means that essentially all approaches either exploit special structure or are approximate in nature. One such approach--the memory function formalism--involves a carefully chosen split into…
We study the resolvent \[ G^z = \left(\frac{1}{n}XX^T - zI_p\right)^{-1}, \qquad z\in\mathbb C,\ \Im(z)>0, \] where $X=(x_1,\ldots,x_n)\in\mathcal M_{p,n}$ is a random matrix with independent, but not necessarily identically distributed,…
We demonstrate that applying an eventual decay to the learning rate (LR) in empirical risk minimization (ERM), where the mean-squared-error loss is minimized using standard gradient descent (GD) for training a two-layer neural network with…
A key property of neural networks is their capacity of adapting to data during training. Yet, our current mathematical understanding of feature learning and its relationship to generalization remain limited. In this work, we provide a…
We study the distribution of singular values of product of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the…
We investigate the sample complexity of bounded two-layer neural networks using different activation functions. In particular, we consider the class $$ \mathcal{H} = \left\{\textbf{x}\mapsto \langle \textbf{v}, \sigma \circ W\textbf{b} +…
We establish novel rates for the Gaussian approximation of random deep neural networks with Gaussian parameters (weights and biases) and Lipschitz activation functions, in the wide limit. Our bounds apply for the joint output of a network…
Random matrix theory has become a cornerstone in modern statistics and data science, providing fundamental tools for understanding high-dimensional covariance structures. Within this framework, the Wishart matrix plays a central role in…
Using Stein's method techniques introduced by Chatterjee (2008) and further extended by Kasprzak and Peccati (2022) and by Lachi\`eze-Rey and Peccati (2017), we derive novel quantitative bounds on the convergence in distribution of…
The paper is concerned with deformed Wigner random matrices. These matrices are closely related to Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly…
Consider a $N\times n$ matrix $\Sigma_n=\frac{1}{\sqrt{n}}R_n^{1/2}X_n$, where $R_n$ is a nonnegative definite Hermitian matrix and $X_n$ is a random matrix with i.i.d. real or complex standardized entries. The fluctuations of the linear…
We consider deep neural networks with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias…
We study the problem of learning an unknown function using random feature models. Our main contribution is an exact asymptotic analysis of such learning problems with Gaussian data. Under mild regularity conditions for the feature matrix,…
We introduce a powerful analytic method to study the statistics of the number $\mathcal{N}_{\textbf{A}}(\gamma)$ of eigenvalues inside any contour $\gamma \in \mathbb{C}$ for infinitely large non-Hermitian random matrices ${\textbf A}$. Our…
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight…
The principal ratio of a connected graph $G$, $\gamma(G)$, is the ratio between the largest and smallest coordinates of the principal eigenvector of the adjacency matrix of $G$. Over all connected graphs on $n$ vertices, $\gamma(G)$ ranges…
This article characterizes the exact asymptotics of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples $n$, their dimension $p$, and the dimension of feature space $N$ are all large and…
Random Matrix Theory (RMT) has successfully modeled diverse systems, from energy levels of heavy nuclei to zeros of $L$-functions; this correspondence has allowed RMT to successfully predict many number theoretic behaviors. However there…
We consider training over-parameterized two-layer neural networks with Rectified Linear Unit (ReLU) using gradient descent (GD) method. Inspired by a recent line of work, we study the evolutions of network prediction errors across GD…