Related papers: A Random Matrix Approach to Neural Networks

We consider the spectrum of additive, polynomially vanishing random perturbations of deterministic matrices, as follows. Let $M_N$ be a deterministic $N\times N$ matrix, and let $G_N$ be a complex Ginibre matrix. We consider the matrix…

Consider an $N\times n$ random matrix $Y_n=(Y^n_{ij})$ where the entries are given by $Y^n_{ij}=\frac{\sigma_{ij}(n)}{\sqrt{n}}X^n_{ij}$, the $X^n_{ij}$ being independent and identically distributed, centered with unit variance and…

We apply the approach of the Google matrix, used in computer science and World Wide Web, to description of properties of neuronal networks. The Google matrix ${\bf G}$ is constructed on the basis of neuronal network of a brain model…

A crucial property for achieving secure, trustworthy and interpretable deep learning systems is their robustness: small changes to a system's inputs should not result in large changes to its outputs. Mathematically, this means one strives…

We consider the following situation: G is a finite directed graph, where to each vertex of G is assigned an element of a finite group Gamma. We consider all walks of length N on G, starting from v_i and ending at v_j To each such walk $w$…

We apply random matrix theory to derive spectral density of large sample covariance matrices generated by multivariate VMA(q), VAR(q) and VARMA(q1,q2) processes. In particular, we consider a limit where the number of random variables N and…

We consider the basic problem of learning Single-Index Models with respect to the square loss under the Gaussian distribution in the presence of adversarial label noise. Our main contribution is the first computationally efficient algorithm…

We investigate random matrices whose entries are obtained by applying a nonlinear kernel function to pairwise inner products between $n$ independent data vectors, drawn uniformly from the unit sphere in $\mathbb{R}^d$. This study is…

We consider the problem of sampling from the Ising model when the underlying interaction matrix has eigenvalues lying within an interval of length $\gamma$. Recent work in this setting has shown various algorithmic results that apply…

We study the extent to which wide neural networks may be approximated by Gaussian processes when initialized with random weights. It is a well-established fact that as the width of a network goes to infinity, its law converges to that of a…

In this article we consider Wigner matrices $X_N$ with variance profiles (also called Wigner-type matrices) which are of the form $X_N(i,j) = \sigma(i/N,j/N) a_{i,j} / \sqrt{N}$ where $\sigma$ is a symmetric real positive function of…

We investigate the sample complexity of networks with bounds on the magnitude of its weights. In particular, we consider the class \[ H=\left\{W_t\circ\rho\circ \ldots\circ\rho\circ W_{1} :W_1,\ldots,W_{t-1}\in M_{d, d}, W_t\in…

We investigate the generalization error of group-invariant neural networks within the Barron framework. Our analysis shows that incorporating group-invariant structures introduces a group-dependent factor $\delta_{G,\Gamma,\sigma} \le 1$…

We prove that the diameter of threshold (zero temperature) Geometric Inhomogeneous Random Graphs (GIRG) is $\Theta(\log n)$. This has strong implications for the runtime of many distributed protocols on those graphs, which often have…

In image generation, generative models can be evaluated naturally by visually inspecting model outputs. However, this is not always the case for graph generative models (GGMs), making their evaluation challenging. Currently, the standard…

The exponential random graph model (ERGM) is a central object in the study of clustering properties in social networks as well as canonical ensembles in statistical physics. Despite some breakthrough works in the mathematical understanding…

We consider large non-Hermitian random matrices $X$ with complex, independent, identically distributed centred entries and show that the linear statistics of their eigenvalues are asymptotically Gaussian for test functions having…

In recent years, graph neural networks (GNNs) have gained increasing popularity and have shown very promising results for data that are represented by graphs. The majority of GNN architectures are designed based on developing new…

We prove density results for neural-network classes on compact sets \(K\subset X\), where \(X\) is a topological vector space whose continuous dual \(X^*\) separates points. Let \(\Psi:\mathbb R\to\mathbb R\) be a continuous squashing…

The rapid adoption of synthetic data for training Large Language Models (LLMs) has introduced the technical challenge of "model collapse"-a degenerative process where recursive training on model-generated content leads to a contraction of…