Related papers: Random matrix analysis of deep neural network weig…

Even when neural networks are widely used in a large number of applications, they are still considered as black boxes and present some difficulties for dimensioning or evaluating their prediction error. This has led to an increasing…

Recurrent neural networks (RNNs) are known to be universal approximators of dynamic systems under fairly mild and general assumptions. However, RNNs usually suffer from the issues of vanishing and exploding gradients in standard RNN…

We propose a novel method for analyzing and visualizing the complexity of standard reinforcement learning (RL) benchmarks based on score distributions. A large number of policy networks are generated by randomly guessing their parameters,…

Neural networks have been successfully used for classification tasks in a rapidly growing number of practical applications. Despite their popularity and widespread use, there are still many aspects of training and classification that are…

We investigate the spectral properties of linear-width feed-forward neural networks, where the sample size is asymptotically proportional to network width. Empirically, we show that the spectra of weight in this high dimensional regime are…

It is described how one comes to the Wigner-Dyson random matrix theory (RMT) starting from a model of a disordered metal. The lectures start with a historical introduction where basic ideas of the RMT and theory of disordered metals are…

We consider the random Markov matrix obtained by assigning i.i.d. non-negative weights to each edge of the complete oriented graph. In this study, the weights have unbounded first moment and belong to the domain of attraction of an…

We study general singular value shrinkage estimators in high-dimensional regression and classification, when the number of features and the sample size both grow proportionally to infinity. We allow models with general covariance matrices…

Despite the popularity and success of deep learning, there is limited understanding of when, how, and why neural networks generalize to unseen examples. Since learning can be seen as extracting information from data, we formally study…

The paper deals with the distribution of singular values of the input-output Jacobian of deep untrained neural networks in the limit of their infinite width. The Jacobian is the product of random matrices where the independent rectangular…

This is an expository account of the edge eigenvalue distributions in random matrix theory and their application in multivariate statistics. The emphasis is on the Painlev\'e representations of these distributions.

We consider fully connected and feedforward deep neural networks with dependent and possibly heavy-tailed weights, as introduced in [26], to address limitations of the standard Gaussian prior. It has been proved in [26] that, as the number…

We discuss non-Gaussian random matrices whose elements are random variables with heavy-tailed probability distributions. In probability theory heavy tails of the distributions describe rare but violent events which usually have dominant…

In random matrix theory (RMT), the Tracy-Widom (TW) distribution describes the behavior of the largest eigenvalue. We consider here two models in which TW undergoes transformations. In the first one disorder is introduced in the Gaussian…

Random contractions (sub-unitary random matrices) appear naturally when considering quantized chaotic maps within a general theory of open linear stationary systems with discrete time. We analyze statistical properties of complex…

The traditional class of elliptical distributions is extended to allow for asymmetries. A completely robust dispersion matrix estimator (the `spectral estimator') for the new class of `generalized elliptical distributions' is presented. It…

We study deep neural networks (DNNs) trained on natural image data with entirely random labels. Despite its popularity in the literature, where it is often used to study memorization, generalization, and other phenomena, little is known…

In this short note we collect together known results on the use of Random Matrix Theory in lattice statistical mechanics. The purpose here is two fold. Firstly the RMT analysis provides an intrinsic characterization of integrability, and…

The Recurrent Neural Networks and their variants have shown promising performances in sequence modeling tasks such as Natural Language Processing. These models, however, turn out to be impractical and difficult to train when exposed to very…

This study delves into the intricate dynamics of trained deep neural networks and their relationships with network parameters. Trained networks predominantly continue training in a single direction, known as the drift mode. This drift mode…