Related papers: On Random Matrices Arising in Deep Neural Networks…
We study the distribution of singular values of product of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the…
The paper deals with the distribution of singular values of the input-output Jacobian of deep untrained neural networks in the limit of their infinite width. The Jacobian is the product of random matrices where the independent rectangular…
This thesis reviews recent progress on products of random matrices from the perspective of exactly solved Gaussian random matrix models. We derive exact formulae for the correlation functions for the eigen- and singular values at arbitrary…
Large random matrices appear in different fields of mathematics and physics such as combinatorics, probability theory, statistics, operator theory, number theory, quantum field theory, string theory etc... In the last ten years, they…
We study the properties of the eigenvalues of real random matrices and their products. It is known that when the matrix elements are Gaussian-distributed independent random variables, the fraction of real eigenvalues tends to unity as the…
Neural network models are one of the most successful approaches to machine learning, enjoying an enormous amount of development and research over recent years and finding concrete real-world applications in almost any conceivable area of…
Random matrix theory has played a major role in several areas of pure and applied mathematics, as well as statistics, physics, and computer science. This lecture aims to describe the intrinsic freeness phenomenon and how it provides new…
We establish, under a moment matching hypothesis, the local universality of the correlation functions associated with products of $M$ independent iid random matrices, as $M$ is fixed, and the sizes of the matrices tend to infinity. This…
We prove two universality results for random tensors of arbitrary rank D. We first prove that a random tensor whose entries are N^D independent, identically distributed, complex random variables converges in distribution in the large N…
Free probability and random matrix theory has shown to be a fruitful combination in many fields of research, such as digital communications, nuclear physics and mathematical finance. The link between free probability and eigenvalue…
Neural networks have been used successfully in a variety of fields, which has led to a great deal of interest in developing a theoretical understanding of how they store the information needed to perform a particular task. We study the…
In order to have a better understanding of finite random matrices with non-Gaussian entries, we study the $1/N$ expansion of local eigenvalue statistics in both the bulk and at the hard edge of the spectrum of random matrices. This gives…
We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices. Specifically, we employ a matrix variate Gaussian \cite{gupta1999matrix} parameter posterior…
We consider fully connected and feedforward deep neural networks with dependent and possibly heavy-tailed weights, as introduced in [26], to address limitations of the standard Gaussian prior. It has been proved in [26] that, as the number…
Random matrices formed from i.i.d. standard real Gaussian entries have the feature that the expected number of real eigenvalues is non-zero. This property persists for products of such matrices, independently chosen, and moreover it is…
Free probability theory started in the 1980s has attracted much attention lately in signal processing and communications areas due to its applications in large size random matrices. However, it involves with massive mathematical concepts…
When inferring parameters from a Gaussian-distributed data set by computing a likelihood, a covariance matrix is needed that describes the data errors and their correlations. If the covariance matrix is not known a priori, it may be…
We show that, under mild assumptions, the spectrum of a sum of independent random matrices is close to that of the Gaussian random matrix whose entries have the same mean and covariance. This nonasymptotic universality principle yields…
This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called \textit{concentrated} random vectors. Further exploiting the fact…
We introduce a new class of large structured random matrices characterized by four fundamental properties which we discuss. We prove that this class is stable under matrix-valued and pointwise non-linear operations. We then formulate an…