Related papers: Beyond Random Matrix Theory for Deep Networks
Random graph models are used to describe the complex structure of real-world networks in diverse fields of knowledge. Studying their behavior and fitting properties are still critical challenges, that in general, require model specific…
This paper presents a study of the properties of a matrix model that was introduced to describe transitions between all Wigner surmises of Random Matrix theory. New results include closed-form exact analytical expressions for the…
This paper explores the use of deep neural networks for semiparametric estimation of economic models of maximizing behavior in production or discrete choice. We argue that certain deep networks are particularly well suited as a…
We consider a Wigner-type ensemble, i.e. large hermitian $N\times N$ random matrices $H=H^*$ with centered independent entries and with a general matrix of variances $S_{xy}=\mathbb E|H_{xy}|^2$. The norm of $H$ is asymptotically given by…
We analyze the neutrino mass matrix entries and their correlations in a probabilistic fashion, constructing probability distribution functions using the latest results from neutrino oscillation fits. Two cases are considered: the standard…
Albeit worryingly underrated in the recent literature on machine learning in general (and, on deep learning in particular), multivariate density estimation is a fundamental task in many applications, at least implicitly, and still an open…
With the recently rapid development in deep learning, deep neural networks have been widely adopted in many real-life applications. However, deep neural networks are also known to have very little control over its uncertainty for unseen…
While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and…
The celebrated Marchenko-Pastur theorem gives the asymptotic spectral distribution of sums of random, independent, rank-one projections. Its main hypothesis is that these projections are more or less uniformly distributed on the first…
This paper introduces a new method to estimate the spectral distribution of a population covariance matrix from high-dimensional data. The method is founded on a meaningful generalization of the seminal Marcenko-Pastur equation, originally…
We analyze recurrent neural networks with diagonal hidden-to-hidden weight matrices, trained with gradient descent in the supervised learning setting, and prove that gradient descent can achieve optimality \emph{without} massive…
This paper studies the spectral behavior of large dimensional Chatterjee's rank correlation matrix when observations are independent draws from a high-dimensional random vector with independent continuous components. We show that the…
A new deep neural network based on the WaveNet architecture (WNN) is presented, which is designed to grasp specific patterns in the NMR spectra. When trained at a fixed non-uniform sampling (NUS) schedule, the WNN benefits from pattern…
The problem of inverse scattering proposed by Angles and Mallat in 2018, concerns training a deep neural network to invert the scattering transform applied to an image. After such a network is trained, it can be used as a generative model…
The advent of deep learning and recurrent neural networks revolutionized the field of time-series processing. Therefore, recent research on spectrum prediction has focused on the use of these tools. However, spectrum prediction, which…
We consider $N\times N$ Hermitian random matrices with independent identically distributed entries (Wigner matrices). The matrices are normalized so that the average spacing between consecutive eigenvalues is of order $1/N$. Under suitable…
Probabilistic models help us encode latent structures that both model the data and are ideally also useful for specific downstream tasks. Among these, mixture models and their time-series counterparts, hidden Markov models, identify…
The attempt to solve inverse scattering problems often leads to optimization and sampling problems that require handling moderate to large amounts of partial differential equations acting as constraints. We focus here on determining…
Recent work introduced deep kernel processes as an entirely kernel-based alternative to NNs (Aitchison et al. 2020). Deep kernel processes flexibly learn good top-layer representations by alternately sampling the kernel from a distribution…
Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the…