Related papers: Random matrix analysis of deep neural network weig…

We conjecture that the inherent difference in generalisation between adaptive and non-adaptive gradient methods in deep learning stems from the increased estimation noise in the flattest directions of the true loss surface. We demonstrate…

We propose a generalization of the random matrix theory following the basic prescription of the recently suggested concept of superstatistics. Spectral characteristics of systems with mixed regular-chaotic dynamics are expressed as weighted…

We compute analytically the probability distribution and moments of the sum and product of the non-zero eigenvalues and singular values of random matrices with (i) non-negative entries, (ii) fixed rank, and (iii) prescribed sums of the…

This article investigates the use of random feature neural networks for learning Kolmogorov partial (integro-)differential equations associated to Black-Scholes and more general exponential L\'evy models. Random feature neural networks are…

We formulate gaussian and circular random-matrix models representing a coupled system consisting of an absorbing and an amplifying resonator, which are mutually related by a generalized time-reversal symmetry. Motivated by optical…

We study heavy-tailed Hermitian random matrices that are unitarily invariant. The invariance implies that the eigenvalue and eigenvector statistics are decoupled. The motivating question has been whether a freely stable random matrix has…

Evaluation of generative models is mostly based on the comparison between the estimated distribution and the ground truth distribution in a certain feature space. To embed samples into informative features, previous works often use…

We employ the random matrix theory (RMT) framework to revisit the distribution of resonance widths in quantum chaotic systems weakly coupled to the continuum via a finite number M of open channels. In contrast to the standard first-order…

Deep learning algorithms are well-known to have a propensity for fitting the training data very well and often fit even outliers and mislabeled data points. Such fitting requires memorization of training data labels, a phenomenon that has…

In a recently published paper [1], it is shown that deep neural networks (DNNs) with random Gaussian weights preserve the metric structure of the data, with the property that the distance shrinks more when the angle between the two data…

Deep neural networks are usually trained in the space of the nodes, by adjusting the weights of existing links via suitable optimization protocols. We here propose a radically new approach which anchors the learning process to reciprocal…

Deep neural networks (DNNs) can easily fit a random labeling of the training data with zero training error. What is the difference between DNNs trained with random labels and the ones trained with true labels? Our paper answers this…

We show how well known rules of back propagation arise from a weighted combination of finite automata. By redefining a finite automata as a predictor we combine the set of all $k$-state finite automata using a weighted majority algorithm.…

Investigating the dynamics of learning in machine learning algorithms is of paramount importance for understanding how and why an approach may be successful. The tools of physics and statistics provide a robust setting for such…

Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what…

We review some applications of central limit theorems and extreme values statistics in the context of disordered systems. We discuss several problems, in particular concerning Random Matrix Theory and the generalisation of the Tracy-Widom…

We derive the sampling properties of random networks based on weights whose pairwise products parameterize independent Bernoulli trials. This enables an understanding of many degree-based network models, in which the structure of realized…

We study the behavior of untrained neural networks whose weights and biases are randomly distributed using mean field theory. We show the existence of depth scales that naturally limit the maximum depth of signal propagation through these…

We analyze statistical properties of complex eigenvalues of random matrices $\hat{A}$ close to unitary. Such matrices appear naturally when considering quantized chaotic maps within a general theory of open linear stationary systems with…

Assumptions about invariances or symmetries in data can significantly increase the predictive power of statistical models. Many commonly used models in machine learning are constraint to respect certain symmetries in the data, such as…