Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…
Quantifying the eigenvalue spectra of large random matrices allows one to understand the factors that contribute to the stability of dynamical systems with many interacting components. This work explores the effect that the interaction…
The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negative eigenvalues. We observe that for wider networks,…
We investigate how in complex systems the eigenpairs of the matrices derived from the correlations of multichannel observations reflect the cluster structure of the underlying networks. For this we use daily return data from the NYSE and…
The spectrum of the non-backtracking matrix plays a crucial role in determining various structural and dynamical properties of networked systems, ranging from the threshold in bond percolation and non-recurrent epidemic processes, to…
In high-dimensional principal component analysis, important inferential targets include both leading spikes and the associated principal eigenspaces. Such problems arise naturally in high-dimensional factor models, where leading principal…
The largest eigenvalue of a network's adjacency matrix and its associated principal eigenvector are key elements for determining the topological structure and the properties of dynamical processes mediated by it. We present a physically…
This paper examines the relationship between sparse random network architectures and neural network stability by examining the eigenvalue spectral distribution. Specifically, we generalise classical eigenspectral results to sparse…
Recently, there has been growing evidence that if the width and depth of a neural network are scaled toward the so-called rich feature learning limit (\mup and its depth extension), then some hyperparameters -- such as the learning rate --…
The study of complex networks has been one of the most active fields in science in recent decades. Spectral properties of networks (or graphs that represent them) are of fundamental importance. Researchers have been investigating these…
Although the spectra of random networks have been studied for a long time, the influence of network topology on the dense limit of network spectra remains poorly understood. By considering the configuration model of networks with four…
The eigenvalues of matrices representing the structure of large-scale complex networks present a wide range of applications, from the analysis of dynamical processes taking place in the network to spectral techniques aiming to rank the…
Quantum neural networks hold promise for tackling computationally challenging tasks that are intractable for classical computers. However, their practical application is hindered by significant optimization challenges, arising from complex…
Covariance and Hessian matrices have been analyzed separately in the literature for classification problems. However, integrating these matrices has the potential to enhance their combined power in improving classification performance. We…
Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be…
We apply state-of-the-art tools in modern high-dimensional numerical linear algebra to approximate efficiently the spectrum of the Hessian of modern deepnets, with tens of millions of parameters, trained on real data. Our results…
The distribution of the ratios of consecutive eigenvalue spacings of random matrices has emerged as an important tool to study spectral properties of many-body systems. This article numerically investigates the eigenvalue ratios…
Stochastic gradients for deep neural networks exhibit strong correlations along the optimization trajectory, and are often aligned with a small set of Hessian eigenvectors associated with outlier eigenvalues. Recent work shows that…
This paper focuses on obtaining clustering information about a distribution from its i.i.d. samples. We develop theoretical results to understand and use clustering information contained in the eigenvectors of data adjacency matrices based…
Using the diagrammatic method, we derive a set of self-consistent equations that describe eigenvalue distributions of large correlated asymmetric random matrices. The matrix elements can have different variances and be correlated with each…
The vibrational behavior of molecules serves as a crucial fingerprint of their structure, chemical state, and surrounding environment. Neutron vibrational spectroscopy provides comprehensive measurements of vibrational modes without…