Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…
We introduce a principled approach for unsupervised structure learning of deep neural networks. We propose a new interpretation for depth and inter-layer connectivity where conditional independencies in the input distribution are encoded…
Hessians of neural network (NN) contain essential information about the curvature of NN loss landscapes which can be used to estimate NN generalization capabilities. We have previously proposed generalization criteria that rely on the…
We study the spectra and eigenvectors of the adjacency matrices of scale-free networks when bi-directional interaction is allowed, so that the adjacency matrix is real and symmetric. The spectral density shows an exponential decay around…
Low dimensional structures appear ubiquitously in the eigenspectra of deep learning matrices in classification networks trained in the overparameterized regime. While theoretical advances have aimed to explain this phenomenology, they…
We study the task of clustering in directed networks. We show that using the eigenvalue/eigenvector decomposition of the adjacency matrix is simpler than all common methods which are based on a combination of data regularization and SVD…
A large variety of dynamical processes that take place on networks can be expressed in terms of the spectral properties of some linear operator which reflects how the dynamical rules depend on the network topology. Often such spectral…
The paper deals with the distribution of singular values of the input-output Jacobian of deep untrained neural networks in the limit of their infinite width. The Jacobian is the product of random matrices where the independent rectangular…
The lack of theoretical results for Layer Normalization and feedforward Hessians has left a gap in the study of Transformer optimization landscapes. We address this by deriving explicit second-order expressions for these components, thereby…
Looking to overcome the limitations of traditional networks, the network science community has lately given much attention to the so-called higher-order networks, where group interactions are modeled alongside pairwise ones. While degree…
Many natural and social systems develop complex networks, that are usually modelled as random graphs. The eigenvalue spectrum of these graphs provides information about their structural properties. While the semi-circle law is known to…
The spiked covariance model has gained increasing popularity in high-dimensional data analysis. A fundamental problem is determination of the number of spiked eigenvalues, $K$. For estimation of $K$, most attention has focused on the use of…
Euclidean random matrices arise in a wide range of physical systems where interactions are determined by spatial configurations, including disordered media and cooperative phenomena in atomic ensembles. Unlike classical random matrix…
This paper focuses on large neural networks whose synaptic connectivity matrices are randomly chosen from certain random matrix ensembles. The dynamics of these networks can be characterized by the eigenvalue spectra of their connectivity…
In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we…
Empirical studies reported that the Hessian matrix of neural networks (NNs) exhibits a near-block-diagonal structure, yet its theoretical foundation remains unclear. In this work, we reveal that the reported Hessian structure comes from a…
The spectral properties of the adjacency matrix provide a trove of information about the structure and function of complex networks. In particular, the largest eigenvalue and its associated principal eigenvector are crucial in the…
We revisit the theoretical performances of Spectral Clustering, a classical algorithm for graph partitioning that relies on the eigenvectors of a matrix representation of the graph. Informally, we show that Spectral Clustering works well as…
The energy landscape of multiverse cosmology is often modeled by a multi-dimensional random Gaussian potential. The physical predictions of such models crucially depend on the eigenvalue distribution of the Hessian matrix at potential…
This work investigates the multiplicity and differentiability of eigenfrequencies in structures with various symmetries. In particular, the study explores how the geometric and design variable symmetries affect the distribution of…
Chaotic behavior or lack thereof in non-Hermitian systems is often diagnosed via spectral analysis of associated complex eigenvalues. Very recently, singular values of the associated non-Hermitian systems have been proposed as an effective…