Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…
Loss landscape analysis is extremely useful for a deeper understanding of the generalization ability of deep neural network models. In this work, we propose a layerwise loss landscape analysis where the loss surface at every layer is…
We explore the block nature of the matrix representation of multiplex networks, introducing a new formalism to deal with its spectral properties as a function of the inter-layer coupling parameter. This approach allows us to derive…
Spectral Embedding (SE) has often been used to map data points from non-linear manifolds to linear subspaces for the purpose of classification and clustering. Despite significant advantages, the subspace structure of data in the original…
We study how the optimization landscape of a neural network deforms as a non-linear activation is introduced through a smooth homotopy. Working first in an abstract local setting - a smooth one-parameter family of objective functions…
We address the amount of information in the non-Gaussian regime of weak lensing surveys by modelling all relevant covariances of the power spectra and bispectra, using 1000 ray-tracing simulation realizations for a Lambda-CDM model and an…
Found in varied contexts from neurons to ants to fish, binary decision-making is one of the simplest forms of collective computation. In this process, information collected by individuals about an uncertain environment is accumulated to…
We study Bayesian inference in the spiked covariance model, where a small number of spiked eigenvalues dominate the spectrum. Our goal is to infer the spiked eigenvalues, their corresponding eigenvectors, and the number of spikes, providing…
We present a deep neural network representation of the AdS/CFT correspondence, and demonstrate the emergence of the bulk metric function via the learning process for given data sets of response in boundary quantum field theories. The…
Network representations are useful for describing the structure of a large variety of complex systems. Although most studies of real-world networks suppose that nodes are connected by only a single type of edge, most natural and engineered…
A Wishart matrix is said to be spiked when the underlying covariance matrix has a single eigenvalue $b$ different from unity. As $b$ increases through $b=2$, a gap forms from the largest eigenvalue to the rest of the spectrum, and with…
This paper is concerned with the interplay between statistical asymmetry and spectral methods. Suppose we are interested in estimating a rank-1 and symmetric matrix $\mathbf{M}^{\star}\in \mathbb{R}^{n\times n}$, yet only a randomly…
Real-world large-scale datasets are heteroskedastic and imbalanced -- labels have varying levels of uncertainty and label distributions are long-tailed. Heteroskedasticity and imbalance challenge deep learning algorithms due to the…
Eigenvalues statistics of various many-body systems have been widely studied using the nearest neighbor spacing distribution under the random matrix theory framework. Here, we numerically analyze eigenvalue ratio statistics of multiplex…
Two ubiquitous aspects of large-scale data analysis are that the data often have heavy-tailed properties and that diffusion-based or spectral-based methods are often used to identify and extract structure of interest. Perhaps surprisingly,…
Kernel spectral clustering corresponds to a weighted kernel principal component analysis problem in a constrained optimization framework. The primal formulation leads to an eigen-decomposition of a centered Laplacian matrix at the dual…
Spectral clustering is one of the most popular, yet still incompletely understood, methods for community detection on graphs. This article studies spectral clustering based on the Bethe-Hessian matrix $H_r = (r^2-1)I_n + D-rA$ for sparse…
The eigendecomposition of the coupling matrix of large biological networks is central to the study of the dynamics of these networks. For neural networks, this matrix should reflect the topology of the network and conform with Dale's law…
Higher-order structures of networks, namely, small subgraphs of networks (also called network motifs), are widely known to be crucial and essential to the organization of networks. There has been a few work studying the community detection…
Doubly-stochastic matrices (DSM) are increasingly utilized in structure-preserving deep architectures -- such as Optimal Transport layers and Sinkhorn-based attention -- to enforce numerical stability and probabilistic interpretability. In…
The Diffusion Map is a nonlinear dimensionality reduction technique used to analyze high-dimensional data, with recent applications extending to datasets from the social sciences. Previous research has given little attention to how the…