Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…

Spectral clustering is a powerful method for finding structure in a dataset through the eigenvectors of a similarity matrix. It often outperforms traditional clustering algorithms such as $k$-means when the structure of the individual…

While using invariant and equivariant maps, it is possible to apply deep learning to a range of primitive data structures, a formalism for dealing with hierarchy is lacking. This is a significant issue because many practical structures are…

We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a…

In the current work, we study the eigenvalue distribution results of a class of non-normal matrix-sequences which may be viewed as a low rank perturbation, depending on a parameter $\beta>1$, of the basic Toeplitz matrix-sequence…

Determining the effect of structural perturbations on the eigenvalue spectra of networks is an important problem because the spectra characterize not only their topological structures, but also their dynamical behavior, such as…

Hyperspectral imaging, a rapidly evolving field, has witnessed the ascendancy of deep learning techniques, supplanting classical feature extraction and classification methods in various applications. However, many researchers employ…

Inspired by the importance of inhibitory and excitatory couplings in the brain, we analyze the largest eigenvalue statistics of random networks incorporating such features. We find that the largest real part of eigenvalues of a network,…

Consider large signal-plus-noise data matrices of the form $S + \Sigma^{1/2} X$, where $S$ is a low-rank deterministic signal matrix and the noise covariance matrix $\Sigma$ can be anisotropic. We establish the asymptotic joint distribution…

Deep clustering has been dominated by flat models, which split a dataset into a predefined number of groups. Although recent methods achieve an extremely high similarity with the ground truth on popular benchmarks, the information contained…

The extreme eigenvalues of adjacency matrices are important indicators on the influences of topological structures to collective dynamical behavior of complex networks. Recent findings on the ensemble averageability of the extreme…

Traditional analyses of gradient descent optimization show that, when the largest eigenvalue of the loss Hessian - often referred to as the sharpness - is below a critical learning-rate threshold, then training is 'stable' and training loss…

We propose an empirical approach centered on the spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to unify and clarify several phenomena in deep learning. We identify a consistent bias in…

Because hyperspectral remote sensing images contain a lot of redundant information and the data structure is highly non-linear, leading to low classification accuracy of traditional machine learning methods. The latest research shows that…

Hypergraphs provide a natural framework to model higher-order interactions in scientific, social, and biological systems. Hypergraph neural networks (HGNNs) aim to learn from such data, yet it remains unclear which higher-order structures…

Linear mixture models are commonly used to represent hyperspectral datacube as a linear combinations of endmember spectra. However, determining of the number of endmembers for images embedded in noise is a crucial task. This paper proposes…

Understanding the power of depth in feed-forward neural networks is an ongoing challenge in the field of deep learning theory. While current works account for the importance of depth for the expressive power of neural-networks, it remains…

This paper investigates the impact of multiscale data on machine learning algorithms, particularly in the context of deep learning. A dataset is multiscale if its distribution shows large variations in scale across different directions.…

The growing size of modern data sets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This paper studies distributed estimation for a fundamental statistical machine…

Understanding the properties of well-generalizing minima is at the heart of deep learning research. On the one hand, the generalization of neural networks has been connected to the decision boundary complexity, which is hard to study in the…

A central problem in data analysis is the low dimensional representation of high dimensional data, and the concise description of its underlying geometry and density. In the analysis of large scale simulations of complex dynamical systems,…