Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…

We study structure, eigenvalue spectra and diffusion dynamics in a wide class of networks with subgraphs (modules) at mesoscopic scale. The networks are grown within the model with three parameters controlling the number of modules, their…

Spectral clustering approaches have led to well-accepted algorithms for finding accurate clusters in a given dataset. However, their application to large-scale datasets has been hindered by computational complexity of eigenvalue…

Graphs (i.e., networks) have become an integral tool for the representation and analysis of relational data. Advances in data gathering have lead to multi-relational data sets which exhibit greater depth and scope. In certain cases, this…

We develop new theoretical results on matrix perturbation to shed light on the impact of architecture on the performance of a deep network. In particular, we explain analytically what deep learning practitioners have long observed…

Spectral correlations in unitary invariant, non-Gaussian ensembles of large random matrices possessing an eigenvalue gap are studied within the framework of the orthogonal polynomial technique. Both local and global characteristics of…

Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights. However, such models face the challenge of Bayesian inference in a high-dimensional and usually…

We explore the spectra and localization properties of the N-site banded one-dimensional non-Hermitian random matrices that arise naturally in sparse neural networks. Approximately equal numbers of random excitatory and inhibitory…

We extend classical time-frequency limiting analysis, historically applied to one-dimensional finite signals, to the multidimensional discrete setting. This extension is relevant for images, videos, and other multidimensional signals, as it…

We study some properties of eigenvalue spectra of financial correlation matrices. In particular, we investigate the nature of the large eigenvalue bulks which are observed empirically, and which have often been regarded as a consequence of…

There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks. We propose and experimentally verify a unified phenomenological model of the loss landscape that incorporates many of them. High…

We study structure formation in the presence of primordial non-Gaussianity of the local type with parameters f_NL and g_NL. We show that the distribution of dark-matter halos is naturally described by a multivariate bias scheme where the…

We study spectral behavior of sparsely connected random networks under the random matrix framework. Sub-networks without any connection among them form a network having perfect community structure. As connections among the sub-networks are…

This paper introduces a novel framework that combines traditional centrality measures with eigenvalue spectra and diffusion processes for a more comprehensive analysis of complex networks. While centrality measures such as degree,…

Modeling uncertainty in deep neural networks, despite recent important advances, is still an open problem. Bayesian neural networks are a powerful solution, where the prior over network weights is a design choice, often a normal…

We consider highly heterogeneous random networks with symmetric interactions in the limit of high connectivity. A key feature of this system is that the spectral density of the corresponding ensemble exhibits a divergence within the bulk.…

Advances in simultaneous recordings of large numbers of neurons have driven significant interest in the structure of neural population activity such as dimension. A key question is how these dynamic features arise mechanistically and their…

We introduce a method for describing eigenvalue distributions of correlation matrices from multidimensional time series. Using our newly developed matrix H theory, we improve the description of eigenvalue spectra for empirical correlation…

Deep declarative networks and other recent related works have shown how to differentiate the solution map of a (continuous) parametrized optimization problem, opening up the possibility of embedding mathematical optimization problems into…

Non-Hermitian random matrices with statistical spectral characteristics beyond the standard Ginibre ensembles have recently emerged in the description of dissipative quantum many-body systems as well as in non-ergodic wave transport in…

Hyperspectral imagery is composed of huge amount of data which creates significant transmission latencies for communication systems. It is vital to decrease the huge data size before transmitting the Hyperspectral imagery. Besides, large…