Related papers: Depth, Not Data: An Analysis of Hessian Spectral B…
Scaling laws for neural networks, in which the loss decays as a power-law in the number of parameters, data, and compute, depend fundamentally on the spectral structure of the data covariance, with power-law eigenvalue decay appearing…
Modern data often arises with multiple modalities. For example, covariates and a network are observed on the same subjects, and both contain useful information. Effectively integrating these modalities is important and challenging,…
We consider a neural field model which consists of a network of an arbitrary number of Wilson-Cowan nodes with homeostatic adjustment of the inhibitory coupling strength and time delayed, excitatory coupling. We extend previous work on this…
Recent work has shown that tight concentration of the entire spectrum of singular values of a deep network's input-output Jacobian around one at initialization can speed up learning by orders of magnitude. Therefore, to guide important…
A neutral network is a subgraph of a Hamming graph, and its principal eigenvalue determines its robustness: the ability of a population evolving on it to withstand errors. Here we consider the most robust small neutral networks: the graphs…
The proper initialization of weights is crucial for the effective training and fast convergence of deep neural networks (DNNs). Prior work in this area has mostly focused on balancing the variance among weights per layer to maintain…
Hyperspectral images are of crucial importance in order to better understand features of different materials. To reach this goal, they leverage on a high number of spectral bands. However, this interesting characteristic is often paid by a…
Because of the significant increase in size and complexity of the networks, the distributed computation of eigenvalues and eigenvectors of graph matrices has become very challenging and yet it remains as important as before. In this paper…
The performance of spectral clustering relies on the fluctuations of the entries of the eigenvectors of a similarity matrix, which has been left uncharacterized until now. In this letter, it is shown that the signal $+$ noise structure of a…
In this paper, we focus on the unsupervised setting for structure learning of deep neural networks and propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience, to guide the…
Understanding the fundamental principles behind the massive success of neural networks is one of the most important open questions in deep learning. However, due to the highly complex nature of the problem, progress has been relatively…
Spectral clustering is a fast and popular algorithm for finding clusters in networks. Recently, Chaudhuri et al. (2012) and Amini et al.(2012) proposed inspired variations on the algorithm that artificially inflate the node degrees for…
Recently Burkhardt et. al. introduced the $k$-checkerboard random matrix ensembles, which have a split limiting behavior of the eigenvalues (in the limit all but $k$ of the eigenvalues are on the order of $\sqrt{N}$ and converge to…
Large batch size training of Neural Networks has been shown to incur accuracy loss when trained with the current methods. The exact underlying reasons for this are still not completely understood. Here, we study large batch size training…
Nonconvex minimization algorithms often benefit from the use of second-order information as represented by the Hessian matrix. When the Hessian at a critical point possesses negative eigenvalues, the corresponding eigenvectors can be used…
The spectral gap of the graph Laplacian with Dirichlet boundary conditions is computed for the graphs of several communication networks at the IP-layer, which are subgraphs of the much larger global IP-layer network. We show that the…
We present a novel approach for computing a variant of eigenvector centrality for multilayer networks with inter-layer constraints on node importance. Specifically, we consider a multilayer network defined by multiple edge-weighted,…
Quantitative ultrasound (QUS) allows estimating the intrinsic tissue properties. Speckle statistics are the QUS parameters that describe the first order statistics of ultrasound (US) envelope data. The parameters of Homodyned K-distribution…
In this work we apply deep neural networks to find the non-equilibrium steady state solution to correlated open quantum many-body systems. Motivated by the ongoing search to find more powerful representations of (mixed) quantum states, we…
We here present a stepping stone towards a deeper understanding of convolutional neural networks (CNNs) in the form of a theory of learning in linear CNNs. Through analyzing the gradient descent equations, we discover that the evolution of…