Related papers: Exploring Neural Network Landscapes: Star-Shaped a…
High-throughput methods for yielding the set of connections in a neural system, the connectome, are now being developed. This tutorial describes ways to analyze the topological and spatial organization of the connectome at the macroscopic…
Using a generalized random recurrent neural network model, and by extending our recently developed mean-field approach [J. Aljadeff, M. Stern, T. Sharpee, Phys. Rev. Lett. 114, 088101 (2015)], we study the relationship between the network…
The cortex is a very large network characterized by a complex connectivity including at least two scales: a microscopic scale at which the interconnections are non-specific and very dense, while macroscopic connectivity patterns connecting…
We discuss several aspects of the loss landscape of regularized neural networks: the structure of stationary points, connectivity of optimal solutions, path with nonincreasing loss to arbitrary global optimum, and the nonuniqueness of…
We study the connected ensemble, a statistical-mechanics framework that characterizes the formation of low-loss paths in rugged landscapes. First introduced in a previous paper, this ensemble allows one to identify when a network can be…
Random geometric networks consist of 1) a set of nodes embedded randomly in a bounded domain $\mathcal{V} \subseteq \mathbb{R}^d$ and 2) links formed probabilistically according to a function of mutual Euclidean separation. We quantify how…
Most social, technological and biological networks are embedded in a finite dimensional space, and the distance between two nodes influences the likelihood that they link to each other. Indeed, in social systems, the chance that two…
Data describing the three-dimensional structure of physical networks is increasingly available, leading to a surge of interest in network science to explore the relationship between the shape and connectivity of physical networks. We…
Systematic relations between multiple objects that occur in various fields can be represented as networks. Real-world networks typically exhibit complex topologies whose structural properties are key factors in characterizing and further…
A simple and accurate relationship is demonstrated that links the average shortest path, nodes, and edges in a complex network. This relationship takes advantage of the concept of link density and shows a large improvement in fitting…
Training Deep Neural Networks relies on the model converging on a high-dimensional, non-convex loss landscape toward a good minimum. Yet, much of the phenomenology of training remains ill understood. We focus on three seemingly disparate…
We draw connections between simple neural networks and under-determined linear systems to comprehensively explore several interesting theoretical questions in the study of neural networks. First, we emphatically show that it is unsurprising…
Artificial neural networks which are inspired from the learning mechanism of brain have achieved great successes in many problems, especially those with deep layers. In this paper, we propose a nucleus neural network (NNN) and corresponding…
We consider dynamical and geometrical aspects of deep learning. For many standard choices of layer maps we display semi-invariant metrics which quantify differences between data or decision functions. This allows us, when considering random…
We study the effects of spatial constraints on the structural properties of networks embedded in one or two dimensional space. When nodes are embedded in space, they have a well defined Euclidean distance $r$ between any pair. We assume…
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we…
Metric graph properties lie in the heart of the analysis of complex networks, while in this paper we study their convexity through mathematical definition of a convex subgraph. A subgraph is convex if every geodesic path between the nodes…
This paper describes how realistic neuromorphic networks can have their connectivity properties fully characterized in analytical fashion. By assuming that all neurons have the same shape and are regularly distributed along the…
Recently, dense connections have attracted substantial attention in computer vision because they facilitate gradient flow and implicit deep supervision during training. Particularly, DenseNet, which connects each layer to every other layer…
The connectivity structure of graphs is typically related to the attributes of the nodes. In social networks for example, the probability of a friendship between two people depends on their attributes, such as their age, address, and…