Related papers: Understanding Mode Connectivity via Parameter Spac…

In this study we establish connections between asymptotic functions and properties of solutions to important problems in wireless networks. We start by introducing a class of self-mappings (called asymptotic mappings) constructed with…

We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss. We show both by count and volume that most activation patterns correspond to…

A mechanism for self-organization of the degree of connectivity in model neural networks is studied. Network connectivity is regulated locally on the basis of an order parameter of the global dynamics which is estimated from an observable…

The parameter space for any fixed architecture of feedforward ReLU neural networks serves as a proxy during training for the associated class of functions - but how faithful is this representation? It is known that many different parameter…

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method…

How does connectivity impact network dynamics? We address this question by linking network characteristics on two scales. On the global scale we consider the coherence of overall network dynamics. We show that such \emph{global coherence}…

We present a generic solution to the fundamental problem of how to connect two points in a plane by a smooth curve that goes through these points with a given slope. The smoothness of any curve depends both on its curvature and its length.…

Spatial networks, in which nodes and edges are embedded in space, play a vital role in the study of complex systems. For example, many social networks attach geo-location information to each user, allowing the study of not only topological…

Symmetries (transformations by group actions) are present in many datasets, and leveraging them holds considerable promise for improving predictions in machine learning. In this work, we aim to understand when and how deep networks -- with…

Many observables of brain dynamics appear to be optimized for computation. Which connectivity structures underlie this fine-tuning? We propose that many of these structures are naturally encoded in the space that more directly relates to…

Most social, technological and biological networks are embedded in a finite dimensional space, and the distance between two nodes influences the likelihood that they link to each other. Indeed, in social systems, the chance that two…

Connectedness, path connectedness, and uniform connectedness are well-known concepts. In the traditional presentation of these concepts there is a substantial difference between connectedness and the other two notions, namely connectedness…

Neural field models are commonly used to describe wave propagation and bump attractors at a tissue level in the brain. Although motivated by biology, these models are phenomenological in nature. They are built on the assumption that the…

We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. It is demonstrated that, if a nonlinear conic approximation…

One surprising trait of neural networks is the extent to which their connections can be pruned with little to no effect on accuracy. But when we cross a critical level of parameter sparsity, pruning any further leads to a sudden drop in…

We explore some mathematical features of the loss landscape of overparameterized neural networks. A priori one might imagine that the loss function looks like a typical function from $\mathbb{R}^n$ to $\mathbb{R}$ - in particular,…

In this work, we look at the symmetry of normal modes in symmetric structures, particularly structures with cyclic symmetry. We show that normal modes of symmetric structures have different levels of symmetry, or symmetricity. One novel…

Dynamic networks have been increasingly used to characterize brain connectivity that varies during resting and task states. In such characterizations, a connectivity network is typically measured at each time point for a subject over a…

Neural networks trained via gradient descent with random initialization and without any regularization enjoy good generalization performance in practice despite being highly overparametrized. A promising direction to explain this phenomenon…

Deep learning models have proven enormously successful at using multiple layers of representation to learn relevant features of structured data. Encoding physical symmetries into these models can improve performance on difficult tasks, and…