Related papers: Explaining Landscape Connectivity of Low-cost Solu…

While training error of most deep neural networks degrades as the depth of the network increases, residual networks appear to be an exception. We show that the main reason for this is the Lyapunov stability of the gradient descent…

The training of neural networks is a complex, high-dimensional, non-convex and noisy optimization problem whose theoretical understanding is interesting both from an applicative perspective and for fundamental reasons. A core challenge is…

Humans are increasingly stressing ecosystems via habitat destruction, climate change and global population movements leading to the widespread loss of biodiversity and the disruption of key ecological services. Ecosystems characterized…

Local optima networks (LONs) capture fitness landscape information. They are typically constructed in a black-box manner; information about the problem structure is not utilised. This also applies to the analysis of LONs: knowledge about…

We study the optimal routing on multilayered communication networks, which are composed of two layers of subnetworks. One is a wireless network, and the other is a wired network. We develop a simple recurrent algorithm to find an optimal…

Here we analyze the topology of the network formed by the minima and transition states on the potential energy landscape of small clusters. We find that this network has both a small-world and scale-free character. In contrast to other…

We study a scenario for route planning in road networks, where the objective to be optimized may change between every shortest path query. Since this invalidates many of the known speedup techniques for road networks that are based on…

The Internet is inherently a multipath network---for an underlying network with only a single path connecting various nodes would have been debilitatingly fragile. Unfortunately, traditional Internet technologies have been designed around…

Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation, sentence similarities to deep learning. Yet, its ability to capture frequently occurring structure beyond the "ground…

Multilayer networks represent systems in which there are several topological levels each one representing one kind of interaction or interdependency between the systems' elements. These networks have attracted a lot of attention recently…

Despite the widespread adoption of neural networks, their training dynamics remain poorly understood. We show experimentally that as the size of the dataset increases, a point forms where the magnitude of the gradient of the loss becomes…

In this paper we present an analytic study of sampled networks in the case of some important shortest-path sampling models. We present analytic formulas for the probability of edge discovery in the case of an evolving and a static network…

We study navigation with limited information in networks and demonstrate that many real-world networks have a structure which can be described as favoring communication at short distance at the cost of constraining communication at long…

Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between…

Highly-optimized complex transport networks serve crucial functions in many man-made and natural systems such as power grids and plant or animal vasculature. Often, the relevant optimization functional is non-convex and characterized by…

By dividing potential energy landscapes into basins of attractions surrounding minima and linking those basins that are connected by transition state valleys, a network description of energy landscapes naturally arises. These networks are…

We show that abrupt structural transitions can arise in functionally optimal networks, driven by small changes in the level of transport congestion. Our results offer an explanation as to why so many diverse species of network structure…

The objective of this paper is to enhance the optimization process for neural networks by developing a dynamic learning rate algorithm that effectively integrates exponential decay and advanced anti-overfitting strategies. Our primary…

Training an artificial neural network involves an optimization process over the landscape defined by the cost (loss) as a function of the network parameters. We explore these landscapes using optimisation tools developed for potential…

Complex networks can be used to represent and model an ample diversity of abstract and real-world systems and structures. A good deal of the research on these structures has focused on specific topological properties, including node degree,…