Related papers: The effect of temporal pattern of injury on disabi…
Why do neural networks trained with large learning rates for a longer time often lead to better generalization? In this paper, we delve into this question by examining the relation between training and testing loss in neural networks.…
This paper examines the impact of static sparsity on the robustness of a trained network to weight perturbations, data corruption, and adversarial examples. We show that, up to a certain sparsity achieved by increasing network width and…
Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called lazy training regime in which the network can be well approximated by its linearization around…
We develop an approach to efficiently grow neural networks, within which parameterization and optimization strategies are designed by considering their effects on the training dynamics. Unlike existing growing methods, which follow simple…
The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally…
Low dose computed tomography (CT) acquisition using reduced radiation or sparse angle measurements is recommended to decrease the harmful effects of X-ray radiation. Recent works successfully apply deep networks to the problem of low dose…
Structure entails function and thus a structural description of the brain will help to understand its function and may provide insights into many properties of brain systems, from their robustness and recovery from damage, to their dynamics…
Transport in complex networks can describe a variety of natural and human-engineered processes including biological, societal and technological ones. However, how the properties of the source and drain nodes can affect transport subject to…
In this paper, we introduce a mathematical framework to assess the impact of damage, defined as the reduction of weight in a specific link, on identical oscillator systems governed by the Kuramoto model and coupled through weighted…
On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is known as a neural scaling law. Of fundamental importance is…
A quadratic approximation of neural network loss landscapes has been extensively used to study the optimization process of these networks. Though, it usually holds in a very small neighborhood of the minimum, it cannot explain many…
We introduce the concept of self-healing in the field of complex networks. Obvious applications range from infrastructural to technological networks. By exploiting the presence of redundant links in recovering the connectivity of the…
Viewing neural network models in terms of their loss landscapes has a long history in the statistical mechanics approach to learning, and in recent years it has received attention within machine learning proper. Among other things, local…
We systematically study and compare damage spreading for random Boolean and threshold networks under small external perturbations (damage), a problem which is relevant to many biological networks. We identify a new characteristic…
Generalization of deep neural networks remains one of the main open problems in machine learning. Previous theoretical works focused on deriving tight bounds of model complexity, while empirical works revealed that neural networks exhibit…
We explore the robustness of complex networks against physical damage. We focus on spatially embedded network models and datasets where links are physical objects or physically transfer some quantity, which can be disrupted at any point…
The study of the response of complex dynamical social, biological, or technological networks to external perturbations has numerous applications. Random Boolean Networks (RBNs) are commonly used a simple generic model for certain dynamics…
Network properties govern the rate and extent of various spreading processes, from simple contagions to complex cascades. Recently, the analysis of spreading processes has been extended from static networks to temporal networks, where nodes…
In this work and the supporting Parts II [2] and III [3], we provide a rather detailed analysis of the stability and performance of asynchronous strategies for solving distributed optimization and adaptation problems over networks. We…
In this paper, we propose that a tree-like network with damage can be modeled as the product of a fractional-order nominal plant and a fractional-order multiplicative disturbance, which is well structured and completely characterized by the…