Related papers: Collective evolution of weights in wide neural net…
Complex networks serve as abstract models for understanding real-world complex systems and provide frameworks for studying structured dynamical systems. This article addresses limitations in current studies on the exploration of individual…
Several problems in machine learning are naturally expressed as the design and analysis of time-evolving probability distributions. This includes sampling via diffusion methods, optimizing the weights of neural networks, and analyzing the…
We consider kinetic systems and prove their stability working in weighted spaces in which the systems are symmetric. We prove stability for various explicit and implicit semi-discrete and fully discrete schemes. The applications include…
For percolating systems, we propose a universal exponent relation connecting the leading corrections to scaling of the cluster size distribution with the dynamic corrections to the asymptotic transport behaviour at criticality. Our…
This paper considers the equilibrium-free stability and performance analysis of discrete-time nonlinear systems. We consider two types of equilibrium-free notions. Namely, the universal shifted concept, which considers stability and…
The largely successful method of training neural networks is to learn their weights using some variant of stochastic gradient descent (SGD). Here, we show that the solutions found by SGD can be further improved by ensembling a subset of the…
Predictive models that generalize well under distributional shift are often desirable and sometimes crucial to building robust and reliable machine learning applications. We focus on distributional shift that arises in causal inference from…
In this paper we study the problem of learning Rectified Linear Units (ReLUs) which are functions of the form $max(0,<w,x>)$ with $w$ denoting the weight vector. We study this problem in the high-dimensional regime where the number of…
In this work we systematically analyze general properties of differential equations used as machine learning models. We demonstrate that the gradient of the loss function with respect to to the hidden state can be considered as a…
We consider an evolution equation of parabolic type in R having a travelling wave solution. We perform an appropriate change of variables which transforms the equation into a non local evolution one having a travelling wave solution with…
We report on recent progress in the study of evolution processes involving degenerate parabolic equations what may exhibit free boundaries. The equations we have selected follow to recent trends in diffusion theory: considering anomalous…
We consider the dynamics of gradient descent (GD) in overparameterized single hidden layer neural networks with a squared loss function. Recently, it has been shown that, under some conditions, the parameter values obtained using GD achieve…
Using a simple model with link removals as well as link additions, we show that an evolving network is scale free with a degree exponent in the range of (2, 4]. We then establish a relation between the network evolution and a set of…
We present a general variational framework for the training of freeform nonlinearities in layered computational architectures subject to some slope constraints. The regularization that we add to the traditional training loss penalizes the…
We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian…
Many experiments have been performed that use evolutionary algorithms for learning the topology and connection weights of a neural network that controls a robot or virtual agent. These experiments are not only performed to better understand…
We study a class of stochastic evolution equations with a dissipative forcing nonlinearity and additive noise. The noise is assumed to satisfy rather general assumptions about the form of the covariance function; our framework covers…
The paper introduces the weighted convolution, a novel approach to the convolution for signals defined on regular grids (e.g., 2D images) through the application of an optimal density function to scale the contribution of neighbouring…
We study a nonlinear recombination model from population genetics as a combinatorial version of the Kac-Boltzmann equation from kinetic theory. Following Kac's approach, the nonlinear model is approximated by a mean field linear evolution…
In this work we study a simple evolutionary model of bipartite networks which its evolution is based on the duplication of nodes. Using analytical results along with numerical simulation of the model, we show that the above evolutionary…