Related papers: Neural Fields and Noise-Induced Patterns in Neuron…

To understand the training dynamics of neural networks (NNs), prior studies have considered the infinite-width mean-field (MF) limit of two-layer NN, establishing theoretical guarantees of its convergence under gradient flow training as…

We establish central and non-central limit theorems for sequences of functionals of the Gaussian output of an infinitely-wide random neural network on the d-dimensional sphere . We show that the asymptotic behaviour of these functionals as…

We study the asymptotic behaviour for asymmetric neuronal dynamics in a network of Hopfield neurons. The randomness in the network is modelled by random couplings which are centered Gaussian correlated random variables. We prove that the…

We review a recent approach to the mean-field limits in neural networks that takes into account the stochastic nature of input current and the uncertainty in synaptic coupling. This approach was proved to be a rigorous limit of the network…

Neural network field theory formulates field theory as a statistical ensemble of fields defined by a network architecture and a density on its parameters. We extend the construction to topological settings via the inclusion of discrete…

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie…

Growing evidence suggests that synaptic weights in the brain follow heavy-tailed distributions, yet most theoretical analyses of recurrent neural networks (RNNs) assume Gaussian connectivity. We systematically study the activity of RNNs…

Despite impressive performance on numerous visual tasks, Convolutional Neural Networks (CNNs) --- unlike brains --- are often highly sensitive to small perturbations of their input, e.g. adversarial noise leading to erroneous decisions. We…

We continue the work of a series of previous studies of a mathematical model that describes the mean-field limit behavior of a homogeneous network of excitatory point spiking neurons. Contrary to other models, here noise is intrinsic to the…

The dynamics of neural networks is often characterized by collective behavior and quasi-synchronous events, where a large fraction of neurons fire in short time intervals, separated by uncorrelated firing activity. These global temporal…

Neurons in the central nervous system are affected by complex and noisy signals due to fluctuations in their cellular environment and in the inputs they receive from many other cells 1,2. Such noise usually increases the probability that a…

An exact low-dimensional system of mean-field equations for an infinite-size network of pulse coupled integrate-and-fire neurons with a bimodal distribution of an excitability parameter is derived. Bifurcation analysis of these equations…

In many modern applications of deep learning the neural network has many more parameters than the data points used for its training. Motivated by those practices, a large body of recent theoretical research has been devoted to studying…

The continuous integration of experimental data into coherent models of the brain is an increasing challenge of modern neuroscience. Such models provide a bridge between structure and activity, and identify the mechanisms giving rise to…

Residual deep neural networks are formulated as interacting particle systems leading to a description through neural differential equations, and, in the case of large input data, through mean-field neural networks. The mean-field…

The Bayesian view of the brain hypothesizes that the brain constructs a generative model of the world, and uses it to make inferences via Bayes' rule. Although many types of approximate inference schemes have been proposed for hierarchical…

We extend previous mean-field approaches for non-equilibrium neural network models to estimate correlations in the system. This offers a powerful tool for approximating the system dynamics as well as a fast method to infer network…

In this paper we present a general framework in which to rigorously study the effect of spatio-temporal noise on traveling waves and stationary patterns. In particular the framework can incorporate versions of the stochastic neural field…

We provide an empirical study of the stability of recurrent neural networks trained to recognize regular languages. When a small amount of noise is introduced into the activation function, the neurons in the recurrent layer tend to saturate…

We present a study of a phase-separation process induced by the presence of spatially-correlated multiplicative noise. We develop a mean-field approach suitable for conserved-order-parameter systems and use it to obtain the phase diagram of…