Related papers: Self-organized criticality in neural networks

Scale invariance is a central organizing principle in physics, underlying phenomena that range from critical behaviour in statistical mechanics to transport and chaos in nonlinear dynamical systems. Here we present a unified and physically…

A minimal model for self-organized critical percolation on directed graphs with activating and de-activating links is studied. Unlike classical self-organized criticality, the variables that determine criticality are separated from the…

The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network's prediction, when confronted with a learning task. This iterative change can be naturally…

Neuromorphic networks can be described in terms of coarse-grained variables, where emergent sustained behaviours spontaneously arise if stochasticity is properly taken in account. For example it has been recently found that a directed…

In this paper we introduce a neural network model of self-organization. This model uses a variation of Hebb rule for updating its synaptic weights, and surely converges to the equilibrium status. The key point of the convergence is the…

The critical brain hypothesis posits that neural systems operate near a phase transition, optimizing the processing of information. While scale invariance and non-Gaussian dynamics--hallmarks of criticality--have been observed in brain…

We study how the connectivity within a recurrent neural network determines and is determined by the multistable solutions of network activity. To gain analytic tractability we let neural activation be a non-smooth Heaviside step function.…

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie…

We show that a network of spiking neurons exhibits robust self-organized criticality if the synaptic efficacies follow realistic dynamics. Deriving analytical expressions for the average coupling strengths and inter-spike intervals, we…

Artificial neural networks which are inspired from the learning mechanism of brain have achieved great successes in many problems, especially those with deep layers. In this paper, we propose a nucleus neural network (NNN) and corresponding…

We study the effect of learning dynamics on network topology. A network of discrete dynamical systems is considered for this purpose and the coupling strengths are made to evolve according to a temporal learning rule that is based on the…

As a promising computational paradigm, occurrence of critical states in artificial and biological neural networks has attracted wide-spread attention. An often-made explicit or implicit assumption is that one single critical state is…

We consider a fully-connected network of leaky integrate-and-fire neurons with spike-timing-dependent plasticity. The plasticity is controlled by a parameter representing the expected weight of a synapse between neurons that are firing…

Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning. However, little effort has been spent on improving RNN architectures and on…

In recent years self organised critical neuronal models have provided insights regarding the origin of the experimentally observed avalanching behaviour of neuronal systems. It has been shown that dynamical synapses, as a form of short-term…

Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why deep neural nets work so well today. In this paper, we…

As deep neural networks grow in size, from thousands to millions to billions of weights, the performance of those networks becomes limited by our ability to accurately train them. A common naive question arises: if we have a system with…

Neural circuits exhibit complex activity patterns, both spontaneously and evoked by external stimuli. Information encoding and learning in neural circuits depend on how well time-varying stimuli can control spontaneous network activity. We…

While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood. In this work, we study the case of binary classification and prove various…

Many natural systems are organized as networks, in which the nodes (be they cells, individuals or populations) interact in a time-dependent fashion. The dynamic behavior of these networks depends on how these nodes are connected, which can…