Related papers: Self-organized criticality in neural networks
We evolve network topology of an asymmetrically connected threshold network by a simple local rewiring rule: quiet nodes grow links, active nodes lose links. This leads to convergence of the average connectivity of the network towards the…
While modern deep networks have demonstrated remarkable versatility, their training dynamics remain poorly understood--often driven more by empirical tweaks than architectural insight. This paper investigates how internal structural choices…
Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological and artificial networks, we analyse a theoretical and a…
Gradient-based algorithms are a cornerstone of artificial neural network training, yet it remains unclear whether biological neural networks use similar gradient-based strategies during learning. Experiments often discover a diversity of…
We investigate the dynamics of two models of biological networks with purely suppressive interactions between the units; species interacting via niche competition and neurons via inhibitory synaptic coupling. In both of these cases,…
We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping…
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task related neural dynamics we study trained Recurrent Neural Networks. We develop a Mean Field Theory for Reservoir Computing…
Autonomous neural systems must efficiently process information in a wide range of novel environments, which may have very different statistical properties. We consider the problem of how to optimally distribute receptors along a…
We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a standard gradient-based training. The resulting network has the structure of a graph tailored to the particular learning…
We propose an explanation for the self-organization towards criticality observed in martensites during the cyclic process known as `training'. The scale-free behavior originates from the interplay between the reversible phase transformation…
Deciphering the underpinnings of the dynamical processes leading to information transmission, processing, and storing in the brain is a crucial challenge in neuroscience. An inspiring but speculative theoretical idea is that such dynamics…
This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infinity, its predictive variance on the training set decreases…
All crucial features of the recently observed real-world weighted networks are obtained in a model where the weight of a link is defined with a single non-linear parameter $\alpha$ as $w_{ij}=(s_is_j)^\alpha$, $s_i$ and $s_j$ are the…
This work develops the global equations of neural networks through stacked piecewise manifolds, fixed-point theory, and boundary-conditioned iteration. Once fixed coordinates and operators are removed, a neural network appears as a…
Random Boolean networks, originally invented as models of genetic regulatory networks, are simple models for a broad class of complex systems that show rich dynamical structures. From a biological perspective, the most interesting networks…
Formation of a layered structure is studied in a globally coupled map of chaotic units with a plastic coupling strength that changes depending on the states of units globally and an external input. In the parameter region characterized by…
The fundamental `plasticity' of the nervous system (i.e high adaptability at different structural levels) is primarily based on Hebbian learning mechanisms that modify the synaptic connections. The modifications rely on neural activity and…
We study a mechanism of activity sustaining on networks inspired by a well-known model of neuronal dynamics. Our primary focus is the emergence of self-sustaining collective activity patterns, where no single node can stay active by itself,…
When a physical system is driven away from equilibrium, the statistical distribution of its dynamical trajectories informs many of its physical properties. Characterizing the nature of the distribution of dynamical observables, such as a…
We are offering a particular interpretation (well within the range of experimentally and theoretically accepted notions) of neural connectivity and dynamics and discuss it as the data-and-process architecture of the visual system. In this…