Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution
Abstract
We propose an analytically tractable neural connectivity model with power-law distributed synaptic strengths. When threshold neurons with biologically plausible number of incoming connections are considered, our model features a continuous transition to chaos and can reproduce biologically relevant low activity levels and scale-free avalanches, i.e. bursts of activity with power-law distributions of sizes and lifetimes. In contrast, the Gaussian counterpart exhibits a discontinuous transition to chaos and thus cannot be poised near the edge of chaos. We validate our predictions in simulations of networks of binary as well as leaky integrate-and-fire neurons. Our results suggest that heavy-tailed synaptic distribution may form a weakly informative sparse-connectivity prior that can be useful in biological and artificial adaptive systems.
Keywords
Cite
@article{arxiv.1910.05780,
title = {Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution},
author = {Łukasz Kuśmierz and Shun Ogawa and Taro Toyoizumi},
journal= {arXiv preprint arXiv:1910.05780},
year = {2020}
}
Comments
Accepted for publication in Physical Review Letters