Related papers: Self-organized criticality in neural networks

A hallmark of living systems is the ability to employ a common set of versatile building blocks that can self-organize into a multitude of different structures, in a way that can be controlled with minimal cost. This capability can only be…

Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \textit{stationary} data distribution. In settings where this…

We construct and analyze a rate-based neural network model in which self-interacting units represent clusters of neurons with strong local connectivity and random inter-unit connections reflect long-range interactions. When sufficiently…

A given neural network in the brain is involved in many different tasks. This implies that, when considering a specific task, the network's connectivity contains a component which is related to the task and another component which can be…

Gradient-based meta-learning algorithms have gained popularity for their ability to train models on new tasks using limited data. Empirical observations indicate that such algorithms are able to learn a shared representation across tasks,…

Attention is drawn to the possibility that self-organizing biological neural networks could spontaneously acquire the capability to carry out sophisticated computations. In particular it is shown that the effective action governing the…

The relation between neural activity and behaviorally relevant variables is at the heart of neuroscience research. When strong, this relation is termed a neural representation. There is increasing evidence, however, for partial…

Scale-free behavior as well as oscillations are frequently observed in the activity of many natural systems. One important example is the cortical tissues of mammalian brain where both phenomena are simultaneously observed. Rhythmic…

The principles of self-organizing the neural networks of optimal complexity is considered under the unrepresentative learning set. The method of self-organizing the multi-layered neural networks is offered and used to train the logical…

Recurrent neural networks (RNNs) trained on low-dimensional tasks have been widely used to model functional biological networks. However, the solutions found by learning and the effect of initial connectivity are not well understood. Here,…

The brain modifies its synaptic strengths during learning in order to better adapt to its environment. However, the underlying plasticity rules that govern learning are unknown. Many proposals have been suggested, including Hebbian…

The presence of noise in non linear dynamical systems can play a constructive role, increasing the degree of order and coherence or evoking improvements in the performance of the system. An example of this positive influence in a biological…

Novel computing hardwares are necessary to keep up with today's increasing demand for data storage and processing power. In this research project, we turn to the brain for inspiration to develop novel computing substrates that are…

Neural networks are known to be highly sensitive to adversarial examples. These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem. To better understand these factors, we…

In this paper, the question how spiking neural network (SNN) learns and fixes in its internal structures a model of external world dynamics is explored. This question is important for implementation of the model-based reinforcement learning…

We consider neural networks from the point of view of dynamical systems theory. In this spirit we review recent results dealing with the following questions, adressed in the context of specific models. 1. Characterizing the collective…

On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is known as a neural scaling law. Of fundamental importance is…

Scale invariance profoundly influences the dynamics and structure of complex systems, spanning from critical phenomena to network architecture. Here, we propose a precise definition of scale-invariant networks by leveraging the concept of a…

We consider a discrete-time nonatomic routing game with variable demand and uncertain costs. Given a routing network with single origin and destination, the cost function of each edge depends on some uncertain persistent state parameter. At…

A massively recurrent neural network responds on one side to input stimuli and is autonomously active, on the other side, in the absence of sensory inputs. Stimuli and information processing depends crucially on the qualia of the…