Related papers: Threshold Disorder as a Source of Diverse and Comp…

Recurrent Neural Networks (RNNs) were recently successfully used to model the way neural activity drives task-related behavior in animals, operating under the implicit assumption that the obtained solutions are universal. Observations in…

Threshold networks are used as models for neural or gene regulatory networks. They show a rich dynamical behaviour with a transition between a frozen and a chaotic phase. We investigate the phase diagram of randomly connected threshold…

In biological evolution complex neural structures grow from a handful of cellular ingredients. As genomes in nature are bounded in size, this complexity is achieved by a growth process where cells communicate locally to decide whether to…

We investigate species-rich mathematical models of ecosystems. While much of the existing literature focuses on the properties of equilibrium fixed points, persistent dynamics (e.g., limit cycles or chaos) have also been observed, both in…

Threshold rules of spreading in binary-state networks lead to cascades. We study persistent cascade-recovery dynamics on quasi-robust networks, i.e., networks which are robust against small trigger but may collapse for larger one. It is…

In specific motifs of three recurrently connected neurons with probabilistic response, the spontaneous information flux, defined as the mutual information between subsequent states, has been shown to increase by adding ongoing white noise…

Recurrent Neural Networks (RNN) are widely used to solve a variety of problems and as the quantity of data and the amount of available compute have increased, so have model sizes. The number of parameters in recent state-of-the-art networks…

Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in…

Deep neural networks (DNNs) frequently present behaviorally irregular patterns, significantly limiting their practical potentials and theoretical validity in travel behavior modeling. This study proposes strong and weak behavioral…

Inhibition is considered to shape neural activity, and broaden its pattern repertoire. In the sensory organs, where the anatomy of neural circuits is highly structured, lateral inhibition sharpens contrast among stimulus properties. The…

Existing analyses of neural network training often operate under the unrealistic assumption of an extremely small learning rate. This lies in stark contrast to practical wisdom and empirical studies, such as the work of J. Cohen et al.…

In this paper, we investigate temporal clusters of extremes defined as subsequent exceedances of high thresholds in a stationary time series. Two meaningful features of these clusters are the probability distribution of the cluster size and…

Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is…

Task-trained recurrent neural networks (RNNs) are widely used in neuroscience and machine learning to model dynamical computations. To gain mechanistic insight into how neural systems solve tasks, prior work often reverse-engineers…

Recurrent Neural Networks (RNNs) are frequently used to model aspects of brain function and structure. In this work, we trained small fully-connected RNNs to perform temporal and flow control tasks with time-varying stimuli. Our results…

Characterizing the in uence of network properties on the global emerging behavior of interacting elements constitutes a central question in many areas, from physical to social sciences. In this article we study a primary model of disordered…

Recurrent neural networks are deep learning topologies that can be trained to classify long documents. However, in our recent work, we found a critical problem with these cells: they can use the length differences between texts of different…

The main objective of this work is to explore aspects of stochastic resonance (SR) in noisy bistable, symmetric systems driven by subthreshold periodic rectangular external signals possessing a large duty cycle of unity. Using a precise…

Low-dimensional yet rich dynamics often emerge in the brain. Examples include oscillations and chaotic dynamics during sleep, epilepsy, and voluntary movement. However, a general mechanism for the emergence of low dimensional dynamics…

The problem of learning long-term dependencies in sequences using Recurrent Neural Networks (RNNs) is still a major challenge. Recent methods have been suggested to solve this problem by constraining the transition matrix to be unitary…