Related papers: Threshold Disorder as a Source of Diverse and Comp…

A key attribute that drives the unprecedented success of modern Recurrent Neural Networks (RNNs) on learning tasks which involve sequential data, is their ability to model intricate long-term temporal dependencies. However, a well…

Disordered elastic networks provide a framework for describing a wide variety of physical systems, ranging from amorphous solids, through polymeric fibrous materials to confluent cell tissues. In many cases, such networks feature two widely…

The temporal activity of many biological systems, including neural circuits, exhibits fluctuations simultaneously varying over a large range of timescales. The mechanisms leading to this temporal heterogeneity are yet unknown. Here we show…

The residual neural network (ResNet) is a popular deep network architecture which has the ability to obtain high-accuracy results on several image processing problems. In order to analyze the behavior and structure of ResNet, recent work…

We study the effect of width on the dynamics of feature-learning neural networks across a variety of architectures and datasets. Early in training, wide neural networks trained on online data have not only identical loss curves but also…

Complex network formalism allows to explain the behavior of systems composed by interacting units. Several prototypical network models have been proposed thus far. The small-world model has been introduced to mimic two important features…

There has been great interest in identifying tractable subclasses of NP complete problems and designing efficient algorithms for these tractable classes. Constraint satisfaction and Bayesian network inference are two examples of such…

Dominance of Milnor attractors in high-dimensional dynamical systems is reviewed, with the use of globally coupled maps. From numerical simulations, the threshold number of degrees of freedom for such prevalence of Milnor attractors is…

In a series of two papers, we investigate the large deviations and asymptotic behavior of stochastic models of brain neural networks with random interaction coefficients. In this first paper, we take into account the spatial structure of…

Recognizing symmetries in data allows for significant boosts in neural network training, which is especially important where training data are limited. In many cases, however, the exact underlying symmetry is present only in an idealized…

Chaos and Noise are ubiquitous in the Brain. Inspired by the chaotic firing of neurons and the constructive role of noise in neuronal models, we for the first time connect chaos, noise and learning. In this paper, we demonstrate Stochastic…

The spectra of random feature matrices provide essential information on the conditioning of the linear system used in random feature regression problems and are thus connected to the consistency and generalization of random feature models.…

We quantify the finite size effects in a stochastic network made up of rate neurons, for several kinds of recurrent connectivity matrices. This analysis is performed by means of a perturbative expansion of the neural equations, where the…

In natural resource management, or more generally in the study of sustainability issues, often the objective is to maintain the state of a given system within a desirable configuration, typically established in terms of standards or…

Recurrent Neural Networks (RNNs) have found widespread applications in machine learning for time series prediction and dynamical systems reconstruction, and experienced a recent renaissance with improved training algorithms and…

Chaos control techniques have been applied to a wide variety of experimental systems, including magneto-elastic ribbons, lasers, chemical reactions, arrhythmic cardiac tissue, and spontaneously bursting neuronal networks. An underlying…

Fault detection in rotating machinery is a complex task, particularly in small and heterogeneous dataset scenarios. Variability in sensor placement, machinery configurations, and structural differences further increase the complexity of the…

Among the versatile forms of dynamical patterns of activity exhibited by the brain, oscillations are one of the most salient and extensively studied, yet are still far from being well understood. In this paper, we provide various structural…

Networks of interconnected neurons display diverse patterns of collective activity. Relating this collective activity to the network's connectivity structure is a key goal of computational neuroscience. We approach this question for…

Recurrent neural network (RNN)'s architecture is a key factor influencing its performance. We propose algorithms to optimize hidden sizes under running time constraint. We convert the discrete optimization into a subset selection problem.…