Related papers: Evaluating the Stability of Recurrent Neural Model…

Reservoir Computing is a class of Recurrent Neural Networks with internal weights fixed at random. Stability relates to the sensitivity of the network state to perturbations. It is an important property in Reservoir Computing as it directly…

Stability of recurrent models is closely linked with trainability, generalizability and in some applications, safety. Methods that train stable recurrent neural networks, however, do so at a significant cost to expressibility. We propose an…

In this paper, we elaborate over the well-known interpretability issue in echo state networks. The idea is to investigate the dynamics of reservoir neurons with time-series analysis techniques taken from research on complex systems.…

Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically,…

Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them…

Eigenvalue analysis is a well-established tool for stability analysis of dynamical systems. However, there are situations where eigenvalues miss some important features of physical models. For example, in models of incompressible fluid…

Reservoir Computing (RC) provides an efficient way for designing dynamical recurrent neural models. While training is restricted to a simple output component, the recurrent connections are left untrained after initialization, subject to…

Recurrent neural networks have been extensively studied in the context of neuroscience and machine learning due to their ability to implement complex computations. While substantial progress in designing effective learning algorithms has…

Convolutional and Recurrent, deep neural networks have been successful in machine learning systems for computer vision, reinforcement learning, and other allied fields. However, the robustness of such neural networks is seldom apprised,…

Recurrent neural networks are capable of learning the dynamics of an unknown nonlinear system purely from input-output measurements. However, the resulting models do not provide any stability guarantees on the input-output mapping. In this…

Quantifying the eigenvalue spectra of large random matrices allows one to understand the factors that contribute to the stability of dynamical systems with many interacting components. This work explores the effect that the interaction…

Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem and to model long-term dependencies of sequences. However,…

In this paper we give a profound insight into the computation capability of delay-based reservoir computing via an eigenvalue analysis. We concentrate on the task-independent memory capacity to quantify the reservoir performance and compare…

In reservoir computing, the coupling strength of the initial untrained recurrent neural network (the reservoir) is an important hyperparameter that can be varied for accurate training. A common heuristic is to set this parameter near the…

For a given stable recurrent neural network (RNN) that is trained to perform a classification task using sequential inputs, we quantify explicit robustness bounds as a function of trainable weight matrices. The sequential inputs can be…

Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task related neural dynamics we study trained Recurrent Neural Networks. We develop a Mean Field Theory for Reservoir Computing…

Water distribution networks are hydraulic infrastructures that aim to meet water demands at their various nodes. Water flows through pipes in the network create nonlinear dynamics on networks. A desirable feature of water distribution…

We study the propagation and distribution of information-carrying signals injected in dynamical systems serving as a reservoir computers. A multivariate correlation analysis in tailored replica tests reveals consistency spectra and…

We consider the problem of training input-output recurrent neural networks (RNN) for sequence labeling tasks. We propose a novel spectral approach for learning the network parameters. It is based on decomposition of the cross-moment tensor…

The topology of a network associated with a reservoir computer is often taken so that the connectivity and the weights are chosen randomly. Optimization is hardly considered as the parameter space is typically too large. Here we investigate…