Related papers: Evaluating the Stability of Recurrent Neural Model…

Recent theoretical developments of reservoir computing have clarified a sufficient condition about which reservoir computing can capture the dynamics of a target system, enabling the reconstruction of dynamical invariants. Even when the…

Recent research studies revealed that neural networks are vulnerable to adversarial attacks. State-of-the-art defensive techniques add various adversarial examples in training to improve models' adversarial robustness. However, these…

We introduce a method that can be used to evolve the topology of a network in a way that preserves both the network's spectral as well as local structure. This method is quite versatile in the sense that it can be used to evolve a network's…

Recurrent neural networks are powerful tools for understanding and modeling computation and representation by populations of neurons. Continuous-variable or "rate" model networks have been analyzed and applied extensively for these…

A new machine learning scheme, termed versatile reservoir computing, is proposed for sustaining the dynamics of heterogeneous complex networks. We show that a single, small-scale reservoir computer trained on time series from a subset of…

This paper presents a method that learns a regionally stable recurrent neural network model from a set of input-output data generated by an unknown dynamical system. Relying on generalized sector conditions on the deadzone activation…

It is widely accepted that the complex dynamics characteristic of recurrent neural circuits contributes in a fundamental manner to brain function. Progress has been slow in understanding and exploiting the computational power of recurrent…

In adaptive dynamical networks, the dynamics of the nodes and the edges influence each other. We show that we can treat such systems as a closed feedback loop between edge and node dynamics. Using recent advances on the stability of…

Artificial Recurrent Neural Networks are a powerful information processing abstraction, and Reservoir Computing provides an efficient strategy to build robust implementations by projecting external inputs into high dimensional dynamical…

We provide an empirical study of the stability of recurrent neural networks trained to recognize regular languages. When a small amount of noise is introduced into the activation function, the neurons in the recurrent layer tend to saturate…

This paper examines Echo State Network, a reservoir computer, performance using four different benchmark problems, then proposes heuristics or rules of thumb for configuring the architecture, as well as the selection of parameters and their…

A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over long time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary. This…

We investigate the important problem of certifying stability of reinforcement learning policies when interconnected with nonlinear dynamical systems. We show that by regulating the input-output gradients of policies, strong guarantees of…

To be effective in sequential data processing, Recurrent Neural Networks (RNNs) are required to keep track of past events by creating memories. While the relation between memories and the network's hidden state dynamics was established over…

It is common to evaluate the performance of a machine learning model by measuring its predictive power on a test dataset. This approach favors complicated models that can smoothly fit complex functions and generalize well from training data…

In this paper we propose a robust Model Predictive Control where a Gated Recurrent Unit network model is used to learn the input-output dynamic of the system under control. Robust satisfaction of input and output constraints and recursive…

We often use perturbations to regularize neural models. For neural encoder-decoders, previous studies applied the scheduled sampling (Bengio et al., 2015) and adversarial perturbations (Sato et al., 2019) as perturbations but these methods…

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a "next-generation"…

This paper studies network resilience against structured additive perturbations to its topology. We consider dynamic networks modeled as linear time-invariant systems subject to perturbations of bounded energy satisfying specific sparsity…

Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning. Continual learning algorithms seek to mitigate this effect by sustaining good performance while maintaining network…