Related papers: Anomalous Random Neural Networks: a Special Renewa…

We study the long-time dynamics in non-Markovian single-population stochastic models, where one or more reactions are modelled as a stochastic process with a fat-tailed non-exponential distribution of waiting times, mimicking long-term…

In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…

Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly,…

Continual lifelong learning requires an agent or model to learn many sequentially ordered tasks, building on previous knowledge without catastrophically forgetting it. Much work has gone towards preventing the default tendency of machine…

Anomalous diffusion occurs in many physical and biological phenomena, when the growth of the mean squared displacement (MSD) with time has an exponent different from one. We show that recurrent neural networks (RNN) can efficiently…

The recurrent neural networks (RNN) with richly distributed internal states and flexible non-linear transition functions, have overtaken the dynamic Bayesian networks such as the hidden Markov models (HMMs) in the task of modeling highly…

Non-Markovian spontaneous recovery processes with a time delay (memory) are ubiquitous in the real world. How does the non-Markovian characteristic affect failure propagation in complex networks? We consider failures due to internal causes…

This paper presents a new algorithm, Evolutionary eXploration of Augmenting Memory Models (EXAMM), which is capable of evolving recurrent neural networks (RNNs) using a wide variety of memory structures, such as Delta-RNN, GRU, LSTM, MGU…

Forgetting is often seen as an unwanted characteristic in both human and machine learning. However, we propose that forgetting can in fact be favorable to learning. We introduce "forget-and-relearn" as a powerful paradigm for shaping the…

We study the large-time asymptotic of renewal-reward processes with a heavy-tailed waiting time distribution. It is known that the heavy tail of the distribution produces an extremely slow dynamics, resulting in a singular large deviation…

We present a novel methodology utilizing Recurrent Neural Networks (RNNs) to classify Markovian and non-Markovian quantum processes, leveraging time series data derived from Choi states. The model exhibits exceptional accuracy, surpassing…

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed…

The diffusion process near low order synchro-betatron resonances driven by beam-beam interactions at a crossing angle is investigated. Macroscopic observables such as beam emittance, lifetime and beam profiles are calculated. These are…

It is our intention to provide via fractional calculus a generalization of the pure and compound Poisson processes, which are known to play a fundamental role in renewal theory, without and with reward, respectively. We first recall the…

Recurrent Neural networks (RNN) have shown promising potential for learning dynamics of sequential data. However, artificial neural networks are known to exhibit poor robustness in presence of input noise, where the sequential architecture…

In the recent publication (arxiv:2007.08063v2 [cs.LG]) a fast prediction algorithm for a single recurrent network (RN) was suggested. In this manuscript we generalize this approach to a chain of RNs and show that it can be implemented in…

Anomalous diffusions arise as scaling limits of continuous-time random walks (CTRWs) whose innovation times are distributed according to a power law. The impact of a non-exponential waiting time does not vanish with time and leads to…

We observe that the standard log likelihood training objective for a Recurrent Neural Network (RNN) model of time series data is equivalent to a variational Bayesian training objective, given the proper choice of generative and inference…

Recurrent neural networks are a widely used class of neural architectures. They have, however, two shortcomings. First, it is difficult to understand what exactly they learn. Second, they tend to work poorly on sequences requiring long-term…

Partially observable environments present an important open challenge in the domain of sequential control learning with delayed rewards. Despite numerous attempts during the two last decades, the majority of reinforcement learning…