Related papers: Stochastic approximation in non-markovian environm…

The success of reinforcement learning in typical settings is predicated on Markovian assumptions on the reward signal by which an agent learns optimal policies. In recent years, the use of reward machines has relaxed this assumption by…

Non-Markovian stochastic Langevin-like equations of motion are compared to their corresponding Markovian (local) approximations. The validity of the local approximation for these equations, when contrasted with the fully nonlocal ones, is…

We study the task of learning from non-i.i.d. data. In particular, we aim at learning predictors that minimize the conditional risk for a stochastic process, i.e. the expected loss of the predictor on the next point conditioned on the set…

We study ergodic properties of a family of traffic maps acting in the space of bi-infinite sequences of real numbers. The corresponding dynamics mimics the motion of vehicles in a simple traffic flow, which explains the name. Using…

Many regenerative arguments in stochastic processes use random times which are akin to stopping times, but which are determined by the future as well as the past behaviour of the process of interest. Such arguments based on "conditioning on…

This is a set of four lectures devoted to simple ideas about turbulent transport, a ubiquitous non-equilibrium phenomenon. In the course similar to that given by the author in 2006 in Warwick [45], we discuss lessons which have been learned…

When learning to act in a stochastic, partially observable environment, an intelligent agent should be prepared to anticipate a change in its belief of the environment state, and be capable of adapting its actions on-the-fly to changing…

The use of attention-based deep learning models in stochastic filtering, e.g. transformers and deep Kalman filters, has recently come into focus; however, the potential for these models to solve stochastic filtering problems remains largely…

The exact dynamics of a system coupled to an environment can be described by an integro-differential stochastic equation of its reduced density. The influence of the environment is incorporated through a mean-field which is both stochastic…

In reinforcement learning, we typically aim to optimize the expected value of the sum of rewards an agent collects over a trajectory. However, if the process generating these rewards is non-ergodic, the expected value, i.e., the average…

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…

We consider the problem of learning by demonstration from agents acting in unknown stochastic Markov environments or games. Our aim is to estimate agent preferences in order to construct improved policies for the same task that the agents…

We consider the problem of learning by demonstration from agents acting in unknown stochastic Markov environments or games. Our aim is to estimate agent preferences in order to construct improved policies for the same task that the agents…

Stochastic approximation (SA) is a key method used in statistical learning. Recently, its non-asymptotic convergence analysis has been considered in many papers. However, most of the prior analyses are made under restrictive assumptions…

Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are, however, difficult to characterize. Here, we…

We propose a random adaptation variant of time-varying distributed averaging dynamics in discrete time. We show that this leads to novel interpretations of fundamental concepts in distributed averaging, opinion dynamics, and distributed…

Non-Markovian reduced dynamics of an open system is investigated. In the case the initial state of the reservoir is the vacuum state, an approximation is introduced which makes possible to construct a reduced dynamics which is completely…

This paper is concerned with correlation functions of stochastic systems with memory, a prominent example being a molecule or colloid moving through a complex (e.g., viscoelastic) fluid environment. Analytical investigations of such systems…

Quantum memory effects can be qualitatively understood as a consequence of an environment-to-system backflow of information. Here, we analyze and compare how this concept is interpreted and implemented in different approaches to quantum…

We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…