Related papers: Stochastic approximation in non-markovian environm…

Non-Markovian dynamics are ubiquitous across physics, biology, and engineering. Yet our understanding of non-Markovian processes significantly lags that of simpler Markovian processes, due largely to a lack of tractable models. In this…

The developing field of stochastic thermodynamics extends concepts of macroscopic thermodynamics such as entropy production and work to the microscopic level of individual trajectories taken by a system through phase space. The scheme…

As a primary contribution, we present a convergence theorem for stochastic iterations, and in particular, Q-learning iterates, under a general, possibly non-Markovian, stochastic environment. Our conditions for convergence involve an…

Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…

This paper introduces a new approach for continual planning and model learning in relational, non-stationary stochastic environments. Such capabilities are essential for the deployment of sequential decision-making systems in the uncertain…

Machine learning methods have proved to be useful for the recognition of patterns in statistical data. The measurement outcomes are intrinsically random in quantum physics, however, they do have a pattern when the measurements are performed…

We present a probabilistic model for stochastic iterative algorithms with the use case of optimization algorithms in mind. Based on this model, we present PAC-Bayesian generalization bounds for functions that are defined on the trajectory…

In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper…

Motivated by the novel paradigm developed by Van Roy and coauthors for reinforcement learning in arbitrary non-Markovian environments, we propose a related formulation and explicitly pin down the error caused by non-Markovianity of…

Supervised learning algorithms take as input a set of labelled examples and return as output a predictive model. Such models are used to estimate labels for future, previously unseen examples drawn from the same generating distribution. In…

Stochastic models in which agents interact with their neighborhood according to a network topology are a powerful modeling framework to study the emergence of complex dynamic patterns in real-world systems. Stochastic simulations are often…

Recent success in developing increasingly general purpose agents based on sequence models has led to increased focus on the problem of deploying computationally limited agents within the vastly more complex real-world. A key challenge…

Stochastic resonance is a phenomenon where the response signal to external driving is enhanced by environment noise. In quantum regime, the effect of environment is often intrinsically non-Markovian. Due to the combination of such…

By modeling the interaction of a system with an environment through a renewal approach, we demonstrate that completely positive non-Markovian dynamics may develop some unexplored non-standard statistical properties. The renewal approach is…

We study a fairly general class of time-homogeneous stochastic evolutions driven by noises that are not white in time. As a consequence, the resulting processes do not have the Markov property. In this setting, we obtain constructive…

We discuss a variant of Thompson sampling for nonparametric reinforcement learning in a countable classes of general stochastic environments. These environments can be non-Markov, non-ergodic, and partially observable. We show that Thompson…

Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…

Stochastic resetting describes dynamics which are reinitialized to a reference state at random times. These protocols are attracting significant interest: they can stabilize nonequilibrium stationary states, generate correlations in…

Partially observable environments present an important open challenge in the domain of sequential control learning with delayed rewards. Despite numerous attempts during the two last decades, the majority of reinforcement learning…

Machine learning techniques not only offer efficient tools for modelling dynamical systems from data, but can also be employed as frontline investigative instruments for the underlying physics. Nontrivial information about the original…