Related papers: Non-Markovianity increases transition path probabi…

Thermo-Kinetic relations bound thermodynamic quantities such as entropy production with statistics of dynamical observables. We introduce a Thermo-Kinetic Relation to bound the entropy production or the non-adiabatic (Hatano-Sasa, excess)…

This work is devoted to the investigation of the most probable transition time between metastable states for stochastic dynamical systems. Such a system is modeled by a stochastic differential equation with non-vanishing Brownian noise, and…

The initial stages of the evolution of an open quantum system encode the key information of its underlying dynamical correlations, which in turn can predict the trajectory at later stages. We propose a general approach based on…

We consider the problem of learning the optimal action-value function in the discounted-reward Markov decision processes (MDPs). We prove a new PAC bound on the sample-complexity of model-based value iteration algorithm in the presence of…

We describe $\omega$-limit sets of completely positive (CP) maps over finite-dimensional spaces. In such sets and in its corresponding convex hulls, CP maps present isometric behavior and the states contained in it commute with each other.…

The linear response of non-equilibrium systems with Markovian dynamics satisfies a generalized fluctuation-dissipation relation derived from time symmetry and antisymmetry properties of the fluctuations. The relation involves the sum of two…

Memoryless processes are ubiquitous in nature, in contrast with the mathematics of open systems theory, which states that non-Markovian processes should be the norm. This discrepancy is usually addressed by subjectively making the…

Controlling phase transitions in quantum systems via coupling to reservoirs has been mostly studied for idealized memory-less environments under the so-called Markov approximation. Yet, most quantum materials and experiments in the solid…

We study the evaluation of a policy under best- and worst-case perturbations to a Markov decision process (MDP), using transition observations from the original MDP, whether they are generated under the same or a different policy. This is…

Recently regular decision processes have been proposed as a well-behaved form of non-Markov decision process. Regular decision processes are characterised by a transition function and a reward function that depend on the whole history,…

Featuring memory of past inputs is a fundamental requirement for machine learning models processing time-dependent data. In quantum reservoir computing, all architectures proposed so far rely on Markovian dynamics, which, as we prove,…

We investigate the reversible diffusion-influenced reaction of an isolated pair in the presence of a non-Markovian generalization of the backreaction boundary condition in two space dimensions. Following earlier work by Agmon and Weiss, we…

Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…

In Markov decision processes (MDPs), quantile risk measures such as Value-at-Risk are a standard metric for modeling RL agents' preferences for certain outcomes. This paper proposes a new Q-learning algorithm for quantile optimization in…

This paper examines a number of solution methods for decision processes with non-Markovian rewards (NMRDPs). They all exploit a temporal logic specification of the reward function to automatically translate the NMRDP into an equivalent…

This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…

Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…

The exploration of far-from-equilibrium systems has been at the forefront of nonequilibrium thermodynamics, with a particular focus on understanding the fluctuations and response of thermodynamic systems to external perturbations. In this…

We derive a geometrical expression of the excess entropy production for quasi-static transitions between nonequilibrium steady states of Markovian jump processes, which can be exactly applied to nonlinear and nonequilibrium situations. The…

We consider a class of partially observable Markov decision processes (POMDPs) with uncertain transition and/or observation probabilities. The uncertainty takes the form of probability intervals. Such uncertain POMDPs can be used, for…