Related papers: Non-Markovianity increases transition path probabi…
We present an overview of the reaction coordinate approach to handling strong system-reservoir interactions in quantum thermodynamics. This technique is based on incorporating a collective degree of freedom of the reservoir (the reaction…
We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the…
We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the…
A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…
The reaction-coordinate mapping is a useful technique to study complex quantum dissipative dynamics into structured environments. In essence, it aims to mimic the original problem by means of an 'augmented system', which includes a suitably…
The non-Markovian nature of polymer motions is accounted for in folding kinetics, using frequency-dependent friction. Folding, like many other problems in the physics of disordered systems, involves barrier crossing on a correlated energy…
This study introduces a comparative modeling framework using stationary and non-stationary transition probabilities within a Markov Decision Process (MDP) to assess COVID-19 disease dynamics. Stationary transition probabilities assume…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
In spatially distributed cellular systems, it is often convenient to represent complicated auxiliary pathways and spatial transport by time-delayed reaction rates. Furthermore, many of the reactants appear in low numbers necessitating a…
From a recent geometric generalization of Thermodynamic Uncertainty Relations (TURs) we derive novel upper bounds on the nonlinear response of an observable of an arbitrary system undergoing a change of probabilistic state. Various…
We construct a large class of completely positive and trace preserving non-Markovian dynamical maps for an open quantum system. These maps arise from a piecewise dynamics characterized by a continuous time evolution interrupted by jumps,…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
Partially observable Markov decision processes (POMDPs) are standard models for dynamic systems with probabilistic and nondeterministic behaviour in uncertain environments. We prove that in POMDPs with long-run average objective, the…
We study upper and lower bounds on the sample-complexity of learning near-optimal behaviour in finite-state discounted Markov Decision Processes (MDPs). For the upper bound we make the assumption that each action leads to at most two…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…
This paper is concerned with collective variables, or reaction coordinates, that map a discrete-in-time Markov process $X_n$ in $\mathbb{R}^d$ to a (much) smaller dimension $k \ll d$. We define the effective dynamics under a given…
Molecule- and particle-based simulations provide the tools to test, in microscopic detail, the validity of classical nucleation theory. In this endeavour, determining nucleation mechanisms and rates for phase separation requires an…
We extend the non-parametric framework of reaction coordinate optimization to non-equilibrium ensembles of (short) trajectories. For example, we show how, starting from such an ensemble, one can obtain an equilibrium free energy profile…
We analyze the connections between the non-Markovianity degree of the most general phase-damping qubit maps and their legitimate mixtures. Using the results for image non-increasing dynamical maps, we formulate the necessary and sufficient…