Related papers: Non-Markovianity increases transition path probabi…

We introduce a rigorous method to microscopically compute the observables which characterize the thermodynamics and kinetics of rare macromolecular transitions for which it is possible to identify a priori a slow reaction coordinate. In…

Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…

The computation of the probability of the first-passage time through a given threshold of a stochastic process is a classic problem that appears in many branches of physics. When the stochastic dynamics is markovian, the probability admits…

We study Boundary time crystals (BTCs) in the presence of non-Markovian dynamics. In contrast to BTCs observed in earlier works in the Markovian regime, we show that non-Markovian dynamics can be highly beneficial for stabilizing BTCs over…

We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…

The reaction coordinate (RC) technique is emerging as a significant tool in the study of quantum dissipative dynamics and quantum thermodynamics. With the objective to further establish this tool, here we explore to what extent the method…

We introduce a new method to accurately and efficiently estimate the effective dynamics of collective variables in molecular simulations. Such reduced dynamics play an essential role in the study of a broad class of processes, ranging from…

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…

The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…

We show that the use of a recently proposed iterative collision model with inter-environment swaps displays a signature of strongly non-Markovian dynamics that is highly dependent on the establishment of system-environment correlations. Two…

We study non-Markovian dynamics of a two level atom using pseudomode method. Because of the memory effect of non-Markovian dynamics, the atom receives back information and excited energy from the reservoir at a later time, which causes more…

In this paper, we consider a piecewise deterministic Markov process (PDMP), with known flow and deterministic transition measure, and unknown jump rate $\lambda$. To estimate nonparametrically the jump rate, we first construct an adaptive…

We propose a modified non-Markovian quantum jump method to overcome the obstacle of dramatically increased trajectory number in conventional quantum trajectory simulations. In our method the trajectories are classified into the trajectory…

We study the dynamics of an open quantum system interacting with a non-thermal bath. Here, "non-thermal" means that the bath modes do not need to have the same temperature, but they have an effective temperature distribution. We find that,…

This paper addresses a key limitation in existing counterfactual inference methods for Markov Decision Processes (MDPs). Current approaches assume a specific causal model to make counterfactuals identifiable. However, there are usually many…

We propose a simple criterion for non-Markovianity: a quantum master equation is non-Markovian if and only if its \textit{trajectory set} contains a \textit{self-intersecting trajectory} (defined in the main text). Since self-intersection…

We present an exact solution for one-dimensional overdamped dynamics near a hard wall, allowing us to connect steady-state distributions under confinement with the extreme value statistics of unconfined stochastic processes. This mapping…

Learning-based approaches to verifying unknown Markov decision processes (MDPs) often employ uncertain MDPs. These models use, for example, confidence intervals to capture transition uncertainty and allow synthesis of policies that are…

In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set. By targeting maximal return under the most adversarial model from that set, RMDPs address performance…

We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…