Related papers: Delayed logistic equation as a limit of long memor…

Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carr\'e du champ of a Markov process recently obtained by…

We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…

Consider a sequence of continuous-time Markov chains $(X^{(n)}_t:t\ge 0)$ evolving on a fixed finite state space $V$. Let $I_n$ be the level two large deviations rate functional for $X^{(n)}_t$, as $t\to\infty$. Under a hypothesis on the…

A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys. Rev. Lett. 90, 110601 (2003) is generalized to the biased case (non equal numbers of…

In Turitsyn, Chertkov, Vucelja (2011) a non-reversible Markov Chain Monte Carlo (MCMC) method on an augmented state space was introduced, here referred to as Lifted Metropolis-Hastings (LMH). A scaling limit of the magnetization process in…

We consider a class of generalized long-range exclusion processes evolving either on $\mathbb Z$ or on a finite lattice with an open boundary. The jump rates are given in terms of a general kernel depending on both the departure and…

We consider a continuous time Markov chain on a countable state space and prove a joint large deviation principle for the empirical measure and the empirical flow, which accounts for the total number of jumps between pairs of states. We…

Large Language Models (LLMs) have achieved significant performance gains through test-time scaling methods. However, existing approaches often incur redundant computations due to the accumulation of historical dependency information during…

We consider an Ito stochastic differential equation with delay, driven by brownian motion, whose solution, by an appropriate reformulation, defines a Markov process $X$ with values in a space of continuous functions $\mathbf C$, with…

We study a Markov decision problem in which the state space is the set of finite marked point configurations in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the…

We consider a stochastic functional delay differential equation, namely an equation whose evolution depends on its past history as well as on its present state, driven by a pure diffusive component plus a pure jump Poisson compensated…

Predicting the near-future delay with accuracy for trains is momentous for railway operations and passengers' traveling experience. This work aims to design prediction models for train delays based on Netherlands Railway data. We first…

We present two main theorems for stochastic processes with a Markov regime-switching model. First, we work on an existence-uniqueness theorem for a Stochastic Differential Delay Equation with Jumps and Regimes (SDDEJRs). Then, we provide…

We consider the task of estimating a structural model of dynamic decisions by a human agent based upon the observable history of implemented actions and visited states. This problem has an inherent nested structure: in the inner problem, an…

At the working heart of policy iteration algorithms commonly used and studied in the discounted setting of reinforcement learning, the policy evaluation step estimates the value of states with samples from a Markov reward process induced by…

The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior…

This paper analyzes the dynamics of a level-dependent quasi-birth-death process ${\cal X}=\{(I(t),J(t)): t\geq 0\}$, i.e., a bi-variate Markov chain defined on the countable state space $\cup_{i=0}^{\infty} l(i)$ with $l(i)=\{(i,j) :…

Markov chain methods are remarkably successful in computational physics, machine learning, and combinatorial optimization. The cost of such methods often reduces to the mixing time, i.e., the time required to reach the steady state of the…

A large class of linear memory differential equations in one dimension, where the evolution depends on the whole history, can be equivalently described as a projection of a Markov process living in a higher dimensional space. Starting with…

Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…