Related papers: Loop-free Markov chains as determinantal point pro…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…

We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable…

We prove a law of large numbers in terms of complete convergence of independent random variables taking values in increments of monotone functions, with convergence uniform both in the initial and the final time. The result holds also for…

Every irreducible discrete-time linear switching system possesses an invariant convex Lyapunov function (Barabanov norm), which provides a very refined analysis of trajectories. Until recently that notion remained rather theoretical apart…

We consider an infinite horizon optimal control problem for a pure jump Markov process $X$, taking values in a complete and separable metric space $I$, with noise-free partial observation. The observation process is defined as $Y_t =…

We establish a new Bernstein-type deviation inequality for general (non-reversible) discrete-time Markov chains via an elementary approach. More robust than existing works in the literature, our result only requires the Markov chain to…

We introduce discrete time Markov chains that preserve uniform measures on boxed plane partitions. Elementary Markov steps change the size of the box from (a x b x c) to ((a-1) x (b+1) x c) or ((a+1) x (b-1) x c). Algorithmic realization of…

We obtain necessary and sufficient conditions for the regular variation of the variance of partial sums of functionals of discrete and continuous-time stationary Markov processes with normal transition operators. We also construct a class…

For near-critical, transient Markov chains on the non-negative integers in the Lamperti regime, where the mean drift at $x$ decays as $1/x$ as $x \to \infty$, we quantify degree of transience via existence of moments for conditional return…

Certain Markov processes, or deterministic evolution equations, have the property that they are dual to a stochastic process that exhibits extinction versus unbounded growth, i.e., the total mass in such a process either becomes zero, or…

We consider a particle system with weights and the scaling limits derived from its occupation time. We let the particles perform independent recurrent L\'evy motions and we assume that their initial positions and weights are given by a…

This paper investigates the limit behavior of Markov Decision Processes (MDPs) made of independent particles evolving in a common environment, when the number of particles goes to infinity. In the finite horizon case or with a discounted…

We prove a boundary Harnack inequality for jump-type Markov processes on metric measure state spaces, under comparability estimates of the jump kernel and Urysohn-type property of the domain of the generator of the process. The result holds…

The cutoff phenomenon was recently shown to systematically follow from non-negative curvature and the product condition, for all Markov diffusions. The proof crucially relied on a classical \emph{chain rule} satisfied by the carr\'e du…

In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

The singular values of a product of $M$ independent Ginibre matrices of size $N\times N$ form a determinantal point process. Near the soft edge, as both $M$ and $N$ go to infinity in such a way that $M/N\to \alpha$, $\alpha>0$, a scaling…

Markov chains and Markov decision processes (MDPs) are well-established probabilistic models. While finite Markov models are well-understood, analysing their infinite counterparts remains a significant challenge. Decisiveness has proven to…

Two aspects of noncolliding diffusion processes have been extensively studied. One of them is the fact that they are realized as harmonic Doob transforms of absorbing particle systems in the Weyl chambers. Another aspect is integrability in…

In this work we introduce the discrete-space broken line process (with discrete and continues parameter values) and derive some of its properties. We explore polygonal Markov fields techniques developed by Arak-Surgailis. The discrete…