Related papers: On Transformations of Markov Chains and Poisson Bo…

There is a well-established theory linking certain semi-Markov chains and continuous-time random walks to time-fractional equations and anomalous diffusion. In this work, we go beyond the semi-Markov framework by considering some…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

We consider an elementary model for self-organised criticality, the activated random walk on the complete graph. We introduce a discrete time Markov chain as follows. At each time step, we add an active particle at a random vertex and let…

In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

For a discrete time Markov chain and in line with Strotz' consistent planning we develop a framework for problems of optimal stopping that are time-inconsistent due to the consideration of a non-linear function of an expected reward. We…

We address the problem of Lyapunov function construction for a class of continuous-time Markov chains with affine transition rates, typically encountered in stochastic chemical kinetics. Following an optimization approach, we take advantage…

We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is…

We study the synchronization behavior of discrete-time Markov chains on countable state spaces. Representing a Markov chain in terms of a random dynamical system, which describes the collective dynamics of trajectories driven by the same…

We give a bound on the mixing time of a uniformly ergodic, reversible Markov chain in terms of the spectral radius of the transition operator. This bound has been established previously in finite state spaces, and is widely believed to hold…

In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…

The partial sum of the states of a Markov chain or more generally a Markov source is asymptotically normally distributed under suitable conditions. One of these conditions is that the variance is unbounded. A simple combinatorial…

We present a method to sample Markov-chain trajectories constrained to both the initial and final conditions, which we term Markov bridges. The trajectories are conditioned to end in a specific state at a given time. We derive the master…

The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…

We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…

We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…

* ACTIVATED RANDOM WALK MODEL * This is a conservative particle system on the lattice, with a Markovian continuous-time evolution. Active particles perform random walks without interaction, and they may as well change their state to…

We consider Backward Stochastic Differential Equations in a setting where noise is generated by a countable state, continuous time Markov chain, and the terminal value is prescribed at a stopping time. We show that, given sufficient…

Let $(X_n)$ be a Markov chain on a standard borelian space $\mathbb{X}$. Any stopping time $\tau$ such that $\mathbb{E}_x\tau$ is finite for all $x\in\mathbb{X}$ induces a Markov chain in $\mathbb{X}$. In this article, we show that there is…

We give a closed form of the discrete-time evolution of a recombination transformation in population genetics. This decomposition allows to define a Markov chain in a natural way. We describe the geometric decay rate to the limit…