Related papers: Optimal stopping time and halting set for total va…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
We prove an upper bound on the total variation mixing time of a finite Markov chain in terms of the absolute spectral gap and the number of elements in the state space. Unlike results requiring reversibility or irreducibility, this bound is…
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…
We study the Markov chain $x_{n+1}=ax_n+b_n$ on a finite field $\mathbb{F}_p$, where $a \in \mathbb{F}_p$ is fixed and $b_n$ are independent and identically distributed random variables in $\mathbb{F}_p$. Conditionally on the Riemann…
A curious connection exists between the theory of optimal stopping for independent random variables, and branching processes. In particular, for the branching process $Z_n$ with offspring distribution $Y$, there exists a random variable $X$…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
Earlier work by Diaconis and Saloff-Coste gives a spectral criterion for a maximum separation cutoff to occur for birth and death chains. Ding, Lubetzky and Peres gave a related criterion for a maximum total variation cutoff to occur in the…
This paper studies Markov chains on the chambers of real hyperplane arrangements, a model that generalizes famous examples, such as the Tsetlin library and riffle shuffles. We discuss cutoff for the Tsetlin library for general weights, and…
The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical…
This article provides the first procedure for computing a fully data-dependent interval that traps the mixing time $t_{\text{mix}}$ of a finite reversible ergodic Markov chain at a prescribed confidence level. The interval is computed from…
We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…
Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…
Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…
The cutoff phenomenon describes a sharp transition in the convergence of a family of ergodic finite Markov chains to equilibrium. Many natural families of chains are believed to exhibit cutoff, and yet establishing this fact is often…
What can one say on convergence to stationarity of a finite state Markov chain that behaves "locally" like a nearest neighbor random walk on ${\mathbb Z}$ ? The model we consider is a version of nearest neighbor lazy random walk on the…
In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main…
A Markov process is registered. At random moment $\theta$ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is…
We investigate the mixing time of the asymmetric Zero Range process on the segment with a non-decreasing rate. We show that the cutoff holds in the totally asymmetric case with a convex flux, and also with a concave flux if the asymmetry is…
We show how to map the states of an ergodic Markov chain to Euclidean space so that the squared distance between states is the expected commuting time. We find a minimax characterization of commuting times, and from this we get monotonicity…
We analyze the notions of monotonicity and complete monotonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions are not equivalent. However,…