Related papers: Optimal stopping time and halting set for total va…
We obtain moment and Gaussian bounds for general Lipschitz functions evaluated along the sample path of a Markov chain. We treat Markov chains on general (possibly unbounded) state spaces via a coupling method. If the first moment of the…
For an infinite-horizon continuous-time optimal stopping problem under non-exponential discounting, we look for an optimal equilibrium, which generates larger values than any other equilibrium does on the entire state space. When the…
This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent…
Consider a discrete time Markov chain with rather general state space which has an invariant probability measure $\mu$. There are several sufficient conditions in the literature which guarantee convergence of all or $\mu$-almost all…
We derive novel results on the ergodic theory of irreducible, aperiodic Markov chains. We show how to optimally steer the network flow to a stationary distribution over a finite or infinite time horizon. Optimality is with respect to an…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…
In this paper, we report a novel approach for studying the effect of optimal uncoupling on the stability of synchronization in coupled chaotic systems. The clipping of phase space of the driven system having an orientation along the…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
Given a reversible Markov chain $P_n$ on $n$ states, and another chain $\tilde{P}_n$ obtained by perturbing each row of $P_n$ by at most $\alpha_n$ in total variation, we study the total variation distance between the two stationary…
This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…
The spectral gap $\gamma$ of a finite, ergodic, and reversible Markov chain is an important parameter measuring the asymptotic rate of convergence. In applications, the transition matrix $P$ may be unknown, yet one sample of the chain up to…
Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…
We study the steady-state patterns of population of the coupled oscillators that sync and swarm, where the interaction distances among oscillators have finite-cutoff in interaction distance. We examine how the static patterns known in the…
Lamperti's maximal branching process is revisited, with emphasis on the description of the shape of the invariant measures in both the recurrent and transient regimes. A truncated version of this chain is exhibited, preserving the…
For birth and death chains, we derive bounds on the spectral gap and mixing time in terms of birth and death rates. Together with the results of Ding et al. in 2010, this provides a criterion for the existence of a cutoff in terms of the…
We consider a continuous time Markov chain on a countable state space and prove a joint large deviation principle for the empirical measure and the empirical flow, which accounts for the total number of jumps between pairs of states. We…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…