Related papers: Skorokhod embeddings for two-sided Markov chains

General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic…

Filyokov and Karpov [Inzhenerno-Fizicheskii Zhurnal 13, 624 (1967)] have proposed a theory of non-equilibrium steady states in direct analogy with the theory of equilibrium states : the principle is to maximize the Shannon entropy…

Consider a stochastic process $\{X(t)\}$ on a finite state space $ {\sf X}=\{1,\dots, d\}$. It is conditionally Markov, given a real-valued `input process' $\{\zeta(t)\}$. This is assumed to be small, which is modeled through the scaling,…

A well-known theorem for an irreducible skip-free chain with absorbing state $d$, under some conditions, is that the hitting (absorbing) time of state $d$ starting from state 0 is distributed as the sum of $d$ independent geometric (or…

Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic…

We consider a family of measure preserving transformations, which act on a common probability space and are chosen at random by a stationary ergodic Markov chain. This setting defines an instance of a random dynamical system (RDS), which…

Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and…

We consider the optimal stopping problem for a Gauss-Markov process conditioned to adopt a prescribed terminal distribution. By applying a time-space transformation, we show it is equivalent to stopping a Brownian bridge pinned at a random…

Potential theory has important applications in various fields such as physics, finance, and biology. In this paper, we investigate the potentials of two classic types of discrete-time skip-free Markov chains: upward skip-free and downward…

There are many Markov chains on infinite dimensional spaces whose one-step transition kernels are mutually singular when starting from different initial conditions. We give results which prove unique ergodicity under minimal assumptions on…

Consider time-homogeneous discrete-time Markov chains $X$, $Y$, and $Z$ on countable state spaces, considered as stochastic processes with specified initial distributions. Suppose for maps $f$ and $g$ that $(f(X_t))_{t \ge 0}$ and…

The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…

In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…

We propose Markov two-components processes (M2CP) as a probabilistic model of asynchronous systems based on the trace semantics for concurrency. Considering an asynchronous system distributed over two sites, we introduce concepts and tools…

We define the spectral gap of a Markov chain on a finite state space as the second-smallest singular value of the generator of the chain, generalizing the usual definition of spectral gap for reversible chains. We then define the relaxation…

In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…

This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its…

This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it…

Let $Q$ be a probability measure on a finite group $G$, and let $H$ be a subgroup of $G$. We show that a necessary and sufficient condition for the random walk driven by $Q$ on $G$ to induce a Markov chain on the double coset space…

A simple lemma bounds $\mathrm{s.d.}(T)/\mathbb{E} T$ for hitting times $T$ in Markov chains with a certain strong monotonicity property. We show how this lemma may be applied to several increasing set-valued processes. Our main result…