Related papers: Markov chains conditioned never to wait too long a…

We study the (weak) equilibrium problem arising from the problem of optimally stopping a one-dimensional diffusion subject to an expectation constraint on the time until stopping. The weak equilibrium problem is realized with a set of…

We consider a vector of $N$ independent binary variables, each with a different probability of success. The distribution of the vector conditional on its sum is known as the conditional Bernoulli distribution. Assuming that $N$ goes to…

Weak Feller property of controlled and control-free Markov chains lead to many desirable properties. In control-free setups this leads to the existence of invariant probability measures for compact spaces and applicability of numerical…

We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in $\mathbb{R}^d$, described in terms of a general stochastic evolution equation. In a setting where the state…

In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of…

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…

Consider an ergodic Markov chain on a countable state space for which the return times have exponential tails. We show that the stationary version of any such chain is a finitary factor of an i.i.d. process. A key step is to show that any…

We consider trap models on Z^d, namely continuous time Markov jump process on Z^d with embedded chain given by a generic discrete time random walk, and whose mean waiting time at x is given by tau_x, with tau = (tau_x, x in Z^d) a family of…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…

Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ taking the steps $(1,0)$, $(-1,1)$ and $(0,-1)$ with probabilities $\lambda < (\mu_1\neq \mu_2)$; in particular, $X$ is assumed stable. Let $\tau_n$ be the first time $X$ hits…

Let $\xi_1$, $\xi_2,\ldots$ be i.i.d. random variables of zero mean and finite variance and $\eta_1$, $\eta_2,\ldots$ positive i.i.d. random variables whose distribution belongs to the domain of attraction of an $\alpha$-stable…

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as…

Classical distribution testing assumes access to i.i.d. samples from the distribution that is being tested. We initiate the study of Markov chain testing, assuming access to a single trajectory of a Markov Chain. In particular, we observe a…

Let $X=(X_t, t\geq 0)$ be a self-similar Markov process taking values in $\mathbb{R}$ such that the state 0 is a trap. In this paper, we present a necessary and sufficient condition for the existence of a self-similar recurrent extension of…

When the unconditioned process is a diffusion process $X(t)$ of drift $\mu(x)$ and of diffusion coefficient $D=1/2$, the local time $A(t)= \int_{0}^{t} d\tau \delta(X(\tau)) $ at the origin $x=0$ is one of the most important time-additive…

In the setting of non-reversible Markov chains on finite or countable state space, exact results on the distribution of the first hitting time to a given set $G$ are obtained. A new notion of "strong metastability time" is introduced to…

We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix…

We consider finite irreducible Markov chains. It was shown that mean hitting time from one state to another satisfies the triangle inequality. Hence, sum of mean hitting time between couple of states in both directions is a metric on the…

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals,…

We consider the time optimal stabilization problem for a nonlinear control system $\dot x=f(x,u)$. Let $\tau(y)$ be the minimum time needed to steer the system from the state $y\in\R^n$ to the origin, and call $\A(T)$ the set of initial…