Related papers: Markov chains conditioned never to wait too long a…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

Consider the class of (functions of) strictly stationary Markov chains in which (i) the second moments are finite and (ii) absolute regularity (beta-mixing) is satisfied with exponential mixing rate. For (functions of) Markov chains in that…

For continuous-time Markov chains we prove that, depending on the notion of effective affinity $F$, the probability of an edge current to ever become negative is either $1$ if $F< 0$ else $\sim \exp - F$. The result generalizes a ``noria''…

Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…

We derive some key extremal features for $k$th order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…

The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…

We consider a sequence of Markov chains $(\mathcal X^n)_{n=1,2,...}$ with $\mathcal X^n = (X^n_\sigma)_{\sigma\in\mathcal T}$, indexed by the full binary tree $\mathcal T = \mathcal T_0 \cup \mathcal T_1 \cup ...$, where $\mathcal T_k$ is…

We study the limit behaviour of a generally non-linear ordinary differential equation whose solution is a superadditive generalisation of a stochastic matrix, and provide necessary and sufficient conditions for this solution to be ergodic,…

For a discrete time Markov chain and in line with Strotz' consistent planning we develop a framework for problems of optimal stopping that are time-inconsistent due to the consideration of a non-linear function of an expected reward. We…

We consider a random walk on a discrete connected graph having some infinite branches plus finitely many vertices with finite degrees. We find the generator of a strong stationary dual in the sense of Fill, and use it to find some…

Let $M$ be an irreducible transition matrix on a finite state space $V$. For a Markov chain $C=(C_k,k\geq 0)$ with transition matrix $M$, let $\tau^{\geq 1}_u$ denote the first positive hitting time of $u$ by $C$, and $\rho$ the unique…

We study the large deviations of Markov chains under the sole assumption that the state space is discrete. In particular, we do not require any of the usual irreducibility and exponential tightness assumptions. Using subadditive arguments,…

This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…

We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…

Many finite-state reversible Markov chains can be naturally decomposed into "projection" and "restriction" chains. In this paper we provide bounds on the total variation mixing times of the original chain in terms of the mixing properties…

We consider a family of Markov chains whose transition dynamics are affected by model parameters. Understanding the parametric dependence of (complex) performance measures of such Markov chains is often of significant interest. The…

We consider discrete-space continuous-time Markov models of reaction networks and provide sufficient conditions for the following stability condition to hold: each state in a closed, irreducible component of the state space is positive…

This paper studies some analytical properties of weak solutions of 3D stochastic primitive equations with periodic boundary conditions. The martingale problem associated to this model is shown to have a family of solutions satisfying the…

The paper studies a probabilistic notion of causes in Markov chains that relies on the counterfactuality principle and the probability-raising property. This notion is motivated by the use of causes for monitoring purposes where the aim is…