Related papers: Ergodic Theory for Controlled Markov Chains with S…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
The authors consider stochastic aspects of the stabilization problem for two and three-dimensional Oseen equations with help of feedback control defined on a part of the fluid boundary. Stochastic issues arise when inevitable unpredictable…
The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…
In this paper, we study Markov chains (MC) on topological spaces within the framework of the operator approach. We extend the Markov operator from the space of countably additive measures to the space of finitely additive measures. Cesaro…
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…
We present a general black box theorem that ensures convergence of a sequence of stationary Markov processes, provided a few assumptions are satisfied. This theorem relies on a control of the resolvents of the sequence of Markov processes,…
We are interested in the quasi-stationarity of the time-inhomogeneous Markov process X t = B t (t + 1) $\kappa$ where (B t) t$\ge$0 is a one-dimensional Brownian motion and $\kappa$ $\in$ (0, $\infty$). We first show that the law of X t…
This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not…
We prove an ergodic theorem for Markov chains indexed by the Ulam-Harris-Neveu tree over large subsets with arbitrary shape under two assumptions: with high probability, two vertices in the large subset are far from each other and have…
Necessary and sufficient conditions for a Markov chain to be ergodic are that the chain is irreducible and aperiodic. This result is manifest in the case of random walks on finite groups by a statement about the support of the driving…
We justify and discuss expressions for joint lower and upper expectations in imprecise probability trees, in terms of the sub- and supermartingales that can be associated with such trees. These imprecise probability trees can be seen as…
We study the problem of stationarity and ergodicity for autoregressive multinomial logistic time series models which possibly include a latent process and are defined by a GARCH-type recursive equation. We improve considerably upon the…
We consider a simple discrete-time Markov chain with values in $[0,\infty)^{Z^d}$. The Markov chain describes various interesting examples such as oriented percolation, directed polymers in random environment, time discretizations of binary…
We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning…
Starting from the overdamped Langevin dynamics in $\mathbb{R}^n$, $$ dX_t = -\nabla V(X_t) dt + \sqrt{2 \beta^{-1}} dW_t, $$ we consider a scalar Markov process $\xi_t$ which approximates the dynamics of the first component $X^1_t$. In the…
Bailey showed that the general pointwise forecasting for stationary and ergodic time series has a negative solution. However, it is known that for Markov chains the problem can be solved. Morvai showed that there is a stopping time sequence…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
Inspired by \citet{Berkes14} and \citet{Wu07}, we prove an almost sure invariance principle for stationary $\beta-$mixing stochastic processes defined on Hilbert space. Our result can be applied to Markov chain satisfying Meyn-Tweedie type…
Let $\{X(t):t\in\mathbb R_+\}$ be a stationary Gaussian process with almost surely (a.s.) continuous sample paths, $\mathbb E X(t) = 0$, $\mathbb E X^2(t) = 1$ and correlation function satisfying (i) $r(t) = 1 - C|t|^{\alpha} +…