Related papers: Ergodic Theory for Controlled Markov Chains with S…
This work develops a rigorous framework for analysing ergodicity and mixing in time-inhomogeneous quantum dynamics. It considers quantum evolutions generated by sequences of quantum channels and examines in detail the relationship between…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…
We prove an invariance principle (functional central limit theorem) for a vector-valued additive functional of a Markov chain for almost every starting point with respect to an ergodic equilibrium distribution. The hypothesis is a moment…
We investigate simple models for strictly non-ergodic stochastic processes $x_t$ ($t$ being the discrete time step) focusing on the expectation value $v$ and the standard deviation $\delta v$ of the empirical variance $v[x]$ of finite time…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…
A nonlinear Markov chain is a discrete time stochastic process whose transitions depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a infinite state space can be identified by a…
Asymptotic behavior of the three-dimensional stochastic Navier-Stokes equations with Markov switching in additive noises is studied for incompressible fluid flow in a bounded domain in the three-dimensional space. To study such a system, we…
We consider a class of stochastic processes $X$ defined by $X\left( t\right) =\int_{0}^{T}G\left( t,s\right) dM\left( s\right) $ for $t\in\lbrack0,T]$, where $M$ is a square-integrable continuous martingale and $G$ is a deterministic…
We study an open discrete-time queueing network that models the collection of data in a multi-hop sensor network. We assume data is generated at the sensor nodes as a discrete-time Bernoulli process. All nodes in the network maintain a…
For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…
We study the three state toric homogeneous Markov chain model and three special cases of it, namely: (i) when the initial state parameters are constant, (ii) without self-loops, and (iii) when both cases are satisfied at the same time.…
We study a system of $N$ interacting particles on $\bf{Z}$. The stochastic dynamics consists of two components: a free motion of each particle (independent random walks) and a pair-wise interaction between particles. The interaction belongs…
We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic…
The invariant measure is a fundamental object in the theory of Markov processes. In finite dimensions a Markov process is defined by transition rates of the corresponding stochastic matrix. The Markov tree theorem provides an explicit…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…
We study stochastic approximation procedures for approximately solving a $d$-dimensional linear fixed point equation based on observing a trajectory of length $n$ from an ergodic Markov chain. We first exhibit a non-asymptotic bound of the…
We consider ergodic backward stochastic differential equations, in a setting where noise is generated by a countable state uniformly ergodic Markov chain. We show that for Lipschitz drivers such that a comparison theorem holds, these…