Related papers: Convergence of Imprecise Continuous-Time Markov Ch…
Adaptive Monte Carlo methods can be viewed as implementations of Markov chains with infinite memory. We derive a general condition for the convergence of a Monte Carlo method whose history dependence is contained within the simulated…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
A new sufficient condition is proposed in the problem of Central Limit Theorem in the array scheme for non-homogeneous Markov Chains.
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…
The effect of perturbations of parameters for uniquely convergent imprecise Markov chains is studied. We provide the maximal distance between the distributions of original and perturbed chain and maximal degree of imprecision, given the…
This work develops a rigorous framework for analysing ergodicity and mixing in time-inhomogeneous quantum dynamics. It considers quantum evolutions generated by sequences of quantum channels and examines in detail the relationship between…
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…
Adaptive and interacting Markov Chains Monte Carlo (MCMC) algorithms are a novel class of non-Markovian algorithms aimed at improving the simulation efficiency for complicated target distributions. In this paper, we study a general…
This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded…
We prove a Chernoff-type bound for sums of matrix-valued random variables sampled via a regular (aperiodic and irreducible) finite Markov chain. Specially, consider a random walk on a regular Markov chain and a Hermitian matrix-valued…
We derive novel results on the ergodic theory of irreducible, aperiodic Markov chains. We show how to optimally steer the network flow to a stationary distribution over a finite or infinite time horizon. Optimality is with respect to an…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
We introduce and study time-inhomogeneous quantum Markov chains with parameter $\zeta \ge 0$ and decoherence parameter $0 \leq p \leq 1$ on finite spaces and their large scale equilibrium properties. Here $\zeta$ resembles the inverse…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden…
This study explores a Gaussian quasi-likelihood approach for estimating parameters of diffusion processes with Markovian regime switching. Assuming the ergodicity under high-frequency sampling, we will show the asymptotic normality of the…
A new approach is developed for evaluating the convergence rate for nonlinear Markov chains (MC) based on the recently developed spectral radius technique of markovian coupling for linear MC and the idea of small nonlinear perturbations of…
The theory of imprecise Markov chains has achieved significant progress in recent years. Its applicability, however, is still very much limited, due in large part to the lack of efficient computational methods for calculating…
A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…