Related papers: A Transience Criterion for Uniformly Bounded Marko…
We develop a central limit theorem (CLT) for a non-parametric estimator of the transition matrices in controlled Markov chains (CMCs) with finite state-action spaces. Our results establish precise conditions on the logging policy under…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
This paper considers the Poisson equation for general state-space Markov chains in continuous time. The main purpose of this paper is to present specific bounds for the solutions of the Poisson equation for general state-space Markov…
In this paper we study the central limit theorem for additive functionals of stationary Markov chains with general state space by using a new idea involving conditioning with respect to both the past and future of the chain. Practically, we…
We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is…
We establish existence of Markov chains of mean-field type with unbounded jump intensities by means of a fixed point argument using the Total Variation distance. We further show existence of nearly-optimal controls and, using a Markov chain…
We prove a transient fluctuation theorem for the currents for continuous-time Markov jump processes with stationary rates, generalizing an asymptotic result by Andrieux and Gaspard [J. Stat. Phys. 127, 107 (2007)] to finite times. The…
Phase I distribution-free runs- and patterns-type control charts are proposed for monitoring the unknown target value (or location parameter) for both continuous and discrete individual observations. Our approach maintains the nominal…
This paper studies limit theorems for Markov Chains with general state space under conditions which imply subgeometric ergodicity. We obtain a central limit theorem and moderate deviation principles for additive not necessarily bounded…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
For general penalized Markov processes with soft killing, we propose a simple criterion ensuring uniform convergence of conditional distributions in Wasserstein distance to a unique quasi-stationary distribution. We give several examples of…
We study the limiting object of a sequence of Markov chains analogous to the limits of graphs, hypergraphs, and other objects which have been studied. Following a suggestion of Aldous, we assign to a sequence of finite Markov chains with…
Using ideas borrowed from topological dynamics and ergodic theory we introduce topological and metric versions of the recurrence property for general Markov chains. The main question of interest here is how large is the set of recurrent…
We establish non-asymptotic error bounds for the classical Maximal Likelihood Estimation of the transition matrix of a given Markov chain. Meanwhile, in the reversible case, we propose a new reversibility-preserving online Symmetric…
We consider the last zero crossing time $T_{\mu,t}$ of a Brownian motion, with drift $\mu \neq 0$ in the time interval $[0, t]$. We prove the large deviation principle of $\{T_{\mu \sqrt r t} : r > 0 \}$ as $r$ tends to infinity. Moreover,…
Given a unichain Markov reward process (MRP), we provide an explicit expression for the bias values in terms of mean first passage times. This result implies a generalization of known Markov chain perturbation bounds for the stationary…
We establish existence of controlled Markov chain of mean-field type with unbounded jump intensities by means of a fixed point argument using the Wasserstein distance. Using a Markov chain entropic backward SDE approach, we further suggest…
A discrete rate theory for general multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three…
We study zero-sum stochastic games for controlled discrete time Markov chains with risk-sensitive average cost criterion with countable state space and Borel action spaces. The payoff function is nonnegative and possibly unbounded. Under a…
In this paper, the problem of non-fragile finite-time stabilization for linear discrete mean-field stochastic systems is studied. The uncertain characteristics in control parameters are assumed to be random satisfying the Bernoulli…