Related papers: Intricacies of Dependence between Components of Mu…

We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are…

Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation…

In this paper we study the additive functionals of Markov chains via conditioning with respect to both past and future of the chain. We shall point out new sufficient projective conditions, which assure that the variance of partial sums of…

A note on the property of weak contraction, which implies that all bounded solutions of a nonlinear system converge to a (possibly non-unique) equilibrium. We provide some simple results about interconnections of such systems, and a brief…

It is well known that any higher order Markov chain can be associated with a first order Markov chain. In this primarily expository article, we present the first fairly comprehensive analysis of the relationship between higher order and…

. Markov chains in time, such as simple random walks, are at the heart of probability. In space, due to the absence of an obvious definition of past and future, a range of definitions of Markovianity have been proposed. In this paper, after…

This article is a tutorial on Markov chain Monte Carlo simulations and their statistical analysis. The theoretical concepts are illustrated through many numerical assignments from the author's book on the subject. Computer code (in Fortran)…

General birth-and-death as well as hopping stochastic dynamics of infinite particle systems in the continuum are considered. We derive corresponding evolution equations for correlation functions and generating functionals. General…

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…

This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC)…

The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…

In this work, we characterise the statistics of Markov chains by constructing an associated sequence of periodic differential operators. Studying the density of states of these operators reveals the absolutely continuous invariant measure…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

A model for the cosmological time dependence of weak interactions is discused and some experimental tests suggested.

This is a supplement to the article "Markov Chain Monte Carlo Based on Deterministic Transformations" available at http://arxiv.org/abs/1106.5850

The extremal behaviour of a Markov chain is typically characterized by its tail chain. For asymptotically dependent Markov chains existing formulations fail to capture the full evolution of the extreme event when the chain moves out of the…

In this paper, we give precise rates of convergence in the strong invariance principle for stationary sequences of bounded real-valued random variables satisfying weak dependence conditions. One of the main ingredients is a new Fuk-Nagaev…

The variability of structure in a finite Markov equivalence class of causally sufficient models represented by directed acyclic graphs has been fully characterized. Without causal sufficiency, an infinite semi-Markov equivalence class of…

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…

We first introduce the concept of weak random periodic solutions of random dynamical systems. Then, we discuss the existence of such periodic solutions. Further, we introduce the definition of weak random periodic measures and study their…