Related papers: Sampling rare fluctuations of discrete-time Markov…
We study the fluctuations of systems modeled by Markov jump processes with periodic generators. We focus on observables defined through time-periodic functions of the system's states or transitions. Using large deviation theory, canonical…
Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested…
This study in centered on models accounting for stochastic deformations of sample paths of random walks, embedded either in $\mathbb{Z}^2$ or in $\mathbb{Z}^3$. These models are immersed in multi-type particle systems with exclusion.…
We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain…
The tail chain of a Markov chain can be used to model the dependence between extreme observations. For a positive recurrent Markov chain, the tail chain aids in describing the limit of a sequence of point processes $\{N_n,n\geq1\}$,…
This article reviews the concepts and methods of variational path sampling. These methods allow computational studies of rare events in systems driven arbitrarily far from equilibrium. Based upon a statistical mechanics of trajectory space…
In this paper we are concerned with hitting times of a family of density-dependent Markov chains. A moderate deviation principle of the hitting time is given. The proof of the main theorem relies heavily on moderate deviations of…
We describe a simple form of importance sampling designed to bound and compute large-deviation rate functions for time-extensive dynamical observables in continuous-time Markov chains. We start with a model, defined by a set of rates, and a…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
Conditions on the generator of a Markov process to control the fluctuations of its bridges are found. In particular, continuous time random walks on graphs and gradient diffusions are considered. Under these conditions, a concentration of…
We consider Markovian models on graphs with local dynamics. We show that, under suitable conditions, such Markov chains exhibit both rapid convergence to equilibrium and strong concentration of measure in the stationary distribution. We…
We consider a system of $N$ identical independent Markov processes, each taking values 0 or 1. The system describes a stochastic dynamics of an ensemble of two-level atoms. The atoms are exposed to a photon flux. Under the photon flux…
Many systems are partially stochastic in nature. We have derived data driven approaches for extracting stochastic state machines (Markov models) directly from observed data. This chapter provides an overview of our approach with numerous…
We study the stochastic dynamics of a system of interacting species in a stochastic environment by means of a continuous-time Markov chain with transition rates depending on the state of the environment. Models of gene regulation in systems…
A Markovian dichotomic system driven by a deterministic time-periodic force is analyzed in terms of the statistical properties of the switching events between the states. The consideration of the counting process of the switching events…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
In this work, sample-based observability of linear discrete-time systems is studied. That is, we consider the case where the system output measurements are not available at every time instance. It is shown that some discrete-time systems…
We establish the existence and uniqueness of quasi-stationary and quasi-ergodic measures for almost surely absorbed discrete-time Markov chains under weak conditions. We obtain our results by exploiting Banach lattice properties of…