Related papers: Markov chains revisited
We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…
This paper investigates stochastic finite matrices and the corresponding finite Markov chains constructed using recurrence matrices for general families of orthogonal polynomials and multiple orthogonal polynomials. The paper explores the…
In this research the technology of complex Markov chains is applied to predict financial time series. The main distinction of complex or high-order Markov Chains and simple first-order ones is the existing of aftereffect or memory. The…
Most successful applications of deep learning involve similar training and test conditions. However, tasks such as biological sequence design involve searching for sequences that improve desirable properties beyond previously known values,…
A reaction network is a chemical system involving multiple reactions and chemical species. Stochastic models of such networks treat the system as a continuous time Markov chain on the number of molecules of each species with reactions as…
In this paper we develop the elements of the theory of algorithmic randomness in continuous-time Markov chains (CTMCs). Our main contribution is a rigorous, useful notion of what it means for an individual trajectory of a CTMC to be random.…
The molecular evolution in a gene regulatory network is classically modeled by Markov jump processes. However, the direct simulation of such models is extremely time consuming. Indeed, even the simplest Markovian model, such as the…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
An asymptotic model for extreme behavior of certain Markov chains is the "tail chain". Generally taking the form of a multiplicative random walk, it is useful in deriving extremal characteristics such as point process limits. We place this…
The recurrence features of persistent random walks built from variable length Markov chains are investigated. We observe that these stochastic processes can be seen as L{\'e}vy walks for which the persistence times depend on some internal…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
Markov chain model is widely applied in many fields, especially the field of prediction. The classical Discrete-time Markov chain(DTMC) is a widely used method for prediction. However, the classical DTMC model has some limitation when the…
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…
We introduce a theoretical framework for sampling from unnormalized densities based on a smoothing scheme that uses an isotropic Gaussian kernel with a single fixed noise scale. We prove one can decompose sampling from a density (minimal…
Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…
We presented in \cite{bl2,bl7} an approach to derive the metastable behavior of continuous-time Markov chains. We assumed in these articles that the Markov chains visit points in the time scale in which it jumps among the metastable sets.…
It has become increasingly easy nowadays to collect approximate posterior samples via fast algorithms such as variational Bayes, but concerns exist about the estimation accuracy. It is tempting to build solutions that exploit approximate…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
We consider a Markov chain $(x_n)$ whose kernel is indexed by a scaling parameter $\gamma>0$, refered to as the step size. The aim is to analyze the behavior of the Markov chain in the doubly asymptotic regime where $n\to\infty$ then…
This is a free textbook suitable for a one-semester course on Markov chains, covering basics of finite-state chains, many classical models, asymptotic behavior and mixing times, Monte Carlo methods, and martingales and harmonic functions.…