Related papers: Sensitivity analysis for finite Markov chains in d…
The aim of this paper is to propose a methodology for testing general hypothesis in a Markovian setting with random sampling. A discrete Markov chain X is observed at random time intervals $\tau$ k, assumed to be iid with unknown…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
We use the $f-divergence$ also called relative entropy as a measure of diversity between probability densities and review its basic properties. In the sequence we define a few objects which capture relevant information from the sample of a…
Numerical observations on a Markov chain and on the continuous Markov process performed by a granular tracer show that the ``usual'' fluctuation relation for a given observable is not verified for finite (but arbitrarily large) times. This…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…
For a reversible and ergodic Markov chain $\{X_n,n\geq0\}$ with invariant distribution $\pi$, we show that a valid confidence interval for $\pi(h)$ can be constructed whenever the asymptotic variance $\sigma^2_P(h)$ is finite and positive.…
We give a closed form of the discrete-time evolution of a recombination transformation in population genetics. This decomposition allows to define a Markov chain in a natural way. We describe the geometric decay rate to the limit…
Markov chain model is widely applied in many fields, especially the field of prediction. The classical Discrete-time Markov chain(DTMC) is a widely used method for prediction. However, the classical DTMC model has some limitation when the…
Dynamical systems are often subject to forcing or changes in their governing parameters and it is of interest to study how this affects their statistical properties. A prominent real-life example of this class of problems is the…
In this paper we study Monte Carlo estimators based on the likelihood ratio approach for steady-state sensitivity. We first extend the result of Glynn and Olvera-Cravioto [doi:doi: 10.1287/stsy.2018.002] to the setting of continuous time…
Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…