Related papers: On discrete-time semi-Markov processes
A block Markov chain is a Markov chain whose state space can be partitioned into a finite number of clusters such that the transition probabilities only depend on the clusters. Block Markov chains thus serve as a model for Markov chains…
We propose and investigate a method for identifying timescales of dissipation in nonequilibrium steady states modeled as discrete-state Markov jump processes. The method is based on how the irreversibility-measured by the statistical…
We provide sharp error bounds for the difference between the transition densities of some multidimensional Continuous Time Markov Chains (CTMC) and the fundamental solutions of some fractional in time Partial (Integro) Differential…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
We establish convergence to an invariant measure as time tends to infinity, for a large class of (possibly non-Markovian) stochastic volatility models. Our arguments are based on a novel coupling idea for Markov chains which also extends to…
We address stability of a class of Markovian discrete-time stochastic hybrid systems. This class of systems is characterized by the state-space of the system being partitioned into a safe or target set and its exterior, and the dynamics of…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
In the paper we consider models of generalized counting processes time-changed by a general inverse subordinator, we characterize their distributions and present governing equations for them. The equations are given in terms of the…
Master equations are typically adopted to describe the dynamics of open quantum systems. Such equations are either in integro-differential or in time-local form, with the latter class more frequently adopted due to the simpler numerical…
We characterise the convergence of a certain class of discrete time Markov processes toward locally Feller processes in terms of convergence of associated operators. The theory of locally Feller processes is applied to L\'evy-type processes…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
In this paper, we study the existence and uniqueness of solutions for general fractional-time parabolic equations of mixture type, and their probabilistic representations in terms of the corresponding inverse subordinators with or without…
The Semi-Markov property of Continuous Time Random Walks (CTRWs) and their limit processes is utilized, and the probability distributions of the bivariate Markov process $(X(t),V(t))$ are calculated: $X(t)$ is a CTRW limit and $V(t)$ a…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…
In this article dedicated to the memory of Igor D. Chueshov, I first summarize in a few words the joint results that we obtained over a period of six years regarding the long-time behavior of solutions to a class of semilinear stochastic…
Given a finite state space E, we build a universal dilation for all possible discrete time Markov chains on E, homogeneous or not: we introduce a second system (an ``environment'') and a deterministic invertible time-homogeneous global…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…