Related papers: Full classification of dynamics for one-dimensiona…
The classical embeddability problem asks whether a given stochastic matrix $T$, describing transition probabilities of a $d$-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the…
We develop a central limit theorem (CLT) for a non-parametric estimator of the transition matrices in controlled Markov chains (CMCs) with finite state-action spaces. Our results establish precise conditions on the logging policy under…
We are concerned with the absolute continuity of stationary distributions corresponding to some piecewise deterministic Markov process, being typically encountered in biological models. The process under investigation involves a…
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
We extend in two directions our previous results about the sampling and the empirical measures of immortal branching Markov processes. Direct applications to molecular biology are rigorous estimates of the mutation rates of polymerase chain…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
We consider the fluctuations of a time-integrated particle current around an atypical value in a generic stochastic Markov process involving classical particles with two-site interaction and hardcore repulsion on a finite one-dimensional…
For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…
Stochastic kinetic models (SKMs) are increasingly used to account for the inherent stochasticity exhibited by interacting populations of species in areas such as epidemiology, population ecology and systems biology. Species numbers are…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
We propose a method based on continuous time Markov chain approximation to compute the distribution of Parisian stopping times and price Parisian options under general one-dimensional Markov processes. We prove the convergence of the method…
Stochastic birth-death processes are described as continuous-time Markov processes in models of population dynamics. A system of infinite, coupled ordinary differential equations (the so-called master equation) describes the time-dependence…
To determine the effect of nonradiative excitation energy transfer on the fluorescence of a rigid multicomponent solution, a new analytical method was developed by treating this transfer as a time-resolved Markov chain (TRMC). In the TRMC…
We consider a stochastic spatial point process with births and deaths on $\mathbb{R}^d$, with the hard-core property that at any time the balls of radius half of any two points do not overlap. We give explicit construction of the process.…
Many economic models feature monotone Markov dynamics on state spaces that may be noncompact. Establishing existence, uniqueness, and stability of stationary distributions in such settings has required a patchwork of sufficient conditions,…
This article aims to investigate sufficient conditions for the stability of stochastic differential equations with a random structure, particularly in contexts involving the presence of concentration points. The proof of asymptotic…