Related papers: Markov Process Jump Times and Their Cox Constructi…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman…
Consider a system of coalescing random walks where each individual performs random walk over a finite graph G, or (more generally) evolves according to some reversible Markov chain generator Q. Let C be the first time at which all walkers…
We derive the equations governing the protocols minimizing the heat released by a continuous-time Markov jump process on a one-dimensional countable state space during a transition between assigned initial and final probability…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained.…
In this paper we consider impulse control of continuous time Markov processes with average cost per unit time functional. This problem is approximated using impulse control problems stopped at the first exit time from increasing sequence of…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
Recently Belopolskaya and Suhov studied Markov process's in a random environment, where the environment changes in likeness to a Markov process. Constructions were made to allow the process to "interact with the environment", this was done…
A jumping process, defined in terms of jump size distribution and waiting time distribution, is presented. The jumping rate depends on the process value. The process, which is Markovian and stationary, relaxes to an equilibrium and is…
It has been noticed that when the waiting time distribution exhibits a transition from an intermediate time power law decay to a long-time exponential decay in the continuous time random walk model, a transition from anomalous diffusion to…
We construct the analogue of the local time -- at a fixed point $x$ -- for Markov processes indexed by Levy trees. We start by proving that Markov processes indexed by Levy trees satisfy a special Markov property which can be thought as a…
This paper considers differential problems with random switching, with specific applications to the motion of cells and centrally coordinated motion. Starting with a differential-equation model of cell motion that was proposed previously,…
Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…
We review some developments concerning Markov and Feller processes with jumps in geometric settings. These include stochastic differential equations in Markus canonical form, the Courr\`{e}ge theorem on Lie groups, and invariant Markov…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates…
This paper discusses tractable development and statistical estimation of a continuous time stochastic process with a finite state space having non-Markov property. The process is formed by a finite mixture of right-continuous Markov jump…
We consider a one-dimensional piecewise deterministic Markov process (PDMP) on $[0,1]$ with resetting at $0$ and depending on a small parameter $\varepsilon>0$. In the singular vanishing limit $\varepsilon \to 0$ we prove that the ``…
We present a closed-form, computable expression for the expected number of times any transition event occurs during the transient phase of a reducible Markov chain. Examples of events include time to absorption, number of visits to a state,…