Related papers: Hitting probabilities in a Markov additive process…
New results on conditional joint probability distributions of first exit times are presented for a continuous-time stochastic process defined as the mixture of Markov jump processes moving at different speeds on the same finite state space,…
Linear systems with many degrees of freedom containing multiplicative and additive noise are considered. The steady state probability distribution for equations of this kind is examined. With multiplicative white noise it is shown that…
We study the persistence probability for processes with stationary increments. Our results apply to a number of examples: sums of stationary correlated random variables whose scaling limit is fractional Brownian motion, random walks in…
In this paper, we consider the discounted continuous-time Markov decision process (CTMDP) with a lower bounding function. In this model, the negative part of each cost rate is bounded by the drift function, say $w$, whereas the positive…
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.
The paper proposes a class of financial market models which are based on inhomogeneous telegraph processes and jump diffusions with alternating volatilities. It is assumed that the jumps occur when the tendencies and volatilities are…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…
Suppose there are $n$ Markov chains and we need to pay a per-step \emph{price} to advance them. The "destination" states of the Markov chains contain rewards; however, we can only get rewards for a subset of them that satisfy a…
We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…
Some classes of increment martingales, and the corresponding localized classes, are studied. An increment martingale is indexed by the real line and its increment processes are martingales. We focus primarily on the behavior as time goes to…
We consider the task of generating draws from a Markov jump process (MJP) between two time-points at which the process is known. Resulting draws are typically termed bridges and the generation of such bridges plays a key role in…
Markov decision processes are widely used for planning and verification in settings that combine controllable or adversarial choices with probabilistic behaviour. The standard analysis algorithm, value iteration, only provides a lower bound…
We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a…
We introduce and investigate a new model of a finite number of particles jumping forward on the real line. The jump lengths are independent of everything, but the jump rate of each particle depends on the relative position of the particle…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…
We present sufficient conditions, in terms of the jumping kernels, for two large classes of conservative Markov processes of pure-jump type to be purely discontinuous martingales with finite second moment. As an application, we establish…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
In this short paper, we connect the procedure of constructing a totally inaccessible stopping time for a given process using the well-known Cox construction, dependent on an independent exponential random variable; with naturally occurring…