Related papers: Numerical method for impulse control of Piecewise …
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…
This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
We are concerned with the absolute continuity of stationary distributions corresponding to some piecewise deterministic Markov process, being typically encountered in biological models. The process under investigation involves a…
Piecewise-deterministic Markov processes combine continuous in time dynamics with jump events, the rates of which generally depend on the continuous variables and thus are not constants. This leads to a problem in a Monte-Carlo simulation…
We investigate a piecewise-deterministic Markov process, evolving on a Polish metric space, whose deterministic behaviour between random jumps is governed by some semi-flow, and any state right after the jump is attained by a randomly…
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics…
In this paper, we investigate the effects of applying generalised (non-exponential) discounting on a long-run impulse control problem for a Feller-Markov process. We show that the optimal value of the discounted problem is the same as the…
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…
The use of stochastic models, in effect piecewise deterministic Markov processes (PDMP), has become increasingly popular especially for the modeling of chemical reactions and cell biophysics. Yet, exact simulation methods, for the…
In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates…
This paper develops numerical methods for finding optimal dividend pay-out and reinsurance policies. A generalized singular control formulation of surplus and discounted payoff function are introduced, where the surplus is modeled by a…
Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially…
This paper deals with discrete-time Markov control processes on a general state space. A long-run risk-sensitive average cost criterion is used as a performance measure. The one-step cost function is nonnegative and possibly unbounded.…
We consider a long-run impulse control problem for a generic Markov process with a multiplicative reward functional. We construct a solution to the associated Bellman equation and provide a verification result. The argument is based on the…
An extension of non-deterministic processes driven by the random telegraph signal is introduced in the framework of "piecewise deterministic Markov processes" [Davis], including a broader category of random systems. The corresponding…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…