Related papers: Numerical method for impulse control of Piecewise …
We establish $L^2$-exponential convergence rate for three popular piecewise deterministic Markov processes for sampling: the randomized Hamiltonian Monte Carlo method, the zigzag process, and the bouncy particle sampler. Our analysis is…
Optimal control problem is typically solved by first finding the value function through Hamilton-Jacobi equation (HJE) and then taking the minimizer of the Hamiltonian to obtain the control. In this work, instead of focusing on the value…
A new branch based on Markov processes is developing in the recent literature of financial time series modeling. In this paper, an Indexed Markov Chain has been used to model high frequency price returns of quoted firms. The peculiarity of…
For Markov jump processes on irreducible networks with finite number of sites, we derive a general and explicit expression of the squared coefficient of variation for the net number of transitions from one site to a connected site in a…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
A new method for stochastic control based on neural networks and using randomisation of discrete random variables is proposed and applied to optimal stopping time problems. The method models directly the policy and does not need the…
Markov jump processes are widely used to model natural and engineered processes. In the context of biological or chemical applications one typically refers to the chemical master equation (CME), which models the evolution of the probability…
The objective of this paper is to study the filtering problem for a system of partially observable processes $(X, Y)$, where $X$ is a non-Markovian pure-jump process representing the signal and $Y$ is a general jump-diffusion which provides…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
In this article, we investigate a dynamic control problem of a production-inventory system. Here, demands arrive at the production unit according to a Poisson process and are processed in an FCFS manner. The processing time of the…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
Markov decision process (MDP) is a decision making framework where a decision maker is interested in maximizing the expected discounted value of a stream of rewards received at future stages at various states which are visited according to…
This paper deals with the numerical approximation of American-style option values governed by partial differential complementarity problems. For a variety of one- and two-asset American options we investigate by ample numerical experiments…
In this paper we consider Bayesian parameter inference associated to a class of partially observed stochastic differential equations (SDE) driven by jump processes. Such type of models can be routinely found in applications, of which we…
The main objective of this paper is the construction of the solution of an impulsive stochastic differential equation, subject to control conditions in the pulse-times and give sufficient conditions for them to be random variables with…
We consider a class of impulse control problems for general underlying strong Markov processes on the real line, which allows for an explicit solution. The optimal impulse times are shown to be of threshold type and the optimal threshold is…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
We first derive the Hamilton-Jacobi theory underlying continuous-time Markov processes, and then use the construction to develop a variational algorithm for estimating escape (least improbable or first passage) paths for a generic…
We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…