Related papers: Numerical method for optimal stopping of piecewise…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…
Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space…
We analyze an optimal stopping problem with random maturity under a nonlinear expectation with respect to a weakly compact set of mutually singular probabilities $\mathcal{P}$. The maturity is specified as the hitting time to level $0$ of…
Piecewise Diffusion Markov Processes (PDifMPs) are valuable for modelling systems where continuous dynamics are interrupted by sudden shifts and/or changes in drift and diffusion. The first-passage time (FPT) in such models plays a central…
A Markov process is registered. At random moment $\theta$ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is…
In this paper we consider a method of solving optimal stopping problems in discrete and continuous time based on their dual representation. A novel and generic simulation-based optimization algorithm not involving nested simulations is…
We study an optimal process control problem with multiple assignable causes. The process is initially in-control but is subject to random transition to one of multiple out-of-control states due to assignable causes. The objective is to find…
We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…
We present a numerical approximation technique for the analysis of continuous-time Markov chains that describe networks of biochemical reactions and play an important role in the stochastic modeling of biological systems. Our approach is…
We develop algorithms for the numerical computation of the quadratic hedging strategy in incomplete markets modeled by pure jump Markov process. Using the Hamilton-Jacobi-Bellman approach, the value function of the quadratic hedging problem…
We introduce a new method, stepwise method for solving optimal con- trol problems. Our first motivation for new approach emanate from limi- tations on continuous time control functions in PMP. Practically in most of the real world models,…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
There is much interest in using partially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements…
In this paper, we focus on numerical approximations of Piecewise Diffusion Markov Processes (PDifMPs), particularly when the explicit flow maps are unavailable. Our approach is based on the thinning method for modelling the jump mechanism…
We study optimal control problems in infinite horizon when the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (inspired by traffic models). We adapt the results in [H. M.…
In this paper we consider discrete and continuous time risk sensitive optimal stopping problem. Using suitable properties of the underlying Feller-Markov process we prove continuity of the optimal stopping value function and provide formula…
Piecewise Deterministic Markov Processes (PDMPs) such as the Bouncy Particle Sampler and the Zig-Zag Sampler, have gained attention as continuous-time counterparts of classical Markov chain Monte Carlo. We study their transient regime under…
Markov jump processes are widely used to model natural and engineered processes. In the context of biological or chemical applications one typically refers to the chemical master equation (CME), which models the evolution of the probability…
This paper studies the optimal control problem for discrete-time nonlinear systems and an approximate dynamic programming-based Model Predictive Control (MPC) scheme is proposed for minimizing a quadratic performance measure. In the…