Related papers: Numerical method for impulse control of Piecewise …
We study the optimal liquidation problem in a market model where the bid price follows a geometric pure jump process whose local characteristics are driven by an unobservable finite-state Markov chain and by the liquidation rate. This model…
This paper studies convergence properties of optimal values and actions for discounted and average-cost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic…
Markov decision processes (MDPs) with multi-dimensional weights are useful to analyze systems with multiple objectives that may be conflicting and require the analysis of trade-offs. We study the complexity of percentile queries in such…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…
We consider piecewise deterministic Markov processes with degenerate transition kernels of the "house-of-cards"-type. We use a splitting scheme based on jump times to prove the absolute continuity, as well as some regularity, of the…
Planning problems where effects of actions are non-deterministic can be modeled as Markov decision processes. Planning problems are usually goal-directed. This paper proposes several techniques for exploiting the goal-directedness to…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest…
This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
A multiplicative relative value iteration algorithm for solving the dynamic programming equation for the risk-sensitive control problem is studied for discrete time controlled Markov chains with a compact Polish state space, and controlled…
In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
We study a constrained stochastic control problem with jumps; the jump times of the controlled process are given by a Poisson process. The cost functional comprises quadratic components for an absolutely continuous control and the…
This paper presents a novel approach to pricing American options using piecewise diffusion Markov processes (PDifMPs), a type of generalised stochastic hybrid system that integrates continuous dynamics with discrete jump processes. Standard…
Controlled discrete time Markov processes are studied first with long run general discounting functional. It is shown that optimal strategies for average reward per unit time problem are also optimal for average generally discounting…
We consider the pricing of derivatives written on accumulated marks, such as weather derivatives or aggregate loss claims, using a self-exciting marked point process. The jump intensity mean-reverts between events and increases at jump…
In this paper we investigate a kind of optimal control problem of coupled forward-backward stochastic system with jumps whose cost functional is defined through a coupled forward-backward stochastic differential equation with Brownian…