On gradual-impulse control of continuous-time Markov decision processes with exponential utility
Abstract
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very general conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies. Policies that we consider allow multiple simultaneous impulses, randomized selection of impulses with random effects, relaxed gradual controls, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the continuous-time gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.
Keywords
Cite
@article{arxiv.1811.11704,
title = {On gradual-impulse control of continuous-time Markov decision processes with exponential utility},
author = {Xin Guo and Aiko Kurushima and Alexey Piunovskiy and Yi Zhang},
journal= {arXiv preprint arXiv:1811.11704},
year = {2023}
}
Comments
The proof of Lemma 4 in the published version was wrong (though the statement is correct). It is now corrected it (see the proof of Lemma 5.4 in the current file)