Policy stability and ultimate stationarity in discounted risk-sensitive stochastic control
Abstract
We study discrete-time Markov Decision Processes (MDPs) on finite state-action spaces and analyze the stability of optimal policies and value functions in the long-run discounted risk-sensitive objective setting. Our analysis addresses robustness with respect to perturbations of the risk-aversion parameter and the discount factor, the emergence of ultimate stationarity, and the interaction between discounted and averaged formulations under suitable mixing assumptions. We further investigate limiting regimes associated with vanishing discount and vanishing risk sensitivity, and discuss the role of Blackwell-type stability properties in the discounted setting. Finally, we provide numerical illustrations that highlight the intrinsic non-stationarity of optimal discounted risk-sensitive policies.
Keywords
Cite
@article{arxiv.2601.06998,
title = {Policy stability and ultimate stationarity in discounted risk-sensitive stochastic control},
author = {Nicole Bäuerle and Marcin Pitera and Łukasz Stettner},
journal= {arXiv preprint arXiv:2601.06998},
year = {2026}
}