Control Design for Markov Chains under Safety Constraints: A Convex Approach
Systems and Control
2012-11-09 v2 Optimization and Control
Abstract
This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is qualified as safe if it is not a forbidden state and the probability of it transitioning to a forbidden state is zero. The main objective is to obtain control policies whose closed loop generates the maximal set of safe recurrent states, which may include multiple recurrent classes. A design method is proposed that relies on a finitely parametrized convex program inspired on entropy maximization principles. A numerical example is provided and the adoption of additional constraints is discussed.
Keywords
Cite
@article{arxiv.1209.2883,
title = {Control Design for Markov Chains under Safety Constraints: A Convex Approach},
author = {Eduardo Arvelo and Nuno C. Martins},
journal= {arXiv preprint arXiv:1209.2883},
year = {2012}
}