Related papers: Risk-sensitive Semi-Markov Decision Problems with …
We consider a utility maximization problem over partially observable Markov ON/OFF channels. In this network instantaneous channel states are never known, and at most one user is selected for service in every slot according to the partial…
The aim of this paper is to investigate risk-averse and distributionally robust modeling of Stochastic Optimal Control (SOC) and Markov Decision Process (MDP). We discuss construction of conditional nested risk functionals, a particular…
We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability…
We study discrete-time Markov Decision Processes (MDPs) on finite state-action spaces and analyze the stability of optimal policies and value functions in the long-run discounted risk-sensitive objective setting. Our analysis addresses…
In this paper, a new method for solving the problem of optimal control of semi-Markov processes with finitely many states is considered. A new form of the assertion on an extremum of a liner-fractional integral functional given on a set of…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
In the last decades, control problems with infinite horizons and discount factors have become increasingly central not only for economics but also for applications in artificial intelligence and machine learning. The strong links between…
We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost…
The minimization of energy-like cost functionals is addressed in the context of optimal control problems. For a general class of dynamical systems, with possibly unstable and nonlinear free dynamics, it is shown that a sequence of solutions…
In this paper we consider a broad class of infinite horizon discrete-time optimal control models that involve a nonnegative cost function and an affine mapping in their dynamic programming equation. They include as special cases classical…
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has…
This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an…
We present metrics for measuring state similarity in Markov decision processes (MDPs) with infinitely many states, including MDPs with continuous state spaces. Such metrics provide a stable quantitative analogue of the notion of…
A class of infinite horizon optimal control problems involving mixed quasi-norms of $L^p$-type cost functionals for the controls is discussed. These functionals enhance sparsity and switching properties of the optimal controls. The…
Markov reward processes (MRPs) are used to model stochastic phenomena arising in operations research, control engineering, robotics, and artificial intelligence, as well as communication and transportation networks. In many of these cases,…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
In the general framework of a semimartingale financial model and a utility function $U$ defined on the positive real line, we compute the first-order expansion of marginal utility-based prices with respect to a ``small'' number of random…
We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
We consider the problem of designing policies for partially observable Markov decision processes (POMDPs) with dynamic coherent risk objectives. Synthesizing risk-averse optimal policies for POMDPs requires infinite memory and thus…