Related papers: Time-Varying Parameters in Sequential Decision Mak…
In this paper we develop a novel, discrete-time optimal control framework for mechanical systems with uncertain model parameters. We consider finite-horizon problems where the performance index depends on the statistical moments of the…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…
In this work, we study discrete-time Markov decision processes (MDPs) under constraints with Borel state and action spaces and where all the performance functions have the same form of the expected total reward (ETR) criterion over the…
In this paper, we propose a unified stochastic optimal control framework that integrates time-optimal control problems with classical stochastic optimal control formulations. Unlike conventional deterministic time-optimal control models,…
We prove a general existence result in stochastic optimal control in discrete time where controls take values in conditional metric spaces, and depend on the current state and the information of past decisions through the evolution of a…
We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. We identify sufficient conditions under which small perturbations in the model…
We study the offline data-driven sequential decision making problem in the framework of Markov decision process (MDP). In order to enhance the generalizability and adaptivity of the learned policy, we propose to evaluate each policy by a…
The (R, s, S) is a stochastic inventory control policy widely used by practitioners. In an inventory system managed according to this policy, the inventory is reviewed at instant R; if the observed inventory position is lower than the…
It has long been observed that the performance of evolutionary algorithms and other randomized search heuristics can benefit from a non-static choice of the parameters that steer their optimization behavior. Mechanisms that identify…
This paper studies temporal planning in probabilistic environments, modeled as labeled Markov decision processes (MDPs), with user preferences over multiple temporal goals. Existing works reflect such preferences as a prioritized list of…
We propose a novel flexible-step model predictive control algorithm for unknown linear time-invariant discrete-time systems. The goal is to asymptotically stabilize the system without relying on a pre-collected dataset that describes its…
This paper studies the computation of robust deterministic policies for Markov Decision Processes (MDPs) in the Lightning Does Not Strike Twice (LDST) model of Mannor, Mebel and Xu (ICML '12). In this model, designed to provide robustness…
In this paper, we study asymptotic properties of problems of control of stochastic discrete time systems (also known as Markov decision processes) with time averaging and time discounting optimality criteria, and we establish that the…
We consider a dynamic server control problem for two parallel queues with randomly varying connectivity and server switchover time between the queues. At each time slot the server decides either to stay with the current queue or switch to…
Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not…
We consider the constrained optimal control problem for the gradual-impulsive CTMDP model with the performance criteria being the expected total undiscounted costs (from the running cost and the cost from each time an impulse being…
Canonical models of Markov decision processes (MDPs) usually consider geometric discounting based on a constant discount factor. While this standard modeling approach has led to many elegant results, some recent studies indicate the…
Planning problems are hard, motion planning, for example, isPSPACE-hard. Such problems are even more difficult in the presence of uncertainty. Although, Markov Decision Processes (MDPs) provide a formal framework for such problems, finding…
This paper deals with the problem of formulating an adaptive Model Predictive Control strategy for constrained uncertain systems. We consider a linear system, in presence of bounded time varying additive uncertainty. The uncertainty is…
Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves…