Related papers: Semilinear Dynamic Programming: Analysis, Algorith…
A matrix optimization problem over an uncertain linear system on finite horizon (abbreviated as MOPUL) is studied, in which the uncertain transition matrix is regarded as a decision variable. This problem is in general NP-hard. By using the…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
Dynamic optimization of mean and variance in Markov decision processes (MDPs) is a long-standing challenge caused by the failure of dynamic programming. In this paper, we propose a new approach to find the globally optimal policy for…
Many real-world decision-theoretic planning problems can be naturally modeled with discrete and continuous state Markov decision processes (DC-MDPs). While previous work has addressed automated decision-theoretic planning for DCMDPs,…
We propose policy gradient algorithms for robust infinite-horizon Markov decision processes (MDPs) with non-rectangular uncertainty sets, thereby addressing an open challenge in the robust MDP literature. Indeed, uncertainty sets that…
We study optimal control problems in infinite horizon when the dynamics belong to a specific class of piecewise deterministic Markov processes constrained to star-shaped networks (inspired by traffic models). We adapt the results in [H. M.…
In this paper, we develop a Topological Approximate Dynamic Programming (TADP) method for planningin stochastic systems modeled as Markov Decision Processesto maximize the probability of satisfying high-level systemspecifications expressed…
Differential Dynamic Programming (DDP) is an efficient trajectory optimization algorithm relying on second-order approximations of a system's dynamics and cost function, and has recently been applied to optimize systems with time-invariant…
Equipping approximate dynamic programming (ADP) with inputconstraints has a tremendous significance. This enables ADP to be applied tothe systems with actuator limitations, which is quite common for dynamicalsystems. In a conventional…
We analyze an optimal stopping problem with a constraint on the expected cost. When the reward function and cost function are Lipschitz continuous in state variable, we show that the value of such an optimal stopping problem is a continuous…
Discrete-time robust optimal control problems generally take a min-max structure over continuous variable spaces, which can be difficult to solve in practice. In this paper, we extend the class of such problems that can be solved through a…
Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately,…
Parallel server systems in transportation, manufacturing, and computing heavily rely on dynamic routing using connected cyber components for computation and communication. Yet, these components remain vulnerable to random malfunctions and…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
The problem of synthesizing stochastic explicit model predictive control policies is known to be quickly intractable even for systems of modest complexity when using classical control-theoretic methods. To address this challenge, we present…
This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical…
Many optimization problems can be naturally represented as (hyper) graphs, where vertices correspond to variables and edges to tasks, whose cost depends on the values of the adjacent variables. Capitalizing on the structure of the graph,…
We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…