Related papers: Affine Monotonic and Risk-Sensitive Models in Dyna…
Discrete-time stochastic systems are an essential modelling tool for many engineering systems. We consider stochastic control systems that are evolving over continuous spaces. For this class of models, methods for the formal verification…
Predictive control is frequently used for control problems involving constraints. Being an optimization based technique utilizing a user specified so-called stage cost, performance properties, i.e., bounds on the infinite horizon…
Models of many real-life applications, such as queuing models of communication networks or computing systems, have a countably infinite state-space. Algorithmic and learning procedures that have been developed to produce optimal policies…
When stochastic control problems do not possess separability and/or monotonicity, the dynamic programming pioneered by Bellman in 1950s fails to work as a time-decomposition solution method. Such cases have posted a great challenge to the…
We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability…
We study computationally and statistically efficient Reinforcement Learning algorithms for the linear Bellman Complete setting. This setting uses linear function approximation to capture value functions and unifies existing models like…
We characterize the optimal control for a class of singular stochastic control problems as the unique solution to a related Skorokhod reflection problem. The considered optimization problems concern the minimization of a discounted cost…
We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock…
In this paper we consider long-run risk sensitive average cost impulse control applied to a continuous-time Feller-Markov process. Using the probabilistic approach, we show how to get a solution to a suitable continuous-time Bellman…
As autonomous robots move into complex, dynamic real-world environments, they must learn to navigate safely in real time, yet anticipating all possible behaviors is infeasible. We propose a composable, model-free reinforcement learning…
This paper studies the optimal control problem for discrete-time nonlinear systems and an approximate dynamic programming-based Model Predictive Control (MPC) scheme is proposed for minimizing a quadratic performance measure. In the…
We consider how to use the Bellman residual of the dynamic programming operator to compute suboptimality bounds for solutions to stochastic shortest path problems. Such bounds have been previously established only in the special case that…
We present a numerical method for generating the state-feedback control policy associated with general undiscounted, constant-setpoint, infinite-horizon, nonlinear optimal control problems with continuous state variables. The method is…
Risk-bounded motion planning is an important yet difficult problem for safety-critical tasks. While existing mathematical programming methods offer theoretical guarantees in the context of constrained Markov decision processes, they either…
In this article we present a general framework for non-concave robust stochastic control problems under model uncertainty in a discrete time finite horizon setting. Our framework allows to consider a variety of different path-dependent…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
We consider the stochastic shortest path planning problem in MDPs, i.e., the problem of designing policies that ensure reaching a goal state from a given initial state with minimum accrued cost. In order to account for rare but important…
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
We introduce and study constrained Markov Decision Processes (cMDPs) with anytime constraints. An anytime constraint requires the agent to never violate its budget at any point in time, almost surely. Although Markovian policies are no…