Related papers: Gradient-Bounded Dynamic Programming for Submodula…
Discrete-time stochastic systems are an essential modelling tool for many engineering systems. We consider stochastic control systems that are evolving over continuous spaces. For this class of models, methods for the formal verification…
We propose empirical dynamic programming algorithms for Markov decision processes (MDPs). In these algorithms, the exact expectation in the Bellman operator in classical value iteration is replaced by an empirical estimate to get `empirical…
We consider the problem of optimally controlling stochastic, Markovian systems subject to joint chance constraints over a finite-time horizon. For such problems, standard Dynamic Programming is inapplicable due to the time correlation of…
This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic…
A large number of recent studies consider a compartmental SIR model to study optimal control policies aimed at containing the diffusion of COVID-19 while minimizing the economic costs of preventive measures. Such problems are non-convex and…
We consider maximization of stochastic monotone continuous submodular functions (CSF) with a diminishing return property. Existing algorithms only guarantee the performance \textit{in expectation}, and do not bound the probability of…
We consider linear programming (LP) problems in infinite dimensional spaces that are in general computationally intractable. Under suitable assumptions, we develop an approximation bridge from the infinite-dimensional LP to tractable finite…
In this paper, we first establish the dynamic programming principle for stochastic optimal control problems defined on compact Riemannian manifolds without boundary. Subsequently, we derive the associated Hamilton-Jacobi-Bellman (HJB)…
Differential Dynamic Programming is an optimal control technique often used for trajectory generation. Many variations of this algorithm have been developed in the literature, including algorithms for stochastic dynamics or state and input…
We propose a method for designing policies for convex stochastic control problems characterized by random linear dynamics and convex stage cost. We consider policies that employ quadratic approximate value functions as a substitute for the…
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…
We investigate the computation of the gradient of the value function in parametric convex optimization problems. We derive general expression for the gradient of the value function in terms of the cost function, constraints and Lagrange…
We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and…
We introduce a novel generative formulation of deep probabilistic models implementing "soft" constraints on their function dynamics. In particular, we develop a flexible methodological framework where the modeled functions and derivatives…
We develop a GPU-accelerated dynamic programming (DP) method for valuing, operating, and bidding energy storage under multistage stochastic electricity prices. Motivated by computational limitations in existing models, we formulate DP…
In the theory of dynamic programming, an optimal policy is a policy whose lifetime value dominates that of all other policies from every possible initial condition in the state space. This raises a natural question: when does optimality…
To model combinatorial decision problems involving uncertainty and probability, we introduce stochastic constraint programming. Stochastic constraint programs contain both decision variables (which we can set) and stochastic variables…
How to compute (super) hedging costs in rather general fi- nancial market models with transaction costs in discrete-time ? Despite the huge literature on this topic, most of results are characterizations of the super-hedging prices while it…
We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a…
In the Dynamic Programming approach to optimal control problems a crucial role is played by the value function that is characterized as the unique viscosity solution of a Hamilton-Jacobi-Bellman (HJB) equation. It is well known that this…