Related papers: Applications of Variable Discounting Dynamic Progr…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
We study identification of dynamic discrete choice models with hyperbolic discounting. We show that the standard discount factor, present bias factor, and instantaneous utility functions for the sophisticated agent are point-identified from…
We provide an overview on how to use the measurable selection techniques to derive the dynamic programming principle for a general stochastic optimal control/stopping problem. By considering its martingale problem formulation on the…
Recent work [Ran22] formulated a class of optimal control problems involving positive linear systems, linear stage costs, and elementwise constraints on control. It was shown that the problem admits linear optimal cost and the associated…
We apply numerical dynamic programming techniques to solve discrete-time multi-asset dynamic portfolio optimization problems with proportional transaction costs and shorting/borrowing constraints. Examples include problems with multiple…
We develop a discrete-time optimal control framework for systems evolving on Lie groups. Our work generalizes the original Differential Dynamic Programming method, by employing a coordinate-free, Lie-theoretic approach for its derivation. A…
We consider infinite horizon dynamic programming problems, where the control at each stage consists of several distinct decisions, each one made by one of several agents. In an earlier work we introduced a policy iteration algorithm, where…
We consider the problem of discounted optimal state-feedback regulation for general unknown deterministic discrete-time systems. It is well known that open-loop instability of systems, non-quadratic cost functions and complex nonlinear…
We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…
We consider convex optimization problems formulated using dynamic programming equations. Such problems can be solved using the Dual Dynamic Programming algorithm combined with the Level 1 cut selection strategy or the Territory algorithm to…
We provide a unifying approximate dynamic programming framework that applies to a broad variety of problems involving sequential estimation. We consider first the construction of surrogate cost functions for the purposes of optimization,…
Critical points of energy functionals, which are of broad interest, for instance, in physics and chemistry, in solid and quantum mechanics, in material science, or in general diffusion-reaction models arise as solutions to the associated…
It has been recently pointed out that dynamical systems depending on future values of the unknowns may be useful in different areas of knowledge. We explore in this context the extension of the concept of order reduction that has been…
In this paper, we consider the classic stochastic (dynamic) knapsack problem, a fundamental mathematical model in revenue management, with general time-varying random demand. Our main goal is to study the optimal policies, which can be…
Iterative optimization algorithms depend on access to information about the objective function. In a differentiable programming framework, this information, such as gradients, can be automatically derived from the computational graph. We…
The problem of computing the smallest fixed point of an order-preserving map arises in the study of zero-sum positive stochastic games. It also arises in static analysis of programs by abstract interpretation. In this context, the discount…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
We consider a broad class of dynamic programming (DP) problems that involve a partially linear structure and some positivity properties in their system equation and cost function. We address deterministic and stochastic problems, possibly…
Though switched dynamical systems have shown great utility in modeling a variety of physical phenomena, the construction of an optimal control of such systems has proven difficult since it demands some type of optimal mode scheduling. In…
What are the functionals of the reward that can be computed and optimized exactly in Markov Decision Processes?In the finite-horizon, undiscounted setting, Dynamic Programming (DP) can only handle these operations efficiently for certain…