Related papers: General limit value in Dynamic Programming
The paper is concerned with two-person dynamic zero-sum games. We investigate the limit of value functions of finite horizon games with long run average cost as the time horizon tends to infinity, and the limit of value functions of…
Building on insights of Jovanovic (1982) and subsequent authors, we develop a comprehensive theory of optimal timing of decisions based around continuation value functions and operators that act on them. Optimality results are provided…
We consider a kind of stochastic exit time optimal control problems, in which the cost function is defined through a nonlinear backward stochastic differential equation. We study the regularity of the value function for such a control…
Value iteration (VI) is a ubiquitous algorithm for optimal control, planning, and reinforcement learning schemes. Under the right assumptions, VI is a vital tool to generate inputs with desirable properties for the controlled system, like…
We explore properties of the value function and existence of optimal stopping times for functionals with discontinuities related to the boundary of an open (possibly unbounded) set $\mathcal{O}$. The stopping horizon is either random, equal…
In this paper, we study an ordinary differential equation with a degenerate global attractor at the origin, to which we add a white noise with a small parameter that regulates its intensity. Under general conditions, for any fixed…
We study the dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems to show that the underlying Bellman operator has a…
We study existence and uniqueness of the fixed points solutions of a large class of non-linear variable discounted transfer operators associated to a sequential decision-making process. We establish regularity properties of these solutions,…
In both high-performance computing (HPC) environments and the public cloud, the duration of time to retrieve or save your results is simultaneously unpredictable and important to your over all resource budget. It is generally accepted…
The principle of optimality is a fundamental aspect of dynamic programming, which states that the optimal solution to a dynamic optimization problem can be found by combining the optimal solutions to its sub-problems. While this principle…
Simple stochastic games can be solved by value iteration (VI), which yields a sequence of under-approximations of the value of the game. This sequence is guaranteed to converge to the value only in the limit. Since no stopping criterion is…
Linear Temporal Logic (LTL) is a formal way of specifying complex objectives for planning problems modeled as Markov Decision Processes (MDPs). The planning problem aims to find the optimal policy that maximizes the satisfaction probability…
This paper discusses a general and useful stability principle which, roughly speaking, says that given a uniformly continuous function defined on an arbitrary metric space, if the function is bounded on the constraint set and we slightly…
The goal of this paper is to investigate new and simple convergence analysis of dynamic programming for linear quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of…
This paper studies value iteration for infinite horizon contracting Markov decision processes under convexity assumptions and when the state space is uncountable. The original value iteration is replaced with a more tractable form and the…
This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results…
While Value Iteration (VI) is one of the most fundamental algorithms in Reinforcement Learning, its theoretical convergence guarantees still exhibit a persistent mismatch with empirical behavior. In the discounted-reward case, classical…
In an optimal control framework, we consider the value $V_T(x)$ of the problem starting from state $x$ with finite horizon $T$, as well as the value $V_\lambda(x)$ of the $\lambda$-discounted problem starting from $x$. We prove that uniform…
We consider discrete-time infinite horizon deterministic optimal control problems with nonnegative cost per stage, and a destination that is cost-free and absorbing. The classical linear-quadratic regulator problem is a special case. Our…
This paper focuses on the value function in the time-optimal problem for a continuity equation in the space of probability measures. We derive the dynamic programming principle for this problem. In particular, we prove that the Kruzhkov…