Related papers: Uniform value in Dynamic Programming
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
We investigate the stability of the equilibrium-induced optimal value in one-dimensional diffusion setting for a time-inconsistent stopping problem under non-exponential discounting. We show that the optimal value is semi-continuous with…
We address the question of why particular laws were selected for the universe, by proposing a mechanism for laws to evolve. Normally in physical theories, timeless laws act on time-evolving states. We propose that this is an approximation,…
In our daily lives and industrial settings, we often encounter dynamic problems that require reasoning over time and metric constraints. These include tasks such as scheduling, routing, and production sequencing. Dynamic logics have…
This paper aims to solve two fundamental problems on finite or infinite horizon dynamic games with perfect or almost perfect information. Under some mild conditions, we prove (1) the existence of subgame-perfect equilibria in general…
Recent work [Ran22] formulated a class of optimal control problems involving positive linear systems, linear stage costs, and elementwise constraints on control. It was shown that the problem admits linear optimal cost and the associated…
We consider infinite horizon dynamic programming problems, where the control at each stage consists of several distinct decisions, each one made by one of several agents. In an earlier work we introduced a policy iteration algorithm, where…
We study both the value function and Q-function formulation of the Linear Programming approach to Approximate Dynamic Programming. The approach is model-based and optimizes over a restricted function space to approximate the value function…
We consider the problem of constructing Lyapunov functions for linear differential equations with delays. For such systems it is known that exponential stability implies the existence of a positive Lyapunov function which is quadratic on…
We present the notion of asymptotically non-terminating initial variable values for linear loop programs. Those values are directly associated to initial variable values for which the corresponding program does not terminate. Our…
We present a continuous-time contract whereby a top-level player can incentivize a hierarchy of players below him to act in his best interest despite only observing the output of his direct subordinate. This paper extends Sannikov's…
We review the major achievements of the dynamical reduction program, showing why and how it provides a unified, consistent description of physical phenomena, from the microscopic quantum domain to the macroscopic classical one. We discuss…
Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of…
A fast convergence in a fixed-time of solutions of nonlinear dynamical systems, for which special requirements are satisfied on the derivative of a quadratic function calculated along the solutions of the system, is proposed. The conditions…
We introduce a model of infinite horizon linear dynamic optimization and obtain results concerning existence of solution and satisfaction of the competitive condition and transversality condition being unconditionally sufficient for…
We study the problem of finding equilibrium strategies in multi-agent games with incomplete payoff information, where the payoff matrices are only known to the players up to some bounded uncertainty sets. In such games, an ex-post…
Tracking the solution of time-varying variational inequalities is an important problem with applications in game theory, optimization, and machine learning. Existing work considers time-varying games or time-varying optimization problems.…
Large-scale Markov decision processes (MDPs) require planning algorithms with runtime independent of the number of states of the MDP. We consider the planning problem in MDPs using linear value function approximation with only weak…
Algorithms that solve zero-sum games, multi-objective agent objectives, or, more generally, variational inequality (VI) problems are notoriously unstable on general problems. Owing to the increasing need for solving such problems in machine…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…