Related papers: General limit value in Dynamic Programming
We consider the problem of optimally stopping a general one-dimensional stochastic differential equation (SDE) with generalised drift over an infinite time horizon. First, we derive a complete characterisation of the solution to this…
We study a class of deterministic mean field games and related optimal control problems, with a finite time horizon and in which the state space is a network. An agent controls her velocity, and, when she occupies a vertex, she can either…
Value iteration is a commonly used and empirically competitive method in solving many Markov decision process problems. However, it is known that value iteration has only pseudo-polynomial complexity in general. We establish a somewhat…
A class of valued constraint satisfaction problems (VCSPs) is characterised by a valued constraint language, a fixed set of cost functions on a finite domain. An instance of the problem is specified by a sum of cost functions from the…
We study the policy evaluation problem in multi-agent reinforcement learning. In this problem, a group of agents works cooperatively to evaluate the value function for the global discounted accumulative reward problem, which is composed of…
Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because…
Motivated by applications in risk-sensitive reinforcement learning, we study mean-variance optimization in a discounted reward Markov Decision Process (MDP). Specifically, we analyze a Temporal Difference (TD) learning algorithm with linear…
We prove a sequence of limiting results about weakly dependent stationary and regularly varying stochastic processes in discrete time. After deducing the limiting distribution for individual clusters of extremes, we present a new type of…
In this paper we consider unbounded solutions of perturbed convolution Volterra summation equations. The equations studied are asymptotically sublinear, in the sense that the state--dependence in the summation is of smaller than linear…
Dynamic programming is a class of algorithms used to compute optimal control policies for Markov decision processes. Dynamic programming is ubiquitous in control theory, and is also the foundation of reinforcement learning. In this paper,…
We propose a single-loop variance-reduced acceleration framework, which relates checkpoint update probabilities to momentum parameters, for solving the composite general convex problem where the smooth part has the finite-sum structure.…
We investigate the stability of equilibrium-induced optimal values with respect to (w.r.t.) reward functions $f$ and transition kernels $Q$ for time-inconsistent stopping problems under nonexponential discounting in discrete time. First,…
Symmetry is an important factor in solving many constraint satisfaction problems. One common type of symmetry is when we have symmetric values. In a recent series of papers, we have studied methods to break value symmetries. Our results…
This note tries to show that a re-examination of a first course in analysis, using the more sophisticated tools and approaches obtained in later stages, can be a real fun for experts, advanced students, etc. We start by going to the…
In this work we derive a variant of the classic Glivenko-Cantelli Theorem, which asserts uniform convergence of the empirical Cumulative Distribution Function (CDF) to the CDF of the underlying distribution. Our variant allows for tighter…
We deal with the convergence of the value function of an approximate control problem with uncertain dynamics to the value function of a nonlinear optimal control problem. The assumptions on the dynamics and the costs are rather general and…
A classic result by Cook, Gerards, Schrijver, and Tardos provides an upper bound of $n \Delta$ on the proximity of optimal solutions of an Integer Linear Programming problem and its standard linear relaxation. In this bound, $n$ is the…
We consider the problem of probabilistic allocation of objects under ordinal preferences. We devise an allocation mechanism, called the vigilant eating rule (VER), that applies to nearly arbitrary feasibility constraints. It is constrained…
We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a $d$-dimensional state space of size $N$, the proposed algorithm outputs a quantum-mechanical…
This paper studies the problem of optimal switching for one-dimensional diffusion, which may be regarded as sequential optimal stopping problem with changes of regimes. The resulting dynamic programming principle leads to a system of…