Related papers: A Temporal Difference Method for Stochastic Contin…
Reinforcement learning algorithms are typically designed for discrete-time dynamics, even though the underlying real-world control systems are often continuous in time. In this paper, we study the problem of continuous-time reinforcement…
In this paper, we study the optimal singular controls for stochastic recursive systems, in which the control has two components: the regular control, and the singular control. Under certain assumptions, we establish the dynamic programming…
This note lays part of the theoretical ground for a definition of differential systems modeling reinforcement learning in continuous time non-Markovian rough environments. Specifically we focus on optimal relaxed control of rough equations…
Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the…
In this paper, a stochastic optimal control problem is investigated in which the system is governed by a stochastic functional differential equation. In the framework of functional It\^o calculus, we build the dynamic programming principle…
This paper introduces a new type of second order stochastic backward Hamilton-Jacobi-Bellman (HJB) equations for optimal stochastic control problems with a currently observable but non-predicable parameter process, in addition to the…
In this manuscript, we study optimal control problems for stochastic delay differential equations using the dynamic programming approach in Hilbert spaces via viscosity solutions of the associated Hamilton-Jacobi-Bellman equations. We show…
We consider an extension of the well-known Hamilton-Jacobi-Bellman (HJB) equation for fractional order dynamical systems in which a generalized performance index is considered for the related optimal control problem. Owing to the…
We provide a data-driven framework for optimal control of a continuous-time stochastic dynamical system. The proposed framework relies on the linear operator theory involving linear Perron-Frobenius (P-F) and Koopman operators. Our first…
In recent times, a variety of Reinforcement Learning (RL) algorithms have been proposed for optimal tracking problem of continuous time nonlinear systems with input constraints. Most of these algorithms are based on the notion of uniform…
Merton portfolio management problem is studied in this paper within a stochastic volatility, non constant time discount rate, and power utility framework. This problem is time inconsistent and the way out of this predicament is to consider…
We propose a refinement of temporal-difference learning that enforces first-order Bellman consistency: the learned value function is trained to match not only the Bellman targets in value but also their derivatives with respect to states…
This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of linear temporal logic (LTL) specifications. Due to the consideration of environment and…
This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB)…
In this paper, we explore a new class of stochastic control problems characterized by specific control constraints. Specifically, the admissible controls are subject to the ratcheting constraint, meaning they must be non-decreasing over…
In this note, we demonstrate that a locally semiconvex viscosity supersolution to a possibly degenerate fully nonlinear elliptic Hamilton-Jacobi-Bellman (HJB) equation is differentiable along the directions spanned by the range of the…
In this paper, we study one kind of stochastic recursive optimal control problem with the obstacle constraints for the cost function where the cost function is described by the solution of one reflected backward stochastic differential…
In this paper, we study policy evaluation in continuous-time reinforcement learning (RL), where the state follows an unknown stochastic differential equation (SDE), but only discrete-time data are available. We first highlight that the…
This paper addresses the problem of learning optimal control policies for systems with uncertain dynamics and high-level control objectives specified as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace…
We consider a dynamic portfolio optimization problem that incorporates predictable returns, instantaneous transaction costs, price impact, and stochastic volatility, extending the classical results of Garleanu and Pedersen (2013), which…