Related papers: Policy Iteration Achieves Regularized Equilibrium …
Optimal control of interacting particles governed by stochastic evolution equations in Hilbert spaces is an open area of research. Such systems naturally arise in formulations where each particle is modeled by stochastic partial…
We study equilibrium feedback strategies for a family of dynamic mean-variance problems with competition among a large group of agents. We assume that the time horizon is random and each agent's risk aversion depends dynamically on the…
We study a class of optimal control problems with state constraints where the state equation is a differential equation with delays. This class includes some problems arising in economics, in particular the so-called models with time to…
In this note, we study a class of indefinite stochastic McKean-Vlasov linear-quadratic (LQ in short) control problem under the control taking nonnegative values. In contrast to the conventional issue, both the classical dynamic programming…
Entropy regularized algorithms such as Soft Q-learning and Soft Actor-Critic, recently showed state-of-the-art performance on a number of challenging reinforcement learning (RL) tasks. The regularized formulation modifies the standard RL…
Policy iteration is one of the classical frameworks of reinforcement learning, which requires a known initial stabilizing control. However, finding the initial stabilizing control depends on the known system model. To relax this requirement…
Policy iteration and value iteration are at the core of many (approximate) dynamic programming methods. For Markov Decision Processes with finite state and action spaces, we show that they are instances of semismooth Newton-type methods to…
The long-time average behavior of the value function in the calculus of variations is known to be connected to the existence of the limit of the corresponding Abel means. Still in the Tonelli case, such a limit is in turn related to the…
We deal with an infinite horizon, infinite dimensional stochastic optimal control problem arising in the study of economic growth in time-space. Such problem has been the object of various papers in deterministic cases when the possible…
We provide a unified approach to find equilibrium solutions for time-inconsistent problems with distribution dependent rewards, which are important to the study of behavioral finance and economics. Our approach is based on {\it equilibrium…
In this paper we study a class of HJB equations which solve for equilibria for general time-inconsistent deterministic linear quadratic control problems within the intra-personal game theoretic framework, where the inconsistency arises from…
We develop a theory for continuous-time non-Markovian stochastic control problems which are inherently time-inconsistent. Their distinguishing feature is that the classical Bellman optimality principle no longer holds. Our formulation is…
We address the problem of applying the Kolmogorov-Sinai method of entropic analysis, expressed in a generalized non-extensive form, to the dynamics of the logistic map at the chaotic threshold, which is known to be characterized by a power…
Asynchronous stochastic approximations (SAs) are an important class of model-free algorithms, tools and techniques that are popular in multi-agent and distributed control scenarios. To counter Bellman's curse of dimensionality, such…
Stochastic optimal principle leads to the resolution of a partial differential equation (PDE), namely the Hamilton-Jacobi-Bellman (HJB) equation. In general, this equation cannot be solved analytically, thus numerical algorithms are the…
This paper addresses a Stackelberg stochastic linear-quadratic (LQ) differential game under closed-loop information, a problem inherently time-inconsistent. Existing approaches rely on solving two coupled Hamilton-Jacobi-Bellman (HJB)…
We address the problem of combined stochastic and impulse control for a market maker operating in a limit order book. The problem is formulated as a Hamilton-Jacobi-Bellman quasi-variational inequality (HJBQVI). We propose an implicit…
In this paper we investigate a dynamic stochastic portfolio optimization problem involving both the expected terminal utility and intertemporal utility maximization. We solve the problem by means of a solution to a fully nonlinear…
This paper characterizes differentiable subgame perfect equilibria in a continuous time intertemporal decision optimization problem with non-constant discounting. The equilibrium equation takes two different forms, one of which is…
This paper studies a continuous-time stochastic linear-quadratic (SLQ) optimal control problem on infinite-horizon. A data-driven policy iteration algorithm is proposed to solve the SLQ problem. Without knowing three system coefficient…