Related papers: Optimal Policy Characterization for a Class of Mul…
We study an optimal stopping problem under non-exponential discounting, where the state process is a multi-dimensional continuous strong Markov process. The discount function is taken to be log sub-additive, capturing decreasing impatience…
We consider policy gradient methods for stochastic optimal control problem in continuous time. In particular, we analyze the gradient flow for the control, viewed as a continuous time limit of the policy gradient method. We prove the global…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
Motivated by the design of fast reinforcement learning algorithms, we study the diffusive limit of a class of pure jump ergodic stochastic control problems. We show that, whenever the intensity of jumps is large enough, the approximation…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
This work addresses an optimal control problem for a semilinear elliptic equation in two-dimensional space, characterized by an exponential nonlinearity and a singular source term. The source is modeled as a finite linear combination of…
We consider the control problem of the stochastic Navier-Stokes equations in multidimensional domains introduced in \cite{ocpc} restricted to noise terms defined by Q-Wiener processes. Using a stochastic maximum principle, we derive a…
We study a stochastic control problem for continuous multidimensional martingales with fixed quadratic variation. In a radially symmetric environment, we are able to find an explicit solution to the control problem and find an optimal…
We present a decentralized ergodic control policy for time-varying area coverage problems for multiple agents with nonlinear dynamics. Ergodic control allows us to specify distributions as objectives for area coverage problems for nonlinear…
Recent work [Ran22] formulated a class of optimal control problems involving positive linear systems, linear stage costs, and elementwise constraints on control. It was shown that the problem admits linear optimal cost and the associated…
In recent years, mutual information optimal control has been proposed as an extension of maximum entropy optimal control. Both approaches introduce regularization terms to render the policy stochastic, and it is important to theoretically…
Many techniques originally developed in the context of deterministic control theory have been recently applied to the quest for optimal protocols in stochastic processes. Given a system subject to environmental fluctuations, one may ask…
State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in…
We formulate and study the infinite dimensional linear programming (LP) problem associated with the deterministic discrete time long-run average criterion optimal control problem. Along with its dual, this LP problem allows one to…
For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…
Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately,…
We obtain a probabilistic solution to linear-quadratic optimal control problems with state constraints. Given a closed set $\mathcal{D}\subseteq [0,T]\times\mathbb{R}^d$, a diffusion $X$ in $\mathbb{R}^d$ must be linearly controlled in…
In this note, we study a class of stochastic control problems where the optimal strategies are described by two parameters. These include a subset of singular control, impulse control, and two-player stochastic games. The parameters are…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
We study the minimization of the expected costs under stochastic constraint at the terminal time. The first and the main result says that for a power type of costs, the value function is the minimal positive solution of a second order…