Related papers: Proper Policies in Infinite-State Stochastic Short…
A temporally abstract action, or an option, is specified by a policy and a termination condition: the policy guides option behavior, and the termination condition roughly determines its length. Generally, learning with longer options (like…
This paper describes the structure of optimal policies for discounted periodic-review single-commodity total-cost inventory control problems with fixed ordering costs for finite and infinite horizons. There are known conditions in the…
This work proposes an optimal safe controller minimizing an infinite horizon cost functional subject to control barrier functions (CBFs) safety conditions. The constrained optimal control problem is reformulated as a minimization problem of…
We consider a class of closed loop stochastic optimal control problems in finite time horizon, in which the cost is an expectation conditional on the event that the process has not exited a given bounded domain. An important difficulty is…
We consider a model of optimal investment and consumption with both habit formation and partial observations in incomplete It\^{o} processes market. The investor chooses his consumption under the addictive habits constraint while only…
Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual objective function. However, compared to dynamic…
This paper is concerned with a stochastic recursive optimal control problem with time delay, where the controlled system is described by a stochastic differential delayed equation (SDDE) and the cost functional is formulated as the solution…
This paper investigates a Hamilton-Jacobi (HJ) analysis to solve finite-horizon optimal control problems for high-dimensional systems. Although grid-based methods, such as the level-set method [1], numerically solve a general class of HJ…
In this paper, we study a stochastic recursive optimal control problem in which the system is governed by a functional forward-backward stochastic differential equation. Under standard assumptions, we establish the dynamic programming…
Using game and probability theories, I study the French popular game 421, a perfect information stochastic stage game. The problem is to find strategies maximizing the probability of some expected utility. I only solve a player's round…
In this paper we consider an energy storage optimization problem in finite time in a model with partial information that allows for a changing economic environment. The state process consists of the storage level controlled by the storage…
We consider a class of exit time stochastic control problems for diffusion processes with discounted criterion, where the controller can utilize a given amount of resource, called "fuel". In contrast to the vast majority of existing…
This paper presents an inverse optimality method to solve the Hamilton-Jacobi-Bellman equation for a class of nonlinear problems for which the cost is quadratic and the dynamics are affine in the input. The method is inverse optimal because…
From the Hamilton-Jacobi-Bellman equation for the value function we derive a non-linear partial differential equation for the optimal portfolio strategy (the dynamic control). The equation is general in the sense that it does not depend on…
We propose an algorithm to calculate the exact solution for utility optimization problems on finite state spaces under a class of non-differentiable preferences. We prove that optimal strategies must lie on a discrete grid in the plane, and…
For a general entropy-regularized stochastic control problem on an infinite horizon, we prove that a policy iteration algorithm (PIA) converges to an optimal relaxed control. Contrary to the standard stochastic control literature, classical…
One often encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for controlled Markov chains. In this paper, we provide a method to construct sub-optimal policies along with a bound…
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general…
We consider infinite-horizon stationary $\gamma$-discounted Markov Decision Processes, for which it is known that there exists a stationary optimal policy. Using Value and Policy Iteration with some error $\epsilon$ at each iteration, it is…
We establish a linear programming formulation for the solution of joint chance constrained optimal control problems over finite time horizons. The joint chance constraint may represent an invariance, reachability or reach-avoid…