Related papers: Discrete-time probabilistic approximation of path-…
A treatment of direct simulation Monte Carlo method (DSMC) as a Markov process with a master equation is given and the corresponding master equation is derived. A hierarchy of equations for the reduced probability distributions is derived…
This paper deals with a stochastic optimal feedback control problem for the controlled stochastic partial differential equations. More precisely, we establish the existence of stochastic optimal feedback control for the controlled…
In this paper, we consider the implementation of multi-level Monte Carlo method to a stochastic optimal control problem with log-normal coefficients and its surrogate model problem. From the perspective of two optimization problems, i.e.,…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems [15], [18], this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic…
We prove convergence of the proximal policy gradient method for a class of constrained stochastic control problems with control in both the drift and diffusion of the state process. The problem requires either the running or terminal cost…
We show several results on convergence of the Monte Carlo method applied to consistent approximations of the isentropic Euler system of gas dynamics with uncertain initial data. Our method is based on combination of several new concepts. We…
This paper firstly presents the necessary and sufficient conditions for a kind of discrete-time robust stochastic optimal control problem with convex control domains. As it is an "inf sup problem", the classical variational method is…
In a previous work (Akian, Fodjo, 2016), we introduced a lower complexity probabilistic max-plus numerical method for solving fully nonlinear Hamilton-Jacobi-Bellman equations associated to diffusion control problems involving a finite…
The Multilevel Monte Carlo method is an efficient variance reduction technique. It uses a sequence of coarse approximations to reduce the computational cost in uncertainty quantification applications. The method is nowadays often considered…
This work addresses the optimal covariance control problem for stochastic discrete-time linear time-varying systems subject to chance constraints. Covariance steering is a stochastic control problem to steer the system state Gaussian…
We develop a convergence theory for non-monotone approximation schemes for fully nonlinear parabolic partial differential equations. Modern computational methods such as kernel-based collocation, spectral methods, physics-informed neural…
In this paper we present a new approach to control variates for improving computational efficiency of Ensemble Monte Carlo. We present the approach using simulation of paths of a time-dependent nonlinear stochastic equation. The core idea…
Stochastic Model Predictive Control addresses uncertainties by incorporating chance constraints that provide probabilistic guarantees of constraint satisfaction. However, simultaneously optimizing over the risk allocation and the feedback…
We analyse a multilevel Monte Carlo method for the approximation of distribution functions of univariate random variables. Since, by assumption, the target distribution is not known explicitly, approximations have to be used. We provide an…
In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…
Accurate simulation of complex physical systems enables the development, testing, and certification of control strategies before they are deployed into the real systems. As simulators become more advanced, the analytical tractability of the…
Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the…
Stochastic Optimal Control (SOC) problems arise in systems influenced by uncertainty, such as autonomous robots or financial models. Traditional methods like dynamic programming are often intractable for high-dimensional, nonlinear systems…
In this paper we present a novel approach towards variance reduction for discretised diffusion processes. The proposed approach involves specially constructed control variates and allows for a significant reduction in the variance for the…