Related papers: Probabilistic Framework of Howard's Policy Iterati…

We propose a time-space discretization scheme for quasi-linear parabolic PDEs. The algorithm relies on the theory of fully coupled forward--backward SDEs, which provides an efficient probabilistic representation of this type of equation.…

We adopt a policy optimization viewpoint towards policy evaluation for robust Markov decision process with $\mathrm{s}$-rectangular ambiguity sets. The developed method, named first-order policy evaluation (FRPE), provides the first unified…

Forward-backward stochastic differential equations (FBSDEs) have been generalized by introducing jumps for better capturing random phenomena, while the resulting FBSDEs are far more intricate than the standard one from every perspective. In…

In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…

Many large MDPs can be represented compactly using a dynamic Bayesian network. Although the structure of the value function does not retain the structure of the process, recent work has shown that value functions in factored MDPs can often…

In reinforcement learning, robust policies for high-stakes decision-making problems with limited data are usually computed by optimizing the percentile criterion, which minimizes the probability of a catastrophic failure. Unfortunately,…

In this paper, we present a backward deep BSDE method applied to Forward Backward Stochastic Differential Equations (FBSDE) with given terminal condition at maturity that time-steps the BSDE backwards. We present an application of this…

In this work, we propose a novel backward differential deep learning-based algorithm for solving high-dimensional nonlinear backward stochastic differential equations (BSDEs), where the deep neural network (DNN) models are trained not only…

We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces. We introduce a property of state space partitions which we call epsilon-homogeneity. Intuitively, an epsilon-homogeneous…

Motivated by many application problems, we consider Markov decision processes (MDPs) with a general loss function and unknown parameters. To mitigate the epistemic uncertainty associated with unknown parameters, we take a Bayesian approach…

The optimal stopping problem is one of the core problems in financial markets, with broad applications such as pricing American and Bermudan options. The deep BSDE method [Han, Jentzen and E, PNAS, 115(34):8505-8510, 2018] has shown great…

Hybrid Bayesian networks (HBN) contain complex conditional probabilistic distributions (CPD) specified as partitioned expressions over discrete and continuous variables. The size of these CPDs grows exponentially with the number of parent…

Kernel learning forward backward SDE filter is an iterative and adaptive meshfree approach to solve the nonlinear filtering problem. It builds from forward backward SDE for Fokker-Planker equation, which defines evolving density for the…

We are concerned with high-dimensional coupled FBSDE systems approximated by the deep BSDE method of Han et al. (2018). It was shown by Han and Long (2020) that the errors induced by the deep BSDE method admit a posteriori estimate…

A new variant of Newton's method for empirical risk minimization is studied, where at each iteration of the optimization algorithm, the gradient and Hessian of the objective function are replaced by robust estimators taken from existing…

We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been…

This project investigates numerical methods for solving fully coupled forward-backward stochastic differential equations (FBSDEs) of McKean-Vlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the…

We provide a stochastic analysis of an overlapping-generations model under incomplete markets. By casting individual optimisation with idiosyncratic income risk into a forward-backward stochastic differential equation (FBSDE) system, we (i)…

We propose a new algorithm to approach weakly the solution of a McKean-Vlasov SDE. Based on the cubature method of Lyons and Victoir 2004, the algorithm is deterministic differing from the the usual methods based on interacting particles.…

We introduce a physics-driven deep latent variable model (PDDLVM) to learn simultaneously parameter-to-solution (forward) and solution-to-parameter (inverse) maps of parametric partial differential equations (PDEs). Our formulation…