Related papers: Gradient boosting-based numerical methods for high…

XGBoost, a scalable tree boosting algorithm, has proven effective for many prediction tasks of practical interest, especially using tabular datasets. Hyperparameter tuning can further improve the predictive performance, but unlike neural…

We present the first deep-learning solver for backward stochastic Volterra integral equations (BSVIEs) and their fully-coupled forward-backward variants. The method trains a neural network to approximate the two solution fields in a single…

Stochastic gradient descent (SGD) or stochastic approximation has been widely used in model training and stochastic optimization. While there is a huge literature on analyzing its convergence, inference on the obtained solutions from SGD…

In this paper we present a unified approach to establish gradient type formulas and Bismut type formulas for backward stochastic differential equations (BSDEs). This approach relies on a mix of derivative formulas with respect to the…

Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artificial neural networks. However very little is known on to what extent SGD is crucial for to the success of this technology and, in…

A splitting scheme for backward doubly stochastic differential equations is proposed. The main idea is to decompose a backward doubly stochastic differential equation into a backward stochastic differential equation and a stochastic…

This is one of our series papers on multistep schemes for solving forward backward stochastic differential equations (FBSDEs) and related problems. Here we extend (with non-trivial updates) our multistep schemes in [W. Zhao, Y. Fu and T.…

Novel multi-step predictor-corrector numerical schemes have been derived for approximating decoupled forward-backward stochastic differential equations (FBSDEs). The stability and high order rate of convergence of the schemes are rigorously…

Understanding the behavior of stochastic gradient methods is a central problem in modern machine learning. Recent work has highlighted diagonal linear networks as a simplified yet expressive setting for analyzing the optimization and…

Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of $L_2$Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introduce so-called…

Stochastic gradient descent method and its variants constitute the core optimization algorithms that achieve good convergence rates for solving machine learning problems. These rates are obtained especially when these algorithms are…

In this paper, we consider a class of backward doubly stochastic differential equations (BDSDE for short) with general terminal value and general random generator. Those BDSDEs do not involve any forward diffusion processes. By using the…

In this paper we introduce the Boosted Double-proximal Subgradient Algorithm (BDSA), a novel splitting algorithm designed to address general structured nonsmooth and nonconvex mathematical programs expressed as sums and differences of…

GPU-based algorithms have greatly accelerated many machine learning methods; however, GPU memory is typically smaller than main memory, limiting the size of training data. In this paper, we describe an out-of-core GPU gradient boosting…

In this paper, we consider the Euler method for backward stochastic Volterra integral equations. First, we approximate the original equation by a family of backward stochastic equations (BSDEs, for short). Then we solve the BSDEs by the…

In this paper,we mainly focus on the numerical solution of high-dimensional stochastic optimal control problem driven by fully-coupled forward-backward stochastic differential equations (FBSDEs in short) through deep learning. We first…

This paper investigates accelerating the convergence of distributed optimization algorithms on non-convex problems. We propose a distributed primal-dual stochastic gradient descent~(SGD) equipped with "powerball" method to accelerate. We…

We introduce and develop the concepts of Geometric Backward Stochastic Differential Equations (GBSDEs, for short) and two-driver BSDEs. We demonstrate their natural suitability for modeling continuous-time dynamic return risk measures. We…

In this paper, we study a class of Type-II backward stochastic Volterra integral equations (BSVIEs). For the adapted M-solutions, we obtain two approximation results, namely, a BSDE approximation and a numerical approximation. The BSDE…

This Paper conducts a thorough simulation study to assess the effectiveness of various acceleration techniques designed to enhance the conjugate gradient algorithm, which is used for solving large linear systems to accelerate Bayesian…