Related papers: A Forward Propagation Algorithm for Online Optimiz…

Physical models with uncertain inputs are commonly represented as parametric partial differential equations (PDEs). That is, PDEs with inputs that are expressed as functions of parameters with an associated probability distribution.…

We consider a variant of the stochastic gradient descent (SGD) with a random learning rate and reveal its convergence properties. SGD is a widely used stochastic optimization algorithm in machine learning, especially deep learning. Numerous…

Stochastic differential equations (SDEs) are increasingly used in longitudinal data analysis, compartmental models, growth modelling, and other applications in a number of disciplines. Parameter estimation, however, currently requires…

In this thesis, we propose new theoretical frameworks for the analysis of stochastic and distributed methods with error compensation and local updates. Using these frameworks, we develop more than 20 new optimization methods, including the…

Multiscale and multiphysics problems need novel numerical methods in order for them to be solved correctly and predictively. To that end, we develop a wavelet based technique to solve a coupled system of nonlinear partial differential…

Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential…

In this article, we introduce and analyze a deep learning based approximation algorithm for SPDEs. Our approach employs neural networks to approximate the solutions of SPDEs along given realizations of the driving noise process. If applied…

Distributed stochastic optimization algorithms can simultaneously process large-scale datasets, significantly accelerating model training. However, their effectiveness is often hindered by the sparsity of distributed networks and data…

We consider the problem of making nonparametric inference in a class of multi-dimensional diffusions in divergence form, from low-frequency data. Statistical analysis in this setting is notoriously challenging due to the intractability of…

We propose a new algorithm for solving parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs) in high dimension, by making an analogy between the BSDE and reinforcement learning with the…

This paper investigates a numerical probabilistic method for the solution of some semilinear stochastic partial differential equations (SPDEs in short). The numerical scheme is based on discrete time approximation for solutions of systems…

We study fully nonlinear second-order (forward) stochastic partial differential equations (SPDEs). They can also be viewed as forward path-dependent PDEs (PPDEs) and will be treated as rough PDEs (RPDEs) under a unified framework. We…

We propose a partial differential-integral equation (PDE) framework for deep neural networks (DNNs) and their associated learning problem by taking the continuum limits of both network width and depth. The proposed model captures the…

Machine learning for scientific applications faces the challenge of limited data. We propose a framework that leverages a priori known physics to reduce overfitting when training on relatively small datasets. A deep neural network is…

In recent years, SPDEs have become a well-studied field in mathematics. With their increase in popularity, it becomes important to efficiently approximate their solutions. Thus, our goal is a contribution towards the development of…

This paper considers a distributed adaptive optimization problem, where all agents only have access to their local cost functions with a common unknown parameter, whereas they mean to collaboratively estimate the true parameter and find the…

Stochastic differential equations (SDEs) using jump-diffusion processes describe many natural phenomena at the microscopic level. Since they are commonly used to model economic and financial evolutions, the calibration and optimal control…

Neural Stochastic Differential Equations (NSDEs) model the drift and diffusion functions of a stochastic process as neural networks. While NSDEs are known to make accurate predictions, their uncertainty quantification properties have been…

In this paper, we consider solving the distributed optimization problem over a multi-agent network under the communication restricted setting. We study a compressed decentralized stochastic gradient method, termed ``compressed exact…

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the…