Related papers: Nonlinear Two-Time-Scale Stochastic Approximation:…

Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning. A key step to tackle these problems is the efficient computation of…

Non-linear least squares solvers are used across a broad range of offline and real-time model fitting problems. Most improvements of the basic Gauss-Newton algorithm tackle convergence guarantees or leverage the sparsity of the underlying…

We look at a stochastic time-varying optimization problem and we formulate online algorithms to find and track its optimizers in expectation. The algorithms are derived from the intuition that standard prediction and correction steps can be…

Recently, the stochastic asymptotical regularization (SAR) has been developed in (\emph{Inverse Problems}, 39: 015007, 2023) for the uncertainty quantification of the stable approximate solution of linear ill-posed inverse problems. In this…

This paper addresses the challenging numerical simulation of nonlinear hybrid stochastic functional differential equations with infinite delays. We first propose an explicit scheme using space and time truncation, requiring only finite…

In this paper, a distributed stochastic approximation algorithm is studied. Applications of such algorithms include decentralized estimation, optimization, control or computing. The algorithm consists in two steps: a local step, where each…

Bayesian optimization is a sample-efficient method for solving expensive, black-box optimization problems. Stochastic programming concerns optimization under uncertainty where, typically, average performance is the quantity of interest. In…

Sequential quadratic optimization algorithms are proposed for solving smooth nonlinear optimization problems with equality constraints. The main focus is an algorithm proposed for the case when the constraint functions are deterministic,…

Multi-stage stochastic programming is a well-established framework for sequential decision making under uncertainty by seeking policies that are fully adapted to the uncertainty. Often such flexible policies are not desirable, and the…

This paper studies an optimal control problem for continuous-time stochastic systems subject to reachability objectives specified in a subclass of metric interval temporal logic specifications, a temporal logic with real-time constraints.…

We introduce novel convergence results for asynchronous iterations that appear in the analysis of parallel and distributed optimization algorithms. The results are simple to apply and give explicit estimates for how the degree of asynchrony…

This work proposes and studies the distributed resource allocation problem in asynchronous and stochastic settings. We consider a distributed system with multiple workers and a coordinating server with heterogeneous computation and…

We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients.…

We focus here on a class of fourth-order parabolic equations that can be written as a system of second-order equations by introducing an auxiliary variable. We design a novel second-order fully discrete mixed finite element method to…

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous…

This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic…

We describe stochastic Newton and stochastic quasi-Newton approaches to efficiently solve large linear least-squares problems where the very large data sets present a significant computational burden (e.g., the size may exceed computer…

Forecasting the evolution of complex systems is one of the grand challenges of modern data science. The fundamental difficulty lies in understanding the structure of the observed stochastic process. In this paper, we show that every…

In this paper we consider a stochastic heavy-ball method for solving linear ill-posed inverse problems. With suitable choices of the step-sizes and the momentum coefficients, we establish the regularization property of the method under {\it…

We describe an asynchronous parallel stochastic coordinate descent algorithm for minimizing smooth unconstrained or separably constrained functions. The method achieves a linear convergence rate on functions that satisfy an essential strong…