Related papers: Scalable Derivative-Free Optimization for Nonlinea…
The focus of this paper is two fold. Firstly, we present a logical approach to graph modification problems such as minimum node deletion, edge deletion, edge augmentation problems by expressing them as an expression in first order (FO)…
Solving large-scale optimization on-the-fly is often a difficult task for real-time computer graphics applications. To tackle this challenge, model reduction is a well-adopted technique. Despite its usefulness, model reduction often…
The aim of this paper and associated presentation is to put forward derivative-free optimization methods for control design. The important element, still ignored at the end of 2011 in systems and control (i.e. this element has apparently…
Least squares form one of the most prominent classes of optimization problems, with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must…
Zeroth-order optimization (ZO) has been a powerful framework for solving black-box problems, which estimates gradients using zeroth-order data to update variables iteratively. The practical applicability of ZO critically depends on the…
Reduced-order models have long been used to understand the behavior of nonlinear partial differential equations (PDEs). Naturally, reduced-order modeling techniques come at the price of computational accuracy for a decrease in computation…
This paper studies sparse nonlinear least squares problems, where the Jacobian matrices are unavailable or expensive to compute, yet have some underlying sparse structures. We construct the Jacobian models by the $ \ell_1 $ minimization…
Ordinary least squares (OLS) is the default method for fitting linear models, but is not applicable for problems with dimensionality larger than the sample size. For these problems, we advocate the use of a generalized version of OLS…
This paper studies adaptive first-order least-squares finite element methods for second-order elliptic partial differential equations in non-divergence form. Unlike the classical finite element method which uses weak formulations of PDEs…
We consider the challenge of black-box optimization within hybrid discrete-continuous and variable-length spaces, a problem that arises in various applications, such as decision tree learning and symbolic regression. We propose DisCo-DSO…
It is shown that the computational efficiency of the discrete least-squares (DLS) approximation of solutions of stochastic elliptic PDEs is improved by incorporating a reduced-basis method into the DLS framework. The goal is to recover the…
We consider the least-squares finite element method (lsfem) for systems of nonlinear ordinary differential equations and establish an optimal error estimate for this method when piecewise linear elements are used. The main assumptions are…
In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are…
Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various…
In this paper, we analyze the accuracy of gradient estimates obtained by linear interpolation when the underlying function is subject to bounded measurement noise. The total gradient error is decomposed into a deterministic component…
Gradient-based optimization is now ubiquitous across graphics, but unfortunately can not be applied to problems with undefined or zero gradients. To circumvent this issue, the loss function can be manually replaced by a ``surrogate'' that…
Zeroth-order optimization is a fundamental research topic that has been a focus of various learning tasks, such as black-box adversarial attacks, bandits, and reinforcement learning. However, in theory, most complexity results assert a…
Stochastic dynamical systems with continuous symmetries arise commonly in nature and often give rise to coherent spatio-temporal patterns. However, because of their random locations, these patterns are not well captured by current order…
Complex design problems are common in the scientific and industrial fields. In practice, objective functions or constraints of these problems often do not have explicit formulas, and can be estimated only at a set of sampling points through…
This paper investigates how to accelerate the convergence of distributed optimization algorithms on nonconvex problems with zeroth-order information available only. We propose a zeroth-order (ZO) distributed primal-dual stochastic…