相关论文: A Trust Region Method with Regularized Barzilai-Bo…
The regularized Barzilai-Borwein (RBB) method represents a promising gradient-based optimization algorithm. In this paper, by splitting the gradient into two parts and analyzing the dynamical system of difference equations governing the…
The Barzilai-Borwein (BB) step sizes have a profound impact on gradient descent methods. In this work, we propose two new gradient step sizes: one longer than the original long BB step size, and the other shorter than the original short BB…
Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are batch methods designed mainly based on the convex optimization, say, the…
A trust-region algorithm is presented for finding approximate minimizers of smooth unconstrained functions whose values and derivatives are subject to random noise. It is shown that, under suitable probabilistic assumptions, the new method…
This paper studies proximal gradient iterations for solving simple bilevel optimization problems where both the upper and the lower level cost functions are split as the sum of differentiable and (possibly nonsmooth) proximable functions.…
The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the…
In this contribution we device and analyze improved variants of the non-conforming dual approach for trust-region reduced basis (TR-RB) approximation of PDE-constrained parameter optimization that has recently been introduced in [Keil et…
The Trust Region Subproblem is a fundamental optimization problem that takes a pivotal role in Trust Region Methods. However, the problem, and variants of it, also arise in quite a few other applications. In this article, we present a…
The main goal of this work is equipping convex and nonconvex problems with Barzilai-Borwein (BB) step size. With the adaptivity of BB step sizes granted, they can fail when the objective function is not strongly convex. To overcome this…
This paper is devoted to minimizing the sum of a smooth function and a nonsmooth $\ell_1$-regularized term. This problem as a special cases includes the $\ell_1$-regularized convex minimization problem in signal processing, compressive…
We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting,…
We propose a globally convergent trust-region bundle method for minimizing lower-$C^2$ functions using higher-order cutting-plane models. Under certain growth assumptions on the objective around its minimum, the method is able to compute…
We investigate a trust-region algorithm to solve a nonconvex optimization problem with $L^p$-regularization for $p\in(0,1)$. The algorithm relies on descent properties of a so-called generalized Cauchy point that can be obtained efficiently…
Bayesian optimization is a powerful tool for solving real-world optimization tasks under tight evaluation budgets, making it well-suited for applications involving costly simulations or experiments. However, many of these tasks are also…
Real-world optimization problems often involve complex objective functions with costly evaluations. While Bayesian optimization (BO) with Gaussian processes is effective for these challenges, it suffers in high-dimensional spaces due to…
There is emerging evidence that trust-region (TR) algorithms are very effective at solving derivative-free nonconvex stochastic optimization problems in which the objective function is a Monte Carlo (MC) estimate. A recent strand of…
In this paper we consider the use of probabilistic or random models within a classical trust-region framework for optimization of deterministic smooth general nonlinear functions. Our method and setting differs from many stochastic…
Historically speaking, it is hard to balance the global and local efficiency of second-order optimization algorithms. For instance, the classical Newton's method possesses excellent local convergence but lacks global guarantees, often…
We propose a stochastic nonconvex optimization algorithm that achieves almost sure $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ iteration complexity for problems with smooth objective functions and gradients only observable with noise. The…
This paper considers the generalized continuation Newton method and thetrust-region updating strategy for the underdetermined system of nonlinear equations. Moreover, in order to improve its computational efficiency, the new method will not…