Related papers: An Additively Preconditioned Trust Region Strategy…
The goal of this work is to construct and study hybrid and multiplicative two-level overlapping Schwarz algorithms with standard coarse spaces for the almost incompressible linear elasticity and Stokes systems, discretized by mixed finite…
In [R. J. Baraldi and D. P. Kouri, Math. Program., 201:1 (2023), pp. 559-598], the authors introduced a trust-region method for minimizing the sum of a smooth nonconvex and a nonsmooth convex function, the latter of which has an analytical…
Historically speaking, it is hard to balance the global and local efficiency of second-order optimization algorithms. For instance, the classical Newton's method possesses excellent local convergence but lacks global guarantees, often…
We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of \citet{sorensen1982newton}. The…
Based on an observation that additive Schwarz methods for general convex optimization can be interpreted as gradient methods, we propose an acceleration scheme for additive Schwarz methods. Adopting acceleration techniques developed for…
We develop an interior-point method for nonsmooth regularized bound-constrained optimization problems. Our method consists of iteratively solving a sequence of unconstrained nonsmooth barrier subproblems. We use a variant of the proximal…
For linear problems, domain decomposition methods can be used directly as iterative solvers, but also as preconditioners for Krylov methods. In practice, Krylov acceleration is almost always used, since the Krylov method finds a much better…
We present a trust-region-based adaptive finite-element algorithm for numerically solving a class of nonsmooth PDE-constrained optimization problems that includes problems with sparsifying regularizers and convex constraints. In particular,…
Prompt tuning prepends a soft prompt to the input embeddings or hidden states and only optimizes the prompt to adapt pretrained models (PTMs) to downstream tasks. The previous work manually selects prompt layers which are far from optimal…
This paper considers an explicit continuation method and the trust-region updating strategy for the unconstrained optimization problem. Moreover, in order to improve its computational efficiency and robustness, the new method uses the…
Adaptive regularized framework using cubics has emerged as an alternative to line-search and trust-region algorithms for smooth nonconvex optimization, with an optimal complexity amongst second-order methods. In this paper, we propose and…
We develop a trust-region method for minimizing the sum of a smooth term $f$ and a nonsmooth term $h$), both of which can be nonconvex. Each iteration of our method minimizes a possibly nonconvex model of $f + h$ in a trust region. The…
Benefiting from prompt tuning, recent years have witnessed the promising performance of pre-trained vision-language models, e.g., CLIP, on versatile downstream tasks. In this paper, we focus on a particular setting of learning adaptive…
This paper introduces new perspectives on analog design space search. To minimize the time-to-market, this endeavor better cast as constraint satisfaction problem than global optimization defined in prior arts. We incorporate model-based…
We propose a trust-region method for finite-sum minimization with an adaptive sample size adjustment technique, which is practical in the sense that it leads to a globally convergent method that shows strong performance empirically without…
Post-training methods, especially Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), play an important role in improving large language models' (LLMs) complex reasoning abilities. However, the dominant two-stage pipeline (SFT…
In this paper, we study a few challenging theoretical and numerical issues on the well known trust region policy optimization for deep reinforcement learning. The goal is to find a policy that maximizes the total expected reward when the…
Gradient boosting, a method of building additive ensembles from weak learners, has established itself as a practical and theoretically-motivated approach to approximate functions, especially using decision tree weak learners. Comparable…
Two-trust-region subproblem (TTRS), which is the minimization of a general quadratic function over the intersection of two full-dimensional ellipsoids, has been the subject of several recent research. In this paper, to solve TTRS, a hybrid…
Substructured domain decomposition (DD) methods have been extensively studied, and they are usually associated with nonoverlapping decompositions. We introduce here a substructured version of Restricted Additive Schwarz (RAS) which we call…