Related papers: Scalable adaptive cubic regularization methods
We consider minimization of indefinite quadratics with either trust-region (norm) constraints or cubic regularization. Despite the nonconvexity of these problems we prove that, under mild assumptions, gradient descent converges to their…
In this paper, we propose a novel adaptive stochastic extended iterative method, which can be viewed as an improved extension of the randomized extended Kaczmarz (REK) method, for finding the unique minimum Euclidean norm least-squares…
This paper presents an accelerated composite gradient (ACG) variant, referred to as the AC-ACG method, for solving nonconvex smooth composite minimization problems. As opposed to well-known ACG variants that are either based on a known…
The Abstraction and Reasoning Corpus (ARC) is designed to assess generalization beyond pattern matching, requiring models to infer symbolic rules from very few examples. In this work, we present a transformer-based system that advances ARC…
We present a new adaptive collocation scheme for solving partial differential equations based on Local Coupled Multiquadrics (LCMQs) within a covers-and-nodes framework. The method, referred to as the Adaptive Ch Method, automatically…
The parallel solution of large scale non-linear programming problems, which arise for example from the discretization of non-linear partial differential equations, is a highly demanding task. Here, a novel solution strategy is presented,…
Over the past decade, stochastic algorithms have emerged as scalable and efficient tools for solving large-scale ill-posed inverse problems by randomly selecting subsets of equations at each iteration. However, due to the ill-posedness and…
The adaptive spectral Koopman (ASK) method was introduced to numerically solve autonomous dynamical systems that lay the foundation of numerous applications across different fields in science and engineering. Although ASK achieves high…
We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the…
LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent CGLS applied to normal equations system, are commonly used for large-scale discrete ill-posed problems. It is well known that LSQR…
We present adaptive gradient methods (both basic and accelerated) for solving convex composite optimization problems in which the main part is approximately smooth (a.k.a. $(\delta, L)$-smooth) and can be accessed only via a (potentially…
We present an augmented Lagrangian trust-region method to efficiently solve constrained optimization problems governed by large-scale nonlinear systems with application to partial differential equation-constrained optimization. At each…
The block Kaczmarz method and its variants are designed for solving the over-determined linear system. They involve iteratively projecting the current point onto the solution space of a subset of constraints. In this work, by alternately…
In this paper, we study convergence rates of the cubic regularized proximal quasi-Newton method (\csr) for solving non-smooth additive composite problems that satisfy the so-called Kurdyka-\L ojasiewicz (K\L ) property with respect to some…
Nonlinear and nonaffine terms in parametric partial differential equations can potentially lead to a computational cost of a reduced order model (ROM) that is comparable to the cost of the original full order model (FOM). To address this,…
We introduce a family of stochastic optimization methods based on the Runge-Kutta-Chebyshev (RKC) schemes. The RKC methods are explicit methods originally designed for solving stiff ordinary differential equations by ensuring that their…
Emerging applications increasingly demand flexible covariate adaptive randomization (CAR) methods that support unequal targeted allocation ratios. While existing procedures can achieve covariate balance, they often suffer from the shift…
In many hierarchical inverse problems, not only do we want to estimate high- or infinite-dimensional model parameters in the parameter-to-observable maps, but we also have to estimate hyperparameters that represent critical assumptions in…
Regularization method and Bayesian inverse method are two dominating ways for solving inverse problems generated from various fields, e.g., seismic exploration and medical imaging. The two methods are related with each other by the MAP…
Adaptive cubic regularization methods for solving nonconvex problems need the efficient computation of the trial step, involving the minimization of a cubic model. We propose a new approach in which this model is minimized in a low…