Related papers: Regularized Nonlinear Acceleration
We present a numerical method for convergence acceleration for multifidelity models of parameterized ordinary differential equations. The hierarchy of models is defined as trajectories computed using different timesteps in a time…
In this paper, we present a novel nonlinear programming-based approach to fine-tune pre-trained neural networks to improve robustness against adversarial attacks while maintaining high accuracy on clean data. Our method introduces…
Stochastic composition optimization draws much attention recently and has been successful in many emerging applications of machine learning, statistical analysis, and reinforcement learning. In this paper, we focus on the composition…
In this paper, we propose an acceleration framework for a class of iterative methods using the Reduced Order Method (ROM). Assuming that the underlying iterative scheme generates a rich basis for the solution space, we construct the next…
We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii)…
This paper proposes an algorithmic framework for solving parametric optimization problems which we call adjoint-based predictor-corrector sequential convex programming. After presenting the algorithm, we prove a contraction estimate that…
We propose a novel algorithm for solving non-convex, nonlinear equality-constrained finite-sum optimization problems. The proposed algorithm incorporates an additional sampling strategy for sample size update into the well-known framework…
This paper aims to accelerate the test-time computation of deep convolutional neural networks (CNNs). Unlike existing methods that are designed for approximating linear filters or linear responses, our method takes the nonlinear units into…
In this article we study the problem of recovering the unknown solution of a linear ill-posed problem, via iterative regularization methods. We review the problem of projection-regularization from a statistical point of view. A basic…
In this brief paper, we present a naive aggregation algorithm for a typical learning problem with expert advice setting, in which the task of improving generalization, i.e., model validation, is embedded in the learning process as a…
In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search…
We are interested in ensemble methods to solve multi-objective optimization problems. An ensemble Kalman method is proposed to solve a formulation of the nonlinear problem using a weighted function approach. An analysis of the mean field…
We present a novel approach to accelerate iterative methods to solve nonlinear Schr\"odinger eigenvalue problems using neural networks. Nonlinear eigenvector problems are fundamental in quantum mechanics and other fields, yet conventional…
We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. In comparison with recent advances in this vein, the…
Nonlinear programming targets nonlinear optimization with constraints, which is a generic yet complex methodology involving humans for problem modeling and algorithms for problem solving. We address the particularly hard challenge of…
A method of numerically evaluating slowly convergent monotone series is described. First, we apply a condensation transformation due to Van Wijngaarden to the original series. This transforms the original monotone series into an alternating…
Anderson acceleration is a well-established and simple technique for speeding up fixed-point computations with countless applications. Previous studies of Anderson acceleration in optimization have only been able to provide convergence…
Stochastic variance reduced optimization methods are known to be globally convergent while they suffer from slow local convergence, especially when moderate or high accuracy is needed. To alleviate this problem, we propose an optimization…
We propose two basic assumptions, under which the rate of convergence of the augmented Lagrange method for a class of composite optimization problems is estimated. We analyze the rate of local convergence of the augmented Lagrangian method…
Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms.…