Related papers: A note on solving nonlinear optimization problems …

We present an adaptive trust-region method for unconstrained optimization that allows inexact solutions to the trust-region subproblems. Our method is a simple variant of the classical trust-region method of \citet{sorensen1982newton}. The…

In this work we introduce the stochastic nonlinear constrained derivative-free optimization method (S)NOWPAC (Stochastic Nonlinear Optimization With Path-Augmented Constraints). The method extends the derivative-free optimizer NOWPAC to be…

Stochastic minimax optimization has drawn much attention over the past decade due to its broad applications in machine learning, signal processing and game theory. In some applications, the probability distribution of uncertainty depends on…

We present two novel classes of fully discrete energy-preserving algorithms for the sine-Gordon equation subject to Neumann boundary conditions. The cosine pseudo-spectral method is first used to develop structure-preserving spatial…

We propose new numerical approach to non-conservative dynamical systems. Our method being of low order, enhances qualitative performance of standard discrete gradient algorithm, thank to new concept of a reservoir. Paper is of explanatory…

Recent advances in nonlinear dynamical systems theory provide a new insight into numerical properties of discrete algorithms developed to solve nonlinear initial value problems. Basic features like accuracy and stability are well pointed…

Constrained quasiconvex optimization problems appear in many fields, such as economics, engineering, and management science. In particular, fractional programming, which models ratio indicators such as the profit/cost ratio as fractional…

Motivated by TRACE algorithm [Curtis et al. 2017], we propose a trust region algorithm for finding second order stationary points of a linearly constrained non-convex optimization problem. We show the convergence of the proposed algorithm…

An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is…

Worst-case complexity guarantees for nonconvex optimization algorithms have been a topic of growing interest. Multiple frameworks that achieve the best known complexity bounds among a broad class of first- and second-order strategies have…

We propose a novel algorithm, TR-SVR, for solving unconstrained stochastic optimization problems. This method builds on the trust-region framework, which effectively balances local and global exploration in optimization tasks. TR-SVR…

This paper introduces a novel approach to system identification for nonlinear input-output models that minimizes the simulation error and frames the problem as a constrained optimization task. The proposed method addresses vanishing…

When considering an unconstrained minimization problem, a standard approach is to solve the optimality system with a Newton method possibly preconditioned by, e.g., nonlinear elimination. In this contribution, we argue that nonlinear…

The ability to differentiate through optimization problems has unlocked numerous applications, from optimization-based layers in machine learning models to complex design problems formulated as bilevel programs. It has been shown that…

Convex and nonconvex finite-sum minimization arises in many scientific computing and machine learning applications. Recently, first-order and second-order methods where objective functions, gradients and Hessians are approximated by…

An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within…

A computational tool for coarse-graining nonlinear systems of ordinary differential equations in time is discussed. Three illustrative model examples are worked out that demonstrate the range of capability of the method. This includes the…

This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [Curtis2019, Curtis2022]. A theoretical analysis that complements the results…

Density functional theory calculations use a significant fraction of current supercomputing time. The resources required scale with the problem size, internal workings of the code and the number of iterations to convergence, the latter…

Stochastic gradient-based optimization is crucial to optimize neural networks. While popular approaches heuristically adapt the step size and direction by rescaling gradients, a more principled approach to improve optimizers requires…