相关论文: Full-low evaluation methods for derivative-free op…
While variance reduction methods have shown great success in solving large scale optimization problems, many of them suffer from accumulated errors and, therefore, should periodically require the full gradient computation. In this paper, we…
A randomized version of the recently developed barycenter method for derivative--free optimization has desirable properties of a gradient search. We developed a complex version to avoid evaluations at high--gradient points. The method,…
Arising in semi-parametric statistics, control applications, and as sub-problems in global optimization methods, certain optimization problems can have objective functions requiring numerical integration to evaluate, yet gradient function…
We consider quantile optimization of black-box functions that are estimated with noise. We propose two new iterative three-timescale local search algorithms. The first algorithm uses an appropriately modified finite-difference-based…
Steepest descent algorithms, which are commonly used in deep learning, use the gradient as the descent direction, either as-is or after a direction shift using preconditioning. In many scenarios calculating the gradient is numerically hard…
We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients.…
We address the problem of zero-order optimization from noisy observations for an objective function satisfying the Polyak-{\L}ojasiewicz or the strong convexity condition. Additionally, we assume that the objective function has an additive…
In this paper, we propose objective-function-free (OFF) variants of the proximal Newton method for nonconvex composite optimization problems and the regularized Newton method for unconstrained optimization problems, respectively, using…
Feedback optimization has emerged as a promising approach for optimizing the steady-state operation of dynamical systems while requiring minimal modeling efforts. Unfortunately, most existing feedback optimization methods rely on knowledge…
A zero-finding technique for solving nonlinear equations more efficiently than they usually are with traditional iterative methods in which the order of convergence is improved is presented. The key idea in deriving this procedure is to…
In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide…
This paper explores a method for solving constrained optimization problems when the derivatives of the objective function are unavailable, while the derivatives of the constraints are known. We allow the objective and constraint function to…
This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It introduces an algorithm designed to optimize differentiable objective functions $f$ whose values can…
In this paper, we consider the minimization of a $C^2-$smooth and strongly convex objective depending on a given parameter, which is usually found in many practical applications. We suppose that we desire to solve the problem with some…
We present an iterative inverse reinforcement learning algorithm to infer optimal cost functions in continuous spaces. Based on a popular maximum entropy criteria, our approach iteratively finds a weight improvement step and proposes a…
It has long been known that the gradient (steepest descent) method may fail on nonsmooth problems, but the examples that have appeared in the literature are either devised specifically to defeat a gradient or subgradient method with an…
The main focus in this paper is exact linesearch methods for minimizing a quadratic function whose Hessian is positive definite. We give a class of limited-memory quasi-Newton Hessian approximations which generate search directions parallel…
The objective of this publication is to reduce the sensitivity of iterative equation solvers on the initial value. To this end, at the hand of Newton's method, we exemplify how to reformulate the initial problem by means of a set of…
We prove that the finite-difference based derivative-free descent (FD-DFD) methods have a capability to find the global minima for a class of multiple minima problems. Our main result shows that, for a class of multiple minima objectives…
Bilevel programming has recently received a great deal of attention due to its abundant applications in many areas. The optimal value function approach provides a useful reformulation of the bilevel problem, but its utility is often limited…