相关论文: Adaptive SOR methods based on the Wolfe conditions

A class of optimization problems characterized by a weighted finite-sum objective function subject to box constraints is considered. We propose a novel stochastic optimization method, named AS-BOX (\text{A}ddi\-ti\-onal \text{S}ampling for…

Adaptable computing is an increasingly important paradigm that specializes system resources to variable application requirements, environmental conditions, or user requirements. Adapting computing resources to variable application…

An algorithm is said to be adaptive to a certain parameter (of the problem) if it does not need a priori knowledge of such a parameter but performs competitively to those that know it. This dissertation presents our work on adaptive…

We consider the problem of minimizing a high-dimensional objective function, which may include a regularization term, using (possibly noisy) evaluations of the function. Such optimization is also called derivative-free, zeroth-order, or…

For solving pseudo-convex global optimization problems, we present a novel fully adaptive steepest descent method (or ASDM) without any hard-to-estimate parameters. For the step-size regulation in an $\varepsilon$-normalized direction, we…

We propose and analyze an adaptive step-size variant of the Davis-Yin three operator splitting. This method can solve optimization problems composed by a sum of a smooth term for which we have access to its gradient and an arbitrary number…

Two complementary techniques for analyzing search spaces are proposed: (i) an algorithm to detect search points with potential to be local optima; and (ii) a slightly adjusted Wang-Landau sampling algorithm to explore larger search spaces.…

In this paper we propose a solution to the problem of parameter estimation of nonlinearly parameterized regressions--continuous or discrete time--and apply it for system identification and adaptive control. We restrict our attention to…

We develop new adaptive algorithms for temporal integration of nonlinear evolution equations on tensor manifolds. These algorithms, which we call step-truncation methods, are based on performing one time step with a conventional…

Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation…

Here, we focus on Model Order Reduction (MOR) of non-parametric second-order dynamical systems. In these MOR algorithms, sequences of large and sparse linear systems arise during the model reduction process. Solving such linear systems is…

This paper presents a model order reduction (MOR) approach for high dimensional problems in the analysis of financial risk. To understand the financial risks and possible outcomes, we have to perform several thousand simulations of the…

A new pattern search method for bound constrained optimization is introduced. The proposed algorithm employs the coordinate directions, in a suitable way, with a nonmonotone line search for accepting the new iterate, without using…

We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints. Instances of this template naturally arise from SDP-relaxations of…

The vector autoregressive (VAR) model is a powerful tool in modeling complex time series and has been exploited in many fields. However, fitting high dimensional VAR model poses some unique challenges: On one hand, the dimensionality,…

This paper develops a parameter-free adaptive proximal bundle method with two important features: 1) adaptive choice of variable prox stepsizes that "closely fits" the instance under consideration; and 2) adaptive criterion for making the…

There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate.…

A fully stochastic second-order adaptive-regularization method for unconstrained nonconvex optimization is presented which never computes the objective-function value, but yet achieves the optimal $\mathcal{O}(\epsilon^{-3/2})$ complexity…

In this paper we propose a (non-linear) smoothing algorithm for group-affine observation systems, a recently introduced class of estimation problems on Lie groups that bear a particular structure. As most non-linear smoothing methods, the…

Although persistent excitation is often acknowledged as a sufficient condition to exponentially converge in the field of adaptive parameter estimation, it must be noted that in practical applications this may be unguaranteed. Recently, more…