Related papers: The regularization continuation method with an ada…
The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: i) Many well-known operator splitting methods, such as…
This paper introduces a generalization of the well-known Riccati recursion for solving the discrete-time equality-constrained linear quadratic optimal control problem. The recursion can be used to compute the solutions as well as optimal…
We develop a novel stepsize based on \BB method for solving some challenging optimization problems efficiently, named regularized \BB (RBB) stepsize. We indicate that RBB stepsize is the close solution to a $\ell_{2}^{2}$-regularized least…
Many inverse problems are phrased as optimization problems in which the objective function is the sum of a data-fidelity term and a regularization. Often, the Hessian of the fidelity term is computationally unavailable while the Hessian of…
We investigate local optimality conditions of first and second order for integer optimal control problems with total variation regularization via a finite-dimensional switching point problem. We show the equivalence of local optimality for…
We present a regularization method to approach a solution of the pessimistic formulation of ill -posed bilevel problems . This allows to overcome the difficulty arising from the non uniqueness of the lower level problems solutions and…
The continuation method is a popular approach in non-convex optimization and computer vision. The main idea is to start from a simple function that can be minimized efficiently, and gradually transform it to the more complicated original…
We analyze the performance of a variant of Newton method with quadratic regularization for solving composite convex minimization problems. At each step of our method, we choose regularization parameter proportional to a certain power of the…
The question of how to incorporate curvature information in stochastic approximation methods is challenging. The direct application of classical quasi- Newton updating techniques for deterministic optimization leads to noisy curvature…
This paper describes a new approach on optimization of constraint satisfaction problems (CSPs) by means of substituting sub-CSPs with locally consistent regular membership constraints. The purpose of this approach is to reduce the number of…
Classical theory for quasi-Newton schemes has focused on smooth deterministic unconstrained optimization while recent forays into stochastic convex optimization have largely resided in smooth, unconstrained, and strongly convex regimes.…
Many inverse problems are concerned with the estimation of non-negative parameter functions. In this paper, in order to obtain non-negative stable approximate solutions to ill-posed linear operator equations in a Hilbert space setting, we…
We propose a new method for optimistic planning in infinite-horizon discounted Markov decision processes based on the idea of adding regularization to the updates of an otherwise standard approximate value iteration procedure. This…
In this work we describe an Adaptive Regularization using Cubics (ARC) method for large-scale nonconvex unconstrained optimization using Limited-memory Quasi-Newton (LQN) matrices. ARC methods are a relatively new family of optimization…
While first-order methods are popular for solving optimization problems that arise in large-scale deep learning problems, they come with some acute deficiencies. To diminish such shortcomings, there has been recent interest in applying…
The augmented Lagrangian method (ALM) is a benchmark for convex programming problems with linear constraints; ALM and its variants for linearly equality-constrained convex minimization models have been well studied in the literature.…
Although first-order stochastic algorithms, such as stochastic gradient descent, have been the main force to scale up machine learning models, such as deep neural nets, the second-order quasi-Newton methods start to draw attention due to…
This paper presents strong worst-case iteration and operation complexity guarantees for Riemannian adaptive regularized Newton methods, a unified framework encompassing both Riemannian adaptive regularization (RAR) methods and Riemannian…
We consider a wide range of regularized stochastic minimization problems with two regularization terms, one of which is composed with a linear function. This optimization model abstracts a number of important applications in artificial…
Optimization problems with norm-bounding constraints arise in a variety of applications, including portfolio optimization, machine learning, and feature selection. A common approach to these problems involves relaxing the norm constraint…