相关论文: A Parameterized Barzilai-Borwein Method via Interp…
The Barzilai-Borwein (BB) gradient method is efficient for solving large-scale unconstrained problems to the modest accuracy and has a great advantage of being easily extended to solve a wide class of constrained optimization problems. In…
A novel gradient stepsize is derived at the motivation of equipping the Barzilai-Borwein (BB) method with two dimensional quadratic termination property. A remarkable feature of the novel stepsize is that its computation only depends on the…
Barzilai-Borwein (BB) steplength is a popular choice in gradient descent method. By observing that the two existing BB steplengths correspond to the ordinary and the data least squares, respectively, we employ the third kind of least…
The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the…
In this paper, we propose AdaBB, an adaptive gradient method based on the Barzilai-Borwein stepsize. The algorithm is line-search-free and parameter-free, and essentially provides a convergent variant of the Barzilai-Borwein method for…
We develop a novel stepsize based on \BB method for solving some challenging optimization problems efficiently, named regularized \BB (RBB) stepsize. We indicate that RBB stepsize is the close solution to a $\ell_{2}^{2}$-regularized least…
Due to simplicity, computational cheapness, and efficiency, the Barzilai and Borwein (BB) gradient method has received a significant amount of attention in different fields of optimization. In the first part of this paper, based on spectral…
We consider a distributed multi-agent optimization problem over a time-invariant undirected graph, where each agent possesses a local objective function and all agents collaboratively minimize the average of all objective functions through…
The Barzilai-Borwein (BB) method is a popular and efficient tool for solving large-scale unconstrained optimization problems. Its search direction is the same as for the steepest descent (Cauchy) method, but its stepsize rule is different.…
One of the major issues in stochastic gradient descent (SGD) methods is how to choose an appropriate step size while running the algorithm. Since the traditional line search technique does not apply for stochastic optimization algorithms,…
The regularized Barzilai-Borwein (RBB) method represents a promising gradient-based optimization algorithm. In this paper, by splitting the gradient into two parts and analyzing the dynamical system of difference equations governing the…
In this paper, we consider the unconstrained multiobjective optimization problem. In recent years, researchers pointed out that the steepest decent method may generate small stepsize which leads to slow convergence rates. To address the…
Variable metric proximal gradient (VM-PG) is a widely used class of convex optimization method. Lately, there has been a lot of research on the theoretical guarantees of VM-PG with different metric selections. However, most such metric…
The Barzilai and Borwein (BB) gradient method is one of the most widely-used line-search gradient methods. It computes the step-size for the current iterate by using the information carried in the previous iteration. Recently, William Kahan…
We propose a family of spectral gradient methods, whose stepsize is determined by a convex combination of the long Barzilai-Borwein (BB) stepsize and the short BB stepsize. Each member of the family is shown to share certain quasi-Newton…
Recent studies show that the two-dimensional quadratic termination property has great potential in improving performance of the gradient method. However, it is not clear whether higher-dimensional quadratic termination leads further…
In this paper, we incorporate the Barzilai-Borwein step size into gradient descent methods used to train deep networks. This allows us to adapt the learning rate using a two-point approximation to the secant equation which quasi-Newton…
In this paper, we consider to improve the stochastic variance reduce gradient (SVRG) method via incorporating the curvature information of the objective function. We propose to reduce the variance of stochastic gradients using the…
The Barzilai-Borwein (BB) method has demonstrated great empirical success in nonlinear optimization. However, the convergence speed of BB method is not well understood, as the known convergence rate of BB method for quadratic problems is…
Variable metric proximal gradient methods with different metric selections have been widely used in composite optimization. Combining the Barzilai-Borwein (BB) method with a diagonal selection strategy for the metric, the diagonal BB…