Related papers: Optimization Methods for Joint Eigendecomposition
Recently double-bracket quantum algorithms have been proposed as a way to compile circuits for approximating eigenstates. Physically, they consist of appropriately composing evolutions under an input Hamiltonian together with diagonal…
We present an algorithm based on continuation techniques that can be applied to solve numerically minimization problems with equality constraints. We focus on problems with a great number of local minima which are hard to obtain by local…
In this paper, we propose a decomposition approach for eigenvalue problems with spatial symmetries, including the formulation, discretization as well as implementation. This approach can handle eigenvalue problems with either Abelian or…
In this paper, we study the decentralized optimization problem of minimizing a finite sum of continuously differentiable and possibly nonconvex functions over a fixed-connected undirected network. We propose a unified decentralized…
We present an algorithm for minimizing a sum of functions that combines the computational efficiency of stochastic gradient descent (SGD) with the second order curvature information leveraged by quasi-Newton methods. We unify these…
Higher-order tensor methods were recently proposed for minimizing smooth convex and nonconvex functions. Higher-order algorithms accelerate the convergence of the classical first-order methods thanks to the higher-order derivatives used in…
Second-order optimization methods are among the most widely used optimization approaches for convex optimization problems, and have recently been used to optimize non-convex optimization problems such as deep learning models. The widely…
In this work, we propose a high-order multiscale method for an elliptic model problem with rough and possibly highly oscillatory coefficients. Convergence rates of higher order are obtained using the regularity of the right-hand side only.…
A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is…
Regularization approaches have demonstrated their effectiveness for solving ill-posed problems. However, in the context of variational restoration methods, a challenging question remains, which is how to find a good regularizer. While total…
In this paper, the problem of decentralized eigenvalue decomposition of a general symmetric matrix that is important, e.g., in Principal Component Analysis, is studied, and a decentralized online learning algorithm is proposed. Instead of…
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to incorporate local curvature information. Since the true…
When training large models, such as neural networks, the full derivatives of order 2 and beyond are usually inaccessible, due to their computational cost. Therefore, among the second-order optimization methods, it is common to bypass the…
Tensor decompositions, which represent an $N$-order tensor using approximately $N$ factors of much smaller dimensions, can significantly reduce the number of parameters. This is particularly beneficial for high-order tensors, as the number…
The problem of minimizing a separable convex function under linearly coupled constraints arises from various application domains such as economic systems, distributed control, and network flow. The main challenge for solving this problem is…
We consider distributed optimization problems where forming the Hessian is computationally challenging and communication is a significant bottleneck. We develop unbiased parameter averaging methods for randomized second order optimization…
This paper focuses on minimizing a smooth function combined with a nonsmooth regularization term on a compact Riemannian submanifold embedded in the Euclidean space under a decentralized setting. Typically, there are two types of approaches…
In recent years, various subspace algorithms have been developed to handle large-scale optimization problems. Although existing subspace Newton methods require fewer iterations to converge in practice, the matrix operations and full…
The modeling of electric machines and power transformers typically involves systems of nonlinear magnetostatics or -quasistatics, and their efficient and accurate simulation is required for the reliable design, control, and optimization of…
We consider the minimization of a continuous function over the intersection of a regular cone with an affine set via a new class of adaptive first- and second-order optimization methods, building on the Hessian-barrier techniques introduced…