Related papers: The Shifting Technique for Solving a Nonsymmetric …
Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is…
Sequential quadratic optimization algorithms are proposed for solving smooth nonlinear optimization problems with equality constraints. The main focus is an algorithm proposed for the case when the constraint functions are deterministic,…
We study acceleration and preconditioning strategies for a class of Douglas-Rachford methods aiming at the solution of convex-concave saddle-point problems associated with Fenchel-Rockafellar duality. While the basic iteration converges…
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and…
This paper addresses the numerical solution of the matrix square root problem. Two fixed point iterations are proposed by rearranging the nonlinear matrix equation $A - X^2 = 0$ and incorporating a positive scaling parameter. The proposals…
The Hadamard decomposition is a powerful technique for data analysis and matrix compression, which decomposes a given matrix into the element-wise product of two or more low-rank matrices. In this paper, we develop an efficient algorithm to…
We consider data-driven reduced-order models of partial differential equations with shift equivariance. Shift-equivariant systems typically admit traveling solutions, and the main idea of our approach is to represent the solution in a…
Algebraic Riccati equations with indefinite quadratic terms play an important role in applications related to robust controller design. While there are many established approaches to solve these in case of small-scale dense coefficients,…
In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex)…
We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form $\min_x \sum_i f_i(x)$. Specifically, we analyze the SAGA algorithm within an Incremental First-order Oracle framework, and show that it…
Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Most existing algorithms rely on one-sided information, such as the convexity (resp. concavity)…
An algorithmic method to exploit a general class of infinitesimal symmetries for reducing stochastic differential equations is presented and a natural definition of reconstruction, inspired by the classical reconstruction by quadratures, is…
Nonconvex-concave min-max problem arises in many machine learning applications including minimizing a pointwise maximum of a set of nonconvex functions and robust adversarial training of neural networks. A popular approach to solve this…
This paper considers a fractional programming problem (P) which minimizes a ratio of quadratic functions subject to a two-sided quadratic constraint. As is well-known, the fractional objective function can be replaced by a parametric family…
This paper introduces a novel Homogeneous Second-order Descent Ascent (HSDA) algorithm for nonconvex-strongly concave minimax optimization problems. At each iteration, HSDA uniquely computes a search direction by solving a homogenized…
We develop a pivot-shifted Carleman linearization framework for quantum algorithms solving quadratic nonlinear ordinary differential equations. By shifting the dynamics by a pivot state prior to Carleman lifting, and combining this with a…
In this paper, we study the problem of high-dimensional sparse quadratic discriminant analysis (QDA). We propose a novel classification method, termed SSQDA, which is constructed via constrained convex optimization based on the sample…
One approach for reducing run time and improving efficiency of machine learning is to reduce the convergence rate of the optimization algorithm used. Shuffling is an algorithm technique that is widely used in machine learning, but it only…
The rotation averaging problem is a fundamental task in computer vision applications. It is generally very difficult to solve due to the nonconvex rotation constraints. While a sufficient optimality condition is available in the literature,…
In this paper, we introduce some adaptive methods for solving variational inequalities with relatively strongly monotone operators. Firstly, we focus on the modification of the recently proposed, in smooth case [1], adaptive numerical…