Related papers: Generalized Reduced Jacobian Method
This paper examines the nonconvex quadratically constrained quadratic programming (QCQP) problems using an iterative method. One of the existing approaches for solving nonconvex QCQP problems relaxes the rank one constraint on the unknown…
In this paper we introduce two conceptual algorithms for minimising abstract convex functions. Both algorithms rely on solving a proximal-type subproblem with an abstract Bregman distance based proximal term. We prove their convergence when…
Low-rank regularization (LRR) has been widely applied in various machine learning tasks, but the associated optimization is challenging. Directly optimizing the rank function under constraints is NP-hard in general. To overcome this…
Variable projection methods prove highly efficient in solving separable nonlinear least squares problems by transforming them into a reduced nonlinear least squares problem, typically solvable via the Gauss-Newton method. When solving…
This chapter presents reduced-rank linearly constrained minimum variance (LCMV) algorithms based on the concept of joint iterative optimization of parameters. The proposed reduced-rank scheme is based on a constrained robust joint iterative…
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite, number of loss functions. In this paper, we propose a novel Riemannian extension of the Euclidean stochastic variance…
By incorporating a new matrix splitting and the momentum acceleration into the relaxed-based matrix splitting (RMS) method \cite{soso2023}, a generalization of the RMS (GRMS) iterative method for solving the generalized absolute value…
In this paper we present a steepest descent method with Armijo's rule for multicriteria optimization in the Riemannian context. The well definedness of the sequence generated by the method is guaranteed. Under mild assumptions on the…
Projection methods aim to reduce the dimensionality of the optimization instance, thereby improving the scalability of high-dimensional problems. Recently, Sakaue and Oki proposed a data-driven approach for linear programs (LPs), where the…
In this paper, we study two general classes of optimization algorithms for kernel methods with convex loss function and quadratic norm regularization, and analyze their convergence. The first approach, based on fixed-point iterations, is…
The residual cutting (RC) method has been proposed as an outer-inner loop iteration for efficiently solving large and sparse linear systems of equations arising in solving numerically problems of elliptic partial differential equations.…
In this paper, we explain a new Iterative Method-Fixed Point and develop its convergence theory for finding approximate solutions of nonlinear equations in the setting of Banach spaces. First, we discuss the convergence analysis of our…
Projected Gradient Descent denotes a class of iterative methods for solving optimization programs. Its applicability to convex optimization programs has gained significant popularity for its intuitive implementation that involves only…
Neural networks are predominantly trained using gradient-based methods, yet in many applications their final predictions remain far from the accuracy attainable within the model's expressive capacity. We introduce Linearized Subspace…
Robust low-rank matrix completion (RMC), or robust principal component analysis with partially observed data, has been studied extensively for computer vision, signal processing and machine learning applications. This problem aims to…
Iterative methods for nonlinear monotone equations do not require the differentiability assumption on the residual function. This special property of the methods makes them suitable for solving large-scale nonsmooth monotone equations. In…
This paper investigates projection-free algorithms for stochastic constrained multi-level optimization. In this context, the objective function is a nested composition of several smooth functions, and the decision set is closed and convex.…
In this paper, we study a nonconvex continuous relaxation of MAP inference in discrete Markov random fields (MRFs). We show that for arbitrary MRFs, this relaxation is tight, and a discrete stationary point of it can be easily reached by a…
We develop a framework for convexifying a fairly general class of optimization problems. Under additional assumptions, we analyze the suboptimality of the solution to the convexified problem relative to the original nonconvex problem and…
We address composite optimization problems, which consist in minimizing the sum of a smooth and a merely lower semicontinuous function, without any convexity assumptions. Numerical solutions of these problems can be obtained by proximal…