Related papers: A weighted global GMRES algorithm with deflation f…
One of the challenges in optimization of high dimensional problems is finding appropriate solutions in a way that are as close as possible to the global optima. In this regard, one of the most common phenomena that occurs is the curse of…
Tensor ring (TR) decomposition has recently received increased attention due to its superior expressive performance for high-order tensors. However, the applicability of traditional TR decomposition algorithms to real-world applications is…
Bilevel optimization has been widely used in decision-making process. However, there still lacks an efficient algorithm to determine an optimal solution of a bilevel optimization problem, especially for a large-size problem. To bridge the…
The efficient solution of large-scale multiterm linear matrix equations is a challenging task in numerical linear algebra, and it is a largely open problem. We propose a new iterative scheme for symmetric and positive definite operators,…
We present a new method for the numerical solution of singular integral equations on the real axis. The method's value stems from an explicit formula for the Cauchy integral of a complex exponential multiplied by a rational function.…
We propose a new algorithm for efficiently solving the damped Fisher matrix in large-scale scenarios where the number of parameters significantly exceeds the number of available samples. This problem is fundamental for natural gradient…
A generalization of the Newton-based matrix splitting iteration method (GNMS) for solving the generalized absolute value equations (GAVEs) is proposed. Under mild conditions, the GNMS method converges to the unique solution of the GAVEs.…
Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization, $\ell_1$ norm regularized optimization, and $\ell_0$ norm regularized…
We consider the problem of maximizing an unknown function over a compact and convex set using as few observations as possible. We observe that the optimization of the function essentially relies on learning the induced bipartite ranking…
The object of this paper is to introduce a new and fascinating method of solving large linear equations, based on Cramer's rule or Gaussian elimination but employing Sylvester's determinant identity in its computation process. In addition,…
The sparse generalized eigenvalue problem arises in a number of standard and modern statistical learning models, including sparse principal component analysis, sparse Fisher discriminant analysis, and sparse canonical correlation analysis.…
We revisit the use of Stochastic Gradient Descent (SGD) for solving convex optimization problems that serve as highly popular convex relaxations for many important low-rank matrix recovery problems such as \textit{matrix completion},…
Sampling-based algorithms are classical approaches to perform Bayesian inference in inverse problems. They provide estimators with the associated credibility intervals to quantify the uncertainty on the estimators. Although these methods…
Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and…
Stellarator optimization is a multi-objective, non-convex problem characterized by a complex objective landscape containing many local minima. The solution resulting from a single optimization is highly sensitive to factors such as the…
Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to factorize the matrix into two compact low-rank factors, and…
The DGMRES method for solving Drazin-inverse solution of singular linear systems is generally used with restarting. But the restarting often slows down the convergence and DGMRES often stagnates. We show that adding some eigenvectors to the…
The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its…
We study distributed training of deep learning models in time-constrained environments. We propose a new algorithm that periodically pulls workers towards the center variable computed as a weighted average of workers, where the weights are…
Consider the generalized linear least squares (GLS) problem $\min\|Lx\|_2 \ \mathrm{s.t.} \ \|M(Ax-b)\|_2=\min$. The weighted pseudoinverse $A_{ML}^{\dag}$ is the matrix that maps $b$ to the minimum 2-norm solution of this GLS problem. By…