Related papers: The Landscape of Matrix Factorization Revisited
The nonnegative rank of a nonnegative matrix is the minimum number of nonnegative rank-one factors needed to reconstruct it exactly. The problem of determining this rank and computing the corresponding nonnegative factors is difficult;…
In this work, we analyze the global convergence property of coordinate gradient descent with random choice of coordinates and stepsizes for non-convex optimization problems. Under generic assumptions, we prove that the algorithm iterate…
The exact nonnegative matrix factorization (exact NMF) problem is the following: given an $m$-by-$n$ nonnegative matrix $X$ and a factorization rank $r$, find, if possible, an $m$-by-$r$ nonnegative matrix $W$ and an $r$-by-$n$ nonnegative…
Low-rank matrix factorizations are a class of linear models widely used in various fields such as machine learning, signal processing, and data analysis. These models approximate a matrix as the product of two smaller matrices, where the…
In this paper we fully describe the trajectory of gradient flow over diagonal linear networks in the limit of vanishing initialisation. We show that the limiting flow successively jumps from a saddle of the training loss to another until…
The low-rank matrix approximation problem with respect to the entry-wise $\ell_{\infty}$-norm is the following: given a matrix $M$ and a factorization rank $r$, find a matrix $X$ whose rank is at most $r$ and that minimizes $\max_{i,j}…
This paper primarily considers the robust estimation problem under Wasserstein distance constraints on the parameter and noise distributions in the linear measurement model with additive noise, which can be formulated as an…
Hermitian and unitary matrices are two representatives of the class of normal matrices whose full eigenvalue decomposition can be stably computed in quadratic computing com plexity. Recently, fast and reliable eigensolvers dealing with low…
In this paper, we consider the geometric landscape connection of the widely studied manifold and factorization formulations in low-rank positive semidefinite (PSD) and general matrix optimization. We establish a sandwich relation on the…
Low rank matrix factorization is a fundamental building block in machine learning, used for instance to summarize gene expression profile data or word-document counts. To be robust to outliers and differences in scale across features, a…
Many problems encountered in science and engineering can be formulated as estimating a low-rank object (e.g., matrices and tensors) from incomplete, and possibly corrupted, linear measurements. Through the lens of matrix and tensor…
Three operations on eigenvalues of real/complex/quaternion (corresponding to $\beta=1,2,4$) matrices, obtained from cutting out principal corners, adding, and multiplying matrices can be extrapolated to general values of $\beta>0$ through…
We study the saddle-points of the $p$-spin model -- the best understood example of a `complex' (rugged) landscape -- when its $N$ variables are complex. These points are the solutions to a system of $N$ random equations of degree $p-1$. We…
Many electronic structure methods rely on the minimization of the energy of the system with respect to the one-body reduced density matrix (1-RDM). To formulate a minimization algorithm, the 1-RDM is often expressed in terms of its…
Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between…
Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative $n \times m$ matrix $M$ into a product of a nonnegative $n \times d$ matrix $W$ and a nonnegative $d \times m$ matrix $H$. Restricted NMF requires in…
We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the…
Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. While the matrix rank is often approximated by the convex nuclear norm, the use of nonconvex low-rank regularizers has…
We show that absolutely minimizing functions relative to a convex Hamiltonian $H:\mathbb{R}^n \to \mathbb{R}$ are uniquely determined by their boundary values under minimal assumptions on $H.$ Along the way, we extend the known equivalences…
The problem of approximate joint diagonalization of a collection of matrices arises in a number of diverse engineering and signal processing problems. This problem is usually cast as an optimization problem, and it is the main goal of this…