Related papers: An NPDo Approach for Principal Joint SVD-type Bloc…
This paper is concerned with Partial Tensor Block-Diagonalization of a multiway tensor by orthonormal matrices so that the extracted block-diagonal part optimally represents the tensor. The basic idea is to maximize the block-diagonal part…
Matrix joint block-diagonalization (JBD) frequently arises from diverse applications such as independent component analysis, blind source separation, and common principal component analysis (CPCA), among others. Particularly, CPCA aims at…
Given a set of $p$ symmetric (real) matrices, the Orthogonal Joint Diagonalization (OJD) problem consists of finding an orthonormal basis in which the representation of each of these $p$ matrices is as close as possible to a diagonal…
Adaptive gradient approaches that automatically adjust the learning rate on a per-feature basis have been very popular for training deep networks. This rich class of algorithms includes Adagrad, RMSprop, Adam, and recent extensions. All…
Joint diagonalization of a set of positive (semi)-definite matrices has a wide range of analytical applications, such as estimation of common principal components, estimation of multiple variance components, and blind signal separation.…
In a recent paper, a new method was proposed to find the common invariant subspaces of a set of matrices. This paper invstigates the more general problem of putting a set of matrices into block triangular or block-diagonal form…
This paper introduces a novel framework for matrix diagonalization, recasting it as a sequential decision-making problem and applying the power of Decision Transformers (DTs). Our approach determines optimal pivot selection during…
Block coordinate descent (BCD) methods and their variants have been widely used in coping with large-scale nonconstrained optimization problems in many fields such as imaging processing, machine learning, compress sensing and so on. For…
It is well known that a set of non-defect matrices can be simultaneously diagonalized if and only if the matrices commute. In the case of non-commuting matrices, the best that can be achieved is simultaneous block diagonalization. Here we…
We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible…
We consider (stochastic) convex-concave saddle point (SP) problems with high-dimensional decision variables, arising in various applications including machine learning problems. To contend with the challenges in computing full gradients, we…
We present a matrix version of a known method of constructing common eigenvectors of two diagonalizable commuting matrices, thus enabling their simultaneous diagonalization. The matrices may have simple eigenvalues of multiplicity greater…
Tensor diagonalization means transforming a given tensor to an exactly or nearly diagonal form through multiplying the tensor by non-orthogonal invertible matrices along selected dimensions of the tensor. It is generalization of approximate…
Fast computation of singular value decomposition (SVD) is of great interest in various machine learning tasks. Recently, SVD methods based on randomized linear algebra have shown significant speedup in this regime. This paper attempts to…
Reducing communication complexity is critical for efficient decentralized optimization. The proximal decentralized optimization (PDO) framework is particularly appealing, as methods within this framework can exploit functional similarity…
The exact/approximate non-orthogonal general joint block diagonalization ({\sc nogjbd}) problem of a given real matrix set $\mathcal{A}=\{A_i\}_{i=1}^m$ is to find a nonsingular matrix $W\in\mathbb{R}^{n\times n}$ (diagonalizer) such that…
We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle…
Sparse optimization is a central problem in machine learning and computer vision. However, this problem is inherently NP-hard and thus difficult to solve in general. Combinatorial search methods find the global optimal solution but are…
Joint diagonalization, the process of finding a shared set of approximate eigenvectors for a collection of matrices, arises in diverse applications such as multidimensional harmonic analysis or quantum information theory. This task is…
Singular Value Decomposition (SVD) is the basic body of many statistical algorithms and few users question whether SVD is properly handling its job. SVD aims at evaluating the decomposition that best approximates a data matrix, given some…