Related papers: The conjugate gradient algorithm on well-condition…
As machine learning systems are increasingly used to make real world legal and financial decisions, it is of paramount importance that we develop algorithms to verify that these systems do not discriminate against minorities. We design a…
We study the fundamental optimization principles of self-attention, the defining mechanism of transformers, by analyzing the implicit bias of gradient-based optimizers in training a self-attention layer with a linear decoder in binary…
Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…
The convergence of deterministic policy gradient under the Hadamard parameterization is studied in the tabular setting and the linear convergence of the algorithm is established. To this end, we first show that the error decreases at an…
We consider properties of determinants of some random symmetric matrices issued from multivariate statistics: Wishart/Laguerre ensemble (sample covariance matrices), Uniform Gram ensemble (sample correlation matrices) and Jacobi ensemble…
Recently Johansson and Johnstone proved that the distribution of the (properly rescaled) largest principal component of the complex (real) Wishart matrix $ X^* \* X (X^t \*X) $ converges to the Tracy-Widom law as $ n, p $ (the dimensions of…
We study the high-dimensional asymptotic regimes of correlated Wishart matrices $d^{-1}\mathcal{Y}\mathcal{Y}^T$, where $\mathcal{Y}$ is a $n\times d$ Gaussian random matrix with correlated and non-stationary entries. We prove that under…
This paper introduces the Nystr\"om PCG algorithm for solving a symmetric positive-definite linear system. The algorithm applies the randomized Nystr\"om method to form a low-rank approximation of the matrix, which leads to an efficient…
We present an iterative method to diagonalise large matrices. The basic idea is the same as the conjugated gradient (CG) method, i.e, minimizing the Rayleigh quotient via its gradient and avoiding reintroduce errors to the directions of…
Given a square, nonsingular matrix of univariate polynomials $\mathbf{F}\in\mathbb{K}[x]^{n\times n}$ over a field $\mathbb{K}$, we give a deterministic algorithm for finding the determinant of $\mathbf{F}$. The complexity of the algorithm…
We propose a simple proof of the worst-case iteration complexity for the Difference of Convex functions Algorithm (DCA) for unconstrained minimization, showing that the global rate of convergence of the norm of the objective function's…
In this work, we provide a fundamental unified convergence theorem used for deriving expected and almost sure convergence results for a series of stochastic optimization methods. Our unified theorem only requires to verify several…
The conjugate gradient (CG) method is an efficient iterative method for solving large-scale strongly convex quadratic programming (QP). In this paper we propose some generalized CG (GCG) methods for solving the $\ell_1$-regularized…
A Wishart kernel density estimator (KDE) is introduced for density estimation in the cone of positive definite matrices. The estimator is boundary-aware and mitigates the boundary bias suffered by conventional KDEs, while remaining simple…
Several problems in modeling and control of stochastically-driven dynamical systems can be cast as regularized semi-definite programs. We examine two such representative problems and show that they can be formulated in a similar manner. The…
This paper is concerned with the problem of approximating the determinant of A for a large sparse symmetric positive definite matrix A. It is shown that an efficient solution of this problem is obtained by using a sparse approximate inverse…
We prove the convergence of the empirical spectral measure of Wishart matrices with size-dependent entries and characterize the limiting law by its moments. We apply our result to the cases where the entries are Bernoulli variables with…
Consider the product of $M$ quadratic random matrices with complex elements and no further symmetry, where all matrix elements of each factor have a Gaussian distribution. This generalises the classical Wishart-Laguerre Gaussian Unitary…
Constructive methods for matrices of multihomogeneous (or multigraded) resultants for unmixed systems have been studied by Weyman, Zelevinsky, Sturmfels, Dickenstein and Emiris. We generalize these constructions to mixed systems, whose…
We study the conjugacy approximation method in the context of Bayesian ranking and selection with unknown correlations. Under the assumption of normal-inverse-Wishart prior distribution, the posterior distribution remains a…