Related papers: On the error estimate of gradient inclusions
Recently, Sharma et al. suggested a method called Layer-SElective-Rank reduction (LASER) which demonstrated that pruning high-order components of carefully chosen LLM's weight matrices can boost downstream accuracy -- without any…
This paper investigates the asymmetric low-rank matrix completion problem, which can be formulated as an unconstrained non-convex optimization problem with a nonlinear least-squares objective function, and is solved via gradient descent…
Estimating the trace of the inverse of a large matrix is an important problem in lattice quantum chromodynamics. A multilevel Monte Carlo method is proposed for this problem that uses different degree polynomials for the levels. The…
In this paper we investigate the generalization error of gradient descent (GD) applied to an $\ell_2$-regularized OLS objective function in the linear model. Based on our analysis we develop new methodology for computationally tractable and…
In the paper, we derive Li-Yau gradient estimates and Souplet Zhang type estimates of the following equation \begin{equation*} \begin{split} u_t= \Delta_\xi p+\lambda u+A(u) , \end{split} \end{equation*} on complete noncompact metric…
We consider robust low rank matrix estimation as a trace regression when outputs are contaminated by adversaries. The adversaries are allowed to add arbitrary values to arbitrary outputs. Such values can depend on any samples. We deal with…
We consider stopping criteria that balance algebraic and discretization errors for the conjugate gradient algorithm applied to high-order finite element discretizations of Poisson problems. Firstly, we introduce a new stopping criterion…
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of…
Minimizing a convex function of a measure with a sparsity-inducing penalty is a typical problem arising, e.g., in sparse spikes deconvolution or two-layer neural networks training. We show that this problem can be solved by discretizing the…
We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that…
The task of estimating a matrix given a sample of observed entries is known as the \emph{matrix completion problem}. Most works on matrix completion have focused on recovering an unknown real-valued low-rank matrix from a random sample of…
We consider the problem of estimating (diagonally dominant) M-matrices as precision matrices in Gaussian graphical models. These models exhibit intriguing properties, such as the existence of the maximum likelihood estimator with merely two…
We study the convergence properties of gradient descent for training deep linear neural networks, i.e., deep matrix factorizations, by extending a previous analysis for the related gradient flow. We show that under suitable conditions on…
Recent years have seen a flurry of activities in designing provably efficient nonconvex procedures for solving statistical estimation problems. Due to the highly nonconvex nature of the empirical loss, state-of-the-art procedures often…
The linear optimization degree gives an algebraic measure of complexity of optimizing a linear objective function over an algebraic model. Geometrically, it can be interpreted as the degree of a projection map on the {affine} conormal…
We investigate linearity of amalgams of subgroups of algebraic groups along intersections with algebraic subgroups. In the process, we establish linearity of certain "doubles" of linear groups, and obtain new examples of finitely generated…
We study the gradient Expectation-Maximization (EM) algorithm for Gaussian Mixture Models (GMM) in the over-parameterized setting, where a general GMM with $n>1$ components learns from data that are generated by a single ground truth…
We investigate the method of conjugate gradients, exploiting inaccurate matrix-vector products, for the solution of convex quadratic optimization problems. Theoretical performance bounds are derived, and the necessary quantities occurring…
Two-phase composites with non-overlapping inclusions randomly embedded in matrix are investigated. A straight forward approach is applied to estimate the effective properties of random 2D composites. First, deterministic boundary value…
We propose a penalized likelihood framework for estimating multiple precision matrices from different classes. Most existing methods either incorporate no information on relationships between the precision matrices, or require this…