Related papers: Fast Differentiable Matrix Square Root
We propose a fast second-order method that can be used as a drop-in replacement for current deep learning solvers. Compared to stochastic gradient descent (SGD), it only requires two additional forward-mode automatic differentiation…
The Fast Multipole Method (FMM) computes pairwise interactions between particles with an efficiency that scales linearly with the number of particles. The method works by grouping particles based on their spatial distribution and…
Forward modeling of wave scattering and radar imaging mechanisms is the key to information extraction from synthetic aperture radar (SAR) images. Like inverse graphics in optical domain, an inherently-integrated forward-inverse approach…
Volker Strassen first suggested an algorithm to multiply matrices with worst case running time less than the conventional $\mathcal{O}(n^3)$ operations in 1969. He also presented a recursive algorithm with which to invert matrices, and…
This course, intended for undergraduates familiar with elementary calculus and linear algebra, introduces the extension of differential calculus to functions on more general vector spaces, such as functions that take as input a matrix and…
Many neural learning algorithms require to solve large least square systems in order to obtain synaptic weights. Moore-Penrose inverse matrices allow for solving such systems, even with rank deficiency, and they provide minimum-norm vectors…
The most popular method for computing the matrix logarithm is a combination of the inverse scaling and squaring method in conjunction with a Pad\'e approximation, sometimes accompanied by the Schur decomposition. The main computational…
This paper introduces an efficient algorithm for computing the general oscillatory matrix functions. These computations are crucial for solving second-order semi-linear initial value problems. The method is exploited using the scaling and…
Low-rank regularization-based deep unrolling networks have achieved remarkable success in various inverse imaging problems (IIPs). However, the singular value decomposition (SVD) is non-differentiable when duplicated singular values occur,…
Robust fine-tuning aims to achieve competitive in-distribution (ID) performance while maintaining the out-of-distribution (OOD) robustness of a pre-trained model when transferring it to a downstream task. Recently, projected gradient…
Random backpropagation (RBP) is a variant of the backpropagation algorithm for training neural networks, where the transpose of the forward matrices are replaced by fixed random matrices in the calculation of the weight updates. It is…
We develop fast approximation algorithms for the minimum-cost version of the Bounded-Degree MST problem (BD-MST) and its generalization the Crossing Spanning Tree problem (Crossing-ST). We solve the underlying LP to within a $(1+\epsilon)$…
In this work we are interested in general linear inverse problems where the corresponding forward problem is solved iteratively using fixed point methods. Then one-shot methods, which iterate at the same time on the forward problem solution…
Aiming to provide a faster and convenient truncated SVD algorithm for large sparse matrices from real applications (i.e. for computing a few of largest singular values and the corresponding singular vectors), a dynamically shifted power…
Matrix functions such as square root, inverse roots, and orthogonalization play a central role in preconditioned gradient methods for neural network training. This has motivated the development of iterative algorithms that avoid explicit…
In this paper we consider symmetric, positive semidefinite (SPSD) matrix $A$ and present two algorithms for computing the $p$-Schatten norm $\|A\|_p$. The first algorithm works for any SPSD matrix $A$. The second algorithm works for…
Forward scatter radar (FSR) has emerged as an effective imaging modality for target detection, utilizing forward scattering (FS) signals to reconstruct two-dimensional shadow profile images of objects. However, real-world FS signals are…
A new O(N) algorithm based on a recursion method, in which the computational effort is proportional to the number of atoms N, is presented for calculating the inverse of an overlap matrix which is needed in electronic structure calculations…
We propose an iterative method for nonlinear semidefinite programs with box constraints. The search direction in the proposed method utilizes the distance from the current point to the boundary of a feasible set. The computation of the…
There are thousands of papers on rootfinding for nonlinear scalar equations. Here is one more, to talk about an apparently new method, which I call ``Inverse Cubic Iteration'' (ICI) in analogy to the Inverse Quadratic Iteration in Richard…