Related papers: A preconditioned inverse iteration with an improve…

We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clipping akin to the widely used gradient clipping technique.…

In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We introduce a family of stochastic algorithms that can handle a…

We propose a variant of the classical conditional gradient method for sparse inverse problems with differentiable measurement models. Such models arise in many practical problems including superresolution, time-series modeling, and matrix…

We study and derive algorithms for nonlinear eigenvalue problems, where the system matrix depends on the eigenvector, or several eigenvectors (or their corresponding invariant subspace). The algorithms are derived from an implicit…

Pickrell has fully characterized the unitarily invariant probability measures on infinite Hermitian matrices, and an alternative proof of this classification has been found by Olshanski and Vershik. Borodin and Olshanski deduced from this…

We present an iterative algorithm for computing an invariant subspace associated with the algebraically smallest eigenvalues of a large sparse or structured Hermitian matrix A. We are interested in the case in which the dimension of the…

We consider parameterized variational inverse problems that are constrained by partial differential equations (PDEs). We seek to efficiently compute the solution of the inverse problem when auxiliary model parameters, which appear in the…

Inverse optimization, determining parameters of an optimization problem that render a given solution optimal, has received increasing attention in recent years. While significant inverse optimization literature exists for convex…

We introduce a new extragradient iterative process, motivated and inspired by [S. H. Khan, A Picard-Mann Hybrid Iterative Process, Fixed Point Theory and Applications, doi:10.1186/1687-1812-2013-69], for finding a common element of the set…

Eigenvalue problems have a distinctive forward-inverse structure and are fundamental to characterizing a system's thermal response, stability, and natural modes. Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative for…

We apply the method of inverse iteration to the Laplace eigenvalue problem with Robin and mixed Dirichlet-Neumann boundary conditions, respectively. For each problem, we prove convergence of the iterates to a non-trivial principal…

We study a nonlinear decomposition of a positive definite matrix into two components: the inverse of another positive definite matrix and a symmetric matrix constrained to lie in a prescribed linear subspace. Equivalently, the inverse…

The Levenberg-Marquardt algorithm is one of the most popular algorithms for finding the solution of nonlinear least squares problems. Across different modified variations of the basic procedure, the algorithm enjoys global convergence, a…

We consider the proximal gradient method on Riemannian manifolds for functions that are possibly not geodesically convex. Starting from the forward-backward-splitting, we define an intrinsic variant of the proximal gradient method that uses…

The analysis of the acceleration behavior of gradient-based eigensolvers with preconditioning presents a substantial theoretical challenge. In this work, we present a novel framework for preconditioning on Riemannian manifolds and introduce…

In this work, we analyze the regularizing property of the stochastic gradient descent for the efficient numerical solution of a class of nonlinear ill-posed inverse problems in Hilbert spaces. At each step of the iteration, the method…

This paper studies a novel algorithm for nonconvex composite minimization which can be interpreted in terms of dual space nonlinear preconditioning for the classical proximal gradient method. The proposed scheme can be applied to additive…

Conditional stability estimates require additional regularization for obtaining stable approximate solutions if the validity area of such estimates is not completely known. In this context, we consider ill-posed nonlinear inverse problems…

We give faster algorithms and improved sample complexities for estimating the top eigenvector of a matrix $\Sigma$ -- i.e. computing a unit vector $x$ such that $x^T \Sigma x \ge (1-\epsilon)\lambda_1(\Sigma)$: Offline Eigenvector…

Ample empirical evidence in deep neural network training suggests that a variety of optimizers tend to find nearly global optima. In this article, we adopt the reversed perspective that convergence to an arbitrary point is assumed rather…