Related papers: A derivative-free Levenberg-Marquardt method for s…
Least squares method is one of the simplest and most popular techniques applied in data fitting, imaging processing and high dimension data analysis. The classic methods like QR and SVD decomposition for solving least squares problems has a…
We give a damped inexact Newton method for entropy-regularized least-squares on the nonnegative orthant that converges globally at a linear rate with $O(\log\epsilon^{-1})$ iteration complexity, locally at a superlinear-to-quadratic rate,…
The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov…
This paper proposes a sparse regression strategy for discovery of ordinary differential equations from incomplete and noisy data. Inference is performed over both equation parameters and state variables using a statistically motivated…
A new Jacobian approximation is developed for use in quasi-Newton methods for solving systems of nonlinear equations. The new hypersecant Jacobian approximation is intended for the special case where the evaluation of the functions whose…
The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on…
We develop a computationally efficient algorithm for the automatic regularization of nonlinear inverse problems based on the discrepancy principle. We formulate the problem as an equality constrained optimization problem, where the…
We propose a general random subspace framework for unconstrained nonconvex optimization problems that requires a weak probabilistic assumption on the subspace gradient, which we show to be satisfied by various random matrix ensembles, such…
We propose a gradient-based Jacobi algorithm for a class of maximization problems on the unitary group, with a focus on approximate diagonalization of complex matrices and tensors by unitary transformations. We provide weak convergence…
A q-Levenberg-Marquardt method is an iterative procedure that blends a q-steepest descent and q-Gauss-Newton methods. When the current solution is far from the correct one the algorithm acts as the q-steepest descent method. Otherwise the…
We describe two algorithms for computing a sparse solution to a least-squares problem where the coefficient matrix can have arbitrary dimensions. We show that the solution vector obtained by our algorithms is close to the solution vector…
Constrained optimization problems exist in many domains of science, such as thermodynamics, mechanics, economics, etc. These problems are classically solved with the help of the Lagrange multipliers and the Lagrangian function. However, the…
Motivated by the increasing availability of high-performance parallel computing, we design a distributed parallel algorithm for linearly-coupled block-structured nonconvex constrained optimization problems. Our algorithm performs…
In this article we propose a novel strategy for choosing the Lagrange multipliers in the Levenberg-Marquardt method for solving ill-posed problems modeled by nonlinear operators acting between Hilbert spaces. Convergence analysis results…
The problem of finding roots or solutions of a nonlinear partial differential equation may be formulated as the problem of minimizing a sum of squared residuals. One then defines an evolution equation so that in the asymptotic limit a…
This study proposes a simple procedure to compute Efficient Pseudo Likelihood (EPL) estimator proposed by Dearing and Blevins (2024) for estimating dynamic discrete games, without computing Jacobians of equilibrium constraints. EPL…
In this paper, we propose an efficient sieving based secant method to address the computational challenges of solving sparse optimization problems with least-squares constraints. A level-set method has been introduced in [X. Li, D.F. Sun,…
We study the Levenberg-Marquardt (L-M) method for solving the highly nonlinear and ill-posed inverse problem of identifying the Robin coefficients in elliptic and parabolic systems. The L-M method transforms the Tikhonov regularized…
This paper argues that the method of least squares has significant unfulfilled potential in modern machine learning, far beyond merely being a tool for fitting linear models. To release its potential, we derive custom gradients that…
Minimizing the sum of a convex function and a composite function appears in various fields. The generalized Levenberg--Marquardt (LM) method, also known as the prox-linear method, has been developed for such optimization problems. The…