Related papers: Improvements to the Levenberg-Marquardt algorithm …
Many recent problems in signal processing and machine learning such as compressed sensing, image restoration, matrix/tensor recovery, and non-negative matrix factorization can be cast as constrained optimization. Projected gradient descent…
In this article we propose a novel strategy for choosing the Lagrange multipliers in the Levenberg-Marquardt method for solving ill-posed problems modeled by nonlinear operators acting between Hilbert spaces. Convergence analysis results…
We consider stochastic variational inequality problems where the mapping is monotone over a compact convex set. We present two robust variants of stochastic extragradient algorithms for solving such problems. Of these, the first scheme…
The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over…
Optimization techniques play a crucial role in estimating parameters and state information for nonlinear systems. However, some critical aspects of these problems have received little attention in previous research. In this paper, we…
This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints…
The paper studies a geometrically robust least-squares problem that extends classical and norm-based robust formulations. Rather than minimizing residual error for fixed or perturbed data, we interpret least-squares as enforcing approximate…
In this paper, a least-squares finite element method for scalar nonlinear hyperbolic balance laws is proposed and studied. The approach is based on a formulation that utilizes an appropriate Helmholtz decomposition of the flux vector and is…
The low-rank matrix approximation problem is ubiquitous in computational mathematics. Traditionally, this problem is solved in spectral or Frobenius norms, where the accuracy of the approximation is related to the rate of decrease of the…
Necessary optimality conditions in Lagrangian form and the sequential minimization framework are extended to mixed-integer nonlinear optimization, without any convexity assumptions. Building upon a recently developed notion of local…
Rational filter functions can be used to improve convergence of contour-based eigensolvers, a popular family of algorithms for the solution of the interior eigenvalue problem. We present a framework for the optimization of rational filters…
We develop a Frank-Wolfe algorithm with corrective steps, generalizing previous algorithms including blended conditional gradients, blended pairwise conditional gradients, and fully-corrective Frank-Wolfe. For this, we prove tight…
Structured Low-Rank Approximation is a problem arising in a wide range of applications in Numerical Analysis and Engineering Sciences. Given an input matrix $M$, the goal is to compute a matrix $M'$ of given rank $r$ in a linear or affine…
We present a simple formula to update the pseudoinverse of a full-rank rectangular matrix that undergoes a low-rank modification, and demonstrate its utility for solving least squares problems. The resulting algorithm can be dramatically…
This paper presents a new approach to solve linear and nonlinear model predictive control (MPC) problems that requires small memory footprint and throughput and is particularly suitable when the model and/or controller parameters change at…
In this paper, we study the equality constrained nonlinear least squares problem, where the Jacobian matrices of the objective function and constraints are unavailable or expensive to compute. We approximate the Jacobian matrices via…
In this contribution, we address the estimation of the frequency-dependent elastic parameters of polymers in the ultrasound range, which is formulated as an inverse problem. This inverse problem is implemented as a nonlinear regression-type…
We consider large-scale nonlinear least squares problems with sparse residuals, each of them depending on a small number of variables. A decoupling procedure which results in a splitting of the original problems into a sequence of…
We propose a new stochastic L-BFGS algorithm and prove a linear convergence rate for strongly convex and smooth functions. Our algorithm draws heavily from a recent stochastic variant of L-BFGS proposed in Byrd et al. (2014) as well as a…
New recursive least squares algorithms with rank two updates (RLSR2) that include both exponential and instantaneous forgetting (implemented via a proper choice of the forgetting factor and the window size) are introduced and systematically…