Related papers: Scalable Subspace Methods for Derivative-Free Nonl…
This work presents a novel version of recently developed Gauss-Newton method for solving systems of nonlinear equations, based on upper bound of solution residual and quadratic regularization ideas. We obtained for such method global…
In this paper, we develop a nonlinear reduction framework based on our recently introduced extended group finite element method. By interpolating nonlinearities onto approximation spaces defined with the help of finite elements, the…
When a physical system is modeled by a nonlinear function, the unknown parameters can be estimated by fitting experimental observations by a least-squares approach. Newton's method and its variants are often used to solve problems of this…
Least squares method is one of the simplest and most popular techniques applied in data fitting, imaging processing and high dimension data analysis. The classic methods like QR and SVD decomposition for solving least squares problems has a…
Solving large-scale optimization on-the-fly is often a difficult task for real-time computer graphics applications. To tackle this challenge, model reduction is a well-adopted technique. Despite its usefulness, model reduction often…
In contrast to the many continuous global optimization methods that assume the objective function and constraints are factorable, we study how to find globally maximal solutions to problems that are not factorable, focusing on a particular…
In this paper, we propose a subspace method based on neural networks for eigenvalue problems with high accuracy and low cost. We first construct a neural network-based orthogonal basis by some deep learning method and dimensionality…
We present an efficient framework for solving algebraically-constrained global non-convex polynomial optimization problems over subsets of the hypercube. We prove the existence of an equivalent nonlinear reformulation of such problems that…
Modern day engineering problems are ubiquitously characterized by sophisticated computer codes that map parameters or inputs to an underlying physical process. In other situations, experimental setups are used to model the physical process…
In linear regression we wish to estimate the optimum linear least squares predictor for a distribution over $d$-dimensional input points and real-valued responses, based on a small sample. Under standard random design analysis, where the…
The paper contains several theoretical results related to the weighted nonlinear least-squares problem for low-rank signal estimation, which can be considered as a Hankel structured low-rank approximation problem. A parameterization of the…
Time series averaging in dynamic time warping (DTW) spaces has been successfully applied to improve pattern recognition systems. This article proposes and analyzes subgradient methods for the problem of finding a sample mean in DTW spaces.…
A novel discontinuous Galerkin (DG) method is developed to solve time-dependent bi-harmonic type equations involving fourth derivatives in one and multiple space dimensions. We present the spatial DG discretization based on a mixed…
This paper proposes a random subspace trust-region algorithm for general convex-constrained derivative-free optimization (DFO) problems. Similar to previous random subspace DFO methods, the convergence of our algorithm requires a certain…
Following early work on Hessian-free methods for deep learning, we study a stochastic generalized Gauss-Newton method (SGN) for training DNNs. SGN is a second-order optimization method, with efficient iterations, that we demonstrate to…
For subspace estimation with an unknown colored noise, Factor Analysis (FA) is a good candidate for replacing the popular eigenvalue decomposition (EVD). Finding the unknowns in factor analysis can be done by solving a non-linear least…
The paper is devoted to the solution of a weighted nonlinear least-squares problem for low-rank signal estimation, which is related to Hankel structured low-rank approximation problems. A modified weighted Gauss-Newton method, which uses…
The finite element method, finite difference method, finite volume method and spectral method have achieved great success in solving partial differential equations. However, the high accuracy of traditional numerical methods is at the cost…
Derivative-free optimization methods are numerical methods for optimization problems in which no derivative information is used. Such optimization problems are widely seen in many real applications. One particular class of derivative-free…
In this paper, we propose a derivative-free Levenberg-Marquardt algorithm for nonlinear least squares problems, where the Jacobian matrices are approximated via orthogonal spherical smoothing. It is shown that the gradient models which use…