Related papers: A Stochastic Iteratively Regularized Gauss-Newton …
For numerous parameter and state estimation problems, assimilating new data as they become available can help produce accurate and fast inference of unknown quantities. While most existing algorithms for solving those kind of ill-posed…
We develop and analyze stochastic inexact Gauss-Newton methods for nonlinear least-squares problems and for nonlinear systems ofequations. Random models are formed using suitable sampling strategies for the matrices involved in the…
We study Newton type methods for inverse problems described by nonlinear operator equations $F(u)=g$ in Banach spaces where the Newton equations $F'(u_n;u_{n+1}-u_n) = g-F(u_n)$ are regularized variationally using a general data misfit…
Computing the regularized solution of Bayesian linear inverse problems as well as the corresponding regularization parameter is highly desirable in many applications. This paper proposes a novel iterative method, termed the Projected Newton…
This paper is concerned with the numerical solution of nonlinear ill-posed operator equations involving convex constraints. We study a Newton-type method which consists in applying linear Tikhonov regularization with convex constraints to…
Non linear regression models are a standard tool for modeling real phenomena, with several applications in machine learning, ecology, econometry... Estimating the parameters of the model has garnered a lot of attention during many years. We…
In this paper we consider the iteratively regularized Gauss-Newton method for solving nonlinear ill-posed inverse problems. Under merely Lipschitz condition, we prove that this method together with an a posteriori stopping rule defines an…
In this paper we investigate adaptive discretization of the iteratively regularized Gauss- Newton method IRGNM. All-at-once formulations considering the PDE and the measurement equation simultaneously allow to avoid (approximate) solution…
In this paper we propose an extension of the iteratively regularized Gauss--Newton method to the Banach space setting by defining the iterates via convex optimization problems. We consider some a posteriori stopping rules to terminate the…
In this paper we consider the Iteratively Regularized Gauss-Newton Method (IRGNM) in its classical Tikhonov version and in an Ivanov type version, where regularization is achieved by imposing bounds on the solution. We do so in a general…
We propose a Randomised Subspace Gauss-Newton (R-SGN) algorithm for solving nonlinear least-squares optimization problems, that uses a sketched Jacobian of the residual in the variable domain and solves a reduced linear least-squares on…
We introduce a new second order stochastic algorithm to estimate the entropically regularized optimal transport cost between two probability measures. The source measure can be arbitrary chosen, either absolutely continuous or discrete,…
This work presents a novel version of recently developed Gauss-Newton method for solving systems of nonlinear equations, based on upper bound of solution residual and quadratic regularization ideas. We obtained for such method global…
In this paper we study adaptive discretization of the iteratively regularized Gauss-Newton method IRGNM with an a posteriori (discrepancy principle) choice of the regularization parameter in each Newton step and of the stopping index. We…
We develop a randomized Newton method capable of solving learning problems with huge dimensional feature spaces, which is a common setting in applications such as medical imaging, genomics and seismology. Our method leverages randomized…
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates…
First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks. Second-order methods, despite their better convergence rate, are rarely used in practice due to the…
We propose a new globally convergent stochastic second order method. Our starting point is the development of a new Sketched Newton-Raphson (SNR) method for solving large scale nonlinear equations of the form $F(x)=0$ with $F:\mathbb{R}^p…
We develop a novel, fundamental and surprisingly simple randomized iterative method for solving consistent linear systems. Our method has six different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random…
An important question in deep learning is how higher-order optimization methods affect generalization. In this work, we analyze a stochastic Gauss-Newton (SGN) method with Levenberg-Marquardt damping and mini-batch sampling for training…