Related papers: Projected Newton method for large-scale Bayesian l…
A zero-finding technique for solving nonlinear equations more efficiently than they usually are with traditional iterative methods in which the order of convergence is improved is presented. The key idea in deriving this procedure is to…
These lecture notes for a graduate class present the regularization theory for linear and nonlinear ill-posed operator equations in Hilbert spaces. Covered are the general framework of regularization methods and their analysis via spectral…
Optimizing deep neural networks (DNNs) often suffers from the ill-conditioned problem. We observe that the scaling-based weight space symmetry property in rectified nonlinear network will cause this negative effect. Therefore, we propose to…
Despite a variety of available techniques the issue of the proper regularization parameter choice for inverse problems still remains one of the biggest challenges. The main difficulty lies in constructing a rule, allowing to compute the…
These lecture notes evolve around mathematical concepts arising in inverse problems. We start by introducing inverse problems through examples such as differentiation, deconvolution, computed tomography and phase retrieval. This then leads…
Newton's method for polynomial root finding is one of mathematics' most well-known algorithms. The method also has its shortcomings: it is undefined at critical points, it could exhibit chaotic behavior and is only guaranteed to converge…
Tikhonov regularization involves minimizing the combination of a data discrepancy term and a regularizing term, and is the standard approach for solving inverse problems. The use of non-convex regularizers, such as those defined by trained…
An evolutionary form of a generalized Bayesian update method, which is strictly derivative- free yet directed through an additive update term based purely on the statistical moments of the design variables, is proposed for nonlinear inverse…
Optimization problems with composite functions consist of an objective function which is the sum of a smooth and a (convex) nonsmooth term. This particular structure is exploited by the class of proximal gradient methods and some of their…
In this paper, we propose an inexact proximal Newton-type method for nonconvex composite problems. We establish the global convergence rate of the order $\mathcal{O}(k^{-1/2})$ in terms of the minimal norm of the KKT residual mapping and…
In this paper we propose a new class of iterative regularization methods for solving ill-posed linear operator equations. The prototype of these iterative regularization methods is in the form of second order evolution equation with a…
Interior-point methods for linear programming problems require the repeated solution of a linear system of equations. Solving these linear systems is non-trivial due to the severe ill-conditioning of the matrices towards convergence. This…
In this paper, we propose a globally convergent Newton type method to solve $\ell_0$ regularized sparse optimization problem. In fact, a line search strategy is applied to the Newton method to obtain global convergence. The Jacobian matrix…
Neural networks functions are supposed to be able to encode the desired solution of an inverse problem very efficiently. In this paper, we consider the problem of solving linear inverse problems with neural network coders. First we…
Nonlinear inverse problems have complicated landscapes. Hence the calculation with naive iterative schemes (e.g., Gauss-Newton or conjugate gradients) is trapped in local minima. The (first) Born approximation can avoid this trapping but…
Screening and working set techniques are important approaches to reducing the size of an optimization problem. They have been widely used in accelerating first-order methods for solving large-scale sparse learning problems. In this paper,…
Second-order optimization methods, such as cubic regularized Newton methods, are known for their rapid convergence rates; nevertheless, they become impractical in high-dimensional problems due to their substantial memory requirements and…
We develop a randomized Newton's method for solving differential equations, based on a fully connected neural network discretization. In particular, the randomized Newton's method randomly chooses equations from the overdetermined nonlinear…
Current methods for regularization in machine learning require quite specific model assumptions (e.g. a kernel shape) that are not derived from prior knowledge about the application, but must be imposed merely to make the method work. We…
The aim of this paper is to investigate the use of an entropic projection method for the iterative regularization of linear ill-posed problems. We derive a closed form solution for the iterates and analyze their convergence behaviour both…