Related papers: On the Convergence of Stochastic Gradient Descent …

In this paper we address the numerical solution of nonlinear ill-posed systems by iterative regularization methods in the classes of Levenberg-Marquardt, trust-region and adaptive quadratic regularization procedures. Both with exact and…

We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…

The present article studies the minimization of convex, L-smooth functions defined on a separable real Hilbert space. We analyze regularized stochastic gradient descent (reg-SGD), a variant of stochastic gradient descent that uses a…

Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…

We consider linear inverse problems under white noise. These types of problems can be tackled with, e.g., iterative regularisation methods and the main challenge is to determine a suitable stopping index for the iteration. Convergence…

Implicit inverse problems, in which noisy observations of a physical quantity are used to infer a nonlinear functional applied to an associated function, are inherently ill posed and often exhibit non uniqueness of solutions. Such problems…

We derive and analyse a new variant of the iteratively regularized Landweber iteration, for solving linear and nonlinear ill-posed inverse problems. The method takes into account training data, which are used to estimate the interior of a…

In this paper, we prove optimal convergence rates results for regularisation methods for solving linear ill-posed operator equations in Hilbert spaces. The result generalises existing convergence rates results on optimality to general…

We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with general regularization and data-fit functions. In particular, we develop an inertial approach of which we…

In nonsmooth optimization, a negative subgradient is not necessarily a descent direction, making the design of convergent descent methods based on zeroth-order and first-order information a challenging task. The well-studied bundle methods…

Both for the theoretical and practical treatment of Inverse Problems, the modeling of the noise is a crucial part. One either models the measurement via a deterministic worst-case error assumption or assumes a certain stochastic behavior of…

Recent studies have shown that many nonconvex machine learning problems satisfy a generalized-smooth condition that extends beyond traditional smooth nonconvex optimization. However, the existing algorithms are not fully adapted to such…

This study investigates leveraging stochastic gradient descent (SGD) to learn operators between general Hilbert spaces. We propose weak and strong regularity conditions for the target operator to depict its intrinsic structure and…

The article deals with gradient-like iterative methods for solving nonlinear operator equations on Hilbert and Banach spaces. The authors formulate a general principle of studying such methods. This principle allows to formulate simple…

We suggest simple implementable modifications of conditional gradient and gradient projection methods for smooth convex optimization problems in Hilbert spaces. Usually, the custom methods attain only weak convergence. We prove strong…

This paper presents a general description of a parameter estimation inverse problem for systems governed by nonlinear differential equations. The inverse problem is presented using optimal control tools with state constraints, where the…

We establish sharp geometric Holder regularity estimates for Gradient for bounded solutions of a class of fully nonlinear elliptic equations with non-homogeneous degeneracy. Such regularity estimates simplify and generalize, to some extent,…

We study learning properties of accelerated gradient descent methods for linear least-squares in Hilbert spaces. We analyze the implicit regularization properties of Nesterov acceleration and a variant of heavy-ball in terms of…

Inverse problems in scientific computing often require optimization over infinite-dimensional Hilbert spaces. A commonly used solver in such settings is stochastic gradient descent (SGD), where gradients are approximated using randomly…

We propose a new gradient descent algorithm with added stochastic terms for finding the global optimizers of nonconvex optimization problems. A key component in the algorithm is the adaptive tuning of the randomness based on the value of…